Image interpolation method and device, equipment and storage medium

Abstract

The invention discloses an image interpolation method and device, equipment and a storage medium. The method comprises the steps of determining a to-be-processed image and a pixel value of each first pixel point in the to-be-processed image; based on the pixel value of each first pixel point, determining the edge intensity corresponding to each first pixel point; based on the edge intensity corresponding to each first pixel point, determining an interpolation mode corresponding to each first pixel point; performing interpolation processing on each first pixel point based on the interpolation mode corresponding to each first pixel point to obtain an interpolation result corresponding to each first pixel point; and obtaining an interpolated image based on the interpolation result corresponding to each first pixel point. According to the technical scheme of the invention, the edge intensity of each pixel point in the to-be-processed image is determined, and then the interpolation modes corresponding to different pixel points are determined based on the edge intensity, so that the image interpolation is completed, and therefore, a self-adaptive interpolation method adopting different interpolation modes for different pixel points can be provided. Therefore, the problems of pseudo-color, sawtooth, noise amplification and the like caused by interpolation are effectively reduced.

Description

Technical Field

[0001] The present disclosure relates to the technical field of image processing, and in particular to an image interpolation method and apparatus, device, and storage medium. Background Art

[0002] Bayer pattern sensors are widely used in the fields of consumer electronics and autonomous driving. Each pixel of a Bayer pattern sensor can only receive channel information of one of the R / G / B channels, and the channel information of the other two channels cannot be obtained.

[0003] To recover the missing information of the other two channels of each pixel from the Bayer array, it is necessary to rely on the demosaic interpolation method to obtain it. However, the demosaic interpolation method has a great impact on the final image quality. For example, an inappropriate demosaic interpolation method will cause serious image problems such as false color, edge jaggedness, and increased image noise. This will cause the accuracy of the perception result to be greatly reduced when the image obtained based on demosaic is processed. Summary of the invention

[0004] In order to solve the above technical problems, the present disclosure provides an image interpolation method and apparatus, a device, and a storage medium, which can solve the problem of poor image interpolation effect caused by traditional de-mosaicing interpolation methods.

[0005] The first aspect of the present disclosure provides an image interpolation method, including: determining an image to be processed, and pixel values ​​of each first pixel in the image to be processed; determining edge strengths corresponding to each first pixel in the image to be processed based on the pixel values ​​of each first pixel in the image to be processed; determining interpolation methods corresponding to each first pixel based on the edge strengths corresponding to each first pixel; performing interpolation processing on each first pixel based on the interpolation methods corresponding to each first pixel to obtain interpolation results corresponding to each first pixel; and obtaining an interpolated image based on the interpolation results corresponding to each first pixel.

[0006] The second aspect of the present disclosure provides an image interpolation device, including: an image determination module, used to determine an image to be processed, and pixel values ​​of each first pixel in the image to be processed; an edge strength determination module, used to determine the edge strength corresponding to each first pixel in the image to be processed based on the pixel value of each first pixel in the image to be processed; an interpolation method determination module, used to determine the interpolation method corresponding to each first pixel based on the edge strength corresponding to each first pixel; an image interpolation module, used to perform interpolation processing on each first pixel based on the interpolation method corresponding to each first pixel to obtain an interpolation result corresponding to each first pixel; and an image processing module, used to obtain an interpolated image based on the interpolation result corresponding to each first pixel.

[0007] According to a third aspect of the present disclosure, a computer-readable storage medium is provided, wherein the storage medium stores a computer program, and the computer program is used to execute the image interpolation method provided in the first aspect.

[0008] The fourth aspect of the present disclosure provides an electronic device, which includes: a processor; a memory for storing processor executable instructions; the processor is used to read the executable instructions from the memory and execute the instructions to implement the image interpolation method provided in the first aspect above.

[0009] A fifth aspect of the present disclosure provides a computer program product. When instructions in the computer program product are executed by a processor, the image interpolation method provided in the first aspect is executed.

[0010] Based on the image interpolation method provided by the present disclosure, by determining the edge strength of each first pixel point in the image to be processed, and then determining the interpolation method corresponding to each first pixel point based on the edge strength of each first pixel point, and finally using the interpolation method corresponding to each first pixel point to complete the image interpolation processing of the image to be processed, an adaptive interpolation method that uses different interpolation methods for different pixels can be provided, thereby effectively reducing the occurrence of problems such as false color, aliasing and noise amplification caused in the image interpolation process, and thus obtaining an image with better image quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 It is a schematic diagram of a scene to which the image enhancement method provided by an exemplary embodiment of the present disclosure is applicable.

[0012] Figure 2 It is a flowchart of an image interpolation method provided by an exemplary embodiment of the present disclosure.

[0013] Figure 3 It is a flowchart of an image interpolation method provided by another exemplary embodiment of the present disclosure.

[0014] Figure 4A It is a flowchart of an image interpolation method provided by yet another exemplary embodiment of the present disclosure.

[0015] Figure 4B is a schematic diagram of a pixel block provided by an exemplary embodiment of the present disclosure.

[0016] Figure 5 It is a flowchart of an image interpolation method provided by yet another exemplary embodiment of the present disclosure.

[0017] Figure 6 It is a flowchart of an image interpolation method provided by yet another exemplary embodiment of the present disclosure.

[0018] Figure 7 It is a flowchart of an image interpolation method provided by yet another exemplary embodiment of the present disclosure.

[0019] Figure 8 It is a flowchart of an image interpolation method provided by yet another exemplary embodiment of the present disclosure.

[0020] Fig. 9 The present invention is a schematic diagram of the structure of an image interpolation device provided by an exemplary embodiment of the present disclosure.

[0021] Fig.10 A schematic structural diagram of an image interpolation device provided by another exemplary embodiment of the present disclosure.

[0022] Fig.11 The present invention is a schematic structural diagram of an image interpolation device provided by yet another exemplary embodiment of the present disclosure.

[0023] Fig.12 The present invention is a schematic structural diagram of an image interpolation device provided by yet another exemplary embodiment of the present disclosure.

[0024] Fig.13 The present invention is a schematic structural diagram of an image interpolation device provided by yet another exemplary embodiment of the present disclosure.

[0025] Fig.14 It is a structural schematic diagram of an electronic device provided by an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] To explain the present disclosure, example embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. It should be understood that the present disclosure is not limited to the example embodiments.

[0027] It should be noted that the relative arrangement of components and steps, the numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless specifically stated otherwise.

[0028] Application Overview

[0029] The image interpolation method provided by the embodiments of the present disclosure can be applied to, for example, the front end of an environmental perception scene of a vehicle, and any other feasible scene.

[0030] like Figure 1 As shown, at least a plurality of image sensors 11 are provided on the vehicle 10, and the image output by the image sensor 11 is an original image without any processing, that is, a RAW domain image. The principle of image acquisition by the image sensor 11 is to set a layer of color filter array (Color Filter Array, CFA) in front of the image sensor 11, and place a single color filter on each pixel at intervals. Therefore, each pixel in the image acquired by the image sensor 11 has only one color component, and the image with only one color component for each pixel is a RAW domain image. Only 1 / 4 of the pixels of the R channel and the B channel of the RAW domain image are filled, and only 1 / 2 of the pixels of the G channel are filled, and both have missing color components. The image interpolation method provided in the embodiment of the present disclosure is an image interpolation based on the RAW domain image. The image interpolation method of the embodiment of the present disclosure can fill the three channels of R / G / B to obtain an interpolated RGB image. And illustratively, the interpolated RGB image can be used as input data for vehicle environment perception; of course, the interpolated RGB image can be further processed by image signal processing (ISP), and the processed image can be used as input data for vehicle environment perception. Figure 1 This is only an example, and the embodiments of the present disclosure are Figure 1 The number, location and function of the multiple image sensors 11 are not limited.

[0031] Image interpolation algorithms have a great impact on the final image quality. Poor image interpolation algorithms (such as bilinear interpolation algorithms) will cause serious image problems such as false color and edge jaggedness, and the overall image clarity will be reduced, seriously affecting the image quality. At the same time, inappropriate image interpolation methods will amplify image noise during interpolation, causing a significant negative impact on image quality.

[0032] In order to solve the above problems, the embodiment of the present disclosure provides an image interpolation method, which determines the edge strength of each first pixel point in the image to be processed, and then determines the interpolation method corresponding to each first pixel point based on the edge strength of each first pixel point, and finally uses the interpolation method corresponding to each first pixel point to complete the image interpolation processing of the image to be processed. In this way, an adaptive interpolation method that uses different interpolation methods for different pixels can be implemented, thereby effectively reducing the occurrence of problems such as false color, aliasing and noise amplification caused by the image interpolation process, and thus obtaining an image with better image quality.

[0033] Exemplary Methods

[0034] Figure 2 The present invention provides a flowchart of an image interpolation method. This embodiment can be applied to electronic devices, such as Figure 2 As shown, the method includes the following steps S201-S204.

[0035] Step S201: Determine an image to be processed and the pixel value of each first pixel in the image to be processed.

[0036] For example, Figure 1 As shown, the RAW domain image captured by the image sensor 11 can be first obtained, and then the RAW domain image can be used as the image to be processed. Among them, each first pixel point in the image to be processed refers to the pixel point in the image to be processed that needs to be interpolated. Furthermore, the pixel value of each first pixel point refers to the pixel value of the pixel point that needs to be interpolated.

[0037] In some examples, the pixel value of a pixel refers to the color or brightness of the pixel.

[0038] Step S202: determining the edge strength corresponding to each first pixel point in the image to be processed based on the pixel value of each first pixel point in the image to be processed.

[0039] Exemplarily, the edge strength corresponding to each first pixel point in the image to be processed refers to the amplitude of the gradient of each first pixel point. Among them, the gradient of the first pixel point reflects the rate of change of the pixel value of the first pixel point, and the magnitude of the rate of change of the pixel value can be used to measure the strength of the edge. For example, if the gradient of the first pixel point is larger, the edge strength of the first pixel point is larger. In the embodiment of the present disclosure, the gradient of each first pixel point can be obtained by determining the rate of change of the pixel value of each first pixel point in the x-direction and the y-direction of the image to be processed, thereby determining the edge strength corresponding to each first pixel point.

[0040] In some examples, the edge strength corresponding to each first pixel point can be obtained by performing a first-order or second-order differential operation on the grayscale change of each first pixel point in the processed image within its neighborhood. For example, the Sobel operator, Prewitt operator, Roberts operator, etc. are first-order differential operators, and the edge strength of the first pixel point can be determined by calculating the gradient of the first pixel point in the x-direction and the y-direction. For another example, the Laplacian operator is a second-order differential operator, and the edge strength of the first pixel point can be determined by finding the zero-crossing point of the second-order derivative through the operator.

[0041] It should be noted that the embodiments of the present disclosure do not limit the specific implementation and use methods of determining the edge strength corresponding to each first pixel in the image to be processed. Those skilled in the art may choose any method to determine the edge strength corresponding to each first pixel in the image to be processed.

[0042] Step S203: Determine the interpolation method corresponding to each first pixel point based on the edge strength corresponding to each first pixel point.

[0043] Exemplarily, two preset values ​​may be set: a first preset value (edge_threshold_high) and a second preset value (edge_threshold_low). Among them, the first pixel point whose edge strength is greater than edge_threshold_high corresponds to the first interpolation method, the first pixel point whose edge strength is between edge_threshold_low and edge_threshold_high corresponds to the second interpolation method, and the first pixel point whose edge strength is less than edge_threshold_high corresponds to the third interpolation method. For example, the first interpolation method may be a gradient interpolation method, that is, the gradient information of the image is used to guide the interpolation process. The second interpolation method may be a linear interpolation method, such as a bilinear interpolation method. The third interpolation method may be a mixed interpolation method, such as a mixture of a gradient interpolation method and a linear interpolation method.

[0044] In some examples, it can be considered that the first pixel point with an edge strength greater than edge_threshold_high belongs to a strong edge area, the first pixel point with an edge strength between edge_threshold_low and edge_threshold_high belongs to a texture area, and the first pixel point with an edge strength less than edge_threshold_low belongs to a flat area. Then, an interpolation method matching different areas is determined, and the interpolation method corresponding to the area is used to interpolate the first pixel point belonging to the area. For example, the interpolation method corresponding to the strong edge area is a gradient interpolation method; the interpolation method corresponding to the flat area is a linear interpolation method; the interpolation method corresponding to the texture area is a mixed interpolation method, and the mixed interpolation method refers to the process of combining the interpolation results of at least two single interpolation methods to realize pixel interpolation processing, for example, using a gradient interpolation method and a linear interpolation method to interpolate the pixels of the texture area respectively, and performing weighted summation of the interpolation results of the gradient interpolation method and the interpolation results of the linear interpolation method to obtain the interpolation result of the pixel point.

[0045] Step S204: performing interpolation processing on each first pixel point based on the interpolation method corresponding to each first pixel point to obtain an interpolation result corresponding to each first pixel point.

[0046] Exemplarily, after determining the interpolation method corresponding to each first pixel point, each first pixel point is interpolated using the interpolation method corresponding to each first pixel point to obtain the interpolation result corresponding to each first pixel point. For example, if the edge strength of the first pixel point is greater than edge_threshold_high, the first pixel point is interpolated using a gradient interpolation method to obtain the interpolation result of the first pixel point. For another example, if the edge strength of the first pixel point is greater than edge_threshold_low and less than edge_threshold_high, the first pixel point is interpolated using a hybrid interpolation method to obtain the interpolation result corresponding to the first pixel point. For another example, if the edge strength of the first pixel point is less than edge_threshold_low, the first pixel point is interpolated using a linear interpolation method to obtain the interpolation result corresponding to the first pixel point.

[0047] In some examples, each first pixel in the image to be processed may be processed pixel by pixel based on the position of the pixel to obtain an interpolation result corresponding to each first pixel. For example, if the size of the image to be processed is m×n pixels, the pixel value of the first pixel A with row and column coordinates (1,1) in the image is first determined, and then the interpolation method corresponding to the first pixel A is determined based on the pixel value and the first pixel A is interpolated. Then the pixel value of the first pixel B with row and column coordinates (1,2) in the image is determined, and then the interpolation method corresponding to the first pixel B is determined based on the pixel value and the first pixel B is interpolated. Then determine the pixel value of the first pixel point C with row and column coordinates (1,3) in the image, and then determine the interpolation method corresponding to the first pixel point C based on the pixel value and perform interpolation processing on the first pixel point C...and so on, until the pixel value of the first pixel point P with row and column coordinates (m,n) in the image is determined, and then determine the interpolation method corresponding to the first pixel point P based on the pixel value and perform interpolation processing on the first pixel point P, so that the interpolation results corresponding to each first pixel point in the image to be processed can be obtained.

[0048] Step S205: Obtain an interpolated image based on the interpolation results corresponding to each first pixel point.

[0049] Exemplarily, the known color components of each first pixel in the image to be processed and the interpolation results corresponding to each first pixel in the image to be processed (ie, each pixel to be interpolated) are combined to obtain an interpolated image.

[0050] The image interpolation method provided by the embodiment of the present disclosure determines the edge strength of each first pixel point in the image to be processed, and then determines the interpolation method corresponding to each first pixel point based on the edge strength of each first pixel point, and finally uses the interpolation method corresponding to each first pixel point to complete the image interpolation processing of the image to be processed. In this way, an adaptive interpolation method using different interpolation methods for different pixel points can be provided. For example, the first pixel point with a larger edge strength adopts a gradient interpolation method to reduce the aliasing and pseudo-color problems at the edge position of the image, the first pixel point with a smaller edge strength adopts a linear interpolation method to achieve fast interpolation and the interpolation process will not introduce new noise, and the first pixel point with an edge strength between the above two adopts a hybrid interpolation method to suppress noise while ensuring texture expression. This effectively reduces the occurrence of pseudo-color, aliasing and noise amplification problems caused in the image interpolation process, and thus obtains an image with better image quality.

[0051] like Figure 3 As shown in the above Figure 2 Based on the illustrated embodiment, step S203 may include the following steps S2031 to S2033.

[0052] Step S2031: In response to the edge strength corresponding to the first pixel being greater than a first preset value, determining that the interpolation mode corresponding to the first pixel is a gradient interpolation mode.

[0053] Exemplarily, if the edge strength corresponding to the first pixel point is greater than the first preset value, the interpolation method of the first pixel point is a gradient interpolation method. Among them, the basic principle of the gradient interpolation method is to use the gradient information of the image to guide the interpolation process. That is, the gradient interpolation method uses gradient information to construct an interpolation algorithm, thereby improving computing efficiency while maintaining image clarity. Compared with traditional interpolation methods, the gradient interpolation method can better handle local changes in the image. In the embodiment of the present disclosure, if the edge strength corresponding to the first pixel point is greater than the first preset value, it is considered that the first pixel point is in a high-frequency edge area, and then the first pixel point is interpolated using a gradient interpolation method.

[0054] Step S2032: In response to the edge strength corresponding to the first pixel being less than a second preset value, determining that the interpolation method corresponding to the first pixel is a linear interpolation method.

[0055] For example, if the edge strength corresponding to the first pixel is less than the second preset value, the interpolation method of the first pixel is a linear interpolation method; wherein, the linear interpolation method is a commonly used image processing technology, and the basic principle of linear interpolation is to estimate the R / G / B value of the pixel to be interpolated by using the values ​​of neighboring pixels. Therefore, compared with other interpolation methods, the linear interpolation method is simple to calculate, easy to implement, and the generated image is smooth.

[0056] In some examples, if the edge strength corresponding to the first pixel is greater than the first preset value, it is considered that the first pixel is in a low-frequency flat area, and then an isotropic linear interpolation method can be used for interpolation, such as bilinear interpolation, bicubic interpolation, etc. Among them, isotropic linear interpolation means that the properties and effects of the interpolation function in all directions are consistent, so that it can be ensured that the interpolation results have the same accuracy and effect in all directions regardless of the data distribution. At the same time, for the flat area, the embodiment of the present disclosure adopts an isotropic linear interpolation method, which can suppress the noise to a certain extent while interpolating, and will not amplify the noise during the interpolation process. In addition, the interpolation method of the low-frequency flat area is lower in computational complexity than that of the strong edge and texture area, so the amount of calculation can be reduced.

[0057] Step S2033: In response to the edge strength corresponding to the first pixel being greater than or equal to the second preset value and less than or equal to the first preset value, determine that the interpolation mode corresponding to the first pixel is a hybrid interpolation mode.

[0058] Exemplarily, if the edge strength corresponding to the first pixel point is greater than or equal to the second preset value and less than or equal to the first preset value, the interpolation method of the first pixel point is a mixed interpolation method; wherein the second preset value is less than the first preset value.

[0059] In some examples, the mixed interpolation method refers to using at least two different interpolation methods to interpolate the first pixel. Among them, each of the at least two different interpolation methods is a single interpolation method, and the single interpolation method includes but is not limited to a gradient interpolation method, a linear interpolation method, an adaptive interpolation method, etc. For example, the first pixel can be interpolated using a gradient interpolation method and a linear interpolation method, respectively, and the interpolation result of the gradient interpolation method and the interpolation result of the linear interpolation method are weighted and summed to obtain a mixed interpolation result of the first pixel.

[0060] The image interpolation method provided by the embodiment of the present disclosure adopts a gradient interpolation method when the edge strength of the first pixel point is greater than a first preset value, adopts a linear interpolation method when the edge strength of the first pixel point is less than a second preset value, and adopts a mixed interpolation method when the edge strength of the first pixel point is greater than or equal to the second preset value and less than or equal to the first preset value. In this way, an adaptive Bayer image de-mosaic interpolation method can be provided, and the method will adaptively adopt the corresponding interpolation method according to the characteristics of each area. That is, an interpolation method guided by edge gradient information is adopted for strong edge areas, an isotropic linear interpolation method is adopted for flat areas, and a mixed interpolation method is adopted for texture detail areas. Furthermore, by adopting an adaptive method that adopts different interpolation methods for different areas, better image quality performance can be obtained, and problems such as pseudo color, aliasing and noise amplification can be effectively reduced.

[0061] like Figure 4A As shown in the above Figure 3 Based on the illustrated embodiment, step S204 may include the following steps S2041a to S2045a.

[0062] Step S2041a: When it is determined that the interpolation mode corresponding to the first pixel point is the gradient interpolation mode, a pixel block including the first pixel point is determined in the image to be processed based on the first pixel point.

[0063] Exemplarily, a pixel block including the first pixel point may be determined with the first pixel point as the center. For example, a 3*3 pixel block with the first pixel point as the center is determined in the image to be processed. For another example, a 5*5 pixel block or a 7*7 pixel block with the first pixel point as the center is determined in the image to be processed.

[0064] It should be noted that the embodiment of the present disclosure does not limit the size and shape of the pixel block including the first pixel point, and those skilled in the art may make relevant settings according to actual usage.

[0065] Step S2042a: Determine the color components of a plurality of second pixels in the pixel block in the first preset channel.

[0066] Exemplarily, the pixel block determined in step S2041a includes at least a first pixel point and a plurality of second pixel points; wherein the first pixel point is a pixel point to be interpolated currently, and the second pixel point is a pixel point different from the first pixel point. It should be noted that the number of the plurality of second pixel points included in the pixel block is not limited in the embodiment of the present disclosure.

[0067] In some examples, the first preset channel is a G channel, and the color component of the first preset channel is a G color component.

[0068] For example, Figure 4B As shown, the pixel block 41 is a 3*3 pixel block determined with the first pixel point (3,4) as the center. Among them, the pixel point (2,4), the pixel point (3,3), the pixel point (3,5) and the pixel point (4,4) are multiple second pixel points in the pixel block 41. And, as Figure 4B The known color components and pixel coordinates of each pixel point in the pixel block 41 are marked on the corresponding pixel block; wherein the mark "G" indicates that the known color component of the pixel point is the G color component, the mark "B" indicates that the known color component of the pixel point is the B color component, and the mark "R" indicates that the known color component of the pixel point is the R color component. It can be further seen that the G color components of the multiple second pixel points in the pixel block 41 in the G channel are all known color components and can be directly obtained.

[0069] Step S2043a: determine a plurality of first gradient values ​​based on the color components of a plurality of second pixels in the pixel block in the first preset channel.

[0070] Exemplarily, a plurality of first gradient values ​​may be determined based on the change values ​​of the color components of a plurality of second pixels in the pixel block in the first preset channel.

[0071] In some examples, the first gradient value may represent the change value of the color component of the first preset channel in different directions within a neighborhood centered at the first pixel. Figure 4B As shown, for pixel block 41, the first first gradient value may represent the change value of the G color component in the horizontal direction in the neighborhood centered on the first pixel point (3,4), that is, the first first gradient value may be determined by the change value of the G color component in the horizontal direction in the neighborhood. The second first gradient value may represent the change value of the G color component in the vertical direction in the neighborhood centered on the first pixel point (3,4), that is, the second first gradient value may be determined by the change value of the G color component in the vertical direction in the neighborhood.

[0072] Step S2044a: determining an interpolation result corresponding to the first pixel point in the first preset channel based on the color components of the plurality of second pixel points in the pixel block in the first preset channel and the plurality of first gradient values.

[0073] For example, the color components of multiple second pixels in the pixel block in the first preset channel are usually known, and then on this basis, multiple first gradient values ​​can be combined to obtain the interpolation result corresponding to the first pixel in the first preset channel. Figure 4B As shown, multiple weights can be determined by multiple first gradient values, and then based on the G color component of pixel point (2,4), the G color component of pixel point (3,3), the G color component of pixel point (3,5) and the G color component of pixel point (4,4) and the multiple weights, the interpolation result of pixel point (3,4) in the G channel is determined.

[0074] Step S2045a: based on the interpolation result of the first pixel point corresponding to the first preset channel, determine the interpolation result of the first pixel point corresponding to the second preset channel.

[0075] For example, the second preset channel may be the R channel or the B channel. In the embodiment of the present disclosure, the order of channel interpolation may be to first interpolate the G channel, which has a larger proportion of known color components, and then further interpolate the R or B channel after completing the interpolation of the G channel. Therefore, based on the interpolation result corresponding to the first pixel point in the first preset channel, the interpolation result corresponding to the first pixel point in the second preset channel may be further determined. For example, Figure 4B As shown, the interpolation result of the pixel point (3, 4) in the G channel can be determined first, and then the interpolation result of the pixel point (3, 4) in the B channel can be determined.

[0076] The image interpolation method provided by the embodiment of the present disclosure first determines a pixel block including a first pixel point, then determines the color components of multiple second pixels in the pixel block in a first preset channel, determines multiple first gradient values ​​based on the color components of the multiple second pixels in the pixel block in the first preset channel, and then determines the interpolation result corresponding to the first pixel point in the first preset channel based on the color components of the multiple second pixels in the pixel block in the first preset channel and the multiple first gradient values; finally, based on the interpolation result corresponding to the first pixel point in the first preset channel, determines the interpolation result corresponding to the first pixel point in the second preset channel. In this way, the adaptive interpolation method combined with the edge gradient can effectively protect the image information at the edge position, thereby effectively reducing the aliasing and pseudo-color problems at the edge position while ensuring that the high contrast and high-frequency line areas of the image have good interpolation performance.

[0077] In some embodiments, the above step S2043a, based on the color components of multiple second pixels in the pixel block in the first preset channel, determines multiple first gradient values, including: determining multiple preset directions; based on the color components of the multiple second pixels in the first preset channel, determining the first difference of the color components of the multiple second pixels in each preset direction in the first preset channel; based on the first difference corresponding to each preset direction, determining the first gradient value corresponding to each preset direction.

[0078] Exemplarily, the plurality of preset directions may include a vertical direction and a horizontal direction. Further, a first difference value A of a color component of a plurality of second pixels in the vertical direction in a first preset channel and a first difference value B of a color component of a plurality of second pixels in the horizontal direction in the first preset channel may be determined. Thus, a first gradient value A corresponding to the vertical direction may be determined based on the first difference value A in the vertical direction, and a first gradient value B corresponding to the horizontal direction may be determined based on the first difference value B in the horizontal direction.

[0079] For example, Figure 4B As shown, the difference of the G color component in the vertical direction can be determined based on the G color component of the pixel point (2,4) and the G color component of the pixel point (4,4), and the difference of the G color component in the vertical direction is determined as the first gradient value corresponding to the vertical direction. The difference of the G color component in the horizontal direction can be determined based on the G color component of the pixel point (3,3) and the G color component of the pixel point (3,5), and the difference of the G color component in the horizontal direction is determined as the first gradient value corresponding to the horizontal direction.

[0080] like Figure 5 As shown in the above Figure 4A Based on the illustrated embodiment, step S2044a may include the following steps S21a to S22a.

[0081] Step S21a: Determine the weight corresponding to each second pixel point based on the multiple first gradient values.

[0082] Exemplarily, based on the multiple first gradient values, the weight corresponding to each of the multiple first gradient values ​​can be determined. The larger the first gradient value, the smaller the corresponding weight; conversely, the smaller the first gradient value, the larger the corresponding weight. In addition, the sum of the weights corresponding to the first gradient values ​​should be equal to 1.

[0083] In some examples, the first gradient value corresponding to each preset direction may be determined based on the first difference of the color component of the first preset channel of the multiple second pixel points in each preset direction in the multiple preset directions. Furthermore, the weight corresponding to the first gradient value in each preset direction may be determined based on the multiple first gradient values, so that the weight corresponding to the first gradient value in each preset direction is used as the weight of the multiple second pixel points in each preset direction. That is, the interpolation of the G channel in the embodiment of the present disclosure uses the gradient change information in the neighborhood as a guide, and the general principle is that the direction with a large gradient has a smaller weighted weight, and the direction with a small gradient has a larger weighted weight.

[0084] For example, Figure 4B As shown, the difference of the G color component in the vertical direction can be determined based on the G color component of the pixel point (2,4) and the G color component of the pixel point (4,4), and the difference of the G color component in the vertical direction is determined as the first gradient value corresponding to the vertical direction. Then, the weight corresponding to the first gradient value in the vertical direction is used as the weight of the pixel point (2,4) and the weight of the pixel point (4,4). Similarly, the difference of the G color component in the horizontal direction can be determined based on the G color component of the pixel point (3,3) and the G color component of the pixel point (3,5), and the difference of the G color component in the horizontal direction is determined as the first gradient value corresponding to the horizontal direction. Then, the weight corresponding to the first gradient value in the horizontal direction is used as the weight of the pixel point (3,3) and the weight of the pixel point (3,5).

[0085] In some examples, the weight corresponding to each second pixel point may be determined based on the first gradient value in the vertical direction and the first gradient value in the horizontal direction using the following formula (1) or formula (2):

[0086]

[0087] Among them, w hori is the weight corresponding to the first gradient value in the horizontal direction, w veri is the weight corresponding to the first gradient value in the vertical direction, grad veri is the first gradient value in the vertical direction, grad hori is the first gradient value in the horizontal direction.

[0088] Step S22a: determining an interpolation result corresponding to the first pixel in the first preset channel based on the color components of the plurality of second pixel points in the first preset channel and the weights corresponding to the second pixel points.

[0089] Exemplarily, the color components of the plurality of second pixels in the first preset channel may be weighted summed according to the weight corresponding to each second pixel to determine the interpolation result corresponding to the first pixel in the first preset channel.

[0090] In some examples, the interpolation result corresponding to the first pixel point in the first preset channel may be determined by the following formula (3):

[0091]

[0092] Where G1 is the interpolation result of the first pixel point corresponding to the first preset channel, N is the number of second pixel points, G 2j is the color component of the jth second pixel in the N second pixels in the first preset channel, w j G 2j Corresponding weights, the first preset channel is the G channel.

[0093] For example, Figure 4B As shown, the interpolation result of the first pixel point (3,4) on the G channel is the weighted average of the G color component of the pixel point (3,3), the G color component of the pixel point (3,5), the G color component of the pixel point (2,4) and the G color component of the pixel point (4,4).

[0094] The image interpolation method provided by the embodiment of the present disclosure first determines the weight corresponding to each second pixel based on multiple first gradient values, and then determines the interpolation result corresponding to the first pixel in the first preset channel based on the color components of the multiple second pixels in the first preset channel and the weights corresponding to each second pixel. In this way, the interpolation processing of the first preset channel can be completed by using an adaptive interpolation method that combines gradient information with different weights, thereby obtaining an accurate interpolation result.

[0095] like Figure 6 As shown in the above Figure 4A Based on the illustrated embodiment, step S2045a may include the following steps S21b to S23b.

[0096] Step S21b: Determine the color components of a plurality of third pixels in the pixel block in the first preset channel.

[0097] Exemplarily, the pixel block includes at least a first pixel point and a plurality of third pixel points; wherein the first pixel point is a pixel point to be interpolated currently, and the third pixel point is a pixel point different from the first pixel point. It should be noted that the number of the third pixel points is not limited in the embodiments of the present disclosure, and those skilled in the art can make relevant settings according to actual use requirements.

[0098] For example, Figure 4B As shown, pixel point (2,4) and pixel point (4,4) are multiple third pixel points in pixel block 41. Alternatively, pixel point (3,3) and pixel point (3,5) are multiple third pixel points in pixel block 41. Furthermore, it is necessary to first determine the color components of the multiple third pixel points on the G preset channel.

[0099] Step S22b: determining a plurality of second gradient values ​​based on the color components of the plurality of third pixels in the first preset channel and the interpolation results corresponding to the first pixel in the first preset channel.

[0100] For example, multiple second gradient values ​​may be determined based on the color components of multiple third pixels in the G channel and the interpolation results corresponding to the first pixel in the G preset channel. Figure 4B As shown, if pixel point (3,3) and pixel point (3,5) are multiple third pixel points in pixel block 41, since the first pixel point (3,4) has been determined before the interpolation result of the G channel, multiple second gradient values ​​can be determined based on the G color component of pixel point (3,3), the G color component of pixel point (3,5) and the interpolation result of the first pixel point (3,4) in the G channel.

[0101] Step S23b: determining the interpolation result of the first pixel corresponding to the second preset channel based on the multiple second gradient values ​​and the interpolation result of the first pixel corresponding to the first preset channel.

[0102] Exemplarily, multiple weights can be determined based on multiple second gradient values, and then the first pixel point is interpolated in the second preset channel based on the interpolation result corresponding to the first pixel point in the first preset channel and combined with the weights corresponding to the multiple second gradient values ​​to obtain the interpolation result corresponding to the first pixel point in the second preset channel.

[0103] In some examples, the interpolation result of the first pixel in the R / B channel can be determined based on the interpolation result of the first pixel in the G channel and multiple second gradient values ​​using the principle of constant color difference. The color difference refers to the difference between the chromaticity signal (i.e., the R / B component) and the brightness signal (i.e., the G component), and the principle of constant color difference means that within a very small range in the image, the color difference of the current pixel is the same as the color difference of the surrounding pixels. That is, the principle of constant color difference can be expressed by the formula R(i,j)-G(i,j)=R(i,j+1)-G(i,j+1) or the formula R(i,j)-G(i,j)=R(i+1,j)-G(i+1,j) or the formula B(i,j)-G(i,j)=B(i,j+1)-G(i,j+1) or the formula B(i,j)-G(i,j)=B(i+1,j)-G(i+1,j).

[0104] The image interpolation method provided by the embodiment of the present disclosure first determines the color components of multiple third pixels in the pixel block in the first preset channel; then determines multiple second gradient values ​​based on the color components of the multiple third pixels in the first preset channel and the interpolation results corresponding to the first pixel in the first preset channel; finally, determines the interpolation result corresponding to the first pixel in the second preset channel based on the multiple second gradient values ​​and the interpolation result corresponding to the first pixel in the first preset channel, so that the interpolation of the second preset channel can be guided based on the first preset channel information obtained by interpolation, so as to conveniently and quickly obtain the interpolation result of the second preset channel.

[0105] In some embodiments, the above step S22b, determining multiple second gradient values ​​based on the color components of multiple third pixels in the first preset channel and the interpolation results corresponding to the first pixel in the first preset channel, includes: determining the second difference between the color component of each third pixel in the first preset channel and the interpolation result corresponding to the first pixel in the first preset channel; determining multiple second gradient values ​​based on the second difference corresponding to each third pixel.

[0106] Exemplarily, a second difference between a color component of each third pixel in a plurality of third pixels and an interpolation result of the first pixel in the G channel is determined, and based on the second difference corresponding to each third pixel, a second gradient value corresponding to each third pixel is determined.

[0107] In some examples, such as Figure 4B As shown, if pixel point (3,3) and pixel point (3,5) are multiple third pixel points in pixel block 41, since the interpolation result of the first pixel point (3,4) in the G channel has been determined, the first second gradient value can be determined based on the difference between the G color component of pixel point (3,3) and the interpolation result of the first pixel point (3,4) in the G channel. Similarly, the second second gradient value can be determined based on the difference between the G color component of pixel point (3,5) and the interpolation result of the first pixel point (3,4) in the G channel.

[0108] In some embodiments, the above-mentioned step S23b, based on multiple second gradient values ​​and the interpolation result corresponding to the first pixel in the first preset channel, determines the interpolation result corresponding to the first pixel in the second preset channel, including: determining the third difference between the color component of each third pixel in the second preset channel and the color component of the first preset channel based on the color components of multiple third pixels in the first preset channel and the color components of multiple third pixels in the second preset channel; determining the weight of each third pixel based on multiple second gradient values; determining the interpolation result corresponding to the first pixel in the second preset channel based on the third difference corresponding to each third pixel, the weight of each third pixel and the interpolation result corresponding to the first pixel in the first preset channel.

[0109] Exemplarily, the weight of the third pixel in the embodiment of the present disclosure refers to the weight corresponding to the color difference channel of the third pixel. The weighted weight of the color difference channel (i.e., RG color difference channel or BG color difference channel) of each third pixel can be determined using the gradient information of the G channel. The larger the gradient, the smaller the weight corresponding to the color difference channel of the third pixel, and the smaller the gradient, the larger the weight corresponding to the color difference channel of the third pixel.

[0110] Exemplarily, the interpolation result of the first pixel in the R channel can be determined based on the color components of the third pixels in the G channel and the R channel, the color components of the first pixel in the G channel, and the weights corresponding to the color difference channels of each third pixel using the color difference constancy principle. Similarly, the interpolation result of the first pixel in the B channel can be determined based on the color components of the third pixels in the G channel and the B channel, the color components of the first pixel in the G channel, and the weights corresponding to the color difference channels of each third pixel using the color difference constancy principle.

[0111] In some examples, the interpolation result of the first pixel in the R channel may be determined by the following formula (4), and the interpolation result of the first pixel in the B channel may be determined by the following formula (5):

[0112]

[0113] Among them, R1 is the interpolation result corresponding to the first pixel in the R channel, B1 is the interpolation result corresponding to the first pixel in the B channel, C is the number of third pixels, G1 is the interpolation result of the first pixel in the G channel, R k is the R color component of the kth third pixel among the C third pixels, B k is the B color component of the kth third pixel among the C third pixels, G k is the G color component of the kth third pixel among the C third pixels, w k is the weight of the kth third pixel.

[0114] For example, Figure 4BAs shown, the first second gradient value can be determined based on the G color component of the pixel point (3,3) and the G color component of the pixel point (3,4). And, the second second gradient value can be determined based on the G color component of the pixel point (3,5) and the G color component of the pixel point (3,4). Then, the weight corresponding to the first second gradient value is used as the weight of the pixel point (3,3), and the weight corresponding to the second second gradient value is used as the weight of the pixel point (3,5). At this time, the interpolation result of the first pixel point (3,4) in the B channel can be determined based on the weight of the pixel point (3,3), the weight of the pixel point (3,5), the interpolation result of the first pixel point (3,4) in the G channel, the G color components of the pixel point (3,3) and the pixel point (3,5), and the B color components of the pixel point (3,3) and the pixel point (3,5).

[0115] Exemplarily, the weight corresponding to the first second gradient value and the weight corresponding to the second second gradient value may be determined by the following formula (6) or formula (7):

[0116]

[0117] Among them, w 21 is the weight corresponding to the first and second gradient values, w 22 is the weight corresponding to the second gradient value, grad 21 is the first second gradient value, grad 22 is the second gradient value.

[0118] It should be noted that if the pixel block is Figure 4B For pixel block 41 in , it is necessary to first calculate the B color components of pixel point (3,3) and pixel point (3,5), and then determine the interpolation result of the first pixel point (3,4) in the B channel. Figure 4B The pixel blocks in are only exemplary. In other examples, if the color components of the pixel point (3,3) and the pixel point (3,5) in the second preset channel are known color components, they can be directly obtained and used without calculation.

[0119] like Figure 7 As shown in the above Figure 3 Based on the illustrated embodiment, step S204 may include the following steps S2041b to S2044b.

[0120] Step S2041b, when determining that the interpolation method corresponding to the first pixel point is a hybrid interpolation method, determine the first weight corresponding to the interpolation of the first pixel point using a gradient interpolation method, and determine the second weight corresponding to the interpolation of the first pixel point using a linear interpolation method.

[0121] Exemplarily, if it is determined that the interpolation method corresponding to the first pixel point is a hybrid interpolation method, and the hybrid interpolation method includes a gradient interpolation method and a linear interpolation method, it is necessary to determine a first weight corresponding to the gradient interpolation method and a second weight corresponding to the linear interpolation method.

[0122] In some examples, for the mid-frequency texture area, a combination of gradient interpolation and linear interpolation weighting is used for interpolation processing. The weighting weight is determined by the edge strength of the first pixel. The weight distribution principle is that when the edge strength of the first pixel is large, the weight of the gradient information guidance method accounts for a higher proportion; conversely, when the edge strength of the first pixel is small, the weight of the linear interpolation method accounts for a higher proportion.

[0123] Step S2042b: interpolate the first pixel point based on a gradient interpolation method to obtain a first interpolation result corresponding to the first pixel point.

[0124] For example, it is possible to use Figure 4A The first pixel point is interpolated using the gradient interpolation method in to obtain a first interpolation result corresponding to the first pixel point. Of course, other gradient interpolation methods can also be used to interpolate the first pixel point to obtain a first interpolation result corresponding to the first pixel point, and the embodiment of the present disclosure does not limit this.

[0125] Step S2043b: interpolate the first pixel point based on a linear interpolation method to obtain a second interpolation result corresponding to the first pixel point.

[0126] For example, the first pixel point may be interpolated using the linear interpolation method in step S2032 to obtain a second interpolation result corresponding to the first pixel point. Of course, the first pixel point may also be interpolated using other linear interpolation methods different from the linear interpolation method in step S2032 to obtain a second interpolation result corresponding to the first pixel point, and the embodiments of the present disclosure are not limited to this.

[0127] Step S2044b: determine the interpolation result corresponding to the first pixel based on the first weight, the second weight, the first interpolation result and the second interpolation result.

[0128] For example, when the interpolation method corresponding to the first pixel point is a hybrid interpolation method, the interpolation result corresponding to the first pixel point can be determined by the following formula (8):

[0129]

[0130] Among them, demosaic is the interpolation result corresponding to the first pixel point, demosaic1 is the first interpolation result, demosaic2 is the second interpolation result, w1 is the first weight corresponding to the first interpolation result, and w2 is the second weight corresponding to the second interpolation result.

[0131] The image interpolation method provided by the embodiment of the present disclosure first determines a first weight corresponding to interpolation of a first pixel point using a gradient interpolation method, and determines a second weight corresponding to interpolation of the first pixel point using a linear interpolation method; then interpolates the first pixel point based on the gradient interpolation method to obtain a first interpolation result corresponding to the first pixel point; interpolates the first pixel point based on the linear interpolation method to obtain a second interpolation result corresponding to the first pixel point; finally, based on the first weight, the second weight, the first interpolation result and the second interpolation result, determines the interpolation result corresponding to the first pixel point, so that the weights corresponding to the gradient interpolation method and the linear interpolation method can be adaptively determined, thereby suppressing noise while retaining image texture details.

[0132] like Figure 8 As shown in the above Figure 7 Based on the illustrated embodiment, step S2041b may include the following steps S21c to S22c.

[0133] Step S21c: When it is determined that the interpolation method corresponding to the first pixel point is the hybrid interpolation method, the edge intensity corresponding to the first pixel point is normalized to obtain a first weight corresponding to the interpolation of the first pixel point using the gradient interpolation method.

[0134] Exemplarily, the edge intensity corresponding to the first pixel point may be normalized to obtain a value of 0-1 distribution, and then the 0-1 distribution value is used as the first weight corresponding to the first pixel point interpolated by gradient interpolation.

[0135] Step S22c: Based on the first weight, determine a second weight corresponding to the linear interpolation method for interpolating the first pixel point.

[0136] Exemplarily, the sum of the first weight and the second weight should be 1, so the first weight can be used to directly determine the second weight corresponding to the interpolation of the first pixel point using linear interpolation.

[0137] The image interpolation method provided by the embodiment of the present disclosure first normalizes the edge strength corresponding to the first pixel point to obtain the first weight corresponding to the interpolation of the first pixel point using the gradient interpolation method, and then determines the second weight corresponding to the interpolation of the first pixel point using the linear interpolation method based on the first weight. In this way, the weights corresponding to different interpolation methods in the hybrid interpolation method can be adaptively determined according to the edge strength of the first pixel point.

[0138] In some embodiments, the method further includes: converting the interpolated image from the first color space to the second color space to obtain a target interpolated image; and performing filtering processing on the chroma channel of the target interpolated image to obtain an optimized target interpolated image.

[0139] Exemplarily, the first color space is an RGB color space, and the second color space is a YUV color space. "Y" represents brightness, that is, grayscale value. "U" and "V" represent chromaticity, which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0140] In some examples, after the interpolation of the image to be processed is completed, the interpolated image to be processed can be subjected to pseudo color suppression processing, that is, the interpolated RGB three-channel image is converted to the YUV domain for processing, and filtering is performed on the UV channel to suppress pseudo color. The filtering can be performed by median filtering or the like.

[0141] Exemplary Devices

[0142] Fig. 9 An image interpolation device provided by an embodiment of the present disclosure, such as Fig. 9 As shown, the image interpolation device 900 includes an image determination module 901 , an edge strength determination module 902 , an interpolation mode determination module 903 , an image interpolation module 904 and an image processing module 905 .

[0143] An image determination module 901 is used to determine an image to be processed and a pixel value of each first pixel point in the image to be processed;

[0144] An edge strength determination module 902 is used to determine the edge strength corresponding to each first pixel point in the image to be processed based on the pixel value of each first pixel point in the image to be processed;

[0145] An interpolation mode determination module 903, configured to determine an interpolation mode corresponding to each first pixel point based on an edge strength corresponding to each first pixel point;

[0146] An image interpolation module 904 is used to perform interpolation processing on each first pixel point based on the interpolation method corresponding to each first pixel point to obtain an interpolation result corresponding to each first pixel point;

[0147] The image processing module 905 is used to obtain an interpolated image based on the interpolation results corresponding to each first pixel point.

[0148] In some embodiments, Fig.10 As shown, the interpolation mode determination module 903 includes a first mode determination unit 9031 , a second mode determination unit 9032 and a third mode determination unit 9033 .

[0149] A first mode determining unit 9031 is configured to determine that the interpolation mode corresponding to the first pixel point is a gradient interpolation mode in response to the edge strength corresponding to the first pixel point being greater than a first preset value;

[0150] A second mode determining unit 9032 is configured to determine that the interpolation mode corresponding to the first pixel point is a linear interpolation mode in response to the edge strength corresponding to the first pixel point being less than a second preset value;

[0151] The third mode determination unit 9033 is used to determine that the interpolation mode corresponding to the first pixel point is a hybrid interpolation mode in response to the edge strength corresponding to the first pixel point being greater than or equal to the second preset value and less than or equal to the first preset value.

[0152] In some embodiments, Fig.11 As shown, the image interpolation module 904 includes a pixel block determination unit 9041a, a color component determination unit 9042a, a first gradient value determination unit 9043a, a first interpolation result determination unit 9044a and a second interpolation result determination unit 9045a.

[0153] A pixel block determining unit 9041a, configured to determine, when determining that the interpolation mode corresponding to the first pixel point is a gradient interpolation mode, a pixel block including the first pixel point in the image to be processed based on the first pixel point;

[0154] A color component determination unit 9042a, configured to determine color components of a plurality of second pixels in a pixel block in a first preset channel;

[0155] A first gradient value determining unit 9043a, configured to determine a plurality of first gradient values ​​based on color components of a plurality of second pixels in a pixel block in a first preset channel;

[0156] A first interpolation result determining unit 9044a is used to determine the interpolation result corresponding to the first pixel point in the first preset channel based on the color components of the second pixel points in the pixel block in the first preset channel and the first gradient values;

[0157] The second interpolation result determining unit 9045a is used to determine the interpolation result corresponding to the first pixel point in the second preset channel based on the interpolation result corresponding to the first pixel point in the first preset channel.

[0158] In some embodiments, the first gradient value determination unit 9043a is specifically used to determine multiple preset directions; based on the color components of multiple second pixel points in the first preset channel, determine the first difference in the color components of multiple second pixel points in each preset direction in the first preset channel; based on the first difference corresponding to each preset direction, determine the first gradient value corresponding to each preset direction.

[0159] In some embodiments, the first interpolation result determination unit 9044a is specifically used to determine the weight corresponding to each second pixel point based on multiple first gradient values; based on the color components of multiple second pixel points in the first preset channel and the weights corresponding to each second pixel point, determine the interpolation result corresponding to the first pixel point in the first preset channel.

[0160] In some embodiments, the second interpolation result determination unit 9045a is used to determine the color components of multiple third pixels in the pixel block in the first preset channel; determine multiple second gradient values ​​based on the color components of the multiple third pixels in the first preset channel and the interpolation results corresponding to the first pixel in the first preset channel; determine the interpolation result corresponding to the first pixel in the second preset channel based on the multiple second gradient values ​​and the interpolation results corresponding to the first pixel in the first preset channel.

[0161] In some embodiments, the second interpolation result determination unit 9045a is specifically used to determine the color components of multiple third pixels in the pixel block in the first preset channel; determine the second difference between the color component of each third pixel in the first preset channel and the interpolation result corresponding to the first pixel in the first preset channel; determine multiple second gradient values ​​based on the second difference corresponding to each third pixel; determine the interpolation result corresponding to the first pixel in the second preset channel based on the multiple second gradient values ​​and the interpolation result corresponding to the first pixel in the first preset channel.

[0162] In some embodiments, the second interpolation result determination unit 9045a is specifically used to determine the color components of multiple third pixels in the pixel block in the first preset channel; determine the second difference between the color component of each third pixel in the first preset channel and the interpolation result corresponding to the first pixel in the first preset channel; determine multiple second gradient values ​​based on the second difference corresponding to each third pixel; determine the third difference between the color component of each third pixel in the second preset channel and the color component of the first preset channel based on the color components of the multiple third pixels in the first preset channel and the color components of the multiple third pixels in the second preset channel; determine the weight of each third pixel based on the multiple second gradient values; determine the interpolation result corresponding to the first pixel in the second preset channel based on the third difference corresponding to each third pixel, the weight of each third pixel and the interpolation result corresponding to the first pixel in the first preset channel.

[0163] In some embodiments, Fig.12 As shown, the image interpolation module 904 includes an interpolation weight determination unit 9041b, a first interpolation processing unit 9042b, a second interpolation processing unit 9043b and an interpolation result determination unit 9044b.

[0164] An interpolation weight determination unit 9041b is used to determine, when the interpolation method corresponding to the first pixel point is a hybrid interpolation method, determine a first weight corresponding to interpolating the first pixel point using a gradient interpolation method, and determine a second weight corresponding to interpolating the first pixel point using a linear interpolation method;

[0165] A first interpolation processing unit 9042b is used to perform interpolation processing on the first pixel point based on a gradient interpolation method to obtain a first interpolation result corresponding to the first pixel point;

[0166] A second interpolation processing unit 9043b is used to perform interpolation processing on the first pixel point based on a linear interpolation method to obtain a second interpolation result corresponding to the first pixel point;

[0167] The interpolation result determination unit 9044b is used to determine the interpolation result corresponding to the first pixel point based on the first weight, the second weight, the first interpolation result and the second interpolation result.

[0168] In some embodiments, the interpolation weight determination unit 9041b is specifically used to determine that when the interpolation method corresponding to the first pixel point is a mixed interpolation method, normalize the edge intensity corresponding to the first pixel point to obtain the first weight corresponding to the interpolation of the first pixel point using the gradient interpolation method; based on the first weight, determine the second weight corresponding to the interpolation of the first pixel point using the linear interpolation method.

[0169] In some embodiments, Fig.13 As shown, the image interpolation device 900 further includes a color space conversion module 906 and an image filtering module 907 .

[0170] A color space conversion module 906 is used to convert the interpolated image from the first color space to the second color space to obtain a target interpolated image;

[0171] The image filtering module 907 is used to perform filtering processing on the chrominance channel of the target interpolation image to obtain an optimized target interpolation image.

[0172] The beneficial technical effects corresponding to the exemplary embodiment of the above-mentioned image interpolation device 900 can be referred to the corresponding beneficial technical effects of the above-mentioned exemplary method part, which will not be repeated here.

[0173] Exemplary Electronic Devices

[0174] Fig.14 A structural diagram of an electronic device 140 provided in an embodiment of the present disclosure includes at least one processor 141 and a memory 142 .

[0175] The processor 141 may be a central processing unit (CPU) or other forms of processing units having data processing capabilities and / or instruction execution capabilities, and may control other components in the electronic device 140 to perform desired functions.

[0176] The memory 142 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory may include, for example, random access memory (RAM) and / or cache memory (cache), etc. Non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, and the processor 141 may execute one or more computer program instructions to implement the image interpolation method and / or other desired functions of the various embodiments of the present disclosure described above.

[0177] In one example, the electronic device 140 may further include: an input device 143 and an output device 144 , and these components are interconnected via a bus system and / or other forms of connection mechanisms (not shown).

[0178] The input device 143 may also include, for example, a keyboard, a mouse, etc.

[0179] The output device 144 can output various information to the outside, and may include, for example, a display, a speaker, a printer, a communication network and its connected remote output device, etc.

[0180] Of course, to simplify, Fig.14 Only some of the components related to the present disclosure in the electronic device 140 are shown, and components such as a bus, an input / output interface, etc. are omitted. In addition, according to specific application situations, the electronic device 140 may also include any other appropriate components.

[0181] Exemplary computer program products and computer-readable storage media

[0182] In addition to the above-mentioned methods and devices, embodiments of the present disclosure may also provide a computer program product, including computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the image interpolation method of various embodiments of the present disclosure described in the above-mentioned "Exemplary Method" section.

[0183] The computer program product may be written in any combination of one or more programming languages ​​to write program code for performing the operations of the disclosed embodiments, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0184] In addition, an embodiment of the present disclosure may also be a computer-readable storage medium having computer program instructions stored thereon. When the computer program instructions are executed by a processor, the processor executes the steps in the image interpolation method of various embodiments of the present disclosure described in the above “Exemplary Method” section.

[0185] Computer readable storage media can adopt any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium is, for example, but not limited to, a system, device or device including electricity, magnetism, light, electromagnetic, infrared, or semiconductor, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0186] The basic principles of the present disclosure are described above in conjunction with specific embodiments. However, the advantages, strengths, effects, etc. mentioned in the present disclosure are only examples and not limitations, and cannot be considered as necessary for each embodiment of the present disclosure. In addition, the specific details disclosed above are only for the purpose of illustration and ease of understanding, rather than limitation, and the above details do not limit the present disclosure to being implemented by adopting the above specific details.

[0187] Those skilled in the art may make various changes and modifications to the present disclosure without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present disclosure claims and their equivalents, the present disclosure is also intended to include these modifications and variations.

Claims

1. An image interpolation method, comprising: Determining an image to be processed and a pixel value of each first pixel in the image to be processed; Determining the edge strength corresponding to each first pixel point in the image to be processed based on the pixel value of each first pixel point in the image to be processed; Determining an interpolation method corresponding to each of the first pixel points based on an edge strength corresponding to each of the first pixel points; Performing interpolation processing on each of the first pixel points based on the interpolation method corresponding to each of the first pixel points to obtain an interpolation result corresponding to each of the first pixel points; Based on the interpolation results corresponding to each of the first pixel points, an interpolated image is obtained.

2. The method according to claim 1, wherein: The determining, based on the edge strength corresponding to each of the first pixel points, the interpolation mode corresponding to each of the first pixel points includes: In response to an edge strength corresponding to the first pixel being greater than a first preset value, determining that the interpolation mode corresponding to the first pixel is a gradient interpolation mode; In response to an edge strength corresponding to the first pixel being less than a second preset value, determining that the interpolation mode corresponding to the first pixel is a linear interpolation mode; In response to the edge strength corresponding to the first pixel being greater than or equal to the second preset value and less than or equal to the first preset value, it is determined that the interpolation mode corresponding to the first pixel is a mixed interpolation mode.

3. The method according to claim 2, wherein: When it is determined that the interpolation method corresponding to the first pixel point is a gradient interpolation method, performing interpolation processing on each of the first pixel points based on the interpolation method corresponding to each of the first pixel points to obtain an interpolation result corresponding to each of the first pixel points includes: Based on the first pixel point, determining a pixel block including the first pixel point in the image to be processed; Determine color components of a plurality of second pixels in the pixel block in a first preset channel; Determine a plurality of first gradient values ​​based on color components of a plurality of second pixels in the pixel block in the first preset channel; Determine an interpolation result corresponding to the first pixel point in the first preset channel based on color components of a plurality of second pixel points in the pixel block in the first preset channel and the plurality of first gradient values; Based on the interpolation result of the first pixel point in the first preset channel, the interpolation result of the first pixel point in the second preset channel is determined.

4. The method according to claim 3, wherein: The determining of a plurality of first gradient values ​​based on color components of a plurality of second pixels in the pixel block in the first preset channel comprises: Determine multiple preset directions; Determine, based on the color components of the plurality of second pixels in the first preset channel, a first difference in the color components of the plurality of second pixels in each of the preset directions in the first preset channel; Based on the first difference values ​​corresponding to the preset directions, first gradient values ​​corresponding to the preset directions are determined.

5. The method according to claim 3, wherein: The determining, based on the color components of the plurality of second pixels in the pixel block in the first preset channel and the plurality of first gradient values, an interpolation result corresponding to the first pixel in the first preset channel comprises: Based on the multiple first gradient values, determining a weight corresponding to each of the second pixel points; Based on the color components of the multiple second pixels in the first preset channel and the weights corresponding to the second pixels, an interpolation result corresponding to the first pixel in the first preset channel is determined.

6. The method according to claim 3, wherein: The determining, based on the interpolation result corresponding to the first pixel point in the first preset channel, the interpolation result corresponding to the first pixel point in the second preset channel includes: Determine color components of a plurality of third pixels in the pixel block in the first preset channel; Determine a plurality of second gradient values ​​based on the color components of the plurality of third pixels in the first preset channel and the interpolation results corresponding to the first pixel in the first preset channel; Based on the multiple second gradient values ​​and the interpolation result of the first pixel point in the first preset channel, the interpolation result of the first pixel point in the second preset channel is determined.

7. The method according to claim 6, wherein: The determining of a plurality of second gradient values ​​based on the color components of the plurality of third pixels in the first preset channel and the interpolation result corresponding to the first pixel in the first preset channel comprises: Determine a second difference between a color component of each of the third pixels in the first preset channel and an interpolation result corresponding to the first pixel in the first preset channel; Based on the second difference value corresponding to each of the third pixel points, a plurality of second gradient values ​​are determined.

8. The method according to claim 6, wherein: The determining, based on the plurality of second gradient values ​​and the interpolation result of the first pixel point in the first preset channel, an interpolation result corresponding to the first pixel point in the second preset channel comprises: Determine, based on the color components of the plurality of third pixels in the first preset channel and the color components of the plurality of third pixels in the second preset channel, a third difference between the color component of each of the third pixels in the second preset channel and the color component of each of the third pixels in the first preset channel; Determining a weight of each of the third pixel points based on the multiple second gradient values; Based on the third difference corresponding to each of the third pixel points, the weight of each of the third pixel points and the interpolation result corresponding to the first pixel point in the first preset channel, the interpolation result corresponding to the first pixel point in the second preset channel is determined.

9. The method according to claim 2, wherein: When it is determined that the interpolation method corresponding to the first pixel point is a mixed interpolation method, performing interpolation processing on each of the first pixel points based on the interpolation method corresponding to each of the first pixel points to obtain an interpolation result corresponding to each of the first pixel points includes: Determine a first weight corresponding to interpolating the first pixel point using the gradient interpolation method, and determine a second weight corresponding to interpolating the first pixel point using the linear interpolation method; Performing interpolation processing on the first pixel point based on the gradient interpolation method to obtain a first interpolation result corresponding to the first pixel point; Performing interpolation processing on the first pixel point based on the linear interpolation method to obtain a second interpolation result corresponding to the first pixel point; Based on the first weight, the second weight, the first interpolation result, and the second interpolation result, an interpolation result corresponding to the first pixel is determined.

10. The method according to claim 9, wherein: The determining of a first weight corresponding to interpolating the first pixel point using the gradient interpolation method, and the determining of a second weight corresponding to interpolating the first pixel point using the linear interpolation method, include: Normalizing the edge intensity corresponding to the first pixel point to obtain a first weight corresponding to the first pixel point interpolated by using the gradient interpolation method; Based on the first weight, a second weight corresponding to the first pixel point being interpolated using the linear interpolation method is determined.

11. The method according to any one of claims 1 to 8, further comprising: Converting the interpolated image from the first color space to the second color space to obtain a target interpolated image; The target interpolation image is filtered on the chrominance channel to obtain an optimized target interpolation image.

12. An image interpolation device, comprising: An image determination module, used to determine an image to be processed and a pixel value of each first pixel point in the image to be processed; An edge strength determination module, used to determine the edge strength corresponding to each first pixel point in the image to be processed based on the pixel value of each first pixel point in the image to be processed; An interpolation mode determination module, used to determine the interpolation mode corresponding to each of the first pixel points based on the edge strength corresponding to each of the first pixel points; An image interpolation module, configured to perform interpolation processing on each of the first pixel points based on the interpolation method corresponding to each of the first pixel points, to obtain an interpolation result corresponding to each of the first pixel points; The image processing module is used to obtain an interpolated image based on the interpolation results corresponding to each of the first pixel points.

13. A computer-readable storage medium storing a computer program, wherein the computer program is used to execute the image interpolation method according to any one of claims 1 to 11.

14. An electronic device, comprising: processor; a memory for storing instructions executable by the processor; The processor is used to read the executable instructions from the memory and execute the instructions to implement the image interpolation method described in any one of claims 1-11.

Images