Image processing method and device, and readable storage medium

Abstract

An image processing method and device thereof, and a readable storage medium, the image processing method comprising: obtaining an original image, the original image comprising pixels to be interpolated; extracting high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction respectively; setting a vertical direction interpolation weight and a horizontal direction interpolation weight of each pixel to be interpolated according to the high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction; obtaining an interpolation pixel value of each pixel to be interpolated according to the vertical direction interpolation weight and the horizontal direction interpolation weight of each pixel to be interpolated.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method and an image processing device, and a readable storage medium. Background Art

[0002] In a color image sensor, each pixel has a color filter of a specific color, and the array composed of these color filters is called a "color filter array" (CFA). Each pixel is only responsible for collecting one color information corresponding to its color filter. Therefore, the color image sensor collects a mosaic image. The most widely used color filter array is a Bayer type pixel array composed of color filters that only transmit red light (R), green light (G), and blue light (B).

[0003] Figure 1 A schematic diagram of a top view structure of a Bayer type pixel array is shown. The pixel array is an RGGB pixel array, where R represents a pixel that only transmits red light, G represents a pixel that only transmits green light, and B represents a pixel that only transmits blue light.

[0004] use Figure 1 The image captured by the image sensor of the color filter array shown is an RGGB mosaic image, which has only one pixel value (R, G or B) at each pixel position. The mosaic image needs to be converted into an image with three pixel values ​​(R, G and B) at each pixel position, that is, a three-channel RGB image, through a de-mosaicing image processing method before it can be displayed.

[0005] However, existing image processing methods are often prone to interpolation errors, especially in texture-dense areas, which are prone to display errors. Summary of the invention

[0006] The problem solved by the present invention is how to reduce interpolation errors and display errors.

[0007] To solve the above problems, the present invention provides an image processing method, comprising:

[0008] An original image is obtained, wherein the original image includes pixels to be interpolated; high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction are extracted; a vertical interpolation weight and a horizontal interpolation weight of each pixel to be interpolated are set according to the high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction; an interpolation pixel value of each pixel to be interpolated is obtained according to the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

[0009] Optionally, the step of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated includes: setting a lower interpolation weight in a direction with more high-frequency texture components.

[0010] Optionally, the step of extracting high-frequency texture components of each pixel to be interpolated on the original image in the vertical and horizontal directions includes: extracting texture components of preset spatial frequencies of each pixel to be interpolated on the original image in the vertical and horizontal directions; the step of setting vertical interpolation weights and horizontal interpolation weights of each pixel to be interpolated includes: taking absolute values ​​or squares of texture components of preset spatial frequencies of each pixel to be interpolated on the original image in the vertical and horizontal directions to obtain energy of preset spatial frequencies of each pixel to be interpolated in the vertical and horizontal directions; and setting vertical interpolation weights and horizontal interpolation weights of each pixel to be interpolated according to the energy of preset spatial frequencies of each pixel to be interpolated in the vertical and horizontal directions.

[0011] Optionally, the step of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated includes: setting a lower interpolation weight in a direction with high energy of a preset spatial frequency on each pixel to be interpolated.

[0012] Optionally, the vertical interpolation weight and the horizontal interpolation weight are:

[0013]

[0014] Among them, (x, y) represents the coordinates of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated, eX(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the horizontal direction, and eY(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the vertical direction.

[0015] Optionally, in the step of extracting the texture components of preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image, the original image is convolved in the spatial domain using a preset vertical detection convolution kernel and a horizontal detection convolution kernel to extract the texture components of preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image, wherein the central spatial frequencies of the vertical detection convolution kernel and the horizontal detection convolution kernel are the preset spatial frequencies.

[0016] Optionally, the preset spatial frequency ranges from 0.25 cycles / pixel to 0.5 cycles / pixel.

[0017] Optionally, the step of extracting texture components of preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image includes: designing a vertical detection convolution kernel and a horizontal detection convolution kernel; the step of designing the vertical detection convolution kernel and the horizontal detection convolution kernel includes: designing a vertical initial convolution kernel and a horizontal initial convolution kernel according to a preset spatial frequency; adjusting the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel to obtain the vertical detection convolution kernel and the horizontal detection convolution kernel.

[0018] Optionally, the step of adjusting the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel includes: giving higher weights to pixel positions close to the center of the convolution kernel; and giving lower weights to pixel positions far from the center of the convolution kernel.

[0019] Optionally, in the step of adjusting the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel, the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel are adjusted by multiplying the vertical initial convolution kernel and the horizontal initial convolution kernel with an adjusted Gaussian function pixel by pixel.

[0020] Optionally, in the step of multiplying the vertical initial convolution kernel and the horizontal initial convolution kernel by the adjusted Gaussian function pixel by pixel, the standard deviation of the Gaussian function in the adjusted Gaussian function is set so that the ratio of the numerical value of the pixel position at the edge of the initial convolution kernel to the numerical value of the pixel position at the center is 10% to 30%.

[0021] Optionally, the step of obtaining the interpolation pixel value of each pixel to be interpolated includes: obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated according to the original image; obtaining the interpolation pixel value of the pixel to be interpolated according to the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, in combination with the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

[0022] Optionally, in the step of obtaining the interpolation pixel value of the pixel to be interpolated, the interpolation pixel value of the pixel to be interpolated is:

[0023] Gi(x,y)=GX(x,y)·weight_X(x,y)+GY(x,y)·weight_Y(x,y)

[0024] Among them, (x, y) represents the coordinates of the pixel to be interpolated, Gi(x, y) represents the interpolation pixel value of the pixel to be interpolated, GX(x, y) represents the horizontal interpolation pixel value of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, GY(x, y) represents the vertical interpolation pixel value of the pixel to be interpolated, and weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated.

[0025] Optionally, in the step of obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated are respectively obtained by using a preset vertical interpolation filter kernel and a horizontal interpolation filter kernel in combination with the original image.

[0026] Optionally, the preset vertical interpolation filter kernel and horizontal interpolation filter kernel are:

[0027] hX=[-1 / 4,1 / 2 1 / 2 1 / 2 -1 / 4]

[0028] hY=hX T

[0029] Among them, hX represents the horizontal interpolation filter kernel; hY represents the vertical interpolation filter kernel, hX T Represents the transpose of hX.

[0030] Optionally, the method further includes: after obtaining the interpolation pixel value of each pixel to be interpolated, obtaining a display image according to the interpolation pixel value of each pixel to be interpolated and the original image.

[0031] Optionally, in the step of obtaining the original image, the original image includes multiple color channels, each of which has its own pixel position to be interpolated; one of the multiple color channels is a reference color channel; in the step of extracting high-frequency texture components of each pixel position to be interpolated on the original image in the vertical and horizontal directions, the high-frequency texture components of each pixel position to be interpolated on the reference color channel in the vertical and horizontal directions are extracted; in the step of obtaining the interpolation pixel value of each pixel position to be interpolated, the interpolation pixel value of the reference color channel of each pixel position to be interpolated on the reference color channel is obtained; the step of obtaining the display image includes: according to the interpolation pixel value of the reference color channel of each pixel position to be interpolated of the reference color channel, combined with the original image, obtaining a channel image of a complete reference channel; according to the channel image of the complete reference channel, combined with the original image, obtaining complete channel images of other color channels; merging the channel image of the complete reference channel and the complete channel images of other color channels to obtain the display image.

[0032] Correspondingly, the present invention also provides an image processing device, specifically comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0033] In addition, the present invention also provides a readable storage medium, specifically including: a computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0034] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0035] In the technical solution of the present invention, after extracting the high-frequency texture components of each pixel to be interpolated in the vertical direction and the horizontal direction, the vertical direction interpolation weight and the horizontal direction interpolation weight are set based on the high-frequency texture components, thereby obtaining the interpolation pixel value of each pixel to be interpolated. The interpolation weight set based on the high-frequency texture component can take into account the influence of the spatial frequency on the interpolation accuracy in the process of obtaining the interpolation pixel value of each pixel to be interpolated, thereby effectively improving the accuracy of gradient direction judgment, effectively solving the problem of interpolation direction error, and being conducive to improving the interpolation accuracy.

[0036] In the optional scheme of the present invention, a lower interpolation weight is set in the direction with more high-frequency texture components, that is, try to interpolate along the direction with fewer high-frequency components of the spatial frequency, so as to improve the accuracy of gradient direction judgment, which can effectively solve the problem of wrong interpolation direction, is conducive to obtaining better image effects, and is conducive to improving interpolation accuracy.

[0037] In the optional solution of the present invention, the detection convolution kernel used in the step of extracting the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction is obtained after the initial convolution kernel is adjusted by weight, and the adjustment process is: giving a higher weight to the pixel position close to the center of the convolution kernel; giving a lower weight to the pixel position far from the center of the convolution kernel. By adjusting the weight, the ringing effect of the image can be effectively reduced.

[0038] In an optional solution of the present invention, interpolation pixel values ​​are obtained based on horizontal interpolation pixel values ​​and vertical interpolation pixel values, and the horizontal interpolation pixel values ​​and vertical interpolation pixel values ​​are obtained by preset vertical interpolation filter kernels and horizontal interpolation filter kernels. Therefore, the acquisition of the interpolation pixel values ​​not only considers the interpolation weights determined by the energy size of a specific spatial frequency, but also considers the information of other pixels, thereby effectively improving the interpolation accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 is a schematic diagram of a top view structure of a Bayer type pixel array;

[0040] Figure 2 It is a G channel image obtained by an image processing method;

[0041] Figure 3 It is the texture of the object being photographed;

[0042] Figure 4 It is a color image sensor with a Bayer-type RGGB pixel array. Figure 3 The original mosaic image obtained by the texture of the object shown;

[0043] Figure 5 yes Figure 4 The result of horizontal interpolation of G pixels at the pixel position to be interpolated in the original mosaic image shown;

[0044] Figure 6 yes Figure 4 The result of vertical interpolation of G pixels at the pixel position to be interpolated in the original mosaic image shown;

[0045] Figure 7 It is a flowchart of an embodiment of an image processing method of the present invention;

[0046] Figure 8 yes Figure 7 A schematic diagram of an original image obtained when the image sensor in the embodiment of the image processing method has the highest spatial frequency in the horizontal direction and the vertical direction;

[0047] Fig. 9 yes Figure 7 A schematic diagram of the structure of a Bayer-type pixel array used in an embodiment of the image processing method shown;

[0048] Fig.10 yes Figure 7 A schematic diagram of a process of designing a vertical detection convolution kernel hGy and a horizontal detection convolution kernel hGx in an embodiment of the image processing method shown;

[0049] Fig.11 yes Figure 7 A schematic flow chart of the steps of setting the vertical direction interpolation weight and the horizontal direction interpolation weight of each pixel to be interpolated in the embodiment of the image processing method shown;

[0050] Fig.12 yes Figure 7 A schematic diagram of a process of obtaining an interpolation pixel value for each pixel to be interpolated in the embodiment of the image processing method shown;

[0051] Fig.13is a schematic diagram of an image around a pixel to be interpolated in the original image when the pixel to be interpolated is an R pixel;

[0052] Fig.14 is a schematic diagram of an image around a pixel to be interpolated in the original image when the pixel to be interpolated is a B pixel;

[0053] Fig.15 yes Figure 7 A schematic diagram of obtaining the R position, B position, and G position in the step of displaying the image according to the interpolation pixel value of each pixel to be interpolated and the original image cfa in the embodiment of the image processing method shown;

[0054] Fig.16 Yes Yes Figure 7 The illustrated embodiment of the image processing method is a schematic diagram of calculation within a 9×9 filter window of a mean filter in the step of obtaining a display image according to the interpolation pixel value of each pixel to be interpolated and the original image cfa. DETAILED DESCRIPTION

[0055] As can be seen from the background technology, the existing image processing methods are prone to interpolation errors. Now, the cause of the interpolation error problem is analyzed in combination with an image processing method.

[0056] In the RGGB mosaic image, G pixels account for the largest proportion, which is 50%. Therefore, the G pixels are usually interpolated first to obtain a complete G channel image; and then the correlation between the G pixels and the R and B pixels is used to obtain the complete R and B channel images.

[0057] The most commonly used method for G pixel interpolation is to determine the horizontal and vertical gradients of the pixel position to be interpolated, and then interpolate along the direction with the smaller gradient. Figure 2 That is, a G channel image obtained by an image processing method is shown. In the texture-dense area, the gradient direction is easy to be misjudged, resulting in an incorrect interpolation direction, that is, interpolation errors are easy to occur in the texture-dense area (such as Figure 2 (shown in the middle circle 21).

[0058] The following is an analysis of the causes of interpolation errors in texture-dense areas using a specific image processing method.

[0059] refer to Figure 3 and Figure 4 ,in Figure 3 is the texture of the object being photographed, Figure 4 It is the original mosaic image captured by a color image sensor with a Bayer-type RGGB pixel array. Figure 4 Pixels without diagonal line filling are G pixels, and pixels with diagonal line filling are R pixels or B pixels.

[0060] Since G pixels account for the highest proportion, reaching 50%, it is necessary to interpolate the G pixels first, that is, to obtain the values ​​of the G pixels at the R pixel and B pixel positions with slashes through interpolation, so that each pixel position has a G pixel value, that is, to obtain a complete G channel image.

[0061] from Figure 3 It can be seen that the texture of the photographed object alternates between black and white in the vertical direction, with more high-frequency components in the spatial frequency, and has no color change in the horizontal direction, with fewer high-frequency components in the spatial frequency.

[0062] Combined with reference Figure 5 and Figure 6 ,in Figure 5 For Figure 4 The result of horizontal interpolation of the G pixel at the position of the pixel to be interpolated; Figure 6 For Figure 4 The result of vertical interpolation of the G pixel at the position of the pixel to be interpolated.

[0063] Compare Figure 5 and Figure 6 It can be seen that the G channel image obtained by horizontal interpolation can well reflect the real texture of the photographed object; while the G channel image obtained by vertical interpolation has a large number of interpolation errors and cannot truly reflect the texture of the photographed object.

[0064] When dealing with dense texture areas, existing image processing methods do not consider the impact of spatial frequency on interpolation accuracy, resulting in incorrect gradient direction judgment and interpolation judgment.

[0065] In order to solve the technical problem, the present invention provides an image processing method, comprising:

[0066] An original image is obtained, wherein the original image includes pixels to be interpolated; high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction are extracted; a vertical interpolation weight and a horizontal interpolation weight of each pixel to be interpolated are set according to the high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction; an interpolation pixel value of each pixel to be interpolated is obtained according to the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

[0067] In the technical solution of the present invention, after extracting the high-frequency texture components of each pixel to be interpolated in the vertical direction and the horizontal direction, the vertical direction interpolation weight and the horizontal direction interpolation weight are set based on the high-frequency texture components, thereby obtaining the interpolation pixel value of each pixel to be interpolated. The interpolation weight set based on the high-frequency texture component can take into account the influence of the spatial frequency on the interpolation accuracy in the process of obtaining the interpolation pixel value of each pixel to be interpolated, thereby effectively improving the accuracy of gradient direction judgment, effectively solving the problem of interpolation direction error, and being conducive to improving the interpolation accuracy.

[0068] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0069] refer to Figure 7 , shows a flow chart of an image processing method according to an embodiment of the present invention.

[0070] The image processing method comprises: firstly executing step S110 to obtain an original image cfa, wherein the original image cfa comprises pixel bits to be interpolated.

[0071] In some embodiments of the present invention, the original image cfa is obtained by using a Bayer type pixel array (such as Figure 1 The original image cfa is an RGGB mosaic image (as shown). Figure 4 In the original image cfa, each pixel position has only one pixel value (R pixel value, G pixel value or B pixel value).

[0072] Therefore, in the step of executing step S110 to obtain the original image cfa, the original image cfa includes multiple color channels, each of the color channels has its own pixel bit to be interpolated; and one of the multiple color channels is a reference color channel.

[0073] Specifically, the original image cfa is a three-channel RGB image, that is, the original image cfa includes three color channels, namely, an R channel, a G channel, and a B channel. In the original image cfa, the pixel bits to be interpolated of the R channel are pixel bits with G pixel values ​​and B pixel values, the pixel bits to be interpolated of the G channel are pixel bits with R pixel values ​​and B pixel values, and the pixel bits to be interpolated of the B channel are pixel bits with R pixel values ​​and G pixel values.

[0074] Since the original image cfa is an RGGB mosaic image, the pixel positions with G pixel values ​​in the G channel account for the highest proportion, up to 50%, so among the R channel, G channel and B channel, the G channel is the reference color channel.

[0075] Continue to refer Figure 7 After obtaining the original image cfa, step S120 is performed to extract the high-frequency texture components of each pixel to be interpolated on the original image cfa in the vertical direction and the horizontal direction respectively.

[0076] It should be noted that, as mentioned above, the original image cfa includes multiple color channels, one of which is a reference color channel. Therefore, in the step of extracting the high-frequency texture components of each pixel to be interpolated on the original image cfa in the vertical direction and the horizontal direction, the high-frequency texture components of each pixel to be interpolated on the reference color channel in the vertical direction and the horizontal direction are extracted;

[0077] In some embodiments of the present invention, the step of extracting high-frequency texture components of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions includes: extracting texture components of preset spatial frequencies of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions.

[0078] According to the structure of the image sensor, the original image cfa obtained by the image sensor has a maximum value of 0.5 cycle / pixel in the horizontal and vertical directions. Figure 8 shown).

[0079] Combined with reference Fig. 9 , showing Figure 7 A schematic diagram of the structure of a Bayer-type pixel array used in an embodiment of the image processing method is shown.

[0080] Based on the spatial arrangement structure of Bayer pixel array, such as Fig. 9 As shown, it can be known that the spatial repetition period of the pixel array in the horizontal direction and the vertical direction is 2 pixels / period, and the corresponding spatial frequency is 0.5 period / pixel.

[0081] In order to avoid the interference of the Bayer-type pixel array's own arrangement structure on the extracted texture components, the preset spatial frequency is less than 0.5 cycles / pixel. Moreover, based on the frequency domain analysis results, it can be known that the extracted texture component of the preset spatial frequency is not a strictly isolated frequency point component, but a frequency broadening with a certain half-width. Therefore, in order to eliminate the interference of the Bayer-type pixel array's own arrangement structure as much as possible, the preset spatial frequency has a certain distance from 0.5 cycles / pixel. In some embodiments of the present invention, the preset spatial frequency is in the range of 0.25 cycles / pixel to 0.5 cycles / pixel. Specifically, the preset spatial frequency is 0.375 cycles / pixel to achieve the best effect.

[0082] In some embodiments of the present invention, the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions are extracted by spatial domain convolution. Specifically, in the step of extracting the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions, spatial domain convolution is performed on the original image cfa by using a preset vertical detection convolution kernel hGy and a horizontal detection convolution kernel hGx to extract the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions, respectively, wherein the central spatial frequencies of the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx are the preset spatial frequencies.

[0083] In some embodiments of the present invention, executing step S120, the step of extracting texture components of preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image cfa also includes: designing a vertical detection convolution kernel hGy and a horizontal detection convolution kernel hGx.

[0084] The vertical detection convolution kernel hGy is suitable for detecting the high-frequency texture components of the original image cfa in the vertical direction; the horizontal detection convolution kernel hGx is suitable for detecting the high-frequency texture of the original image cfa in the horizontal direction.

[0085] Combined with reference Fig.10 , showing Figure 7 The figure is a schematic diagram of the process of designing the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx in the embodiment of the image processing method.

[0086] like Fig.10As shown, the steps of designing the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx include: first executing step S121, designing the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 according to the preset spatial frequency; secondly executing step S122, adjusting the weights of different pixel positions of the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 to obtain the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx.

[0087] Since the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx are both used to detect high-frequency texture components, that is, to detect periodically changing components, in some embodiments of the present invention, step S121 is executed to design the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0. Based on the preset spatial frequency, the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 are implemented by a cosine function so that the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 have the function of detecting the size of the preset spatial frequency components.

[0088] Specifically, the value of each pixel position in the horizontal initial convolution kernel hGx0 can be expressed as: hGx0(n)=cos(2·n·π·f), where f represents the preset spatial frequency and n represents the pixel position in the horizontal initial convolution kernel hGx0.

[0089] The size of the convolution kernel should be an odd number greater than or equal to 3. Moreover, the larger the convolution kernel, the smaller its frequency broadening and the higher the detection accuracy, but the greater the amount of calculation. In some embodiments of the present invention, the size of the horizontal detection convolution kernel hGx is equal to 11, that is, the size of the horizontal initial convolution kernel hGx0 is equal to 11, then n = -5, -4, -3, ..., 0, ..., 3, 4, 5, where n = 0 represents the pixel position of the center of the horizontal initial convolution kernel.

[0090] Specifically, the horizontal initial convolution kernel hGx0 is expressed as:

[0091] hGx0=[0.707 -1.000 0.707 0 -0.707 1.000 -0.707 0 0.707 -1.000 0.707]

[0092] like Fig.10 As shown, after the initial convolution kernel is designed, step S122 is performed to adjust the weights of different pixel positions of the initial convolution kernel to obtain a detection convolution kernel.

[0093] In some embodiments of the present invention, the step of adjusting the weights of different pixel positions of the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 includes: giving a higher weight to the pixel position close to the center of the convolution kernel; giving a lower weight to the pixel position far from the center of the convolution kernel. Increasing the weight of the pixel position close to the center and reducing the weight of the pixel position far from the center can reduce the image ringing effect and improve the image processing effect.

[0094] In some embodiments of the present invention, in the step of adjusting the weights of different pixel positions of the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0, the weights of different pixel positions of the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 are adjusted by multiplying the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 with the adjusted Gaussian function pixel by pixel.

[0095] It should be noted that, in the step of multiplying the vertical initial convolution kernel hGy0 and the horizontal initial convolution kernel hGx0 by the adjusted Gaussian function pixel by pixel, the standard deviation σ of the Gaussian function in the adjusted Gaussian function is set so that the ratio of the numerical value of the pixel position at the edge of the initial convolution kernel to the numerical value of the pixel position at the center is 10% to 30%. If the ratio is too small, the numerical value at the edge of the convolution kernel has almost no effect, resulting in a waste of computing resources; if the ratio is too large, the adjusted Gaussian function is too flat, and loses its beneficial effect of reducing the ringing effect.

[0096] In some embodiments of the present invention, the standard deviation of the Gaussian function is σ=3. The discrete adjusted Gaussian function is expressed as:

[0097] Gau=[0.033 0.055 0.081 0.106 0.126 0.133 0.126 0.106 0.081 0.0550.033]

[0098] Therefore, the horizontal initial convolution kernel hGx0 is multiplied by the adjusted Gaussian function pixel by pixel, and the obtained horizontal detection convolution kernel hGx is expressed as:

[0099] hGx=[0.023 -0.055 0.057 0 -0.089 0.133 -0.089 0 0.057 -0.055 0.023]

[0100] The horizontal detection convolution kernel hGx is transposed to obtain the vertical detection convolution kernel hGy, hGy = hGx T .

[0101] After obtaining the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy, the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy are used to perform spatial domain convolution on the original image cfa respectively to extract texture components.

[0102] Continue to refer Figure 7 After extracting the high-frequency texture components, execute step S130 to set the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated on the original image cfa according to the high-frequency texture components of each pixel to be interpolated in the vertical and horizontal directions.

[0103] In some embodiments of the present invention, the step of setting the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated includes: setting a lower interpolation weight in a direction with more high-frequency texture components.

[0104] In the direction with more high-frequency texture components, a lower interpolation weight is set, that is, try to interpolate along the direction with fewer high-frequency components of the spatial frequency, improve the accuracy of gradient direction judgment, effectively solve the problem of incorrect interpolation direction, and help obtain better image effects and improve interpolation accuracy.

[0105] Combined with reference Fig.11 , showing Figure 7 The illustrated embodiment of the image processing method is a flowchart of the steps of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

[0106] In some embodiments of the present invention, step S130 is executed to set the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated, including: first executing step S131, taking the absolute value or square of the texture component of the preset spatial frequency of each pixel to be interpolated on the original image cfa in the vertical and horizontal directions, so as to obtain the energy of the preset spatial frequency of each pixel to be interpolated in the vertical and horizontal directions; then executing step S132, setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated according to the energy of the preset spatial frequency of each pixel to be interpolated in the vertical and horizontal directions.

[0107] Specifically, the absolute value or square of the result of the convolution of the horizontal detection convolution kernel hGx and the original image cfa is taken to obtain the energy eX of the preset spatial frequency of each pixel to be interpolated in the horizontal direction; the absolute value or square of the result of the convolution of the vertical detection convolution kernel hGy and the original image cfa is taken to obtain the energy eY of the preset spatial frequency of each pixel to be interpolated in the vertical direction.

[0108] The energy value eX of the preset spatial frequency in the horizontal direction of each pixel to be interpolated represents the strength of the texture of the preset spatial frequency in a small area in the horizontal direction centered on the pixel; the energy value eY of the preset spatial frequency in the vertical direction of each pixel to be interpolated represents the strength of the texture of the preset spatial frequency in a small area in the vertical direction centered on the pixel.

[0109] Specifically, the step of setting the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated includes: for each pixel to be interpolated, setting a lower interpolation weight in a direction with higher energy of a preset spatial frequency.

[0110] In some embodiments of the present invention, the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X are respectively:

[0111]

[0112] Among them, (x, y) represents the coordinates of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated, eX(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the horizontal direction, and eY(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the vertical direction.

[0113] The vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X set in the above manner have the following characteristics: at the position of the pixel to be interpolated, in the direction of higher energy of the preset spatial frequency, a lower interpolation weight is assigned; and in the direction of lower energy of the preset spatial frequency, a higher interpolation weight is assigned, thereby achieving the purpose of interpolating as much as possible in the direction with fewer high-frequency components of the spatial frequency, thereby overcoming the problem of wrong interpolation direction.

[0114] Continue to refer Figure 7After setting the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated, step S140 is performed to obtain the interpolation pixel value of each pixel to be interpolated according to the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated.

[0115] As mentioned above, in some embodiments of the present invention, in the step of obtaining the original image CFA, the original image CFA includes multiple color channels, each of the color channels has its own pixel position to be interpolated; one of the multiple color channels is a reference color channel; therefore, in the step of obtaining the interpolation pixel value of each pixel position to be interpolated, the interpolation pixel value of the reference color channel for each pixel position to be interpolated of the reference color channel is obtained.

[0116] Specifically, the original image cfa is a three-channel RGB image, that is, the original image cfa includes three color channels of R channel, G channel and B channel, wherein the G channel is the reference color channel. In these embodiments, the pixel to be interpolated is the G pixel to be interpolated, that is, the position of the R pixel and the position of the B pixel in the original image cfa are interpolated to obtain the G pixel value, so as to obtain a complete G channel image.

[0117] Combined with reference Fig.12 , showing Figure 7 The figure is a schematic diagram of a process of obtaining an interpolation pixel value for each pixel to be interpolated in the embodiment of the image processing method.

[0118] In some embodiments of the present invention, the step of obtaining the interpolation pixel value of each pixel to be interpolated includes: first executing step S141, obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated according to the original image cfa; then executing step S142, obtaining the interpolation pixel value of the pixel to be interpolated according to the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, combined with the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated.

[0119] Specifically, in some embodiments of the present invention, the image processing method is to interpolate the image of the G channel, and insert the G pixel value at the position of the R pixel and the B pixel. Therefore, the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated are the vertical interpolation G pixel value GY and the horizontal interpolation G pixel value GX of each pixel to be interpolated.

[0120] In some embodiments of the present invention, in the step of obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated are obtained respectively by using the preset vertical interpolation filter kernel hY and the horizontal interpolation filter kernel hX in combination with the original image cfa.

[0121] Specifically, the preset horizontal interpolation filter kernel hX=[-1 / 4, 1 / 2, 1 / 2, 1 / 2, -1 / 4]. Through the horizontal interpolation filter kernel hX, combined with the original image cfa, the process of obtaining the horizontal interpolation pixel value of each pixel to be interpolated is: in the original image cfa, with the pixel to be interpolated as the center, the 2 pixels on the left, the 2 pixels on the right and the pixel at the center position in the horizontal direction, a total of 5 pixels, are multiplied and added with the horizontal interpolation filter kernel hX accordingly, as the horizontal interpolation G pixel value GX of the pixel to be interpolated.

[0122] Similarly, the preset vertical interpolation filter kernel hY = hX T , where hX T Indicates that hX is transposed. The process of obtaining the vertical interpolation pixel value of each pixel to be interpolated is as follows: in the original image cfa, with the pixel to be interpolated as the center, the 2 pixels above, 2 pixels below and the pixel at the center in the vertical direction, a total of 5 pixels, are multiplied and added with the vertical interpolation filter kernel hY to obtain the vertical interpolation G pixel value GY of the pixel to be interpolated.

[0123] Combined with reference Fig.13 and Fig.14 ,in Fig.13 FIG. 1 shows a schematic diagram of an image around a pixel to be interpolated in an original image when the pixel to be interpolated is an R pixel. Fig.14 A schematic diagram of an image around a pixel to be interpolated in an original image when the pixel to be interpolated is a B pixel is shown.

[0124] The coordinates of the pixel to be interpolated are (x, y). In the original image cfa, the pixel to be interpolated with the coordinates (x, y) may be an R pixel (e.g. Fig.13 ), it may also be a B pixel (as shown Fig.14 shown).

[0125] When the pixel to be interpolated with coordinates (x, y) is an R pixel (such as Fig.13 As shown), the horizontal interpolation pixel value GX of the pixel to be interpolated is:

[0126]

[0127] The vertical interpolation G pixel value GY of the pixel to be interpolated is:

[0128]

[0129] Similarly, when the pixel to be interpolated with coordinates (x, y) is a B pixel (such as Fig.14 As shown), the horizontal interpolation pixel value GX of the pixel to be interpolated is:

[0130]

[0131] The vertical interpolation G pixel value GY of the pixel to be interpolated is:

[0132]

[0133] Among them, R(x, y), G(x, y), and B(x, y) are respectively the R pixel value, G pixel value, and B pixel value of the coordinates (x, y) in the original image cfa.

[0134] Therefore, for the pixel to be interpolated in the original image cfa, the horizontal interpolation G pixel value GX of the pixel to be interpolated is

[0135]

[0136] The vertical interpolation G pixel value GY of the pixel to be interpolated is:

[0137]

[0138] Among them, cfa(x, y) is the pixel value of the pixel position with coordinates (x, y) in the original image cfa.

[0139] Continue to refer Fig.12 After obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, execute step S142 to obtain the interpolation pixel value of the pixel to be interpolated by combining the vertical interpolation weight weight_Y and the horizontal interpolation weight weight_X of each pixel to be interpolated.

[0140] In some embodiments of the present invention, in the step of obtaining the interpolation pixel value of the pixel to be interpolated, the interpolation pixel value of the pixel to be interpolated is:

[0141] Gi(x,y)=GX(x,y)·weight_X(x,y)+GY(x,y)·weight_Y(x,y)

[0142] Among them, (x, y) represents the coordinates of the pixel to be interpolated, Gi(x, y) represents the interpolation pixel value of the pixel to be interpolated, GX(x, y) represents the horizontal interpolation pixel value of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, GY(x, y) represents the vertical interpolation pixel value of the pixel to be interpolated, and weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated.

[0143] Continue to refer Figure 7 After obtaining the interpolated pixel value of each pixel to be interpolated, step S150 is performed to obtain a display image according to the interpolated pixel value of each pixel to be interpolated and the original image cfa.

[0144] Specifically, executing step S150, the step of obtaining a display image includes: obtaining a complete channel image of the reference channel based on the interpolation pixel value of the reference color channel of each pixel to be interpolated in the reference color channel, combined with the original image; obtaining complete channel images of other color channels based on the channel image of the complete reference channel, combined with the original image; merging the complete channel image of the reference channel and the complete channel images of other color channels to obtain a display image.

[0145] After obtaining the interpolated pixel value of each pixel to be interpolated, combined with the pixel values ​​of other pixel positions in the original image, a channel image of a complete reference color channel can be obtained. Specifically, the reference color channel is the G channel, so after obtaining the interpolated G pixel value of each pixel to be interpolated in the G channel, combined with the G pixel value of the G pixel position in the original image cfa, a complete G channel image Gc can be obtained.

[0146] Moreover, in the process of obtaining the interpolation pixel value of the pixel to be interpolated, on the one hand, the influence of the preset spatial frequency on the interpolation accuracy is taken into account by introducing the vertical interpolation weight and the horizontal interpolation weight; and when obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, the interpolation pixel value is obtained based on the preset filter kernel, thereby not only utilizing the pixel value of the reference color channel in the original image cfa, that is, the adjacent G pixel value, but also utilizing the information of the surrounding pixel positions of other channels, that is, the R pixel value and the B pixel value of the surrounding pixel positions in the R channel and the B channel. Therefore, the channel image of the complete reference color channel obtained has higher interpolation accuracy.

[0147] After obtaining the channel image of the complete reference color channel, then, based on the channel image of the complete reference channel, complete channel images of other color channels are obtained.

[0148] In some embodiments of the present invention, interpolation is performed using the characteristic that the difference between the reference color channel and other color channels varies slowly along space to obtain complete channel images of other color channels.

[0149] Specifically, the reference color channel is the G channel, so the difference between the G channel and the R channel and the B channel changes slowly along the space to perform interpolation to obtain a complete R channel channel image and a complete B channel channel image. The obtained R channel and B channel are respectively expressed as:

[0150]

[0151] Among them, Rc, Gc, and Bc represent the complete R channel image, the complete G channel image, and the complete B channel image, respectively; (cfa-Gc) represents the pixel-by-pixel subtraction of the original image cfa from the complete G channel image, and the result of (cfa-Gc) is a slowly varying image, which is an image of the same size as the original image cfa, that is, the slowly varying image has the same number of rows and columns as the original image.

[0152] interpolateR(cfa-Gc) means interpolating only according to the RG pixel values ​​at the R position in the slowly varying image (cfa-Gc) to obtain the slowly varying R pixel value mR at each pixel position of the slowly varying image, that is, to obtain a complete slowly varying R channel image; interpolateB(cfa-Gc) means interpolating only according to the BG pixel values ​​at the B position in the slowly varying image (cfa-Gc) to obtain the slowly varying B pixel value mB at each pixel position of the slowly varying image, that is, to obtain a complete slowly varying B channel image. Fig.15 As shown, the R position represents the position where the R pixel in the original image cfa is the same, and the B position represents the position where the B pixel in the original image cfa is the same.

[0153] In some embodiments of the present invention, in the steps of executing interpolateR(cfa-Gc) and interpolateB(cfa-Gc), at least one interpolation method selected from bilinear interpolation, bicubic interpolation, mean filtering, and median filtering may be used to interpolate to obtain a complete slowly varying channel image. In other embodiments of the present invention, other interpolation methods may also be used for interpolation. In this embodiment, interpolateR(cfa-Gc) and interpolateB(cfa-Gc) are performed by the mean filtering method to obtain a complete slowly varying R channel image and a complete slowly varying B channel image, respectively.

[0154] Specifically, combined with reference Fig.16, the filter window size of the mean filter is 9×9 pixels, then interpolateR(cfa-Gc) sets the average pixel value of the pixels at the R position of the slowly varying image (cfa-Gc) in the filter window to the value mR at the center of the filter window; interpolateB(cfa-Gc) sets the average pixel value of the pixels at the B position of the slowly varying image (cfa-Gc) in the filter window to the value mB at the center of the filter window, which is expressed as follows:

[0155]

[0156] Among them, K R Indicates the total number of R position pixels contained in the slowly varying image (cfa-Gc) in the current filtering window; K B Represents the total number of pixels at position B contained in the slowly varying image (cfa-Gc) within the current filtering window; (R i -G i ) represents the i-th (RG) pixel of the slowly varying image (cfa-Gc) in the current filtering window; (B i -G i ) represents the i-th (BG) pixel of the slowly varying image (cfa-Gc) within the current filtering window.

[0157] Make the filter window traverse the slowly varying image (cfa-G Y ), at each pixel position of the image 2 , a slowly varying R pixel value mR and a slowly varying B pixel value mB are set at each pixel position, so as to obtain a complete slowly varying R channel image and a complete slowly varying B channel image; then, the complete slowly varying R channel image mR and the complete slowly varying B channel image mB are respectively added pixel by pixel to the G channel image Gc, so as to obtain a complete R channel image Rc and a complete B channel image Bc.

[0158] After obtaining the complete channel images of other color channels, the channel images of all color channels are merged to obtain the display image, that is, the complete G channel image Gc, the complete R channel image Rc and the complete B channel image Bc are merged to obtain the display image RGB. c , which shows the image RGB c is a color image with full RGB channels.

[0159] Since the channel image of the complete reference color channel has higher interpolation accuracy, that is, the complete G channel image Gc has higher difference accuracy, the channel images of other color channels obtained based on the channel image of the reference color channel have higher accuracy and lower possibility of generating false colors, so the display image RGB with complete RGB channels is obtained. c With less false colors and higher accuracy.

[0160] Correspondingly, the present invention also provides an image processing device, specifically comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0161] Specifically, when the processor executes the program, the processor executes the steps of the image processing method of the present invention. Therefore, the specific technical solution of the processor refers to the embodiment of the aforementioned image processing method, and the present invention will not be repeated here.

[0162] In addition, the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0163] Specifically, the computer instructions stored on the readable storage medium execute the steps of the image processing method of the present invention when they are executed. Therefore, the specific technical solution of the readable storage medium refers to the embodiment of the aforementioned image processing method, and the present invention will not be repeated here.

[0164] In summary, in the technical solution of the present invention, after extracting the high-frequency texture components of each pixel to be interpolated in the vertical direction and the horizontal direction, the vertical direction interpolation weight and the horizontal direction interpolation weight are set based on the high-frequency texture components, respectively, to obtain the interpolation pixel value of each pixel to be interpolated. The interpolation weight set based on the high-frequency texture component can take into account the influence of the spatial frequency on the interpolation accuracy in the process of obtaining the interpolation pixel value of each pixel to be interpolated, thereby effectively improving the accuracy of gradient direction judgment, effectively solving the problem of interpolation direction error, and helping to improve interpolation accuracy.

[0165] Moreover, in the direction with more high-frequency texture components, a lower interpolation weight is set. In other words, try to interpolate along the direction with fewer high-frequency components of the spatial frequency to improve the accuracy of gradient direction judgment. This can effectively solve the problem of incorrect interpolation direction, which is conducive to obtaining better image effects and improving interpolation accuracy.

[0166] In addition, the detection convolution kernel used in the step of extracting the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction is obtained after the initial convolution kernel is adjusted by weight, and the adjustment process is: giving a higher weight to the pixel position close to the center of the convolution kernel; giving a lower weight to the pixel position far from the center of the convolution kernel. By adjusting the weight, the ringing effect of the image can be effectively reduced.

[0167] In addition, the interpolation pixel value is obtained based on the horizontal interpolation pixel value and the vertical interpolation pixel value, and the horizontal interpolation pixel value and the vertical interpolation pixel value are obtained by a preset horizontal interpolation filter kernel and a vertical interpolation filter kernel. Therefore, the acquisition of the interpolation pixel value not only considers the interpolation weight determined by the energy size of a specific spatial frequency, but also considers the information of other pixels, thereby effectively improving the interpolation accuracy.

[0168] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. An image processing method, characterized in that: include: Obtaining an original image, wherein the original image includes pixel bits to be interpolated; Extracting high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction, respectively. The step of extracting high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction comprises: extracting texture components of preset spatial frequencies of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction; According to the high-frequency texture components of each pixel to be interpolated on the original image in the vertical and horizontal directions, the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated are set. The step of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated comprises: taking the absolute value or the square of the texture components of the preset spatial frequency of each pixel to be interpolated on the original image in the vertical and horizontal directions respectively, so as to obtain the energy of the preset spatial frequency of each pixel to be interpolated in the vertical and horizontal directions respectively; according to the energy of the preset spatial frequency of each pixel to be interpolated in the vertical and horizontal directions respectively, the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated are set; The interpolation pixel value of each pixel to be interpolated is obtained according to the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

2. The image processing method according to claim 1, characterized in that: The steps of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated include: In the direction with more high-frequency texture components, set a lower interpolation weight.

3. The image processing method according to claim 1, characterized in that: The steps of setting the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated include: For each pixel to be interpolated, a lower interpolation weight is set in the direction of high energy of the preset spatial frequency.

4. The image processing method according to claim 3, characterized in that: The vertical interpolation weights and horizontal interpolation weights are: Among them, (x, y) represents the coordinates of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated, eX(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the horizontal direction, and eY(x, y) represents the energy of the preset spatial frequency of the pixel to be interpolated in the vertical direction.

5. The image processing method according to claim 1, wherein: In the step of extracting the texture components of the preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image, the original image is convolved in the spatial domain by using a preset vertical detection convolution kernel and a horizontal detection convolution kernel to extract the texture components of the preset spatial frequencies in the vertical and horizontal directions for each pixel to be interpolated on the original image, wherein the central spatial frequencies of the vertical detection convolution kernel and the horizontal detection convolution kernel are the preset spatial frequencies.

6. The image processing method according to any one of claims 1 to 5, characterized in that: The preset spatial frequency ranges from 0.25 cycles / pixel to 0.5 cycles / pixel.

7. The image processing method according to claim 5, characterized in that: The step of extracting the texture components of the preset spatial frequencies of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction respectively comprises: designing a vertical detection convolution kernel and a horizontal detection convolution kernel; The steps of designing vertical detection convolution kernel and horizontal detection convolution kernel include: According to the preset spatial frequency, design the vertical initial convolution kernel and the horizontal initial convolution kernel; The weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel are adjusted to obtain the vertical detection convolution kernel and the horizontal detection convolution kernel.

8. The image processing method according to claim 7, characterized in that: The step of adjusting the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel includes: Give higher weights to pixels closer to the center of the convolution kernel; Give lower weights to pixel locations that are far from the center of the convolution kernel.

9. The image processing method according to claim 7 or 8, characterized in that: In the step of adjusting the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel, the weights of different pixel positions of the vertical initial convolution kernel and the horizontal initial convolution kernel are adjusted by multiplying the vertical initial convolution kernel and the horizontal initial convolution kernel by the adjusted Gaussian function pixel by pixel.

10. The image processing method according to claim 9, characterized in that: In the step of multiplying the vertical initial convolution kernel and the horizontal initial convolution kernel by the adjusted Gaussian function pixel by pixel, the standard deviation of the Gaussian function in the adjusted Gaussian function is set so that the ratio of the numerical value of the pixel position at the edge of the initial convolution kernel to the numerical value of the pixel position at the center is 10% to 30%.

11. The image processing method according to claim 1, wherein: The steps of obtaining the interpolation pixel value of each pixel to be interpolated include: According to the original image, a vertical interpolation pixel value and a horizontal interpolation pixel value of each pixel to be interpolated are obtained; According to the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, the interpolation pixel value of the pixel to be interpolated is obtained in combination with the vertical interpolation weight and the horizontal interpolation weight of each pixel to be interpolated.

12. The image processing method according to claim 11, characterized in that: In the step of obtaining the interpolation pixel value of the pixel to be interpolated, the interpolation pixel value of the pixel to be interpolated is: Gi(x,y)=GX(x,y)·weight_X(x,y)+GY(x,y)·weight_Y(x,y) Among them, (x, y) represents the coordinates of the pixel to be interpolated, Gi(x, y) represents the interpolation pixel value of the pixel to be interpolated, GX(x, y) represents the horizontal interpolation pixel value of the pixel to be interpolated, weight_X(x, y) represents the horizontal interpolation weight of the pixel to be interpolated, GY(x, y) represents the vertical interpolation pixel value of the pixel to be interpolated, and weight_Y(x, y) represents the vertical interpolation weight of the pixel to be interpolated.

13. The image processing method according to claim 11, characterized in that: In the step of obtaining the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated, the vertical interpolation pixel value and the horizontal interpolation pixel value of each pixel to be interpolated are obtained respectively by using a preset vertical interpolation filter kernel and a preset horizontal interpolation filter kernel in combination with the original image.

14. The image processing method according to claim 13, characterized in that: The preset vertical interpolation filter kernels and horizontal interpolation filter kernels are: hX=[-1 / 4,1 / 21 / 21 / 2-1 / 4] hY=hX T Among them, hX represents the horizontal interpolation filter kernel; hY represents the vertical interpolation filter kernel, hX T Represents the transpose of hX.

15. The image processing method according to claim 1, characterized in that: Also includes: After the interpolation pixel value of each pixel to be interpolated is obtained, a display image is obtained according to the interpolation pixel value of each pixel to be interpolated and the original image.

16. The image processing method according to claim 15, characterized in that: In the step of obtaining the original image, the original image includes a plurality of color channels, each of the color channels has its own pixel position to be interpolated; one of the plurality of color channels is a reference color channel; In the step of extracting the high-frequency texture components of each pixel to be interpolated on the original image in the vertical direction and the horizontal direction respectively, the high-frequency texture components of each pixel to be interpolated on the reference color channel in the vertical direction and the horizontal direction respectively are extracted; In the step of obtaining the interpolation pixel value of each pixel to be interpolated, the interpolation pixel value of the reference color channel of each pixel to be interpolated of the reference color channel is obtained; The steps to obtain the displayed image include: According to the interpolation pixel value of the reference color channel of each pixel to be interpolated in the reference color channel, combined with the original image, a channel image of a complete reference channel is obtained; According to the channel image of the complete reference channel, combined with the original image, a complete channel image of other color channels is obtained; The complete channel image of the base channel and the complete channel images of the other color channels are merged to obtain the display image.

17. An image processing device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein when the processor executes the program, the processor executes the steps of the image processing method according to any one of claims 1 to 16.

18. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, wherein: When the computer instructions are executed, the steps of the image processing method according to any one of claims 1 to 16 are executed.

Images