Image Processing Method, Image Processing Apparatus, Terminal, and Readable Storage Medium

Abstract

The present application discloses an image processing method, an image processing device, a terminal, and a readable storage medium. The image processing method includes: obtaining a first image, where the first image includes a plurality of first color pixels, second color pixels, and third color pixels, the first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same; taking the second color pixels and the third color pixels in the first image as pixels to be converted, and performing, for each pixel to be converted: converting the pixel to be converted into a first color pixel according to the flat region confidence of the local image centered on the pixel to be converted, the gradient values of the pixel to be converted corresponding to different categories of gradients in n directions respectively, and the color difference values of the pixel to be converted corresponding to n directions respectively, so as to form a second image, where 2 ≤ n ≤ 8; and processing the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

Description

Technical Field

[0001] This application relates to the field of image processing technologies, and in particular, to an image processing method, an image processing apparatus, a terminal, and a computer-readable storage medium. Background Art

[0002] In electronic devices such as mobile phones, cameras can be provided to implement a photographing function. An image sensor for receiving light can be provided in the camera. A filter array can be provided in the image sensor. In order to improve the image quality of images obtained by electronic devices such as mobile phones, the pixels in some image sensors are arranged in a Quad bayer array. However, in the process of image processing, the image processor can only directly process images in the Bayer array, and cannot directly process images arranged in the Quad bayer array. Therefore, how to convert a non-Bayer array (for example, a Quad bayer array) into a Bayer array has become an urgent problem to be solved in the field of image processing. Summary of the Invention

[0003] Embodiments of this application provide an image processing method, an image processing apparatus, a terminal, and a computer-readable storage medium.

[0004] The image processing method according to the embodiments of this application includes: obtaining a first image, where the first image includes a plurality of first color pixels, second color pixels, and third color pixels, the first image includes a plurality of pixel units, and the colors of the pixels in each pixel unit are the same; using the second color pixels and the third color pixels in the first image as pixels to be converted, and performing, for each pixel to be converted: converting the pixel to be converted into a first color pixel according to the flat area confidence of the local image centered on the pixel to be converted, the gradient values of the pixel to be converted corresponding to different categories of gradients in n directions, and the color difference values of the pixel to be converted corresponding to n directions, so as to form a second image, where 2≤n≤8; and processing the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0005] The image processing apparatus according to the embodiment of the present application includes an acquisition module, a first processing module, and a second processing module. The acquisition module is configured to acquire a first image, where the first image includes a plurality of first color pixels, second color pixels, and third color pixels, the first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same. The first processing module is configured to use the second color pixels and the third color pixels in the first image as pixels to be converted, and perform, for each pixel to be converted: converting the pixel to be converted into a first color pixel according to the flat region confidence of the local image centered on the pixel to be converted, the gradient values of different types of gradients corresponding to the pixel to be converted in n directions, and the color difference values corresponding to the pixel to be converted in n directions, so as to form a second image, where 2 ≤ n ≤ 8. The second processing module is configured to process the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0006] The terminal according to the embodiment of the present application includes one or more processors, a memory, and one or more programs. One or more of the programs are stored in the memory and executed by one or more of the processors. The programs include those for executing an image processing method. The image processing method includes: acquiring a first image, where the first image includes a plurality of first color pixels, second color pixels, and third color pixels, the first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same; using the second color pixels and the third color pixels in the first image as pixels to be converted, and performing, for each pixel to be converted: converting the pixel to be converted into a first color pixel according to the flat region confidence of the local image centered on the pixel to be converted, the gradient values of different types of gradients corresponding to the pixel to be converted in n directions, and the color difference values corresponding to the pixel to be converted in n directions, so as to form a second image, where 2 ≤ n ≤ 8; and processing the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0007] The non - volatile computer - readable storage medium of the embodiments of the present application contains a computer program. When the computer program is executed by one or more processors, the processors execute the following image - processing method: obtaining a first image, where the first image includes a plurality of first - color pixels, second - color pixels, and third - color pixels, the first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same; taking the second - color pixels and the third - color pixels in the first image as pixels to be converted, and for each pixel to be converted, performing: converting the pixel to be converted into a first - color pixel according to the flat - area confidence of the local image centered on the pixel to be converted, the gradient values of different - category gradients corresponding to the pixel to be converted in n directions respectively, and the color - difference values corresponding to the pixel to be converted in n directions respectively, so as to form a second image, where 2 ≤ n ≤ 8; and processing the first image and the second image to obtain a target image, where the target image contains a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0008] For the image - processing method, image - processing device, terminal, and readable storage medium of the present application, by performing, for each pixel to be converted: converting the pixel to be converted into a first - color pixel according to the flat - area confidence of the local image centered on the pixel to be converted, the gradient values of different - category gradients corresponding to the pixel to be converted in n directions respectively, and the color - difference values corresponding to the pixel to be converted in n directions respectively, so as to form a second image; subsequently, processing the first image and the second image to obtain a target image. In this way, the first image arranged in a four - Bayer pattern can be converted into a target image arranged in a Bayer array, solving the problem that an image processor cannot directly process an image with pixels arranged in a non - Bayer array.

[0009] Additional aspects and advantages of the embodiments of the present application will be given in part in the following description, become apparent in part from the following description, or be learned through the practice of the present application. Description of the Drawings

[0010] The above - mentioned and / or additional aspects and advantages of the present application will become apparent and be easily understood from the description of the embodiments in conjunction with the following drawings, where:

[0011] Figure 1 is a schematic flowchart of the image - processing method of some embodiments of the present application;

[0012] Figure 2 is a schematic structural diagram of the image - processing device of some embodiments of the present application;

[0013] Figure 3 is a schematic structural diagram of the terminal of some embodiments of the present application;

[0014] Figure 4Schematic diagram of the first image of some embodiments of the present application;

[0015] Figure 5 Schematic diagram of converting a pixel to be converted in the first image into a first color pixel in some embodiments of the present application;

[0016] Figure 6 Schematic diagram of obtaining a second image in some embodiments of the present application;

[0017] Figure 7 And Figure 8 Schematic flowchart of an image processing method in some embodiments of the present application;

[0018] Figure 9 Schematic diagram of the first calculation window in some embodiments of the present application;

[0019] Figures 10 to 12 Schematic flowchart of an image processing method in some embodiments of the present application;

[0020] Figure 13 Schematic diagram of the second calculation window and n directions in some embodiments of the present application;

[0021] Figure 14 Schematic flowchart of an image processing method in some embodiments of the present application;

[0022] Figures 15 to 20 Schematic diagram of obtaining gradient values of different categories of gradients corresponding to a pixel to be converted in n directions respectively in some embodiments of the present application;

[0023] Figure 21 Schematic flowchart of an image processing method in some embodiments of the present application;

[0024] Figure 22a And Figure 22b Schematic diagram of obtaining the color difference value of a pixel to be converted in some embodiments of the present application;

[0025] Figure 23 Schematic flowchart of an image processing method in some embodiments of the present application;

[0026] Figure 24 Schematic diagram of obtaining the color difference value of a pixel to be converted in some embodiments of the present application;

[0027] Figures 25 to 27 Schematic flowchart of an image processing method in some embodiments of the present application;

[0028] Figures 28 to 29 Schematic diagram of obtaining a second color intermediate image according to the first image and the second image in some embodiments of the present application;

[0029] Figure 30 is a schematic diagram of obtaining a second color intermediate image according to a first image and a second image in some embodiments of the present application;

[0030] Figure 31 is a schematic diagram of obtaining a target image according to a second image, a second color intermediate image and a third color intermediate image in some embodiments of the present application;

[0031] Figure 32 It is a schematic diagram of the interaction between a non-volatile computer-readable storage medium and a processor according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0033] See also Figure 1 , the embodiment of the present application provides an image processing method. The image processing method includes:

[0034] 01: Acquire a first image, wherein the first image includes a plurality of first color pixels A, a second color pixel B, and a third color pixel C. The first image includes a plurality of pixel units, and a plurality of pixels in each pixel unit have the same color;

[0035] 02: taking the second color pixel B and the third color pixel C in the first image as the pixel Z to be converted, performing for each pixel Z to be converted: converting the pixel Z to be converted into the first color pixel A according to the flat area confidence of the local image centered on the pixel Z to be converted, the gradient values ​​of different categories of gradients corresponding to the pixel Z to be converted in n directions, and the color difference values ​​corresponding to the pixel Z to be converted in n directions, so as to form a second image, wherein 2≤n≤8; and

[0036] 03: Process the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels arranged in a Bayer array.

[0037] See also Figure 2, an embodiment of the present application also provides an image processing apparatus 100. The image processing apparatus 100 includes an acquisition module 10, a first processing module 20, and a second processing module 30. The method in 01 can be implemented by the acquisition module 10, the method in 02 can be implemented by the first processing module 20, and the method in 03 can be implemented by the second processing module 30. That is to say, the acquisition module 10 can be used to acquire a first image, which includes a plurality of first color pixels A, second color pixels B, and third color pixels C. The first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same; the first processing module 20 can be used to use the second color pixels B and third color pixels C in the first image as the pixels Z to be converted, and perform the following operations on each pixel Z to be converted: according to the flat area confidence of the local image centered on the pixel Z to be converted, the gradient values of the different category gradients corresponding to the pixel Z to be converted in n directions respectively, and the color difference values corresponding to the pixel Z to be converted in n directions respectively, convert the pixel Z to be converted into the first color pixel A to form a second image, where 2 ≤ n ≤ 8; the second processing module 30 can be used to process the first image and the second image to obtain a target image, and the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0038] Please refer to Figure 3 , an embodiment of the present application also provides a terminal 1000. The terminal 1000 includes one or more processors 200, a memory 300, and one or more programs. Among them, the one or more programs are stored in the memory 300 and are executed by the one or more processors 200. The programs include instructions for executing the image processing method of the embodiment of the present application. That is to say, the processor 200 can implement the methods in 01, 02, and 03, that is, the processor 200 can be used to acquire a first image, which includes a plurality of first color pixels A, second color pixels B, and third color pixels C. The first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same; use the second color pixels B and third color pixels C in the first image as the pixels Z to be converted, and perform the following operations on each pixel Z to be converted: according to the flat area confidence of the local image centered on the pixel Z to be converted, the gradient values of the different category gradients corresponding to the pixel Z to be converted in n directions respectively, and the color difference values corresponding to the pixel Z to be converted in n directions respectively, convert the pixel Z to be converted into the first color pixel A to form a second image, where 2 ≤ n ≤ 8; and process the first image and the second image to obtain a target image, and the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0039] The image processing method, image processing apparatus 100, and terminal 1000 of the present application perform the following operations on each pixel Z to be converted: Based on the flat region confidence of the local image centered on the pixel Z to be converted, the gradient values of the pixel Z to be converted corresponding to different categories of gradients in n directions, and the color difference values of the pixel Z to be converted corresponding to n directions, the pixel Z to be converted is converted into a first color pixel A to form a second image; subsequently, the first image and the second image are processed to obtain a target image. In this way, on the one hand, it is possible to convert the first image arranged in a four-Bayer pattern into a target image arranged in a Bayer array, solving the problem that the image processor cannot directly process an image with pixels arranged in a non-Bayer array. On the other hand, in the process of converting the pixel Z to be converted into the first color pixel A, the flat region confidence is introduced, which is beneficial to making the pixel value of the converted first color pixel A closer to the value of the first color channel actually corresponding to this position, thereby being beneficial to improving the image quality of the subsequent obtained target image.

[0040] The processor 200 (or the acquisition module 10) acquires a first image, and a plurality of color pixels in the first image are arranged in a four-Bayer array. Specifically, in some embodiments, the first image includes a plurality of first color pixels A, second color pixels B, and third color pixels C, and the first image includes a plurality of pixel units, and the colors of the pixels in each pixel unit are the same. As Figure 4 shown, the first image includes a first pixel unit UA, a second pixel unit UB, and a third pixel unit UC. The first pixel unit UA includes first color pixels A arranged in a 2×2 array, the second pixel unit UB includes second color pixels B arranged in a 2×2 array, and the third pixel unit UC includes third color pixels C arranged in a 2×2 array. A plurality of first pixel units UA are arranged along the first diagonal D1 direction, and a plurality of second pixel units UB and a plurality of third pixel units UC are arranged along the second diagonal D2 direction.

[0041] In some embodiments, the first color pixel A may be a green pixel G; the second color pixel B may be a red pixel R; the third color pixel C may be a blue pixel Bu. In some embodiments, the first color pixel A may be a yellow pixel Y; the second color pixel B may be a red pixel R; the third color pixel C may be a blue pixel Bu. In some embodiments, the first color pixel A may be a cyan pixel Cy, the second color pixel B may be a magenta pixel M; the third color pixel C may be a yellow pixel Y. In the embodiments of the present application, it is described by taking the first color as green, the second color as red, and the third color as blue, that is, the first color pixel A is a green pixel G; the second color pixel B is a red pixel R; the third color pixel C is a blue pixel Bu as an example.

[0042] After obtaining the first image, the processor 200 (or the first processing module 20) uses the second-color pixels B and the third-color pixels C in the first image as the pixels Z to be converted, and performs the following operations on each pixel Z to be converted: According to the flat-region confidence of the local image centered on the pixel Z to be converted, the gradient values of different categories of gradients corresponding to the pixel Z to be converted in n directions respectively, and the color difference values corresponding to the pixel Z to be converted in n directions respectively, convert the pixel Z to be converted into the first-color pixel A to form a second image. Where 2 ≤ n ≤ 8, that is, the number of directions can be 2, 3, 4, 5, 6, 7, and 8, and there is no limit here. Figure 5 Shown is the conversion of one of the pixels Z to be converted in the first image into the first-color pixel A. After all the pixels Z to be converted in the first image are converted into the first-color pixel A, that is, after all the second-color pixels B and the third-color pixels C in the first image are converted into the first-color pixel A, in this way, a second image containing only the first-color pixel A is obtained (as Figure 6 shown).

[0043] The following takes the conversion of one of the pixels Z to be converted in the first image into the first-color pixel A as an example for detailed description.

[0044] Specifically, please refer to Figure 7 , according to the flat-region confidence of the local image centered on the pixel Z to be converted, the gradient values of different categories of gradients corresponding to the pixel Z to be converted in n directions respectively, and the color difference values corresponding to the pixel Z to be converted in n directions respectively, converting the pixel Z to be converted into the first-color pixel A includes:

[0045] 021: Obtain the flat-region confidence of the local image centered on the pixel Z to be converted;

[0046] 022: According to the gradient values of different categories of gradients corresponding to the pixel Z to be converted in n directions respectively, and the color difference values corresponding to the pixel Z to be converted in n directions respectively, obtain the intermediate pixel value for converting the pixel Z to be converted into the first-color pixel A;

[0047] 023: According to the flat-region confidence and the intermediate pixel value, obtain the target pixel value for converting the pixel Z to be converted into the first-color pixel A.

[0048] Please combine Figure 2 and Figure 3, in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 021, 022, and 023. That is, the first processing module 20 and the processor 200 can also be used to obtain the flat area confidence of the local image centered on the pixel Z to be converted; obtain the intermediate pixel value of converting the pixel Z to be converted into the first color pixel A according to the gradient values of different categories corresponding to the pixel Z to be converted in n directions and the color difference values corresponding to the pixel Z to be converted in n directions; and obtain the target pixel value of converting the pixel Z to be converted into the first color pixel A according to the flat area confidence and the intermediate pixel value.

[0049] It should be noted that, in some embodiments, the flat area confidence of the local image centered on the pixel Z to be converted can be used to characterize the degree of the local image centered on the pixel Z to be converted being a flat area.

[0050] Exemplarily, please refer to Figure 8 , in some embodiments, obtaining the flat area confidence of the local image centered on the pixel Z to be converted includes:

[0051] 0211: Preset a first calculation window F1 centered on the pixel Z to be converted;

[0052] 0212: Obtain the pixel values of all the first color pixels A in the first calculation window F1, and calculate the average value wave and the variance value var of the multiple pixel values; and

[0053] 0213: Determine the flat area confidence according to the average value, the variance value, and the preset parameters.

[0054] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 0211, 0212, and 0213. That is, the first processing module 20 and the processor 200 can also be used to preset a first calculation window F1 centered on the pixel Z to be converted; obtain the pixel values of all the first color pixels A in the first calculation window F1, and calculate the average value wave and the variance value var of the multiple pixel values; and determine the flat area confidence according to the average value, the variance value, and the preset parameters.

[0055] Please refer to Figure 9, a first calculation window F1 centered on the pixel Z to be converted is preset. For example, assume that the first calculation window F1 is a window of size 9*9. At this time, the pixel Z to be converted is placed at the center position of the first calculation window F1, that is, the pixel Z to be converted is placed at the 5th row and 5th column of the first calculation window F1. Subsequently, the pixel values of all the first color pixels A within the first calculation window F1 are obtained. It should be noted that the image composed of multiple pixels within the first calculation window F1 is the local image centered on the pixel Z to be converted. The first calculation window F1 is a virtual calculation window and not an actual existing structure; and the size of the first calculation window F1 can be arbitrarily changed according to actual needs, and all the calculation windows mentioned below are the same, which will not be elaborated here.

[0056] After obtaining the pixel values of all the first color pixels A within the first calculation window F1, the average value and variance value of the multiple pixel values are calculated. Subsequently, according to the average value, variance value, and preset parameters, the flat area confidence is determined. Specifically, in some embodiments, the preset parameters include a first initial parameter, a second initial parameter, a first constant, a second parameter, a first flat area threshold, a second flat area threshold, a third flat area threshold, a fourth flat area threshold, and a fifth flat area threshold. After obtaining the average value and variance value of the pixel values of all the first color pixels A within the first calculation window F1. The processor 200 (or the first processing module 20) can calculate a first reference value according to the average value, the first constant, and the first initial parameter; calculate a second reference value according to the average value, the second constant, and the second initial parameter. Among them, the first reference value = (average value + first constant) / first initial value; the second parameter value = (average value + second parameter) / second initial value.

[0057] After obtaining the first parameter value and the second parameter value, it is determined whether the variance is less than or equal to the sum of the second reference value and the first flat area threshold. If the variance is less than or equal to the sum of the second reference value and the first flat area threshold, the flat area confidence corresponding to the pixel Z to be converted can be determined as the first confidence. If the variance is greater than the sum of the second reference value and the first flat area threshold, the magnitudes of the first reference value and the second flat area threshold are compared, and the smaller value is selected and summed with the second reference value to obtain a first result. It is determined whether the variance is less than or equal to the first result. If the variance is less than or equal to the first result, the flat area confidence corresponding to the pixel Z to be converted can be determined as the second confidence. If the variance is greater than the first result, the magnitudes of twice the first reference value and the third flat area threshold are compared, and the smaller value is selected and summed with the second reference value to obtain a second result. It is determined whether the variance is less than or equal to the second result. If the variance is less than or equal to the second result, the flat area confidence corresponding to the pixel Z to be converted can be determined as the third confidence. If the variance is greater than the second result, the magnitudes of three times the first reference value and the fourth flat area threshold are compared, and the smaller value is selected and summed with the second reference value to obtain a third result. It is determined whether the variance is less than or equal to the third result. If the variance is less than or equal to the third result, the flat area confidence corresponding to the pixel Z to be converted can be determined as the fourth confidence. If the variance is greater than the third result, the magnitudes of four times the first reference value and the fifth flat area threshold are compared, and the smaller value is selected and summed with the second reference value to obtain a fourth result. It is determined whether the variance is less than or equal to the fourth result. If the variance is less than or equal to the fourth result, the flat area confidence corresponding to the pixel Z to be converted can be determined as the fifth confidence; if the variance is greater than the fourth result, the flat area confidence corresponding to the pixel Z to be converted can be determined as the sixth confidence.

[0058] It should be noted that the preset parameters can be changed according to the actual situation, that is, the first initial parameter, the second initial parameter, the first constant, the second parameter, the first flat area threshold, the second flat area threshold, the third flat area threshold, the fourth flat area threshold, and the fifth flat area threshold can be changed according to the actual situation. In particular, in some embodiments, the first parameter can be set to 4; the second parameter can be set to 2; the first confidence can be set to 8, the second confidence can be set to 6, the third confidence can be set to 4, the fourth confidence can be set to 2, the fifth confidence can be set to 1, and the sixth confidence can be set to 0.

[0059] Please refer to Figure 10 , in some embodiments, according to the gradient values of different categories corresponding to the pixel Z to be converted in n directions and the color difference values corresponding to the pixel Z to be converted in n directions, obtaining the intermediate pixel value for converting the pixel Z to be converted into the first color pixel includes:

[0060] 0221: Calculate the gradient weights corresponding to the to-be-converted pixel Z in n directions respectively according to the gradient values of different categories of gradients corresponding to the n directions of the to-be-converted pixel Z.

[0061] 0222: Obtain the intermediate pixel value of converting the to-be-converted pixel Z into the first color pixel A according to the color difference values corresponding to the n directions of the to-be-converted pixel Z respectively and the gradient weights.

[0062] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 0221 and 0222. That is, both the first processing module 20 and the processor 200 are also used to calculate the gradient weights corresponding to the to-be-converted pixel Z in n directions respectively according to the gradient values of different categories of gradients corresponding to the n directions of the to-be-converted pixel Z; and obtain the intermediate pixel value of converting the to-be-converted pixel Z into the first color pixel according to the color difference values corresponding to the n directions of the to-be-converted pixel Z respectively and the gradient weights.

[0063] Specifically, in some embodiments, after obtaining the first image, the processor 200 (or the first processing module 20) can first obtain the gradient values of different categories of gradients corresponding to the to-be-converted pixel Z in n directions respectively and the color difference values corresponding to the to-be-converted pixel Z in n directions respectively. By way of example, please refer to Figure 10 , in some embodiments, obtaining the intermediate pixel value of converting the to-be-converted pixel Z into the first color pixel according to the gradient values of different categories of gradients corresponding to the to-be-converted pixel Z in n directions respectively and the color difference values corresponding to the to-be-converted pixel Z in n directions respectively may further include:

[0064] 0223: Obtain the gradient values of different categories of gradients corresponding to the to-be-converted pixel Z in n directions respectively and the color difference values corresponding to the to-be-converted pixel Z in n directions respectively according to the pixel values of multiple pixels in the first image.

[0065] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the method in 0223. That is, both the first processing module 20 and the processor 200 are also used to obtain the gradient values of different categories of gradients corresponding to the to-be-converted pixel Z in n directions respectively and the color difference values corresponding to the to-be-converted pixel Z in n directions respectively according to the pixel values of multiple pixels in the first image.

[0066] After obtaining the first image, the processor 200 (or the first processing module 20) can directly obtain the gradient values of different categories of gradients corresponding to the pixel to be converted Z in n directions and the color difference values corresponding to the pixel to be converted Z in n directions according to the pixel values of multiple pixels in the first image. It should be noted that the number of gradient values belonging to the same category of gradients is the same as the number of directions, and the number of color difference values is also the same as the number of directions. For example, assuming there are two different categories of gradients, namely the first category of gradient and the second category of gradient, and there are two different directions, namely the first direction and the second direction, then it is necessary to calculate the gradient value of the first category of gradient corresponding to the pixel to be converted Z in the first direction, the gradient value of the second category of gradient, and the color difference value, and the gradient value of the first category of gradient corresponding to the second direction, the gradient value of the second category of gradient, and the color difference value.

[0067] In particular, in some embodiments, the processor 200 (or the first processing module 20) can also first weight the pixel values of the second color pixel B and the third color pixel C in the first image, and then obtain the gradient values of different categories of gradients corresponding to the pixel to be converted Z in n directions and the color difference values corresponding to the pixel to be converted Z in n directions. By way of example, please refer to Figure 11 , in some embodiments, obtaining the intermediate pixel value of converting the pixel to be converted Z into the first color pixel according to the gradient values of different categories of gradients corresponding to the pixel to be converted Z in n directions and the color difference values corresponding to the pixel to be converted Z in n directions may further include:

[0068] 0224: Weight the pixel values of the second color pixel B and the third color pixel C in the first image respectively according to the gain parameters corresponding to the second color and the third color to obtain the processed first image; and obtain the gradient values of different categories of gradients corresponding to the pixel to be converted Z in n directions and the color difference values corresponding to the pixel to be converted Z in n directions according to the pixel values of multiple pixels in the processed first image.

[0069] Please combine Figure 2 and Figure 3 , in some embodiments, both the first processing module 20 and the processor 200 can also be used to execute the method in 0224. That is, both the first processing module 20 and the processor 200 are also used to weight the pixel values of the second color pixel B and the third color pixel C in the first image respectively according to the gain parameters corresponding to the second color and the third color to obtain the processed first image; and obtain the gradient values of different categories of gradients corresponding to the pixel to be converted Z in n directions and the color difference values corresponding to the pixel to be converted Z in n directions according to the pixel values of multiple pixels in the processed first image.

[0070] It should be noted that in some embodiments, while the processor 200 (or the acquisition module 10) acquires the first image, it can also acquire the shooting parameters when shooting the first image. The shooting parameters include the gain parameter corresponding to the second color and the gain parameter corresponding to the third color. Among them, the magnitudes of the gain parameter corresponding to the second color and the gain parameter corresponding to the third color are related to the light source color temperature of the shooting environment when shooting the first image.

[0071] After acquiring the first image, the gain parameter corresponding to the second color, and the gain parameter corresponding to the third color, the processor 200 (or the first processing module 20) performs weighted processing on the pixel values of the second color pixels and the third color pixels in the first image respectively according to the gain parameter corresponding to the second color and the gain parameter corresponding to the third color, so as to obtain the processed first image. In this way, compared with directly obtaining the gradient values of different categories corresponding to the n directions of the pixel to be converted Z and the color difference values corresponding to the n directions of the pixel to be converted Z based on the original pixel values of multiple pixels in the original first image (not processed), the accuracy of the obtained gradient values and color difference values can be improved, which is beneficial to improving the image quality of the finally obtained target image.

[0072] Specifically, in some embodiments, the pixel value of each second color pixel B in the first image is multiplied by the gain parameter corresponding to the second color to update the pixel value of the pixel; and the pixel value of each third color pixel C in the first image is multiplied by the gain parameter corresponding to the third color to update the pixel value of the pixel, so as to obtain the processed first image. For example, among them, the pixel value of the third color pixel C arranged in the first row and first column of the processed first image is equal to the product of the gain parameter corresponding to the third color and the pixel value of the third color pixel C arranged in the first row and first column of the first image. The pixel value of the second color pixel B arranged in the third row and third column of the processed first image is equal to the product of the gain parameter corresponding to the second color and the pixel value of the second color pixel B arranged in the third row and third column of the first image.

[0073] It should be noted that in the process of calculating the gradient values and color difference values of different categories in detail below, the pixel values of the second color pixels and the third color pixels in the first image involved can be either the original pixel values of the unprocessed second color pixels and third color pixels or the pixel values of the second color pixels and third color pixels after weighted processing, which will not be elaborated further.

[0074] Please refer to Figure 12 , in some embodiments, different categories of gradients have different priorities. According to the gradient values of different categories of gradients corresponding to the n directions of the pixel to be converted Z, calculate the gradient weights corresponding to the n directions of the pixel to be converted Z, including:

[0075] 041: If the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is greater than a preset difference, then take the gradient value of the first-priority gradient as the target gradient value corresponding to the direction;

[0076] 042: If the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is less than the preset difference, then obtain the weighted gradient values corresponding to n directions respectively for the pixel Z to be converted according to the gradient value of the first-priority gradient and the gradient value of the second-priority gradient;

[0077] 043: If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is greater than the preset difference, then take the weighted gradient value as the target gradient value corresponding to the direction. If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is less than the preset difference, then successively accumulate the gradient values of the next-priority gradient to update the previous weighted gradient value until the difference between the two smallest updated weighted gradient values is greater than the preset difference and stop the cumulative update, then take the last updated weighted gradient value as the target gradient value corresponding to the direction; and

[0078] 044: Obtain the gradient weights corresponding to n directions according to the target gradient values corresponding to n directions.

[0079] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 041, 042, 043, and 044. That is, the first processing module 20 and the processor 200 can also be used to: if the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is greater than the preset difference, then take the gradient value of the first-priority gradient as the target gradient value corresponding to the direction; if the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is less than the preset difference, then obtain the weighted gradient values corresponding to n directions respectively for the pixel Z to be converted according to the gradient value of the first-priority gradient and the gradient value of the second-priority gradient. If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is greater than the preset difference, then take the weighted gradient value as the target gradient value corresponding to the direction; if the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is less than the preset difference, then successively accumulate the gradient values of the next-priority gradient to update the previous weighted gradient value until the difference between the two smallest updated weighted gradient values is greater than the preset difference and stop the cumulative update, then take the last updated weighted gradient value as the target gradient value corresponding to the direction; and obtain the gradient weights corresponding to n directions according to the target gradient values corresponding to n directions.

[0080] In some embodiments, different categories of gradients have different priorities. The processor 200 (or the first processing module 20) first compares the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions. If the difference between the two smallest gradient values is greater than a preset difference, the gradient value of the first-priority gradient is directly used as the target gradient value corresponding to the direction. For example, assuming that the gradient value of the first-priority gradient corresponding to the first direction is g11, the target gradient value corresponding to the first direction is g11.

[0081] Since if the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is greater than the preset difference, that is, the two smallest gradient values are quite different, it indicates that the first-priority gradient has separability. At this time, the texture direction can be determined only by the first-priority gradient. Therefore, in this embodiment, when the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is greater than the preset difference, the gradient value of the first-priority gradient is directly used as the target gradient value corresponding to the direction. In this way, there is no need to superimpose gradients of other categories, which is beneficial to improving the image processing speed and reducing the complexity of image processing.

[0082] If the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is less than the preset difference, weighted gradient values corresponding to the n directions of the pixel Z to be converted are obtained according to the gradient values of the first-priority gradients and the gradient values of the second-priority gradients. The weighted gradient value corresponding to a certain direction is obtained according to the gradient value of the first-priority gradient corresponding to the direction and the gradient value of the second-priority gradient corresponding to the direction.

[0083] In some embodiments, obtaining the weighted gradient values corresponding to the n directions of the pixel Z to be converted according to the gradient values of the first-priority gradients and the gradient values of the second-priority gradients may include the following steps: (1) First, group the gradient values of the first-priority gradients corresponding to n directions according to their corresponding directions, where the directions corresponding to the gradient values of the first-priority gradients in the same group are opposite; (2) Subsequently, extract the smallest gradient value from each group, and then obtain the separability weight according to the two smallest gradient values among the extracted gradient values; (3) Obtain the modulation parameter corresponding to the second-priority gradient, and obtain the weighted gradient value corresponding to the direction according to the separability weight, the modulation parameter corresponding to the second-priority gradient, and the gradient values of the first-priority gradient and the second-priority gradient corresponding to the same direction. Among them, in some embodiments, the weighted gradient value corresponding to a certain direction = the gradient value of the first-priority gradient corresponding to the direction + (the separability weight × the modulation parameter × the gradient value of the second-priority gradient corresponding to the direction).

[0084] Since the divisibility weight is first obtained according to the gradient values of the first-priority gradients in n directions, and then the divisibility weight is introduced in the process of weighted superposition of the second-priority gradients on the first-priority gradients, the second-priority gradients are not weighted with fixed values, which is beneficial to improving the resolution of the color texture directions, and thus is beneficial to improving the image quality of the target image obtained subsequently.

[0085] Exemplarily, taking n as 8, that is, 8 directions as an example for illustration. As Figure 13As shown, the eight directions are the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD. Among them, the first horizontal direction E is opposite to the second horizontal direction W, the first vertical direction N is opposite to the second vertical direction S, the first diagonal direction AU is opposite to the second diagonal direction AD, and the third diagonal direction DU is opposite to the fourth diagonal direction DD. Extract the smaller gradient value from the gradient value of the first-priority gradient corresponding to the first horizontal direction E and the gradient value of the first-priority gradient corresponding to the first horizontal direction E; extract the smaller gradient value from the gradient value of the first-priority gradient corresponding to the first vertical direction N and the gradient value of the first-priority gradient corresponding to the second vertical direction S; extract the smaller gradient value from the gradient value of the first-priority gradient corresponding to the first diagonal direction AU and the gradient value of the first-priority gradient corresponding to the second diagonal direction AD; extract the smaller gradient value from the gradient value of the first-priority gradient corresponding to the third diagonal direction DU and the gradient value of the first-priority gradient corresponding to the fourth diagonal direction DD. Subsequently, select the smallest gradient value among the four extracted gradient values as the first value; select the second smallest gradient value among the four extracted gradient values as the second value. After obtaining the first value and the second value, compare the fourth parameter with the quotient of the first value divided by the third constant, and take the larger value of the two as the comparison parameter. If the second value is less than or equal to the sum of the first value and the comparison parameter, the separability weight can be determined as the first separability weight; if the second value is greater than the sum of the first value and the comparison parameter, and the second value is less than or equal to the sum of the first value and twice the comparison parameter, the separability weight can be determined as the second separability weight; if the second value is greater than the sum of the first value and twice the comparison parameter, and the second value is less than or equal to the sum of the first value and three times the comparison parameter, the separability weight can be determined as the third separability weight; if the second value is greater than the sum of the first value and three times the comparison parameter, and the second value is less than or equal to the sum of the first value and four times the comparison parameter, the separability weight can be determined as the fourth separability weight; if the second value is greater than the sum of the first value and four times the comparison parameter, and the second value is less than or equal to the sum of the first value and five times the comparison parameter, the separability weight can be determined as the fifth separability weight; if the second value is greater than the sum of the first value and five times the comparison parameter, and the second value is less than or equal to the sum of the first value and six times the comparison parameter, the separability weight can be determined as the sixth separability weight; if the second value is greater than the sum of the first value and six times the comparison parameter, and the second value is less than or equal to the sum of the first value and seven times the comparison parameter, the separability weight can be determined as the seventh separability weight; if the second value is greater than the sum of the first value and seven times the comparison parameter, and the second value is less than or equal to the sum of the first value and eight times the comparison parameter, the separability weight can be determined as the eighth separability weight; if the second value is greater than the sum of the first value and eight times the comparison parameter, the separability weight can be determined as the ninth separability weight.Among them, the first divisibility weight, the second divisibility weight, the third divisibility weight, the fourth divisibility weight, the fifth divisibility weight, the sixth divisibility weight, the seventh divisibility weight, the eighth divisibility weight, the ninth divisibility weight, the third constant, and the fourth constant can be arbitrarily changed according to actual needs. In some embodiments, the third constant can be 8, the fourth constant can be 3, the first divisibility weight is 8 / 8, the second divisibility weight is 7 / 8, the third divisibility weight is 6 / 8, the fourth divisibility weight is 5 / 8, the fifth divisibility weight is 4 / 8, the sixth divisibility weight is 3 / 8, the seventh divisibility weight is 2 / 8, the eighth divisibility weight is 1 / 8, and the ninth divisibility weight is 0.

[0086] It should be noted that different types of gradients have corresponding modulation parameters, which can be pre-stored in the terminal 1000 (or the image processing device 100). When accumulating gradients of different types, the adjustment parameter corresponding to the accumulated gradient type can be called automatically. Among them, the adjustment parameters corresponding to different types of gradients can be the same or different. In addition, the preset difference can be pre-set before the terminal 1000 (or the image processing device 100) leaves the factory, or can be set by the user according to requirements, and there is no limitation here. If the difference between the two smallest gradient values among the gradient values of the first-priority gradients corresponding to n directions is equal to the preset difference, either the gradient value of the first-priority gradient can be directly used as the target gradient value for the corresponding direction, or the weighted gradient values corresponding to the n directions of the pixel Z to be converted can be obtained according to the gradient value of the first-priority gradient and the gradient value of the second-priority gradient, and the subsequent steps can be continued, and there is no limitation here either.

[0087] After obtaining the weighted gradient values corresponding to n directions, if the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is greater than the preset difference, it indicates that the first-priority gradient and the second-priority gradient have divisibility after superposition, and the weighted gradient value is used as the target gradient value for the corresponding direction.

[0088] If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is still less than the preset difference, the gradient value of the next-priority gradient is successively accumulated to update the previous weighted gradient value until the difference between the two smallest updated weighted gradient values is greater than the preset difference and the cumulative update stops. Then, the last updated weighted gradient value is used as the target gradient value for the corresponding direction. That is, each time the weighted gradient value is updated, it is judged whether the difference between the two smallest updated weighted gradient values is greater than the preset difference. If it is greater, the update stops; if it is not greater, the previous weighted gradient value is updated according to the gradient value of the next-priority gradient. The specific implementation of updating the previous weighted gradient value according to the gradient value of the next-priority gradient is the same as the specific implementation of obtaining the weighted gradient values corresponding to n directions respectively by the gradient values of the first-priority gradient and the second-priority gradient in the above embodiment, and will not be elaborated here. Of course, in some embodiments, after the cumulative update ends at a certain time, if the smallest gradient value among the weighted gradient values corresponding to n directions is greater than the preset gradient value, the cumulative update can also be stopped, and the last updated weighted gradient value is used as the target gradient value for the corresponding direction.

[0089] In particular, when the color difference gradient is cumulatively superimposed, during the process of obtaining or updating the weighted gradient value, local gray-scale confidence discrimination needs to be introduced. The specific implementation of introducing local gray-scale confidence discrimination during the process of cumulatively superimposing the color difference gradient will be introduced in detail below and will not be elaborated here.

[0090] Since the separability of the updated weighted gradient value is judged each time a category of gradient value is superimposed, compared with directly superimposing all categories of gradients with fixed weights, it is beneficial to improve the resolution of the color texture direction in image processing, thereby being beneficial to improving the image quality of the subsequent obtained target image.

[0091] It should be noted that if the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is equal to the preset difference, the weighted gradient value can either directly be used as the target gradient value for the corresponding direction or the gradient value of the next-priority gradient can be successively accumulated, and there is no limitation here. In addition, in some embodiments, the gradient values of all categories of gradients corresponding to n directions can be calculated first. In some embodiments, the gradient values of the first-priority gradient corresponding to n directions can also be calculated first, and when the gradient values of other priority gradients need to be accumulated subsequently, the gradient values of other priority gradients corresponding to n directions are calculated. In this way, compared with calculating the gradient values of all categories of gradients corresponding to n directions first, it is beneficial to improve the image processing speed.

[0092] After obtaining the target gradient values corresponding to n directions, according to the target gradient values corresponding to the n directions, obtain the gradient weights corresponding to the n directions. Specifically, in some embodiments, first select the smallest target gradient value from the target gradient values corresponding to the n directions, and then calculate the product of the smallest target gradient value and a fifth constant. Subsequently, compare the size between a sixth constant and the obtained product, and select the smaller value as the first data. After obtaining the first data, compare the size between the first data and a seventh constant, and select the larger value as the second data. Among them, the fifth constant, the sixth constant, and the seventh constant can be changed according to actual needs. In some embodiments, the fifth constant can be The sixth constant can be 50, and the seventh constant can be 10.

[0093] Particularly, when the smallest target gradient value is greater than the preset target gradient value, the second data can be updated according to the following calculation formula: delta2 = M8 + M9×(minGrad - M10). Where delta2 represents the second data, minGrad represents the smallest gradient value, and M8, M9, and M10 respectively represent the eighth constant, the ninth constant, and the tenth constant. Similarly, the preset target gradient value, the eighth constant, the ninth constant, and the tenth constant can all be changed according to actual needs. In some embodiments, the eighth constant = the seventh constant = 10, The tenth constant = the preset target gradient value = the sixth constant = 50. If the smallest target gradient value is not greater than the preset target gradient value, there is no need to update the second data. If the second data (including the unupdated second data and the updated second data) is greater than the preset data, then use the preset data as the third data. If the second data is not greater than the preset data, then use the second data as the third data. Among them, in some embodiments, the preset data can be 100.

[0094] After obtaining the third data, traverse each direction, and determine the gradient weight corresponding to the current direction according to the condition satisfied by the target gradient value corresponding to the current direction. For example, in some embodiments, if the target gradient value corresponding to the current direction satisfies the first condition, the gradient weight corresponding to the current direction can be determined as the first weight value, where the first condition is that the target gradient value corresponding to the current direction is greater than or equal to the minimum target gradient value, and the target gradient value corresponding to the current direction is less than or equal to the sum of the minimum target gradient value and the third data. If the target gradient value corresponding to the current direction satisfies the second condition, the gradient weight corresponding to the current direction can be determined as the second weight value, where the second condition is that the target gradient value corresponding to the current direction is greater than the sum of the minimum target gradient value and the third data, and the target gradient value corresponding to the current direction is less than or equal to the sum of the minimum target gradient value and twice the third data. If the target gradient value corresponding to the current direction satisfies the third condition, the gradient weight corresponding to the current direction can be determined as the third weight value, where the third condition is that the target gradient value corresponding to the current direction is greater than the sum of the minimum target gradient value and twice the third data, and the target gradient value corresponding to the current direction is less than or equal to the sum of the minimum target gradient value and three times the third data. If the target gradient value corresponding to the current direction satisfies the fourth condition, the gradient weight corresponding to the current direction can be determined as the fourth weight value, where the fourth condition is that the target gradient value corresponding to the current direction is greater than the sum of the minimum target gradient value and three times the third data, and the target gradient value corresponding to the current direction is less than or equal to the sum of the minimum target gradient value and four times the third data. If the target gradient value corresponding to the current direction does not satisfy any of the above first condition, second condition, third condition, and fourth condition, the gradient weight corresponding to the current direction can be determined as the fifth weight value.

[0095] Similarly, the magnitudes of the first weight value, the second weight value, the third weight value, the fourth weight value, and the fifth weight value can be changed according to actual requirements. In some embodiments, the first weight value, the second weight value, the third weight value, the fourth weight value, and the fifth weight value can be 8, 4, 2, 1, and 0 respectively.

[0096] It should be noted that the priority levels of different categories of gradients are preset before the terminal 1000 (or the image processing device 100) leaves the factory. Further, in some embodiments, different categories of gradients include high-frequency gradients, color difference gradients, second-order gradients, low-frequency gradients, first gradients, and second gradients. Among them, the 1st priority gradient is the high-frequency gradient, the 2nd priority gradient is the color difference gradient, the 3rd priority is the second-order gradient, the 4th priority gradient is the low-frequency gradient, the 5th priority gradient is the first gradient, and the 6th priority gradient is the second gradient.

[0097] Exemplarily, if the difference between the two smallest gradient values among the gradient values of the high-frequency gradients corresponding to n directions is greater than a preset difference, that is, the gradient value of the smallest high-frequency gradient is much smaller than the gradient value of the second smallest high-frequency gradient, it indicates that the current high-frequency gradient has strong separability and there is no need to superimpose gradients of other categories. At this time, directly use the gradient value of the high-frequency gradient as the target gradient value corresponding to the direction. If the difference between the two smallest gradient values among the gradient values of the high-frequency gradients corresponding to n directions is less than the preset difference, it indicates that the current high-frequency gradient has relatively weak separability. Then, obtain the weighted gradient values corresponding to the n directions of the pixel to be converted Z according to the gradient value of the high-frequency gradient and the gradient value of the color difference gradient.

[0098] Please refer to Figure 14 , in some embodiments, when the first-priority gradient is a high-frequency gradient and the second-priority gradient is a color difference gradient, obtain the weighted gradient values corresponding to the n directions of the pixel to be converted Z according to the gradient value of the first-priority gradient and the gradient value of the second-priority gradient, that is, obtain the weighted gradient values corresponding to the n directions of the pixel to be converted Z according to the gradient value of the high-frequency gradient and the gradient value of the color difference gradient, including:

[0099] 0421: Preset a second calculation window F2 centered on the pixel to be converted Z, and obtain the gray-scale confidence of the pixel to be converted Z according to the color difference values corresponding to the pixel to be converted Z and the n directions and the average brightness of all the first-color pixels A in the second calculation window F2;

[0100] 0422: Obtain the separability weight according to the high-frequency gradient values corresponding to the pixel to be converted Z and the n directions; and

[0101] 0423: Obtain the weighted gradient value according to the gray-scale confidence, the separability weight, the modulation parameter corresponding to the color difference gradient, and the gradient values of the high-frequency gradient and the color difference gradient corresponding to the same direction.

[0102] Please combine Figure 2 and Figure 3, in some embodiments, when the first priority gradient is a high-frequency gradient and the second priority gradient is a color difference gradient, both the first processing module 20 and the processor 200 can also be used to execute the methods in 0421, 0422, and 0423. That is, both the first processing module 20 and the processor 200 can also be used to preset a second calculation window F2 centered on the pixel Z to be converted, and obtain the gray-scale confidence corresponding to the pixel Z to be converted according to the color difference values corresponding to the pixel Z to be converted in n directions and the average brightness of all the first color pixels A in the second calculation window F2; obtain the separability weight according to the high-frequency gradient values corresponding to the pixel Z to be converted in n directions; and obtain the weighted gradient value according to the gray-scale confidence, the separability weight, the modulation parameter corresponding to the color difference gradient, and the gradient values of the high-frequency gradient and the color difference gradient corresponding to the same direction.

[0103] Please refer to Figure 13 , preset a second calculation window F2 centered on the pixel Z to be converted. For example, assume that the second calculation window F2 is a 9*9 window. At this time, place the pixel Z to be converted at the center position of the second calculation window F2, that is, place the pixel Z to be converted at the 5th row and 5th column of the second calculation window F2. Subsequently, obtain the brightness values of all the first color pixels A in the second calculation window F2, and calculate the average value of the multiple brightness values, so as to obtain the average brightness of all the first color pixels A in the second calculation window F2. It should be noted that in some embodiments, the size of the second calculation window F2 is the same as that of the first calculation window F1 in the above embodiments. That is, if the first calculation window F1 is a 9*9 window, the second calculation window F2 is also a 9*9 window.

[0104] Obtaining the gray-scale confidence corresponding to the pixel Z to be converted according to the color difference values corresponding to the pixel Z to be converted in n directions and the average brightness of all the first color pixels A in the second calculation window F2 includes the following steps: (1) First, group the color difference values corresponding to the n directions according to their corresponding directions, where the directions corresponding to the color difference values in the same group are opposite; (2) Subsequently, calculate the average color difference of each group, and then obtain the first quotient according to the minimum average color difference, the average brightness of all the first color pixels A in the second calculation window F2, and the preset brightness parameter; (3) Compare the first quotient with multiple preset gray-scale thresholds to obtain the gray-scale confidence corresponding to the pixel Z to be converted.

[0105] Exemplarily, take n as 8, that is, 8 directions as an example for illustration. As Figure 13As shown, the eight directions are the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD. Among them, the first horizontal direction E is opposite to the second horizontal direction W, the first vertical direction N is opposite to the second vertical direction S, the first diagonal direction AU is opposite to the second diagonal direction AD, and the third diagonal direction DU is opposite to the fourth diagonal direction DD. Based on the directions corresponding to multiple color differences, the first horizontal color difference value corresponding to the first horizontal direction E and the second horizontal color difference value corresponding to the second horizontal direction W can be divided into the same group, and the average color difference of this group is equal to the sum of the first horizontal color difference value and the second horizontal color difference value divided by 2; the first vertical color difference value corresponding to the first vertical direction N and the second vertical color difference value corresponding to the second vertical direction W are divided into the same group, and the average color difference of this group is equal to the sum of the first vertical color difference value and the second vertical color difference value divided by 2; the first diagonal color difference value corresponding to the first diagonal direction AU and the second diagonal color difference value corresponding to the second diagonal direction AD are divided into the same group, and the average color difference of this group is equal to the sum of the first diagonal color difference value and the second diagonal color difference value divided by 2; the third diagonal color difference value corresponding to the third diagonal direction DU and the fourth diagonal color difference value corresponding to the fourth diagonal direction DD are divided into the same group, and the average color difference of this group is equal to the sum of the third diagonal color difference value and the fourth diagonal color difference value divided by 2.

[0106] After obtaining multiple color difference means, select the smallest color difference mean among the multiple color difference means. The first quotient = the smallest color difference mean / (the average brightness of all first color pixels A within the second calculation window F2 + the brightness parameter). After obtaining the first quotient, compare the first quotient with multiple preset gray level thresholds to obtain the gray level confidence corresponding to the pixel Z to be converted. For example, if the first quotient is less than the first gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the first gray level confidence; if the first quotient is greater than or equal to the first gray level threshold and less than the second gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the second gray level confidence; if the first quotient is greater than or equal to the second gray level threshold and less than the third gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the third gray level confidence; if the first quotient is greater than or equal to the third gray level threshold and less than the fourth gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the fourth gray level confidence; if the first quotient is greater than or equal to the fourth gray level threshold and less than the fifth gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the fifth gray level confidence; if the first quotient is greater than or equal to the fifth gray level threshold and less than the sixth gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the sixth gray level confidence; if the first quotient is greater than or equal to the sixth gray level threshold and less than the seventh gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the seventh gray level confidence; if the first quotient is greater than or equal to the seventh gray level threshold, it can be determined that the gray level confidence corresponding to the pixel Z to be converted is the eighth gray level confidence.

[0107] It should be noted that the brightness parameter, the first gray level threshold, the second gray level threshold, the third gray level threshold, the fourth gray level threshold, the fifth gray level threshold, the sixth gray level threshold, and the seventh gray level threshold can be changed according to the actual situation. In particular, in some embodiments, the brightness parameter can be 0.001; the first gray level threshold can be 2 / 8, the second gray level threshold can be 3 / 8, the third gray level threshold can be 4 / 8, the fourth gray level threshold can be 5 / 8, the fifth gray level threshold can be 6 / 8, the sixth gray level threshold can be 7 / 8, the seventh gray level threshold can be 8 / 8; the first gray level confidence can be 8 / 8, the second gray level confidence can be 7 / 8, the third gray level confidence can be 6 / 8, the fourth gray level confidence can be 5 / 8, the fifth gray level confidence can be 4 / 8, the sixth gray level confidence can be 3 / 8, the seventh gray level confidence can be 2 / 8, and the eighth gray level confidence can be 1 / 8.

[0108] The separability weights are obtained according to the high-frequency gradient values corresponding to the pixel Z to be converted in n directions. The specific implementation manner is the same as that of obtaining the separability weights according to the gradient values of the first-priority gradient in the above embodiment, and will not be described herein again. After obtaining the gray-scale confidence and the separability weights corresponding to the pixel Z to be converted, the weighted gradient values are obtained according to the gray-scale confidence, the separability weights, the modulation parameters corresponding to the color-difference gradient, and the gradient values of the high-frequency gradient and the color-difference gradient corresponding to the same direction. Specifically, in some embodiments, in the process of accumulating the gradient values of the color-difference gradient on the basis of the gradient values of the high-frequency gradient, the weighted gradient value corresponding to a certain direction = the gradient value of the high-frequency gradient corresponding to this direction + (the separability weight × the modulation parameter corresponding to the color-difference gradient × the gray-scale confidence) × the gradient value of the color-difference gradient corresponding to this direction. Since the local gray-scale confidence is introduced in the process of accumulating the color-difference gradient, it is possible to avoid applying the color-difference gradient in the color texture, thereby further improving the resolution of each color texture direction and improving the image quality of the finally obtained target image.

[0109] After obtaining the weighted gradient values corresponding to the n directions, if the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to the n directions is greater than the preset difference, the weighted gradient value is used as the target gradient value corresponding to the direction. If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to the n directions is still less than the preset difference, the second-order gradient, the low-frequency gradient, the first gradient, and the second gradient are sequentially accumulated to update the previous weighted gradient value until the difference between the two smallest of the updated weighted gradient values is greater than the preset difference and the accumulation update stops. Then, the finally updated weighted gradient value is used as the target gradient value corresponding to the direction. Taking the accumulation of the second-order gradient to update the previous weighted gradient value as an example, the updated weighted gradient value corresponding to a certain direction = the weighted gradient value corresponding to this direction + (the separability weight × the modulation parameter) × the gradient value of the second-order gradient corresponding to this direction.

[0110] The following takes n = 8, that is, 8 directions, and the 8 directions are respectively Figure 13 the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD shown in the figure as an example to illustrate in detail how to calculate the gradient values of the high-frequency gradient, the color-difference gradient, the second-order gradient, the low-frequency gradient, the first gradient, and the second gradient corresponding to these 8 directions of the pixel Z to be converted.

[0111] For example, the high-frequency gradient is related to the pixel difference between two adjacent pixels of the same color. In some embodiments, calculating the gradient values of the high-frequency gradients corresponding to 8 directions of the pixel Z to be converted includes the following steps: preset a high-frequency calculation window F3 centered on the pixel Z to be converted; calculate the pixel difference between each pixel in the high-frequency calculation window F3 and the adjacent pixel of the same color in a different direction from it; subsequently, obtain the gradient values of the high-frequency gradients corresponding to 8 directions of the pixel Z to be converted according to multiple pixel differences.

[0112] Further, please refer to Figure 15 , in some embodiments, assuming that the high-frequency calculation window F3 is a 6*6 window, the gradient values of the high-frequency gradients in 8 directions corresponding to the pixel Z to be converted at the coordinate position (5, 5) among the 9*9 pixels shown in Figure 15 can be calculated according to the following formula:

[0113]

[0114]

[0115]

[0116]

[0117] grad_High_AU = abs(I(4, 5) - I(3, 6)) + abs(I(6, 5) - I(5, 6)) + 2 × abs(I(4, 7) - I(3, 8)) + abs(I(6, 7) - I(5, 8));

[0118] grad_High_AD = abs(I(6, 3) - I(5, 4)) + 2 × abs(I(8, 3) - I(7, 4)) + abs(I(6, 5) - I(5, 6)) + abs(I(8, 5) - I(7, 6));

[0119] grad_High_DU = abs(I(3, 5) - I(4, 6)) + 2 × abs(I(3, 3) - I(4, 4)) + abs(I(5, 3) - I(6, 4)) + abs(I(5, 5) - I(6, 6));

[0120] grad_High_DD = abs(I(5, 7) - I(6, 8)) + 2 × abs(I(5, 5) - I(6, 6)) + abs(I(7, 7) - I(8, 8)) + abs(I(7, 5) - I(8, 6));

[0121] Among them, I(i,j) represents the pixel value of the pixel arranged in the i-th row and the j-th column (as described above, the pixel value here can be the original pixel value in the first image or the weighted pixel value in the processed first image, and the same will not be repeated hereinafter), abs() represents the absolute value, grad_High_E represents the gradient value of the high-frequency gradient corresponding to the first horizontal direction E, grad_High_W represents the gradient value of the high-frequency gradient corresponding to the second horizontal direction W, grad_High_N represents the gradient value of the high-frequency gradient corresponding to the first vertical direction N, grad_High_S represents the gradient value of the high-frequency gradient corresponding to the second vertical direction S, grad_High_AU represents the gradient value of the high-frequency gradient corresponding to the first diagonal direction AU, grad_High_AD represents the gradient value of the high-frequency gradient corresponding to the second diagonal direction AD, grad_High_DU represents the gradient value of the high-frequency gradient corresponding to the third diagonal direction DU, and grad_High_DD represents the gradient value of the high-frequency gradient corresponding to the fourth diagonal direction DD.

[0122] For another example, in some embodiments, calculating the color difference gradients corresponding to 8 directions of the pixel Z to be converted includes the following steps: calculating the pixel differences between the pixel Z to be converted and the adjacent pixels with different colors in its 8 directions, and selecting multiple pixels around the pixel Z to be converted, and respectively calculating the pixel differences between each selected pixel and the adjacent pixels with different colors in its 8 directions; subsequently, obtaining the gradient values of the color differences corresponding to 8 directions of the pixel Z to be converted according to the multiple pixel differences.

[0123] Further, please refer to Figure 16 , in some embodiments, the gradient values of the color difference gradients in 8 directions corresponding to the pixel Z to be converted with the coordinate position (5,5) in the 9*9 pixels shown in Figure 16 can be calculated according to the following formula.

[0124] grad_H_color_diff_E = abs(I(4,6) - I(4,7)) + abs(I(5,6) - I(5,7)) + abs(I(6,6) - I(6,7));

[0125] grad_H_color_diff_W = abs(I(4,4) - I(4,5)) + abs(I(5,4) - I(5,5)) + abs(I(6,4) - I(6,5));

[0126] grad_V_color_diff_N = abs(I(4,4) - I(5,4)) + abs(I(4,5) - I(5,5)) + abs(I(4,6) - I(5,6));

[0127] grad_V_color_diff_S = abs(I(6, 4) - I(7, 4)) + abs(I(6, 5) - I(7, 5)) + abs(I(6, 6) - I(7, 6));

[0128] grad_D_color_diff_U = abs(I(5, 5) - I(4, 4)) + abs(I(4, 5) - I(5, 6)) + abs(I(6, 5) - I(5, 4));

[0129] grad_D_color_diff_D = abs(I(6, 6) - I(5, 5)) + abs(I(5, 6) - I(6, 7)) + abs(I(6, 5) - I(7, 6));

[0130] grad_A_color_diff_U = abs(I(5, 5) - I(4, 6)) + abs(I(5, 6) - I(4, 7)) + abs(I(5, 7) - I(6, 6));

[0131] grad_A_color_diff_D = abs(I(6, 4) - I(4, 4)) + abs(I(6, 5) - I(7, 4)) + abs(I(6, 6) - I(7, 5));

[0132] Among them, I(i, j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, abs() represents the absolute value, grad_H_color_diff_E represents the gradient value of the color difference gradient corresponding to the first horizontal direction E, grad_H_color_diff_W represents the gradient value of the color difference gradient corresponding to the second horizontal direction W, grad_V_color_diff_N represents the gradient value of the color difference gradient corresponding to the first vertical direction N, grad_V_color_diff_S represents the gradient value of the color difference gradient corresponding to the second vertical direction S, grad_A_color_diff_U represents the gradient value of the color difference gradient corresponding to the first diagonal direction AU, grad_A_color_diff_D represents the gradient value of the color difference gradient corresponding to the second diagonal direction AD, grad_D_color_diff_U represents the gradient value of the color difference gradient corresponding to the third diagonal direction DU, and grad_D_color_diff_D represents the gradient value of the color difference gradient corresponding to the fourth diagonal direction DD.

[0133] For another example, in some embodiments, calculating the gradient values of the second-order gradients corresponding to the 8 directions of the pixel Z to be converted includes: obtaining multiple groups of adjacent pixel groups in the 8 directions of the pixel Z to be converted; wherein, each group of pixel groups includes two adjacent pixels, and the line connecting or the extension of the line connecting the two pixels in each group of pixel groups can pass through the pixel Z to be converted or be parallel to a certain direction of the pixel Z to be converted. Subsequently, obtaining the first pixel difference according to the pixel values of two adjacent pixels in each group of pixels; and then obtaining the gradient values of the second-order gradients corresponding to the 8 directions of the pixel Z to be converted according to the multiple first pixel differences.

[0134] Further, please refer to Figure 17 , in some embodiments, the gradient values of the second-order gradients in 8 directions corresponding to the pixel Z to be converted at the coordinate position (5, 5) in the 9*9 pixels shown in Figure 17 can be calculated according to the following formula.

[0135]

[0136]

[0137]

[0138]

[0139] grad_High_AU_Sec_cdf = abs[abs(I(4, 5) - I(3, 6) - abs(I(2, 7) - I(1, 8))] + abs[abs(I(3, 5) - I(2, 6)) - abs(I(2, 6) - I(1, 7))] + abs[abs((I(4, 6) - I(3, 7)) - abs(I(3, 7) - I(2, 8))];

[0140] grad_High_AD_Sec_cdf

[0141] = abs[abs(I(5, 4) - I(6, 3) - abs(I(7, 2) - I(8, 1))] + abs[abs(I(5, 3) - I(6, 2)) - abs(I(6, 2) - I(7, 1))] + abs[abs((I(6, 4) - I(7, 3)) - abs(I(7, 3) - I(8, 2))]

[0142] grad_High_DU_Sec_cdf = abs[abs(I(1,1)-I(2,2)-abs(I(3,3)-I(4,4))]+abs[abs(I(2,1)-I(3,2))-abs(I(3,2)-I(4,3))]+abs[abs((I(1,2)-I(2,3))-abs(I(2,3)-I(3,4))]

[0143] grad_High_DD_Sec_cdf = abs[abs(I(5,5)-I(6,6)-abs(I(7,7)-I(8,8))]+abs[abs(I(5,6)-I(6,7))-abs(I(6,7)-I(7,8))]+abs[abs((I(6,5)-I(7,6))-abs(I(7,6)-I(8,7))]

[0144] Among them, I(i,j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, abs() represents the absolute value, grad_High_E_Sec_cdf represents the gradient value of the second-order gradient corresponding to the first horizontal direction E, grad_High_W_Sec_cdf represents the gradient value of the second-order gradient corresponding to the second horizontal direction W, grad_High_N_Sec_cdf represents the gradient value of the second-order gradient corresponding to the first vertical direction N, grad_High_S_Sec_cdf represents the gradient value of the second-order gradient corresponding to the second vertical direction S, grad_High_AU_Sec_cdf represents the gradient value of the second-order gradient corresponding to the first diagonal direction AU, grad_High_AD_Sec_cdf represents the gradient value of the second-order gradient corresponding to the second diagonal direction AD, grad_High_DU_Sec_cdf represents the gradient value of the second-order gradient corresponding to the third diagonal direction DU, and grad_High_DD_Sec_cdf represents the gradient value of the second-order gradient corresponding to the fourth diagonal direction DD.

[0145] For another example, in some embodiments, calculating the gradient values of the low-frequency gradients corresponding to 8 directions of the pixel Z to be converted includes: obtaining the gradient values of the low-frequency gradients corresponding to 8 directions of the pixel Z to be converted according to the pixel differences between multiple pixels of the same color in 8 directions of the pixel Z to be converted.

[0146] Further, please refer to Figure 18 , in some embodiments, the gradient values of the low-frequency gradients in 8 directions corresponding to the pixel Z to be converted at the coordinate position (5,5) in the 9*9 pixels shown in Figure 18 can be calculated according to the following formula.

[0147] grad_Low_E = abs(I(5,5) - I(5,9) + abs(I(6,5) - I(6,9))

[0148] grad_Low_W = abs(I(5,5) - I(5,1) + abs(I(6,5) - I(6,1))

[0149] grad_Low_N = abs(I(5,5) - I(1,5) + abs(I(5,6) - I(1,6))

[0150] grad_Low_S = abs(I(5,5) - I(9,5) + abs(I(5,6) - I(9,6))

[0151] grad_Low_AU = [abs(I(4,5) - I(1,8) + abs(I(5,5) - I(1,9)) + abs(I(4,6) - I(2,8)) + abs(I(5,6) - I(2,9)) + abs(I(5,7) - I(3,9))] / 2

[0152] grad_Low_AD = [abs(I(8,1) - I(5,4) + abs(I(9,1) - I(5,5)) + abs(I(8,2) - I(6,4)) + abs(I(9,2) - I(6,5)) + abs(I(9,3) - I(7,2))] / 2

[0153] grad_Low_DU = [abs(I(1,3) - I(4,6) + abs(I(2,3) - I(4,5)) + abs(I(1,1) - I(5,5)) + abs(I(3,2) - I(5,4)) + abs(I(3,1) - I(6,4))] / 2

[0154] grad_Low_DD = [abs(I(5,7) - I(7,9) + abs(I(6,7) - I(8,9)) + abs(I(5,5) - I(9,9)) + abs(I(7,6) - I(9,8)) + abs(I(7,5) - I(9,7))] / 2

[0155] Wherein, I(i, j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, abs() represents the absolute value, grad_Low_E represents the gradient value of the low-frequency gradient corresponding to the first horizontal direction E, grad_Low_W represents the gradient value of the low-frequency gradient corresponding to the second horizontal direction W, grad_Low_N represents the gradient value of the low-frequency gradient corresponding to the first vertical direction N, grad_Low_S represents the gradient value of the low-frequency gradient corresponding to the second vertical direction S, grad_Low_AU represents the gradient value of the low-frequency gradient corresponding to the first diagonal direction AU, grad_Low_AD represents the gradient value of the low-frequency gradient corresponding to the second diagonal direction AD, grad_Low_DU represents the gradient value of the low-frequency gradient corresponding to the third diagonal direction DU, and grad_Low_DD represents the gradient value of the low-frequency gradient corresponding to the fourth diagonal direction DD.

[0156] For another example, in some embodiments, the gradient value of the first gradient corresponding to the first horizontal direction E is the same as the gradient value of the first gradient corresponding to the second horizontal direction W; the gradient value of the first gradient corresponding to the first vertical direction N is the same as the gradient value of the first gradient corresponding to the second vertical direction S; the gradient value of the first gradient corresponding to the first diagonal direction AU is the same as the gradient value of the first gradient corresponding to the second diagonal direction AD; the gradient value of the first gradient corresponding to the third diagonal direction DU is the same as the gradient value of the first gradient corresponding to the fourth diagonal direction DD. The gradient values of the first gradients corresponding to the 8 directions of the to-be-converted pixel Z can be obtained according to the pixel values of the pixels that are adjacent to the to-be-converted pixel Z and have different colors from the to-be-converted pixel Z in the 8 directions of the to-be-converted pixel Z. In some embodiments, the gradient values of the first gradients corresponding to the n directions can be obtained according to multiple pixels that are closest to the to-be-converted pixel Z and have different colors from the to-be-converted pixel Z in the n directions of the to-be-converted pixel Z.

[0157] Further, please refer to Figure 19 , in some embodiments, the following formula can be used to calculate the gradient values of the first gradients in the 8 directions corresponding to the to-be-converted pixel Z at the coordinate position (5, 5) in the 9*9 pixels shown in Figure 19 .

[0158] grad_H_Add_E = grad_H_Add_W = abs(I(5, 4) - I(5, 7));

[0159] grad_V_Add_N = grad_V_Add_S = abs(I(4, 5) - I(7, 5));

[0160] grad_A_Add_U = grad_A_Add_D = abs(I(6, 4) - I(4, 6));

[0161] grad_D_Add_U = grad_D_Add_D = abs(I(4, 4) - I(7, 7));

[0162] Wherein, I(i, j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, abs() represents the absolute value, grad_H_Add_E represents the gradient value of the first gradient corresponding to the first horizontal direction E, grad_H_Add_W represents the gradient value of the first gradient corresponding to the second horizontal direction W, grad_V_Add_N represents the gradient value of the first gradient corresponding to the first vertical direction N, grad_V_Add_S represents the gradient value of the first gradient corresponding to the second vertical direction S, grad_A_Add_U represents the gradient value of the first gradient corresponding to the first diagonal direction AU, grad_A_Add_D represents the gradient value of the first gradient corresponding to the second diagonal direction AD, grad_D_Add_U represents the gradient value of the first gradient corresponding to the third diagonal direction DU, and grad_D_Add_D represents the gradient value of the first gradient corresponding to the fourth diagonal direction DD.

[0163] For another example, in some embodiments, the second gradient refers to the high-frequency gradient of the different-pixel difference, that is, it is related to the pixel difference between two adjacent pixels of different colors. Among them, in some embodiments, the gradient value of the second gradient corresponding to the first horizontal direction E is the same as the gradient value of the second gradient corresponding to the second horizontal direction W; the gradient value of the second gradient corresponding to the first vertical direction N is the same as the gradient value of the second gradient corresponding to the second vertical direction S; the gradient value of the second gradient corresponding to the first diagonal direction AU is the same as the gradient value of the second gradient corresponding to the second diagonal direction AD; the gradient value of the second gradient corresponding to the third diagonal direction DU is the same as the gradient value of the second gradient corresponding to the fourth diagonal direction DD.

[0164] Further, please refer to Figure 20 , in some embodiments, the gradient values of the second gradients in 8 directions corresponding to the to-be-converted pixel Z at the coordinate position (5, 5) in the 9*9 pixels shown in Figure 20 can be calculated according to the following formula:

[0165]

[0166]

[0167] grad_High_A_cdf_U = grad_High_A_cdf_D

[0168] = absm(absm(I(5,5), I(4,1), absm(I(5,5), I(6,4))) + absm(absm(I(4,5), I(3,6), absm(I(5,4), I(6,6))) + absm(absm(I(4,4), I(3,5), absm(I(4,4), I(5,3))) + absm(absm(I(5,6), I(4,7), absm(I(6,5), I(7,3))) + absm(absm(I(6,6), I(5,7), absm(I(6,6), I(7,5))))

[0169] grad_High_D_cdf_U = grad_High_D_cdf_D

[0170] = absm(absm(I(5,5), I(4,4), absm(I(6,6), I(7,7))) + absm(absm(I(5,6), I(4,6), absm(I(5,6), I(6,7))) + absm(absm(I(4,6), I(3,5), absm(I(5,7), I(6,8))) + absm(absm(I(6,5), I(5,4), absm(I(6,5), I(7,6))) + absm(absm(I(6,4), I(5,3), absm(I(7,5), I(8,6))))

[0171] Among them, I(i, j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, and absm() represents the absolute value of the subtraction of the two. For example, absm(I1, I2) represents the absolute value of the difference between I1 and I2. grad_High_H_cdf__E represents the gradient value of the second gradient corresponding to the first horizontal direction E, grad_High_H_cdf_W represents the gradient value of the second gradient corresponding to the second horizontal direction W, grad_High_V_cdf_N represents the gradient value of the second gradient corresponding to the first vertical direction N, grad_High_V_cdf_S represents the gradient value of the second gradient corresponding to the second vertical direction S, grad_High_A_cdf_U represents the gradient value of the second gradient corresponding to the first diagonal direction AU, grad_High_A_cdf_D represents the gradient value of the second gradient corresponding to the second diagonal direction AD, grad_High_D_cdf_U represents the gradient value of the second gradient corresponding to the third diagonal direction DU, and grad_High_D_cdf_D represents the gradient value of the second gradient corresponding to the fourth diagonal direction DD.

[0172] Please refer toFigure 21 , in some embodiments, obtaining color difference values corresponding to the to-be-converted pixel Z in n directions includes:

[0173] 051: Traverse each direction and perform: If there is a first-color pixel A in the current direction of the to-be-converted pixel Z, then obtain the color difference value corresponding to the current direction according to the first-color pixel A located in the current direction of the to-be-converted pixel Z and the first candidate pixel P1, where the first candidate pixel P1 has the same color as the to-be-converted pixel Z;

[0174] 052: If there is no first-color pixel A in the current direction of the to-be-converted pixel Z, then obtain the color difference value corresponding to the current direction of the to-be-converted pixel Z according to the second candidate pixel P2, two first-color pixels A adjacent to the second candidate pixel P2, and two third candidate pixels P3 adjacent to the second candidate pixel P2. The second candidate pixel P2 is the pixel closest to the to-be-converted pixel Z in the current direction of the to-be-converted pixel Z, and the second candidate pixel P2 has the same color as the third candidate pixel P3.

[0175] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 051 and 052. That is, the first processing module 20 and the processor 200 can also be used to traverse each direction and perform: If there is a first-color pixel in the current direction of the to-be-converted pixel Z, then obtain the color difference value corresponding to the current direction according to the first-color pixel A located in the current direction of the to-be-converted pixel Z and the first candidate pixel P1, where the first candidate pixel P1 has the same color as the to-be-converted pixel Z, and the first candidate pixel P1 has the same color as the to-be-converted pixel Z; and if there is no first-color pixel A in the current direction of the to-be-converted pixel Z, then obtain the color difference value corresponding to the current direction of the to-be-converted pixel Z according to the second candidate pixel P2, two first-color pixels A adjacent to the second candidate pixel P2, and two third candidate pixels P3 adjacent to the second candidate pixel P2. The second candidate pixel P2 is the pixel closest to the to-be-converted pixel Z in the current direction of the to-be-converted pixel Z, and the second candidate pixel P2 has the same color as the third candidate pixel P3.

[0176] In the process of obtaining the color difference values corresponding to the to-be-converted pixel Z in n directions, traverse each direction and perform: If there is a first-color pixel A in the current direction of the to-be-converted pixel Z, then the color difference value corresponding to the current direction can be directly obtained according to the first-color pixel A located in the current direction of the to-be-converted pixel Z and the first candidate pixel P1, where the first candidate pixel P1 has the same color as the to-be-converted pixel Z.

[0177] Specifically, in some embodiments, if there is a first color pixel A in the current direction of the pixel Z to be converted, and in the current direction of the pixel Z to be converted, the color of the pixel closest to the pixel Z to be converted is the same as the color of the pixel Z to be converted (as Figure 22a shown), then the color difference value T corresponding to the current direction can be calculated according to the formula: T = a1 + (2×p11 + p12) / 3. Wherein, T represents the color difference value corresponding to the current direction; a1 represents the pixel value of the first color pixel A closest to the pixel Z to be converted in the current direction of the pixel Z to be converted; p11 represents the pixel value of the first candidate pixel P1 closest to the pixel Z to be converted in the current direction of the pixel Z to be converted; p12 represents the pixel value of the first candidate pixel P1 that is the second closest to the pixel Z to be converted in the current direction of the pixel Z to be converted. If there is a first color pixel A in the current direction of the pixel Z to be converted, and in the current direction of the pixel Z to be converted, the color of the pixel closest to the pixel Z to be converted is not the same as the color of the pixel Z to be converted (as Figure 22b shown), then the color difference value T corresponding to the current direction can be calculated according to the formula: T = a1 + (2×z + p11) / 3. Wherein, T represents the color difference value corresponding to the current direction; a1 represents the pixel value of the first color pixel A closest to the pixel Z to be converted in the current direction of the pixel Z to be converted; p11 represents the pixel value of the first candidate pixel P1 closest to the pixel Z to be converted in the current direction of the pixel Z to be converted; z represents the pixel value of the pixel Z to be converted.

[0178] For example, please refer to Figure 22a and Figure 22b , in some embodiments, taking the example of including 8 directions, and the 8 directions are the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD, to calculate the color difference value of the pixel Z to be converted at the coordinate position (5, 5) in FIG. 22:

[0179] As Figure 22a shown, when calculating the color difference value corresponding to the first horizontal direction E of the pixel Z to be converted, since there is a first color pixel A in the first horizontal direction E of the pixel Z to be converted, and the color of the pixel closest to the pixel Z to be converted (i.e., the pixel arranged in the 5th row and 6th column) is the same as the color of the pixel Z to be converted, then the color difference value corresponding to the first horizontal direction E can be calculated according to the formula: G_E_delata = I(5, 7) - [2×I(5, 6) + I(5, 9)] / 3. Wherein, I(i, j) represents the pixel value of the pixel arranged in the i-th row and j-th column, and G_E_delata represents the color difference value corresponding to the first horizontal direction E.

[0180] As shown Figure 22b in the figure, when calculating the color difference corresponding to the second horizontal direction W of the pixel Z to be converted, since there is a first color pixel A in the second horizontal direction W of the pixel Z to be converted, and the color of the pixel closest to the pixel Z to be converted (i.e., the coordinate is at position (5, 4)) is not the same as the color of the pixel Z to be converted, the color difference corresponding to the second horizontal direction W can be calculated according to the calculation formula: G_W_delata = I(5, 4) - [2×I(5, 5) + I(5, 2)] / 3. Among them, I(i, j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, and G_W_delata represents the color difference corresponding to the second horizontal direction W.

[0181] If there is no first color pixel A in the current direction of the pixel Z to be converted, it is necessary to estimate the color difference corresponding to the current direction according to the pixel value of the non-first color pixel A closest to it in the current direction and the pixel values of the first color pixels A around it. That is, if there is no first color pixel A in the current direction of the pixel Z to be converted, according to the second candidate pixel P2, the two first color pixels A adjacent to the second candidate pixel P2, and the two third candidate pixels P3 of the second candidate pixel P2, the color difference corresponding to the current direction of the pixel Z to be converted is obtained, where the second candidate pixel P2 is the pixel closest to the pixel Z to be converted in the current direction of the pixel Z to be converted, and the color of the second candidate pixel P2 is the same as the color of the third candidate pixel P3.

[0182] Specifically, please refer to Figure 23 , in some embodiments, according to the second candidate pixel P2, the two first color pixels A closest to the second candidate pixel P2, and the two third candidate pixels P3 closest to the second candidate pixel P2, the color difference corresponding to the current direction of the pixel Z to be converted includes:

[0183] 0521: Obtain a first value according to the two first color pixels A closest to the second candidate pixel P2 and the two third candidate pixels P3 closest to the second candidate pixel P2;

[0184] 0522: Obtain a second value according to the first value and the second candidate pixel P2, and obtain a third value according to the two first color pixels A closest to the second candidate pixel P2;

[0185] 0523: Obtain a first weight according to the first value and the corresponding relationship between the first value and the first weight preset, and obtain a second weight according to the pixel value of the second candidate pixel P2 and the corresponding relationship between the pixel value of the adjacent pixel and the second weight preset;

[0186] 0524: Obtain the color difference value corresponding to the current direction of the pixel Z to be converted according to the second value, the third value, the first weight, and the second weight.

[0187] Please combine Figure 2 and Figure 3 , in some embodiments, the first processing module 20 and the processor 200 can also be used to execute the methods in 0521, 0522, 0523, and 0524. That is, both the first processing module 20 and the processor 200 are also used to obtain the first value according to the two first color pixels A that are closest to the second candidate pixel P2 and the two third candidate pixels P3 that are closest to the second candidate pixel P2; obtain the second value according to the first value and the second candidate pixel P2, and obtain the third value according to the two first color pixels A that are closest to the second candidate pixel P2; obtain the first weight according to the first ratio and the preset corresponding relationship between the first value and the first weight, and obtain the second weight according to the pixel value of the second candidate pixel P2 and the preset corresponding relationship between the pixel values of adjacent pixels and the second weight; obtain the color difference value corresponding to the current direction of the pixel Z to be converted according to the second value, the third value, the first weight, and the second weight.

[0188] If there is no first color pixel A in the current direction of the pixel Z to be converted, obtain the first value according to the two first color pixels A that are closest to the second candidate pixel P2 and the two third candidate pixels P3 that are closest to the second candidate pixel P2.

[0189] Specifically, please refer to Figure 24 , in some embodiments, the pixel closest to the pixel Z to be converted in the current direction of the pixel Z to be converted is used as the second candidate pixel P2, and the two pixels adjacent to the second candidate pixel P2 and having the same color as the second candidate pixel P2 are used as the third candidate pixels P3. After obtaining the second candidate pixel P2 and the third candidate pixels P3, calculate the first value according to the calculation formula Q1 = (ratio(a1, p31)) + ratio(a2, p32)) / 2. Where Q1 represents the first value corresponding to the current direction of the pixel Z to be converted, a1 represents the pixel value of the first pixel A1, a2 represents the pixel value of the first pixel A2, p31 represents the pixel value of the third candidate pixel P31, p32 represents the pixel value of the third candidate pixel P32, ratio() represents the ratio between the two, for example, ratio(A, B) = A / B. The first pixel A1 and the first pixel A2 are adjacent to the first candidate pixel P1, and the connection line between the first pixel A1 and the third candidate pixel P31 and the connection line between the first pixel A2 and the third candidate pixel P32 are both parallel to the current direction of the pixel Z to be converted.

[0190] After obtaining the first value, a second value is obtained based on the first value and the second candidate pixel P2, and a third value is obtained based on the two first color pixels A that are closest to the second candidate pixel P2. Specifically, in some embodiments, the second value can be calculated according to the calculation formula Q2 = (p2 × Q1) / q1, and the third value can be calculated according to the calculation formula Q3 = (a1 + a2) / 2. Wherein, Q2 represents the second value corresponding to the current direction of the pixel Z to be converted, Q3 represents the third value corresponding to the current direction of the pixel Z to be converted, p2 represents the pixel value of the second candidate pixel P2, a1 and a2 respectively represent the pixel values of the two first color pixels A adjacent to the second candidate pixel P2, and q1 represents a preset value. In some embodiments, the preset value can be 1024.

[0191] After obtaining the first value, the second value, and the third value, according to the first value and the preset correspondence between the first value and the first weight, the first weight corresponding to the current first value is obtained; and according to the pixel value of the second candidate pixel P2 and the preset correspondence between the pixel value of the neighboring pixel and the second weight, the second weight corresponding to the pixel value of the current second candidate pixel P2 is obtained. Wherein, the correspondence between the first value and the first weight and the correspondence between the pixel value of the neighboring pixel and the second weight are both preset, and it is satisfied that if the first value is larger, the corresponding first weight is smaller; if the pixel value of the second candidate pixel P2 is smaller, the corresponding second weight is larger.

[0192] After obtaining the first weight and the second weight, according to the second value, the third value, the first weight, and the second weight, the color difference value corresponding to the current direction of the pixel Z to be converted is obtained. Specifically, in some embodiments, the color difference value T corresponding to the current direction can be calculated according to the calculation formula T = [Y1 × Y2 - Q2 + (q2 - Y1 × Y2) × Q3] / q2. Wherein, T represents the color difference value corresponding to the current direction, Y1 and Y2 respectively represent the first weight and the second weight, Q2 and Q3 respectively represent the second value and the third value, and q2 represents a normalization parameter, where the value of q2 is related to the first weight and the second weight. In some embodiments, the value ranges of the first weight and the second weight are both from 0 to 8, then q2 is equal to 64.

[0193] For example, in some embodiments, there are 8 directions, and the 8 directions are respectively the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD, to calculate Figure 24 the color difference value of the pixel Z to be converted at the coordinate position (5, 5) as an example:

[0194] Such as Figure 24As shown, when calculating the color difference corresponding to the third diagonal direction DU of the pixel Z to be converted, since there is no first color pixel A in the third diagonal direction DU of the pixel Z to be converted, the color difference corresponding to the third diagonal direction DU can be calculated according to the following calculation formula:

[0195] Ratio_DU = (ratio(I(4,5), I(3,4)) + ratio(I(5,4), I(4,3))) / 2;

[0196] G_DU_ratio = (I(4,4) × Ratio_DU);

[0197] G_DU_avg = (I(4,5) + I(5,4)) / 2;

[0198] G_DU = [wei_ratio_DU × wei_val_DU × G_DU_ratio + (q2 - wei_ratio_DU × wei_val_DU) × G_DU_ratio] / q2;

[0199] Wherein, I(i,j) represents the pixel value of the pixel arranged in the i-th row and the j-th column, ratio() represents the ratio between the two, Ratio_DU, G_DU_ratio, and G_DU_avg respectively represent the first value, the second value, and the third value corresponding to the third diagonal direction DU of the pixel Z to be converted, wei_ratio_DU represents the first weight, wei_val_DU represents the second weight, and q2 represents the normalization parameter, where the value of q2 is related to the first weight and the second weight. In some embodiments, the value ranges of the first weight and the second weight are both 0 to 8, and q2 is equal to 64.

[0200] After obtaining the color differences and gradient weights corresponding to the n directions of the pixel Z to be converted respectively, the intermediate pixel value of converting the pixel Z to be converted into the first color pixel A is obtained. For example, in some embodiments, the color difference and the gradient weight corresponding to the same direction are multiplied to obtain the product corresponding to that direction. Subsequently, the products corresponding to the n directions are added respectively to obtain the intermediate pixel value of converting the pixel Z to be converted into the first color pixel A.

[0201] In particular, please refer to Figure 25 , in the case where the n directions include the first horizontal direction E, the second horizontal direction W, the first vertical direction N, the second vertical direction S, the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD, according to the color differences and gradient weights corresponding to the n directions of the pixel Z to be converted respectively, obtaining the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A includes:

[0202] 02221: Traverse the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S to perform: According to the distance difference between the first color pixel A closest to the pixel Z to be converted and the pixel Z to be converted in the current direction of the conversion pixel Z, obtain the distance weight corresponding to the current direction;

[0203] 02222: According to the distance weight, the gradient weight, and the color difference value, obtain the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A.

[0204] Please combine Figure 2 and Figure 3 In some embodiments, both the first processing module 20 and the processor 200 can also be used to execute the methods in 02221 and 02222. That is, both the first processing module 20 and the processor 200 can also be used to: Traverse the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S to perform: According to the distance difference between the first color pixel A closest to the pixel Z to be converted and the pixel Z to be converted in the current direction of the conversion pixel Z, obtain the distance weight corresponding to the current direction; and according to the distance weight, the gradient weight, and the color difference value, obtain the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A.

[0205] After obtaining the color difference values and gradient weights corresponding to the pixel Z to be converted in n directions respectively, traverse the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S to perform: According to the distance difference between the first color pixel A closest to the pixel Z to be converted and the pixel Z to be converted in the current direction of the conversion pixel Z, obtain the distance weight corresponding to the current direction.

[0206] In some embodiments, if the distance between the first color pixel A closest to the pixel Z to be converted and the pixel Z to be converted in the current direction of the conversion pixel Z is greater than one pixel, that is, the pixel closest to the pixel Z to be converted in the current direction of the conversion pixel Z is not the first color pixel A, then determine the distance weight as the first distance weight; if the distance between the first color pixel A closest to the pixel Z to be converted and the pixel Z to be converted in the current direction of the conversion pixel Z is equal to one pixel, that is, the pixel closest to the pixel Z to be converted in the current direction of the conversion pixel Z is the first color pixel A, then determine the distance weight as the second distance weight. Among them, both the first distance weight and the second distance weight are preset values and can be changed according to actual needs. In particular, in some embodiments, the first distance weight is 1 and the second distance weight is 2.

[0207] For example, take Figure 9For example, for the pixel Z to be converted arranged in the 5th row and 5th column, in its first horizontal direction E, the first color pixel A closest to it is arranged in the 5th row and 7th column, and the distance between the two is greater than one pixel. Therefore, the distance weight corresponding to the first horizontal direction E of the pixel Z to be converted is the first distance weight. In the second horizontal direction W of the pixel Z to be converted, the first color pixel A closest to it is arranged in the 5th row and 4th column, and the distance between the two is not greater than one pixel. Therefore, the distance weight corresponding to the second horizontal direction W of the pixel Z to be converted is the second distance weight.

[0208] After obtaining the distance weights corresponding to the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S respectively, according to the distance weights, gradient weights, and color difference values, obtain the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A.

[0209] Specifically, in some embodiments, obtaining the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A according to the distance weights, gradient weights, and color difference values may include the following steps:

[0210] (1) Obtain the distance weights and gradient weights corresponding to the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S respectively, multiply the distance weight and gradient weight corresponding to the same direction to obtain the product corresponding to this direction, and then add the products corresponding to the four directions to obtain the first intermediate value (wei_sum_ad). Add the gradient weights corresponding to the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S to obtain the second intermediate value (wei_sum_hv).

[0211] (2) Obtain the distance weights, gradient weights, and color difference values corresponding to the first horizontal direction E, the second horizontal direction W, the first vertical direction N, and the second vertical direction S respectively, multiply the distance weight, gradient weight, and color difference value corresponding to the same direction to obtain the product corresponding to this direction. Subsequently, divide the product corresponding to the four directions by the first intermediate value (wei_sum_ad) to obtain the third intermediate value (edge_result_hv).

[0212] (3) Obtain the gradient weights and color difference values corresponding to the first diagonal direction AU, the second diagonal direction AD, the third diagonal direction DU, and the fourth diagonal direction DD respectively, multiply the gradient weight and color difference value corresponding to the same direction to obtain the product corresponding to this direction. Then add the products corresponding to the four directions to obtain the fourth intermediate value (edge_result_ad_sum).

[0213] (4) After obtaining the first intermediate value, the second intermediate value, the third intermediate value, and the fourth intermediate value, according to the calculation formula:

[0214] The intermediate pixel value after converting the pixel Z to be converted into the first color pixel A is calculated by edge_result = (edge_result_hv × wei_sum_hv + edge_result_ad_sum) / (wei_sum_hv + wei_sum_ad). Among them, edge_result represents the intermediate pixel value after converting the pixel Z to be converted into the first color pixel A, edge_result_hv represents the third intermediate value, wei_sum_hv represents the second intermediate value, edge_result_ad_sum represents the fourth intermediate value, and wei_sum_ad represents the first intermediate value.

[0215] After obtaining the flat area confidence of the local image centered on the pixel Z to be converted and the intermediate pixel value of converting the pixel Z to be converted into the first color pixel A, the target pixel value of converting the pixel Z to be converted into the first color pixel A is obtained according to the flat area confidence and the intermediate pixel value. Specifically, in some embodiments, the target pixel value of converting the pixel Z to be converted into the first color pixel A can be calculated by the formula G_Expand = [(8 - flat_confidence) × edge_result + flat_confidence × wave] / 8. Among them, G_Expand represents the target pixel value of converting the pixel Z to be converted into the first color pixel A, flat_confidence represents the flat area confidence of the local image centered on the pixel Z to be converted, edge_result represents the intermediate pixel value of converting the pixel Z to be converted into the first color pixel A, and wave represents the average value of the pixel values of all the first color pixels A within the first calculation window F1 centered on the pixel Z to be converted. It should be noted that the 8 in the above calculation formula is not fixed and can be adjusted according to actual needs.

[0216] After converting all the pixels to be converted in the first image into the first color pixel A, the second image as shown in Figure 6 can be obtained. After obtaining the second image, the first image and the second image are processed to obtain the target image (as shown in Figure 31 ). The target image contains multiple color pixels, and the multiple color pixels are arranged in a Bayer array. Since the multiple color pixels in the target image are arranged in a Bayer array, the target image can be input into the image processor 200 for subsequent processing.

[0217] Specifically, please refer to Figure 26 , in some embodiments, processing the first image and the second image to obtain the target image, the target image contains multiple color pixels, and the multiple color pixels are arranged in a Bayer array, includes:

[0218] 031: Process the second image according to the first image to obtain a second color intermediate image and a third color intermediate image, where the second color intermediate image includes second color pixels B, and the third color intermediate image includes third color pixels C; and

[0219] 032: Fuse the second image, the second color intermediate image, and the third color intermediate image to obtain a target image.

[0220] Please combine Figure 2 and Figure 3 , in some embodiments, the second processing module 30 and the processor 200 can also be used to execute the methods in 031 and 032. That is, the second processing module 30 and the processing are also used to process the second image according to the first image to obtain a second color intermediate image and a third color intermediate image, where the second color intermediate image includes second color pixels B, and the third color intermediate image includes third color pixels C; and fuse the second image, the second color intermediate image, and the third color intermediate image to obtain a target image.

[0221] Specifically, please refer to Figure 27 , in some embodiments, processing the second image according to the first image to obtain a second color intermediate image and a third color intermediate image, where the second color intermediate image includes second color pixels B, and the third color intermediate image includes third color pixels C, includes:

[0222] 0311: Perform bilateral filtering on the first image and the second image to obtain a second color intermediate image and a third color intermediate image.

[0223] Please combine Figure 2 and Figure 3 , in some embodiments, the second processing module 30 and the processor 200 can also be used to execute the method in 0311. That is, the second processing module 30 and the processing are also used to perform bilateral filtering on the first image and the second image to obtain a second color intermediate image and a third color intermediate image.

[0224] Specifically, please refer to Figure 28 , the first image includes a plurality of second color pixels B and a plurality of third color pixels C. The plurality of second color pixels B are arranged to form a second color original image, and the plurality of third color pixels C are arranged to form a third color original image. Perform bilateral filtering on the second color original image and the second image to obtain a second color intermediate image; perform bilateral filtering on the third color original image and the second image to obtain a third color intermediate image. Among them, Figure 28 , Figure 29 and Figure 30 The N pixels in represent that there are no pixels here, and the pixel value at this place is zero.

[0225] Taking the bilateral filtering process of the second-color original image and the second image to obtain the second-color intermediate image as an example for illustration. In some embodiments, please refer to Figure 29 , the joint bilateral filtering algorithm is where k p = ∑ q∈Ω f(||p - q||)g(||I p ′ - I q ′||), J p is the output pixel value, k p is the total weight, Ω is the filtering window, p is the coordinate of the pixel to be filtered in the second-color original image, q is the coordinate of the pixel in the filtering window in the second-color original image, I q is the pixel value corresponding to point q, I p ′ is the pixel value corresponding to the pixel to be filtered in the second image, I q ′ is the pixel value corresponding to point q in the second image, and both f and g are weight distribution functions, and the weight distribution functions include Gaussian functions.

[0226] Specifically, the joint bilateral filtering algorithm determines the first filtering weight (f(||p - q||)) through the difference between the coordinate of the pixel p to be filtered and the coordinate of a pixel q in the filtering window. The coordinate difference between the exemplified p point and q point in the figure can be 2. The second filtering weight (g(||I p ′ - I q ′||)) is determined by the difference between the pixel value I p ′ corresponding to point p and the pixel value I q ′ corresponding to point q in the second image. According to the first filtering weight, the second filtering weight, the pixel value I q corresponding to point q in the second-color original image, and the total weight k p for each pixel in the filtering window, the output pixel value J p is determined.

[0227] It should be noted that in the second-color original image, there is no position where the second-color pixel is set, and its pixel value is 0. The output pixel value J p is set at the position corresponding to the pixel p to be filtered in the second-color intermediate image. After one output is completed, the filtering window moves to the next pixel position until all pixels in the second-color original image are filtered. In this way, the second-color intermediate image containing only the second-color pixels is obtained. The specific method of performing bilateral filtering on the third-color original image and the second image to obtain the third-color intermediate image is the same as the specific method of performing bilateral filtering on the second-color original image and the second image to obtain the second-color intermediate image, and will not be elaborated here.

[0228] Of course, in some embodiments, other methods may also be used to obtain the second color intermediate image and the third color intermediate image according to the first image and the second image, such as guided filtering guidance, etc., which are not limited herein. In addition, during the bilateral filtering process based on the second color original image and the second image, all positions in the second color original image that are not the second color pixel B can be interpolated to the second color pixel B (as Figure 29 shown), or only the pixels corresponding to the positions of the second color pixel B in the subsequent target image can be interpolated to the second color pixel B (as Figure 30 shown). In this way, compared with interpolating all positions to the second color pixel B, it is beneficial to improve the image processing speed.

[0229] Please refer to Figure 31 . After obtaining the second image, the second color intermediate image, and the third color intermediate image, the second image, the second color intermediate image, and the third color intermediate image are fused to obtain the target image. Exemplarily, in some embodiments, the terminal 1000 (or the image processing device 100) pre-stores the position information corresponding to the first color, the position information corresponding to the second color, and the position information corresponding to the third color. The processor 200 (or the second processing module 30) extracts the first color pixel A located at the corresponding position in the second image according to the position information corresponding to the first color; extracts the second color pixel B located at the corresponding position in the second color intermediate image according to the position information corresponding to the second color; extracts the third color pixel C located at the corresponding position in the third color intermediate image according to the position information corresponding to the third color. Then, the extracted multiple first color pixels A, multiple second color pixels B, and multiple third color pixels C are arranged to obtain the target image. Among them, the pixel value of the first color pixel A in the target image is the same as the pixel value of the first color pixel A located at the same position in the second image; the pixel value of the second color pixel B in the target image is the same as the pixel value of the second color pixel B located at the same position in the second color intermediate image; the pixel value of the third color pixel C in the target image is the same as the pixel value of the third color pixel C located at the same position in the third color intermediate image.

[0230] Please refer to Figure 32, an embodiment of the present application further provides a non - volatile computer - readable storage medium 400 containing a computer program 401. When the computer program 401 is executed by one or more processors 200, the processors 200 are caused to execute the image - processing method described in any of the above - mentioned embodiments. Exemplarily, when the computer program 401 is executed by one or more processors 200, the processors 200 are caused to execute: 01, 02, 03, 021, 022, 023, 0211, 0212, 0213, 0221, 0222, 0223, 0224, 041, 042, 043, 044, 0421, 0422, 0423, 051, 052, 0521, 0522, 0523, 0524, 02221, 02222, 031, 032, 0311 in the image - processing method.

[0231] For example, please refer to Figure 1 , when the computer program 401 is executed by one or more processors 200, the processors 200 are caused to execute the following method:

[0232] 01: Obtain a first image, where the first image includes a plurality of first - color pixels A, second - color pixels B, and third - color pixels C. The first image includes a plurality of pixel units, and the colors of the plurality of pixels in each pixel unit are the same;

[0233] 02: Take the second - color pixels B and third - color pixels C in the first image as pixels Z to be converted, and perform the following for each pixel Z to be converted: According to the flat - area confidence of the local image centered on the pixel Z to be converted, the gradient values of the pixel Z to be converted corresponding to different categories of gradients in n directions respectively, and the color - difference values of the pixel Z to be converted corresponding to n directions respectively, convert the pixel Z to be converted into a first - color pixel A to form a second image, where 2 ≤ n ≤ 8; and

[0234] 03: Process the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

[0235] In the description of this specification, the description with reference to terms such as "one embodiment", "some embodiments", "illustrative embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials, or characteristics described in connection with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic descriptions of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials, or characteristics described can be combined in a suitable manner in any one or more embodiments or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments or examples described in this specification and the features of different embodiments or examples.

[0236] Any process or method description shown in a flowchart or otherwise described herein can be understood to represent a module, segment, or portion of code including one or more executable instructions for implementing a specific logical function or process. The scope of the preferred embodiments of the present application includes additional implementations, where the functions can be executed in a substantially simultaneous manner or in the reverse order according to the functions involved, rather than in the order shown or discussed. This should be understood by those skilled in the art to which the embodiments of the present application belong.

[0237] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and should not be construed as limiting the present application. Those of ordinary skill in the art can make changes, modifications, substitutions, and variations to the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that, Including: Obtain a first image, where the first image includes a plurality of first - color pixels, second - color pixels, and third - color pixels, and the first image includes a plurality of pixel units, and the colors of the pixels in each pixel unit are the same; Use the second - color pixels and the third - color pixels in the first image as pixels to be converted, and perform the following operations on each pixel to be converted: obtain the flat - area confidence of the local image centered on the pixel to be converted; calculate the first weights corresponding to the pixel to be converted in n directions according to the gradient values of different - category gradients corresponding to the pixel to be converted in n directions; Obtain the intermediate pixel value of the pixel to be converted into the first - color pixel according to the color - difference values corresponding to the pixel to be converted in n directions and the first weights; Obtain the target pixel value of the pixel to be converted into the first - color pixel according to the flat - area confidence and the intermediate pixel value to form a second image, where 2 ≤ n ≤ 8; and Process the first image and the second image to obtain a target image, where the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

2. The image processing method according to claim 1, wherein The operation of obtaining the flat - area confidence of the local image centered on the pixel to be converted includes: Preset a first calculation window centered on the pixel to be converted; Obtain the pixel values of all first - color pixels within the first calculation window, and calculate the average value and variance value of the plurality of pixel values; and Determine the flat - area confidence according to the average value, the variance value, and a preset parameter.

3. The image processing method according to claim 1, wherein The method further includes: Obtain the gradient values of different - category gradients corresponding to the pixel to be converted in n directions and the color - difference values corresponding to the pixel to be converted in n directions according to the pixel values of the plurality of pixels in the first image; or Weight the pixel values of the second - color pixels and the third - color pixels in the first image respectively according to the gain parameters corresponding to the second color and the third color to obtain the processed first image; and obtain the gradient values of different - category gradients corresponding to the pixel to be converted in n directions and the color - difference values corresponding to the pixel to be converted in n directions according to the pixel values of the plurality of pixels in the processed first image.

4. The image processing method according to claim 1, characterized in that, Different - category gradients have different priorities; the operation of calculating the first weights corresponding to the pixel to be converted in n directions according to the gradient values of different - category gradients corresponding to the pixel to be converted in n directions includes: If the difference between the two smallest gradient values among the gradient values of the first - priority gradient corresponding to n directions is greater than a preset difference, then use the gradient values of the first - priority gradient as the target gradient values corresponding to the respective directions; If the difference between the two smallest gradient values among the gradient values of the first - priority gradient corresponding to n directions is less than the preset difference, then obtain the weighted gradient values corresponding to the pixel to be converted in n directions according to the gradient values of the first - priority gradient and the gradient values of the second - priority gradient; If the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is greater than the preset difference, then use the weighted gradient value as the target gradient value for the corresponding direction; if the difference between the two smallest weighted gradient values among the weighted gradient values corresponding to n directions is less than the preset difference, then successively accumulate the gradient values of the next-priority gradient to update the previous weighted gradient value until the difference between the two smallest updated weighted gradient values is greater than the preset difference and stop the cumulative update, and then use the last updated weighted gradient value as the target gradient value for the corresponding direction; and Obtain the first weights corresponding to the n directions according to the target gradient values corresponding to the n directions.

5. The image processing method according to claim 4, wherein In the case where the first-priority gradient is a high-frequency gradient and the second-priority gradient is a color difference gradient, obtaining the weighted gradient values corresponding to the to-be-converted pixel in each of the n directions according to the gradient value of the first-priority gradient and the gradient value of the second-priority gradient includes: Preset a second calculation window centered on the to-be-converted pixel, and obtain the gray-scale confidence of the to-be-converted pixel according to the color difference values corresponding to the to-be-converted pixel in each of the n directions and the average brightness of all the first-color pixels in the second calculation window. Obtain the separability weights according to the high-frequency gradient values corresponding to the to-be-converted pixel in each of the n directions; and Obtain the weighted gradient values according to the gray-scale confidence, the separability weights, the modulation parameters corresponding to the color difference gradient, and the gradient values of the high-frequency gradient and the color difference gradient corresponding to the same direction.

6. The image processing method according to claim 3, wherein Obtaining the color difference values corresponding to the to-be-converted pixel in each of the n directions includes: Traverse each direction and perform: if there is a first-color pixel in the current direction of the to-be-converted pixel, then obtain the color difference value corresponding to the current direction according to the first-color pixel and the first candidate pixel located in the current direction of the to-be-converted pixel, and the first candidate pixel has the same color as the to-be-converted pixel. If there is no first-color pixel in the current direction of the to-be-converted pixel, then obtain the color difference value corresponding to the current direction of the to-be-converted pixel according to the second candidate pixel, two first-color pixels adjacent to the second candidate pixel, and two third candidate pixels adjacent to the second candidate pixel, where the second candidate pixel is the pixel closest to the to-be-converted pixel in the current direction of the to-be-converted pixel, and the color of the second candidate pixel is the same as the color of the third candidate pixel.

7. The image processing method according to claim 6, characterized in that, The obtaining the color difference value corresponding to the current direction of the to-be-converted pixel according to the second candidate pixel, two first-color pixels closest to the second candidate pixel, and two third candidate pixels closest to the second candidate pixel includes: Obtain a first value according to two first-color pixels closest to the second candidate pixel and two third candidate pixels closest to the second candidate pixel. Obtain a second value according to the first value and the second candidate pixel, and obtain a third value according to two first color pixels that are closest to the second candidate pixel; Obtain a first weight according to the first value and a corresponding relationship between the first value and the first weight preset, and obtain a second weight according to the pixel value of the second candidate pixel and a corresponding relationship between the pixel value of an adjacent pixel and the second weight preset; Obtain a color difference value corresponding to the current direction of the pixel to be converted according to the second value, the third value, the first weight, and the second weight.

8. The image processing method according to claim 1, characterized in that, When the n directions include a first horizontal direction, a second horizontal direction, a first vertical direction, a second vertical direction, a first diagonal direction, a second diagonal direction, a third diagonal direction, and a fourth diagonal direction, the step of obtaining an intermediate pixel value for converting the pixel to be converted into the first color pixel according to color difference values corresponding to the pixel to be converted in the n directions respectively and the first weight includes: Traverse the first horizontal direction, the second horizontal direction, the first vertical direction, and the second vertical direction and perform: obtain a distance weight corresponding to the current direction according to a distance difference between the first color pixel that is closest to the pixel to be converted and the pixel to be converted in the current direction of the converted pixel; Obtain an intermediate pixel value after converting the pixel to be converted into the first color pixel according to the distance weight, the first weight, and the color difference value.

9. The image processing method according to claim 1, wherein The processing the first image and the second image to obtain a target image, the target image including a plurality of color pixels, the plurality of color pixels being arranged in a Bayer array, includes: Process the second image according to the first image to obtain a second color intermediate image and a third color intermediate image, the second color intermediate image including the second color pixels, and the third color intermediate image including the third color pixels; and Fuse the second image, the second color intermediate image, and the third color intermediate image to obtain a target image.

10. The image processing method according to claim 9, wherein The processing the second image according to the first image to obtain a second color intermediate image and a third color intermediate image, the second color intermediate image including the second color pixels, and the third color intermediate image including the third color pixels, includes: Perform bilateral filtering on the second image according to the first image to obtain a second color intermediate image and a third color intermediate image.

11. An image processing apparatus, characterized in that, Includes: An acquisition module, configured to acquire a first image, the first image including a plurality of first color pixels, second color pixels, and third color pixels, the first image including a plurality of pixel units, and colors of a plurality of pixels in each pixel unit being the same; The first processing module is configured to use the second color pixels and the third color pixels in the first image as pixels to be converted, and perform, for each of the pixels to be converted: obtaining the flat region confidence of the local image centered on the pixel to be converted; calculating the first weights corresponding to the pixel to be converted in n directions respectively according to the gradient values of different category gradients corresponding to the pixel to be converted in n directions; obtaining an intermediate pixel value for converting the pixel to be converted into the first color pixel according to the color difference values corresponding to the pixel to be converted in n directions respectively and the first weights; obtaining a target pixel value for converting the pixel to be converted into the first color pixel according to the flat region confidence and the intermediate pixel value to form a second image, where 2≤n≤8; The second processing module is configured to process the first image and the second image to obtain a target image, and the target image includes a plurality of color pixels, and the plurality of color pixels are arranged in a Bayer array.

12. A terminal, characterized in that, Including: one or more processors, a memory; and one or more programs, wherein the one or more programs are stored in the memory and are executed by the one or more processors, and the programs include the image processing method according to any one of claims 1 to 10.

13. A non-volatile computer-readable storage medium storing a computer program, which implements the image processing method according to any one of claims 1 to 10 when the computer program is executed by one or more processors.

Images