Image processing apparatus, imaging device, method, electronic device, and storage medium

Abstract

Embodiments of the present application provide an image processing device, an imaging device, a method, an electronic device and a storage medium, and relate to the technical field of computer vision. The device comprises: a channel difference value determination module configured to determine a channel difference value of a target pixel point of an original image; the channel difference value comprises an average color difference value between a guide channel and a to-be-guided channel; a guide filter coefficient determination module configured to determine a guide filter coefficient according to the channel difference value and a preset first correspondence relationship; the guide filter coefficient comprises a color ratio coefficient and a color difference coefficient; the first correspondence relationship comprises a correspondence relationship between the average color difference value and the color ratio coefficient; and an image recovery module configured to perform recovery processing on the original image in the to-be-guided channel according to the guide filter coefficient to obtain a target image. The embodiments of the present application reduce the ASIC hardware cost, have a wider application range, improve the image recovery effect of the area where the color difference or color ratio is not constant, and improve the image quality of the target image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular, the present application relates to an image processing device, imaging equipment, method, electronic equipment and storage medium. BACKGROUND

[0002] Each pixel point of a color image usually contains pixel values of three channels of R (Red), G (Green) and B (Blue). However, a photoelectric sensor can only record the intensity of light and cannot simultaneously collect image information of three channels. In the art, a single sensor is usually used, and a CFA (Color Filter Array) is superimposed on the top, so that the filter on the top of each light sensing position of the photoelectric sensor can transmit one of R, G and B, and filter out the other two. The commonly used CFA is a Bayer filter, so the obtained image is called a Bayer image.

[0003] Since only a single channel color value is obtained at each light sensing position, if the Bayer image is directly output, there will be obvious mosaic phenomenon. The process of restoring the Bayer image to an RGB image is called demosaicing.

[0004] Existing demosaicing methods include algorithms based on color difference or color ratio constant assumption and algorithms based on guided filtering and residual. Among them, the algorithm based on color difference or color ratio constant assumption can achieve good results in most regions of the Bayer image, but for regions where the color difference or color ratio constant assumption does not hold, false color phenomenon is easy to appear; the algorithm based on guided filtering and residual although does not limit the image region, but involves division operation and a large number of multiplication operation, the calculation complexity is high, and the ASIC (Application Specific Integrated Circuit) hardware implementation cost is high. SUMMARY

[0005] The embodiments of the present application provide an image processing device, imaging equipment, method, electronic equipment and storage medium, which can solve the problems of high calculation complexity and high ASIC hardware implementation cost of the existing Bayer image demosaicing method.

[0006] The technical solution is as follows:

[0007] According to an aspect of an embodiment of the present application, an image processing device is provided, which comprises:

[0008] a channel difference value determination module configured to determine a channel difference value of a target pixel point of the original image, wherein the channel difference value comprises an average color difference value between a guide channel and a to-be-guided channel;

[0009] a guide filter coefficient determination module configured to determine a guide filter coefficient according to the channel difference value and a preset first correspondence relationship, wherein the guide filter coefficient comprises a color ratio coefficient and a color difference coefficient, and the first correspondence relationship comprises a correspondence relationship between the average color difference value and the color ratio coefficient;

[0010] an image recovery module configured to perform a recovery process on the original image in the to-be-guided channel according to the guide filter coefficient, to obtain a target image.

[0011] According to an aspect of an embodiment of the present application, there is provided an imaging device comprising the image processing apparatus and an image acquisition apparatus, wherein the image acquisition apparatus is configured to acquire an original image.

[0012] According to another aspect of an embodiment of the present application, there is provided an image processing method, comprising:

[0013] determining a channel difference value of a target pixel point of the original image, wherein the channel difference value comprises an average color difference value between a guide channel and a to-be-guided channel;

[0014] determining a guide filter coefficient according to the channel difference value and a preset first correspondence relationship, wherein the guide filter coefficient comprises a color ratio coefficient and a color difference coefficient, and the first correspondence relationship comprises a correspondence relationship between the average color difference value and the color ratio coefficient;

[0015] performing a recovery process on the original image in the to-be-guided channel according to the guide filter coefficient, to obtain a target image.

[0016] According to another aspect of an embodiment of the present application, there is provided an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the image processing method when executing the program.

[0017] According to still another aspect of an embodiment of the present application, there is provided a computer readable storage medium having a computer program stored thereon, wherein the computer program is executable on a processor to implement the steps of the image processing method.

[0018] The technical scheme provided by the embodiments of the present application has the following beneficial effects:

[0019] By determining the channel difference value of the target pixel point of the original image, and according to the channel difference value and the preset first corresponding relationship, the guide filter coefficient is determined, which avoids the division and a large number of multiplication operations required for calculating the guide filter coefficient in the prior art, simplifies the calculation degree, greatly reduces the ASIC hardware cost, and has a wider application range. At the same time, for the area where the color difference or color ratio constant assumption does not hold, the pseudo-color phenomenon and zipper effect are reduced, the image recovery effect of the above-mentioned area can be improved, and the image quality of the target image is improved. BRIEF DESCRIPTION OF DRAWINGS

[0020] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments of the present application will be briefly introduced.

[0021] Figure 1 is a schematic diagram of a Bayer image;

[0022] Figure 2 is a flowchart of an algorithm of the prior art;

[0023] Figure 3 is a structural schematic diagram of an image processing device provided by the embodiments of the present application;

[0024] Figure 4 is a schematic diagram of the relationship between the average color difference value and the color ratio coefficient provided by the embodiments of the present application;

[0025] Figure 5 is a schematic diagram of a first neighborhood provided by the embodiments of the present application;

[0026] Figure 6 is a schematic diagram of a third neighborhood provided by the embodiments of the present application;

[0027] Figure 7 is a schematic diagram of a window region provided by the embodiments of the present application;

[0028] Figure 8 is a flowchart of an image processing method provided by the embodiments of the present application;

[0029] Figure 9 is a structural schematic diagram of an electronic device provided by the embodiments of the present application. DETAILED DESCRIPTION

[0030] The embodiments of the present application will be described below in conjunction with the drawings in the present application. It should be understood that the embodiments described below in conjunction with the drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0031] Those skilled in the art can understand that the singular forms "a," "an," and "the" used herein include plural forms unless specifically stated otherwise. It should be further understood that the terms "include" and "contain" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements and / or components, but do not exclude other features, information, data, steps, operations, elements, components and / or combinations thereof supported by the present technology. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can mean that the element and the other element are connected through an intermediate element. In addition, "connected" or "coupled" used herein can include wireless connection or wireless coupling. The term "and / or" used herein indicates that at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0032] Since each pixel in the Bayer image has only a single channel of color value, the Bayer image needs to be up-sampled and interpolated to restore the two missing color values in each channel, and then the three-channel color values are merged into a complete color RGB image. This process of generating an RGB image from a Bayer image is called demosaicing.

[0033] In the prior art, the commonly used demosaicing method is to use bilinear interpolation to estimate the missing color value by averaging the color values of the same color channel in the neighborhood. However, this method of recovering data only from its own channel has low image quality after demosaicing, and has serious pseudo-color and zipper effect. Therefore, algorithms based on color difference or color ratio constant assumption and algorithms based on guided filtering and residual are proposed.

[0034] The principles of the above two algorithms are described in detail as follows:

[0035] Algorithm based on color difference or color ratio theory assumption:

[0036] Assume that the color plane is perfectly correlated in a small enough neighborhood. In this local neighborhood, there is the following equation:

[0037] Color difference formula: G = B + k -> G - B = k

[0038] G = R + j -> G - R = j

[0039] Color ratio formula: G = m * B -> G / B = m

[0040] G = n * R -> G / R = n

[0041] Taking the center B pixel point as an example, Figure 1For Bayer image, as shown in Figure 1 Figure 1 The white square in the center represents the B pixel point, and the diagonal filled square represents the G pixel point. The G channel value of the center B pixel point can be calculated by the following method:

[0042] For color difference theory:

[0043] Horizontal direction G channel value:

[0044]

[0045]

[0046]

[0047] Vertical direction G channel value:

[0048]

[0049]

[0050]

[0051] Through color gradient or color difference gradient, the weights of two directions are obtained, denoted as W h and W v , then the final G channel value is:

[0052]

[0053] For color ratio theory:

[0054] Horizontal direction G channel value

[0055]

[0056]

[0057]

[0058] Vertical direction G channel value

[0059]

[0060]

[0061]

[0062] Similarly, through color gradient or color difference gradient, the weights of two directions are obtained, denoted as W h and W v , then the final G channel value is: ​

[0063]

[0064] In real images, the color differences k and j, and the color ratios m and n are not smooth in some areas and are sensitive to sharp color transitions, so the algorithms based on the color difference or color ratio theory assumption perform poorly.

[0065] Algorithm based on guided filtering and residual:

[0066] The residual is defined as the difference between the initial acquired color value (i.e., the true value) and the estimated value obtained by applying the guided filter, as follows: The advantage of this strategy is that if all the estimated values are very close to the ground truth, the resulting residual will be very small, so the residual field is very smooth. In other words, this method is less sensitive to sharp color transitions.

[0067] Figure 2 For the flowchart of the prior art algorithm, the R channel is taken as an example, and the B channel is similar, and the specific process is as follows:

[0068] (1) Interpolate the G channel by means of other interpolation algorithms, i.e., interpolate the color values at the R and B pixel points Use the interpolated G channel as the guide image, and define the R channel as the target image.

[0069] (2) With the guide image as the reference, perform guided filtering up-sampling on the target image to obtain the estimated value of all pixels in the target image The guided method is as follows:

[0070] Assume that the relationship between R and is a first-order function, i.e.: Then N equations at the R pixel points in the neighborhood of the center pixel point can be listed. Since the number of equations is greater than the number of unknowns, the equation set has no solution, but the optimal solution of a and b can generally be obtained by the least squares method. Let be I and R be p, and the solution method is as follows:

[0071] 1) mean I = f mean (I)

[0072] mean p = f mean (p)

[0073] corr I = f mean (I.*I)

[0074] corr Ip = f mean (I.*p)

[0075] 2) var I = corr I - mean I * mean I

[0076] cov Ip = corr Ip - mean I * mean p

[0077] 3) a = cov Ip / (var I + ∈)

[0078] b = mean p - a * mean I

[0079] 4) mean a = f mean (a)

[0080] mean b = f mean (b)

[0081] 5) q = mean a * I + mean b

[0082] Therefore, substituting and G into the equation gives the estimated value

[0083]

[0084] (3) Residual interpolation is performed on the estimated value to compensate for the estimation error, to obtain the final recovered image.

[0085] For the N R pixel points, the difference between the true value and the estimated value is calculated, i.e., the residual ΔR is obtained:

[0086]

[0087] By performing linear interpolation on the residual, the residual interpolation at the intermediate pixel point is obtained

[0088]

[0089] Finally, the R channel recovery value of the center pixel point is obtained

[0090]

[0091] In summary, the Bayer image can be up-sampled to an RGB image by the demosaicing process, but the division operation and a large number of multiplication operations are involved in the guided filter, so that the cost of ASIC hardware implementation is high.

[0092] In the algorithm based on the color difference or color ratio constant assumption, a series of theoretical assumptions need to be established, including local color ratio constant (R / G and B / G constant), local color difference constant (R-G and B-G constant), color ratio gradient, color difference gradient, etc., but these theoretical assumptions do not hold in some areas of the image, resulting in false color phenomenon.

[0093] The algorithm based on guided filter and residual is to introduce guided filter into the demosaicing method, and proposes the concept of residual interpolation (RI). The algorithm can achieve excellent image quality, and the false color and zipper effect are greatly reduced, but the guided filter used in the algorithm is relatively complex, involving division operation and a large number of multiplication operations, so that the cost of ASIC hardware implementation is high.

[0094] The image processing device, imaging equipment, method, electronic equipment and storage medium provided by the present application aim to solve the above technical problems of the prior art.

[0095] The technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application will be described below by describing several exemplary embodiments. It should be pointed out that the following embodiments can be mutually referenced, borrowed or combined, and the same terms, similar features and similar implementation steps in different embodiments will not be described repeatedly.

[0096] Figure 3 A structural schematic diagram of an image processing device provided by an embodiment of the present application is shown in FIG. 1, which comprises: Figure 3

[0097] The channel difference value determination module 101 is configured to determine a channel difference value of a target pixel point of the original image, wherein the channel difference value comprises an average color difference value between a guided channel and a channel to be guided.

[0098] Specifically, according to the guided filter formula p=a*I+b, wherein I is the guided channel, and p is the channel to be guided, in the case where the pixel value of the guided channel is known, the channel to be guided can be recovered by the guided filter formula.

[0099] ​In the embodiments of the present application, the original image can be a Bayer image to be recovered, that is, each pixel point value in the original image only includes a pixel value of a single channel; any pixel point in the original image is taken as a target pixel point, a color channel currently included by the target pixel point is defined as a guide channel, that is, the pixel value of the target pixel point on the guide channel is known; a color channel missing in the target pixel point is defined as a channel to be guided, that is, the pixel value of the target pixel point on the channel to be guided is missing. For example, the guide channel of the target pixel point R1 is the R channel, and the channels to be guided are the G channel and the B channel, so the target pixel point R1 only includes the pixel value on the R channel, and the pixel values on the G channel and the B channel are missing.

[0100] The channel difference value of the target pixel point is used to represent the difference between the guide channel and the channel to be guided, and the channel difference value can include an average color difference value between the guide channel and the channel to be guided.

[0101] In the embodiments of the present application, the original image can be a Bayer image to be recovered, that is, each pixel point value in the original image only includes a pixel value of a single channel; any pixel point in the original image is taken as a target pixel point, a color channel currently included by the target pixel point is defined as a guide channel, that is, the pixel value of the target pixel point on the guide channel is known; a color channel missing in the target pixel point is defined as a channel to be guided, that is, the pixel value of the target pixel point on the channel to be guided is missing. For example, the guide channel of the target pixel point R1 is the R channel, and the channels to be guided are the G channel and the B channel, so the target pixel point R1 only includes the pixel value on the R channel, and the pixel values on the G channel and the B channel are missing.

[0102] The guide filter coefficient determination module 102 is configured to determine a guide filter coefficient according to the channel difference value and a preset first correspondence relationship; the guide filter coefficient includes a color ratio coefficient and a color difference coefficient; the first correspondence relationship includes a correspondence relationship between the average color difference value and the color ratio coefficient.

[0103] Specifically, according to the guide filter formula p=a*I+b, to calculate the pixel value of the channel to be guided p, in addition to the pixel value of the guide channel I, the guide filter coefficients a and b also need to be known, where a is the color ratio coefficient and b is the color difference coefficient.

[0104] Based on the above guide filter formula, it can be known that:

[0105] When a=0, p=b, at this time, the I channel does not participate in the calculation, and the estimation of the p channel depends on the p channel itself instead of the relationship between the p channel and the I channel;

[0106] When a=1, p=I+b, the guide filter formula is a color difference formula, and the estimation of the p channel depends on the color difference relationship between the p channel and the I channel.

[0107] When b=0, p=a*I, the guide filter formula is a color ratio formula, and the estimation of the p channel depends on the color ratio relationship between the p channel and the I channel.

[0108] Therefore, when the correlation between the guide channel and the to-be-guided channel is large, i.e., the average color difference value between the guide channel and the to-be-guided channel is small, the color ratio coefficient a is large, so that the estimation of the to-be-guided channel depends more on the guide channel; when the correlation between the guide channel and the to-be-guided channel is small, i.e., the average color difference value between the guide channel and the to-be-guided channel is large, the color ratio coefficient a is small, so that the estimation of the to-be-guided channel depends less on the guide channel.

[0109] Based on the first correspondence relationship between the average color difference value between the guide channel and the to-be-guided channel and the color ratio coefficient, the color ratio coefficient can be determined according to the average color difference value between the guide channel and the to-be-guided channel, i.e., the channel difference value. The first correspondence relationship can be a functional relationship between the average color difference value and the color ratio coefficient, or a relationship table between the average color difference value and the color ratio coefficient, which is not limited in the embodiments of the present application. In the present application, the first correspondence relationship can be determined by a person skilled in the art based on experience and by analyzing a large amount of image data in advance.

[0110] In the prior art, the least square method is usually used to calculate the guide filter coefficient, but the method involves division operation and a large number of multiplication operations, and has high computational complexity and high ASIC hardware implementation cost. In the embodiments of the present application, the guide filter coefficient is determined according to the first correspondence relationship, which avoids the division and a large number of multiplication operations required for calculating the guide filter coefficient in the prior art, simplifies the calculation degree, greatly reduces the ASIC hardware cost, and has a wider application range. At the same time, for the area where the color difference or color ratio constant assumption does not hold, the false color phenomenon and zipper effect are reduced, the image restoration effect of the above-mentioned area is improved, and the image quality of the restored image is improved.

[0111] The image restoration module 103 is configured to perform restoration processing on the to-be-guided channel of the original image according to the guide filter coefficient, to obtain a target image.

[0112] Specifically, after obtaining the guide filter coefficient, the pixel value of the to-be-guided channel can be calculated according to the pixel value of the guide channel based on the guide filter formula. Any one of the R, G and B channels can be selected as the to-be-guided channel, and one of the other two channels can be selected as the guide channel. Further, the R, G and B channels can be sequentially selected as the to-be-guided channel, and the pixel values of the R, G and B channels are restored, and then the pixel values of the three channels are superimposed, so that the original Bayer image can be restored to an RGB image, i.e., a target image.

[0113] Optionally, since the CFA is configured with 1 red light filter, 1 blue light filter and 2 green light filters (25% R, 25% B, 50% G), the pixels with pixel values in the G channel account for 1 / 2 in the Bayer image, and the pixels with pixel values in the R or B channel each account for 1 / 4. On this basis, the G channel can be restored first, and then the other two channels R and B are restored, that is, the R or B channel can be taken as the guide channel to restore the pixel values of each pixel point in the G channel; and then the restored G channel is taken as the guide channel to obtain the pixel values of each pixel point in the R or B channel.

[0114] It should be noted that in the process of restoring the original image, the target pixel point can be taken as a unit for processing, that is, each pixel point is taken as a target pixel point in turn to calculate the pixel values of each pixel point in the three channels; or a neighborhood region of the target pixel point can be taken as a unit for processing, that is, the pixel values of the neighborhood region in any channel are restored first, and then the pixel values of the neighborhood region in other channels are restored in turn; or the entire original image can be taken as a unit for processing, that is, the pixel values of the entire original image in any channel are restored first, and then the pixel values of the entire original image in other channels are restored in turn, which is not limited in the embodiments of the present application.

[0115] In the embodiments of the present application, the channel difference value of the target pixel point of the original image is determined, and the guide filter coefficient is determined according to the channel difference value and the preset first corresponding relationship, which avoids the division and a large number of multiplication operations required for calculating the guide filter coefficient in the prior art, simplifies the calculation degree, greatly reduces the ASIC hardware cost, and has a wider application range; at the same time, for the region where the color difference or color ratio constant assumption does not hold, the false color phenomenon and zipper effect are reduced, the image restoration effect of the above-mentioned region is improved, and the image quality of the target image is improved.

[0116] As an optional embodiment, the guide filter coefficient determination module 102 comprises:

[0117] a color ratio coefficient determination sub-module, configured to determine a color ratio coefficient corresponding to the channel difference value according to the channel difference value and the preset first corresponding relationship;

[0118] a color difference coefficient determination sub-module, configured to determine a color difference coefficient according to the color ratio coefficient and a preset second corresponding relationship; wherein the second corresponding relationship comprises a corresponding relationship between the color ratio coefficient and the color difference coefficient.

[0119] Specifically, the guide filter coefficient comprises a color ratio coefficient and a color difference coefficient, and the first corresponding relationship is used to reflect the relationship between the channel difference value and the color ratio coefficient, so that the color ratio coefficient corresponding to the channel difference value can be determined according to the channel difference value and the first corresponding relationship.

[0120] The guiding filter formula p=a*I+b can be deformed as b=p-a*I. In the case that the pixel value of the guiding channel and the pixel value of the channel to be guided are known, the color difference coefficient can be determined according to the deformation of the guiding filter formula, that is, according to the color ratio coefficient. The corresponding relationship between the color ratio coefficient and the color difference coefficient is taken as the second corresponding relationship.

[0121] For example, the G channel is taken as the channel to be guided, the R or B channel is taken as the guiding channel, and the guiding channel is denoted as I. The color difference coefficient b can be calculated according to the following formula:

[0122] b=G_mean-a*I_mean Formula (1)

[0123] In the formula, G_mean is the pixel value to be guided, I_mean is the guiding pixel value, and a is the color ratio coefficient.

[0124] Optionally, the color ratio coefficient can be corrected according to the size relationship between the color ratio coefficient and the ratio of the guiding pixel value to the pixel value to be guided.

[0125] The color ratio coefficient is corrected by the following formula:

[0126] Wherein k∈[1,2] Formula (2)

[0127] In the formula, k is the correction coefficient.

[0128] As an optional embodiment, the first corresponding relationship includes: an average color difference lookup table and / or a linear function between the channel difference value and the color ratio coefficient.

[0129] The average color difference lookup table includes the color ratio coefficient corresponding to the channel difference value.

[0130] Specifically, the first corresponding relationship can be represented as an average color difference lookup table. The average color difference lookup table can include the channel difference value and the color ratio coefficient corresponding to the channel difference value. Optionally, the average color difference lookup table can be obtained by interpolating the preset channel difference value and the preset color ratio coefficient.

[0131] The first corresponding relationship can also be represented as a linear function between the channel difference value and the color ratio coefficient. The linear function between the channel difference value and the color ratio coefficient can be determined based on the maximum and minimum values of the channel difference value and the maximum and minimum values of the color ratio coefficient corresponding thereto. When the first corresponding relationship is represented as a linear function, the average color difference lookup table is not needed, thereby saving the resources for table lookup.

[0132] Optionally, since the color ratio coefficient can take a smaller value when the average color difference value is larger, and the color ratio coefficient can take a larger value when the average color difference value is smaller, on this basis, the average color difference threshold thr0 and thr1 can be set, the color ratio coefficient takes the maximum value a_max when the average color difference value is less than thr0, and the color ratio coefficient takes the minimum value a_min when the average color difference value is greater than thr1. For example, a_max is 1, and a_min is 0.

[0133] When the average color difference value is between thr0 and thr1, the color ratio coefficient a can be determined by the average color difference lookup table, or the color ratio coefficient a can be determined by the linear function between the average color difference and the color ratio coefficient, wherein the linear function is determined by linear interpolation of the average color difference threshold thr0 and thr1, and the maximum value a_max of the color ratio coefficient and the minimum value a_min of the color ratio coefficient.

[0134] Figure 4 The schematic diagram of the relationship between the average color difference value and the color ratio coefficient provided by the embodiments of the present application is shown in Figure 4 As shown, the range of the average color difference value diff_mean is thr0 to thr1, and the range of the color ratio coefficient a is a_min to a_max, the color ratio coefficient takes the maximum value a_max when the average color difference value is less than thr0, the color ratio coefficient takes the minimum value a_min when the average color difference value is greater than thr1, and the color ratio coefficient a presents a linear relationship when the average color difference value is between thr0 and thr1.

[0135] As an optional embodiment, the channel difference value determination module 101 includes:

[0136] The pixel point determination sub-module is configured to determine the first neighborhood of the target pixel point and determine the guide pixel point and the pixel point to be guided in the first neighborhood.

[0137] The channel difference value determination sub-module is configured to determine the channel difference value according to the difference between the pixel values of the guide pixel point and the pixel point to be guided.

[0138] Specifically, based on the target pixel point, the first neighborhood of the target pixel point is determined, wherein the first neighborhood can be a rectangular region with a preset length as the length and width, for example, 5x5, 7x7, 9x9, and the size of the first neighborhood is not limited in the embodiments of the present application.

[0139] The guide channel and the pixel point to be guided are determined according to the target pixel point, the color channel currently included by the target pixel point is defined as the guide channel, and the color channel missing in the target pixel point is defined as the pixel point to be guided. The first neighborhood can include the guide pixel point and the pixel point to be guided, the guide pixel point is the pixel point with the channel type of the guide channel, and the pixel point to be guided is the pixel point with the channel type of the pixel point to be guided.

[0140] Determine the guiding pixel value based on the pixel value of the guiding pixel in the first neighborhood; determine the pixel value to be guided based on the pixel value of the pixel to be guided in the first neighborhood; determine the channel difference value based on the difference between the guiding pixel value and the pixel value to be guided.

[0141] In this embodiment, the average pixel value of the guiding pixels included in the first neighborhood can be used as the guiding pixel value, and the average pixel value of the pixels to be guided in the first neighborhood can be used as the pixel value to be guided. For example, the pixel values ​​of multiple guiding pixels or pixels to be guided can be directly averaged, or a weighted average can be performed on the pixel values ​​of multiple guiding pixels or pixels to be guided. This embodiment of the application does not limit the specific approach.

[0142] Figure 5 A schematic diagram of the first neighborhood provided in the embodiments of this application is shown below. Figure 5 As an example, such as Figure 5 As shown, the target pixel is I, and the first neighborhood is a 5×5 neighborhood centered on I (I can represent the R or B channel). Then, the pixel value I_mean of the guiding channel and the pixel value G_mean of the channel to be guided can be calculated using the following formula:

[0143]

[0144]

[0145] The average color difference value, diff_mean, can then be calculated using the following formula:

[0146] diff_mean=|I_mean-G_mean| Formula (5)

[0147] As an optional embodiment, the image restoration module 103 includes:

[0148] The first recovery value determination submodule is used to determine the first recovery value of the target pixel in the channel to be guided based on the guiding filter coefficients.

[0149] The second recovery value determination submodule is used to determine the second recovery value of the target pixel on the non-target guide channel based on the first recovery value of the target pixel; wherein, the non-target guide channel is a guide channel with a different channel type than the target pixel.

[0150] The target image determination submodule is used to determine the target image based on the first and second recovery values ​​of the target pixels.

[0151] Specifically, since the pixel points in the original image only have single-channel pixel values, in order to restore the RGB image, two missing channels of each pixel point need to be restored. After the to-be-guided channel and the guide channel are determined, the pixel value of the target pixel point on the to-be-guided channel can be calculated based on the pixel value of the target pixel point on the guide channel, and the guide filter coefficient and the guide filter formula, that is, the first restoration value. At this time, the pixel values of the target pixel point on the guide channel and the to-be-guided channel are known, and on this basis, the second restoration value of the target pixel point on the non-target guide channel can be further restored according to the first restoration value of the target pixel point.

[0152] wherein the channel type of the target pixel point is taken as the target guide channel, and the non-target guide channel is a guide channel different from the type of the target guide channel. For example, the G channel is the to-be-guided channel, and the channel type of the target pixel point is the R channel, and then the non-target guide channel is the B channel.

[0153] Optionally, the non-target guide channel is taken as a new to-be-guided channel, the new guide filter coefficient is calculated based on the steps described in the above embodiments, and then the pixel value of the target pixel point on the non-target guide channel, that is, the second restoration value, is calculated according to the pixel value or the first restoration value of the target pixel point.

[0154] For example, the to-be-guided channel is the G channel, the target guide channel is the R channel, and the non-target guide channel is the B channel, and then the first restoration value of the target pixel point on the G channel can be calculated first. At this time, the pixel values of the target pixel point on the G channel and the R channel are known, and then the B channel can be taken as a new to-be-guided channel, and the G channel or the R channel is taken as a guide channel to calculate the second restoration value of the target pixel point on the B channel.

[0155] Optionally, the second restoration value of the target pixel point is calculated according to the first restoration value of the target pixel point and the channel difference value between the non-target guide channel and the to-be-guided channel.

[0156] For example, the to-be-guided channel is the G channel, the target guide channel is the R channel, and the non-target guide channel is the B channel, and then the first restoration value of the target pixel point on the G channel can be calculated first. At this time, the pixel values of the target pixel point on the G channel and the R channel are known, and then the B channel can be taken as a new to-be-guided channel, and the G channel or the R channel is taken as a guide channel to calculate the second restoration value of the target pixel point on the B channel.

[0157] It should be noted that the guide channel is relative to the to-be-guided channel, that is, the channel type different from the to-be-guided channel type is the guide channel. For example, when the G channel is the to-be-guided channel, then the R channel and the B channel are guide channels; when the R channel is the to-be-guided channel, then the G channel and the B channel are guide channels.

[0158] After obtaining the first recovery value and the second recovery value of the target pixel point, that is, obtaining the pixel values of the target pixel point in the R, G, and B channels, the above steps can be performed on each pixel point in the original image, that is, the pixel values of each pixel point in the original image in the three channels can be recovered, and the pixel values of each pixel point in the original image in the three channels are superimposed, so that the original Bayer image can be restored to an RGB image, that is, a target image.

[0159] As an optional embodiment, the first recovery value determination sub-module comprises:

[0160] The second neighborhood determination unit is configured to determine a second neighborhood of the target pixel point.

[0161] The target interpolation direction determination unit is configured to perform interpolation on the to-be-guided pixel point in the second neighborhood to obtain a target interpolation direction.

[0162] The first recovery value determination unit is configured to take a pixel value corresponding to the target interpolation direction as the first recovery value of the target pixel point.

[0163] Specifically, to further accurately calculate the pixel value of the target pixel point in the to-be-guided channel, the second neighborhood can be determined based on the target guide pixel point. The second neighborhood can be a rectangular region with a preset length as the length and width, for example, 3x3, 5x5, 7x7, and the size of the second neighborhood is not limited in the embodiments of the present application. The second neighborhood can include the guide pixel point and the to-be-guided pixel point.

[0164] The pixel value of the to-be-guided pixel point in the guide channel is obtained by performing interpolation on the guide pixel point in the second neighborhood. The estimated value of the to-be-guided pixel point in the to-be-guided channel can be calculated by substituting the pixel value of the to-be-guided pixel point in the guide channel into the guide filter formula. Since the pixel value of the to-be-guided pixel point is the pixel value of the to-be-guided channel, the residual value of the to-be-guided pixel point can be determined according to the pixel value and the estimated value of the to-be-guided pixel point in the to-be-guided channel.

[0165] According to the residual value of the to-be-guided pixel point, residual interpolation values of the target pixel point in at least two directions are calculated, and initial values of the target pixel point in the at least two directions are calculated. Then, a color difference gradient sum of the target pixel point in the at least two directions is calculated, and the target interpolation direction is determined according to the color difference gradient sum in the at least two directions, and a pixel value corresponding to the target interpolation direction is taken as the first recovery value of the target pixel point.

[0166] The following takes the G channel as the to-be-guided channel and the R or B channel as the guided channel, denoted as I, as an example for description.

[0167] On the basis of Figure 5 The second neighborhood of the target pixel point I is a 3*3 neighborhood centered on I,

[0168] (1) The pixel value of the G pixel point in the I channel in the second neighborhood is calculated by first-order interpolation:

[0169]

[0170] In the formula, I2, I5, I0 and I7 are pixel values of the pixel points I2, I5, I0 and I7 in the I channel, G3, G4, G1 and G6 are pixel values of the pixel points G3, G4, G1 and G6 in the I channel;

[0171] (2) The G pixel point is estimated according to the guided filter formula:

[0172]

[0173] In the formula, G3, G4, G1 and G6 are estimated values of the pixel points G3, G4, G1 and G6 in the G channel;

[0174] (3) The residual of the G pixel point is calculated:

[0175]

[0176] In the formula, G3, G4, G1 and G6 are pixel values of the pixel points G3, G4, G1 and G6 in the G channel, and ΔG3, ΔG4, ΔG1 and ΔG6 are residual values of G3, G4, G1 and G6 in the G channel;

[0177] (4) The residual interpolation values of the target pixel point in the horizontal direction and the vertical direction are calculated:

[0178]

[0179] In the formula, ΔG H is the residual interpolation value of the target pixel point in the horizontal direction, and ΔG Vinterpolating the residual of the target pixel in the vertical direction;

[0180] Among them, the horizontal direction and the vertical direction can be selected, or at least two directions can be selected from above, below, left, right, upper left, lower left, upper right, lower right, etc. The embodiments of the present application are not limited here.

[0181] (5) Calculate the initial value of the G channel of the target pixel in the horizontal and vertical directions:

[0182]

[0183] In the formula, The initial value of the G channel of the target pixel based on the guided filter formula The estimated value of G on the G channel calculated, G H The initial value of the G channel of the target pixel in the horizontal direction, G V The initial value of the G channel of the target pixel in the vertical direction.

[0184] According to the above steps, the initial value of the G channel in the horizontal and vertical directions of all non-G pixel points in the first neighborhood of the target pixel can be obtained, and then the target interpolation direction of the target pixel needs to be judged to obtain the first recovery value.

[0185] (1) Calculate the color difference gradient sum of the target pixel in the horizontal and vertical directions;

[0186] a. Calculate the color difference value of all non-G pixel points in the first neighborhood of the target pixel:

[0187]

[0188] In the formula, Δ H (y, x) is the color difference value of the pixel point at coordinates (y, x) and the target pixel in the I channel and the G channel in the horizontal direction, Δ V (y, x) is the color difference value of the pixel point at coordinates (y, x) and the target pixel in the I channel and the G channel in the vertical direction, I(y, x) is the pixel value of the pixel point at coordinates (y, x) in the I channel, G H (y, x) is the initial value of the G channel of the pixel point at coordinates (y, x) in the horizontal direction, G V (y, x) is the initial value of the G channel of the pixel point at coordinates (y, x) in the vertical direction;

[0189] b. Calculate the gradient value of the adjacent two color difference values in the horizontal and vertical directions:

[0190]

[0191] In the formula, D H(y, x) represents the color difference gradient between I and G of the pixel point at the (y, x) position in the horizontal direction; Δ H (y, x-1), Δ H (y, x+1) respectively represent the color difference between I channel pixel value and G channel pixel value in the horizontal direction of two pixel points at (y, x-1), (y, x+1) positions adjacent to (y, x) in the horizontal direction;

[0192] D v (y, x) represents the color difference gradient between I and G of the pixel point at the (y, x) position in the vertical direction; Δ V (y-1, x)-Δ V (y+1, x) respectively represent the color difference between I channel pixel value and G channel pixel value in the vertical direction of two pixel points at (y-1, x), (y+1, x) positions adjacent to (y, x) in the vertical direction;

[0193] c. Calculate the color difference gradient sum in the horizontal and vertical directions:

[0194]

[0195] In the formula, sum_D H is the color difference gradient sum in the horizontal direction, sum_D V is the color difference gradient sum in the vertical direction.

[0196] (2) Determine the target interpolation direction of the target pixel point, and then calculate the pixel value in the G channel of all non-G pixel points in the first neighborhood of the target pixel point;

[0197] The direction with a small color difference gradient sum is selected as the target interpolation direction of the target pixel point, and the pixel value corresponding to the target interpolation direction is taken as the pixel value in the G channel. Taking the central I pixel point as an example:

[0198]

[0199] Further, the target interpolation direction can be taken as the interpolation direction of the non-G pixel points in the first neighborhood of the target pixel point.

[0200] As an optional embodiment, the second recovery value determination sub-module comprises:

[0201] A third neighborhood determination unit is configured to determine a third neighborhood of the target pixel point.

[0202] A non-target channel difference value determination unit is configured to determine a non-target channel difference value according to a non-target guide pixel point in the third neighborhood; wherein the non-target channel difference value comprises an average color difference value between a non-target guide channel and a to-be-guided channel.

[0203] The second recovery value determination unit is used to determine the second recovery value of the target pixel based on the first recovery value of the target pixel and the difference value of the non-target channel.

[0204] Specifically, after the target pixel has recovered the pixel value on the channel to be guided, the pixel value of the guide pixel in the original image on the channel to be guided can be recovered simultaneously. At this time, in order to further recover the non-target guide channel, a third neighborhood of the target pixel can be determined based on the target pixel. The third neighborhood can be a rectangular area with a preset length and width centered on the target pixel, such as 3×3, 5×5, or 7×7. The size of the third neighborhood is not limited in this embodiment.

[0205] The difference value of the non-target channel is determined based on the non-target guiding pixels in the third neighborhood. The non-target channel difference value is the average color difference between the non-target guiding channel and the channel to be guided. The non-target channel difference value can be determined by the difference between the pixel values ​​of the non-target guiding pixels in the third neighborhood on the non-target guiding channel and the channel to be guided.

[0206] Since the difference value of the non-target channel reflects the difference between the channel to be guided and the non-target guidance channel, when the pixel value of the target pixel on the channel to be guided is known, the second recovery value of the target pixel on the non-target guidance channel can be obtained based on the first recovery value of the target pixel and the difference value of the non-target channel.

[0207] Figure 6 A schematic diagram of the third neighborhood provided in the embodiments of this application, as shown below. Figure 6 As shown, the target pixel's channel type is R channel, the channel to be guided is G channel, and the non-target channel to be guided is B channel. Given that the target pixel's G channel pixel value has been recovered, to further recover the target pixel's B channel pixel value, a 3×3 region is selected as the third neighborhood. The specific steps are as follows:

[0208] (1) Calculate the color difference value of non-target guiding pixels in the third neighborhood of the target pixel:

[0209]

[0210] In the formula, G0, G1, G2, and G3 are the restored values ​​of pixels B0, B1, B2, and B3 in the G channel, and diff0, diff1, diff2, and diff3 are the color difference values ​​of pixels B0, B1, B2, and B3 between the G channel and the B channel.

[0211] (2) Calculate the average color difference:

[0212]

[0213] (3) calculating the pixel value of the target pixel point R in the B channel:

[0214] B = G + diff avg Equation (17)

[0215] In the formula, G is the restored value of the target pixel point R in the G channel.

[0216] It should be noted that, since the proportion of the pixel points corresponding to the G channel in the Bayer image is larger, the method provided in the embodiments of the present application is applicable to the case of continuing to restore the R or B channel after the G channel is restored.

[0217] Compared with re-computing the R or B channel as a new guided channel, in the embodiments of the present application, the non-target channel difference value is determined, and the second restored value of the target pixel point is determined according to the first restored value of the target pixel point and the non-target channel difference value, thereby reducing the complexity of the computation and further reducing the ASIC hardware cost.

[0218] As an optional embodiment, the embodiments of the present application provide an imaging device, which comprises the image processing apparatus provided in the above embodiments and an image acquisition apparatus, wherein the image acquisition apparatus can be used to acquire an original image, and the image acquisition apparatus can be any type of image acquisition device, such as a CMOS image sensor, a CCD image sensor, etc., and the type of the image acquisition apparatus is not limited in the embodiments of the present application.

[0219] As an optional embodiment, the image processing method provided in the embodiments of the present application comprises the following steps:

[0220] For the input original image, the image can be scanned in a "sliding window" manner to obtain the window region; for example, the window is slid with a vertical direction step of 1 and a horizontal direction step of 2, and correspondingly, the sliding is performed in units of two pixel points. When the R and B channels are guided to restore the pixel values of the pixel points in the G channel, the target pixel point can be the pixel point that is not included in the pixel value of the G channel in the two pixel points. For example, the center two pixel points include an R pixel point (i.e., a pixel point including the pixel value of the R channel, which is referred to as an R pixel point herein) and a G pixel point (i.e., a pixel point including the pixel value of the G channel, which is referred to as a G pixel point herein), and a window region with a size of 9x10 is taken with the above two pixel points as the center, Figure 7 a schematic diagram of the window region provided in the embodiments of the present application is shown in FIG. 2, Figure 7 the rectangular region with the black thick line in the center is the sliding window region.

[0221] In addition, for the pixel points located at the image boundary, when the image is scanned by the sliding window, the window region can be beyond the image region, at this time, the mirror image method can be used to fill the region beyond the image boundary in the window region.

[0222] In each sliding window, taking the center two pixel points (R, G) as an example, the steps of processing the center two pixel points include:

[0223] (1) interpolating the G channel;

[0224] (1.1) calculating the average color difference value of the target pixel point;

[0225] The average values of all R or B channels and G channels in the region with the first neighborhood size of 5*5 are calculated with the R or B pixel point as the target pixel point. When the R pixel point is taken as the target pixel point, the R channel pixel value R_mean can be calculated based on the R channel pixel points with the coordinates of (2, 2), (2, 4), (2, 6), (4, 2), (4, 4), (4, 6), (6, 2), (6, 4), (6, 6) in the first neighborhood; when the B pixel point is taken as the target pixel point, the B channel pixel value B_mean can be calculated based on the B channel pixel points with the coordinates of (3, 3), (3, 5), (3, 7), (5, 3), (5, 5), (5, 7) in the first neighborhood.

[0226] Meanwhile, the G channel pixel value G_mean can be calculated based on the G pixel points in the first neighborhood, and then the average color difference value between the R channel and the G channel or the average color difference value between the B channel and the G channel can be calculated.

[0227] (1.2) calculating the color ratio coefficient a and the color difference coefficient b;

[0228] For each target pixel point, after the average color difference value is obtained, the color ratio coefficient a can be obtained based on the average color difference lookup table or the linear relationship between the average color difference and the color ratio coefficient, and then the color difference coefficient b can be calculated.

[0229] (1.3) calculating the estimated value and the residual value of the target pixel point in the G channel;

[0230] For each target pixel point, the second neighborhood can be set as a region with a size of 3*3, and then the estimated value and the residual value of the target pixel point in the G channel can be calculated.

[0231] (1.4) calculating the initial value of the G channel of the target pixel point in the horizontal direction and the vertical direction;

[0232] For each target pixel point, according to the calculated residual value, residual interpolation of the target pixel point in the horizontal direction and the vertical direction is calculated, and then the initial value of the G channel of the target pixel point in the horizontal direction and the vertical direction is calculated.

[0233] (1.5) calculating the color difference gradient sum of the horizontal and vertical directions of the target pixel point;

[0234] For each target pixel point, the color difference of the horizontal and vertical directions of each target pixel point is calculated, and the R pixel point is calculated and The B pixel point is calculated and Then the gradient values of the two adjacent center points in the same row and column are calculated and summed, and the R pixel point is calculated The B pixel point is calculated

[0235] (1.6) calculating the first recovery value of the target pixel point in the G channel;

[0236] For each target pixel point, the direction represented by the smaller color difference gradient sum is selected as the target interpolation direction, and the pixel value corresponding to the target interpolation direction is taken as the first recovery value of the target pixel point in the G channel.

[0237] (2) interpolating the R and B channels;

[0238] (2.1) interpolating the B channel at the R pixel point;

[0239] The R pixel point with coordinates (4, 4) is selected as the target pixel point, the average color difference value between the B channel and the G channel is calculated based on the B pixel points in the third neighborhood with a size of 3x3, and then the second recovery value of the R pixel point with coordinates (4, 4) in the B channel is calculated.

[0240] (2.2) interpolating the R and B channels at the G pixel point

[0241] The G pixel point with coordinates (4, 5) is selected as the target pixel point, the average color difference value between the B channel and the G channel, and the average color difference value between the R channel and the G channel are calculated based on the B pixel points and the R pixel points in the third neighborhood with a size of 3x3, and then the pixel value of the G pixel point with coordinates (4, 5) in the R and B channels is calculated.

[0242] It should be noted that the present application embodiment takes the center two pixel points (R, G) as an example for illustration, and the center two pixel points (B, G), (G, R), and (G, B) can be processed in the same way, which will not be described here.

[0243] The detailed description of each step in the embodiments of the present application can be referred to the corresponding description in the foregoing, which will not be repeated here.

[0244] Figure 8 A flowchart of an image processing method provided by the embodiments of the present application is shown in FIG. 2, which includes the following steps. Figure 8

[0245] In step 201, a channel difference value of a target pixel point of an original image is determined, wherein the channel difference value includes an average color difference value between a guide channel and a to-be-guided channel.

[0246] In step 202, a guide filter coefficient is determined according to the channel difference value and a preset first correspondence relationship, wherein the guide filter coefficient includes a color ratio coefficient and a color difference coefficient, and the first correspondence relationship includes a correspondence relationship between the average color difference value and the color ratio coefficient.

[0247] In step 203, a recovery processing of the to-be-guided channel is performed on the original image according to the guide filter coefficient, so as to obtain a target image.

[0248] As an optional embodiment, the determining of the guide filter coefficient according to the channel difference value and the preset first correspondence relationship includes:

[0249] determining a color ratio coefficient corresponding to the channel difference value according to the channel difference value and the preset first correspondence relationship;

[0250] determining a color difference coefficient according to the color ratio coefficient and a preset second correspondence relationship, wherein the second correspondence relationship includes a correspondence relationship between the color ratio coefficient and the color difference coefficient.

[0251] As an optional embodiment, the determining of the channel difference value of the target pixel point of the original image includes:

[0252] determining a first neighborhood of the target pixel point and determining a guide pixel point and a to-be-guided pixel point in the first neighborhood;

[0253] determining the channel difference value according to a difference between pixel values of the guide pixel point and the to-be-guided pixel point.

[0254] As an optional embodiment, the first correspondence relationship includes an average color difference lookup table and / or a linear function between the channel difference value and the color ratio coefficient.

[0255] The average color difference lookup table includes the color ratio coefficient corresponding to the channel difference value.

[0256] ​As an optional embodiment, the recovering the original image according to the guided filter coefficient to obtain the target image comprises:

[0257] determining a first recovery value of the target pixel point on the to-be-guided channel according to the guided filter coefficient;

[0258] determining a second recovery value of the target pixel point on a non-target guided channel according to the first recovery value of the target pixel point; wherein the non-target guided channel is a guided channel different from the channel type of the target pixel point;

[0259] determining the target image according to the first recovery value and the second recovery value of the target pixel point.

[0260] As an optional embodiment, the determining the first recovery value of the target pixel point on the to-be-guided channel according to the guided filter coefficient comprises:

[0261] determining a second neighborhood of the target pixel point;

[0262] interpolating the to-be-guided pixel points in the second neighborhood to obtain a target interpolation direction;

[0263] taking the pixel value corresponding to the target interpolation direction as the first recovery value of the target pixel point.

[0264] As an optional embodiment, the determining the second recovery value of the target pixel point on the non-target guided channel according to the first recovery value of the target pixel point comprises:

[0265] determining a third neighborhood of the target pixel point;

[0266] determining a non-target channel difference value according to the non-target guided pixel points in the third neighborhood; wherein the non-target channel difference value comprises an average color difference value between the non-target guided channel and the to-be-guided channel;

[0267] determining the second recovery value of the target pixel point according to the first recovery value of the target pixel point and the non-target channel difference value.

[0268] An electronic device is provided in the embodiments of the present application, including a memory, a processor and a computer program stored in the memory, the processor executes the computer program to implement the steps of the image processing method. In some embodiments, the electronic device can be a camera, a scanner, a scanner pen, a mobile phone or a tablet computer, etc. Compared with the related art, the following advantages can be achieved: by determining the channel difference value of the target pixel point of the original image, and according to the channel difference value and the preset first corresponding relationship, the guide filter coefficient is determined, which avoids the division and a large number of multiplication operations required for calculating the guide filter coefficient in the prior art, simplifies the calculation degree, greatly reduces the ASIC hardware cost, and has a wider application range; at the same time, for the area where the color difference or color ratio constant assumption does not hold, the pseudo-color phenomenon and zipper effect are reduced, the image recovery effect of the area is improved, and the image quality of the target image is improved.

[0269] In an optional embodiment, an electronic device is provided, as shown in Figure 9 Figure 9 The electronic device 4000 shown in the figure includes a processor 4001 and a memory 4003. The processor 4001 and the memory 4003 are connected, such as through a bus 4002. Optionally, the electronic device 4000 can also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as data transmission and / or data reception, etc. It should be noted that in actual application, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of the present application.

[0270] The processor 4001 can be a CPU (Central Processing Unit, central processor), a general-purpose processor, an ISP (Image Signal Processor, image signal processor), a DSP (Digital Signal Processor, data signal processor), an ASIC (Application Specific Integrated Circuit, application specific integrated circuit), an FPGA (Field Programmable Gate Array, field programmable gate array) or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It can implement or execute various exemplary logical blocks, modules and circuits described in combination with the disclosure. The processor 4001 can also be a combination of computing functions, such as one or more microprocessor combinations, combinations of DSP and microprocessor, etc.

[0271] ​The bus 4002 can include a path over which the above-described components transmit information. The bus 4002 can be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, or the like. The bus 4002 can be divided into an address bus, a data bus, a control bus, or the like. For convenience of representation, Figure 9 Only one thick line is used in the middle, but it does not mean that there is only one bus or one type of bus.

[0272] The memory 4003 can be a ROM (Read Only Memory) or other type of static storage device that can store static information and instructions, a RAM (Random Access Memory) or other type of dynamic storage device that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical disk storage, a magnetic disk storage medium, other magnetic storage device, or any other medium that can be used to carry or store computer programs and that can be accessed by a computer, without limitation.

[0273] The memory 4003 is used to store a computer program for implementing the embodiments of the present application, and is controlled by the processor 4001 to perform. The processor 4001 is used to execute the computer program stored in the memory 4003 to realize the steps shown in the foregoing method embodiments.

[0274] The embodiments of the present application provide a computer readable storage medium, and the computer readable storage medium stores a computer program. The computer program is executed by a processor to realize the steps and corresponding contents of the foregoing method embodiments.

[0275] The terms "first", "second", "third", "fourth", "1", "2", and the like (if any) in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that shown or described.

[0276] It should be understood that although each operation step in the flowchart of the embodiments of the present application is indicated by an arrow, the implementation order of the steps is not limited to the order indicated by the arrow. Unless explicitly stated herein, in some implementation scenarios of the embodiments of the present application, the implementation steps in each flowchart can be executed in other orders as required. In addition, part or all of the steps in each flowchart can include multiple sub-steps or multiple stages based on the actual implementation scenario. Part or all of these sub-steps or stages can be executed at the same time, and each of these sub-steps or stages can also be executed at different times. In the scenario where the execution times are different, the execution order of these sub-steps or stages can be flexibly configured as required, and the embodiments of the present application do not limit this.

[0277] The above is only an optional implementation of some implementation scenarios of the present application. It should be pointed out that, for ordinary skilled persons in the technical field, other similar implementation means based on the technical idea of the present application without departing from the technical concept of the present application also belong to the protection scope of the embodiments of the present application.

Claims

1. An image processing apparatus, characterized in that, include: The channel difference value determination module is used to determine the original image, and take any pixel in the original image as the target pixel. The difference between the guided channel pixel value and the unguided channel pixel value of the target pixel is taken as the channel difference value of the target pixel. The original image is a Bayer image to be restored; the guided channel of the target pixel is the color channel included in the target pixel; the unguided channel of the target pixel is the color channel missing from the target pixel; the guided channel pixel value is determined based on pixels around the target pixel with the same channel type as the guided channel of the target pixel; the unguided channel pixel value is determined based on pixels around the target pixel with the same channel type as the unguided channel of the target pixel. The guided filter coefficient determination module is used to determine the color ratio coefficient in the guided filter coefficient based on the channel difference value and a preset first correspondence relationship; substitute the color ratio coefficient, the pixel value of the guided channel, and the pixel value of the channel to be guided into the guided filter formula to obtain the color difference coefficient in the guided filter coefficient; the first correspondence relationship includes: an average color difference lookup table and / or a linear function between the channel difference value and the color ratio coefficient; the average color difference lookup table includes the color ratio coefficient corresponding to the channel difference value; The image restoration module is used to restore the channel to be guided in the original image according to the guiding filter coefficients to obtain the target image.

2. The image processing apparatus according to claim 1, characterized in that, The channel difference value determination module includes: The pixel determination submodule is used to determine the first neighborhood of the target pixel and to determine the guiding pixel and the pixel to be guided within the first neighborhood. The channel difference value determination submodule is used to determine the channel difference value based on the difference between the pixel values ​​of the guiding pixel and the pixel to be guided.

3. The image processing apparatus according to claim 1 or 2, characterized in that, The image restoration module includes: The first recovery value determination submodule is used to determine the first recovery value of the target pixel on the channel to be guided based on the guiding filter coefficients. The second recovery value determination submodule is used to determine the second recovery value of the target pixel on the non-target guide channel based on the first recovery value of the target pixel; wherein the non-target guide channel is a guide channel with a different channel type than the target pixel. The target image determination submodule is used to determine the target image based on the first recovery value and the second recovery value of the target pixel.

4. The image processing apparatus according to claim 3, characterized in that, The first recovery value determination submodule includes: The second neighborhood determination unit is used to determine the second neighborhood of the target pixel. The target interpolation direction determination unit is used to interpolate the pixel points to be guided in the second neighborhood to obtain the target interpolation direction; The first recovery value determination unit is used to take the pixel value corresponding to the target interpolation direction as the first recovery value of the target pixel.

5. The image processing apparatus according to claim 3, characterized in that, The second recovery value determination submodule includes: The third neighborhood determination unit is used to determine the third neighborhood of the target pixel. The non-target channel difference value determination unit is used to determine the non-target channel difference value based on the non-target guiding pixels in the third neighborhood; wherein, the non-target channel difference value includes the average color difference value between the non-target guiding channel and the channel to be guided; The second recovery value determination unit is used to determine the second recovery value of the target pixel based on the first recovery value of the target pixel and the difference value of the non-target channel.

6. An imaging device, characterized in that, The image processing apparatus and image acquisition apparatus include any one of claims 1 to 5, wherein the image acquisition apparatus is used to acquire an original image.

7. An image processing method, characterized in that, include: A raw image is determined, and any pixel in the raw image is taken as the target pixel. The difference between the guided channel pixel value and the unguided channel pixel value of the target pixel is taken as the channel difference value of the target pixel. The raw image is a Bayer image to be restored. The guided channel of the target pixel is the color channel included in the target pixel, and the unguided channel of the target pixel is the color channel missing from the target pixel. The guided channel pixel value is determined based on pixels surrounding the target pixel with the same channel type as the guided channel of the target pixel. The unguided channel pixel value is determined based on pixels surrounding the target pixel with the same channel type as the unguided channel of the target pixel. Based on the channel difference value and a preset first correspondence, the color ratio coefficient in the guiding filter coefficient is determined; the color ratio coefficient, the guiding channel pixel value, and the pixel value of the channel to be guided are substituted into the guiding filter formula to obtain the color difference coefficient in the guiding filter coefficient; the first correspondence includes: an average color difference lookup table and / or a linear function between the channel difference value and the color ratio coefficient; the average color difference lookup table includes the color ratio coefficient corresponding to the channel difference value; Based on the guiding filter coefficients, the original image is processed to restore the channel to be guided, thereby obtaining the target image.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the steps of the method of claim 7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method of claim 7.

Images