Image processing method and device for removing mosaics, and readable storage medium

Abstract

An image processing method for removing mosaics, a device thereof, and a readable storage medium, the image processing method comprising: obtaining an original image; performing white balance on the original image to obtain a white balance image; calculating the brightness component of the white balance image; detecting a dense texture area in the white balance image; using the dense texture area in the white balance image to replace the corresponding area in the brightness component to obtain a replaced brightness component. Since the dense texture area in the white balance image has a higher resolution and is clearer, the dense texture area in the obtained brightness component is clearer, has less pseudo-color, and has a higher image quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to an image processing method for removing mosaics, a device thereof, and a readable storage medium. Background Art

[0002] Figure 1 A schematic diagram of a top view structure of a Bayer type pixel array is shown. The pixel array is an RGGB pixel array, where R represents a pixel that only transmits red light, G represents a pixel that only transmits green light, and B represents a pixel that only transmits blue light.

[0003] use Figure 1 The image captured by the image sensor with the color filter array is an RGGB mosaic image, which has only one pixel value (R, G or B) at each pixel position. The mosaic image needs to be converted into an image with three pixel values ​​(R, G and B) at each pixel position, i.e., a three-channel RGB image, by a de-mosaicing image processing method before it can be displayed. Since only light of corresponding colors can be collected through the color filter, the image sensor with the color filter array receives fewer photons per pixel, and when shooting low-light images, the signal-to-noise ratio of the obtained image is low.

[0004] An existing method to improve the poor signal-to-noise ratio problem of images at low illumination is to introduce a transparent color filter (W) into the color filter array to form an RGBW pixel array. In an image sensor with an RGBW pixel array, pixels with an R color filter, a G color filter, a B color filter, and a W color filter are respectively called R pixels, G pixels, B pixels, and W pixels. Since W pixels can receive almost all R, G, and B wavelengths of light, the number of photons entering the pixel can be increased, thereby improving the signal-to-noise ratio of images at low illumination.

[0005] like Figure 2 As shown, the Bayer type RGBW pixel array replaces one G pixel in the RGGB pixel array with a W pixel, forming a Bayer type RGBW pixel array in which R pixels, G pixels, B pixels, and W pixels each account for 25%.

[0006] After an image sensor with an RGBW pixel array captures an image, it has only one pixel value (R, G, B or W) at each pixel position. A demosaicing image processing method is needed to convert the mosaic image into an image with four pixel values ​​(R, G, B and W) at each pixel position, that is, a complete RGBW image.

[0007] However, the complete RGBW image obtained by the existing mosaic removal image processing method is prone to low resolution and blur, and is prone to more pseudo colors, resulting in low image quality. Summary of the invention

[0008] The problem solved by the present invention is how to improve the resolution of a complete RGBW image and reduce pseudo-color.

[0009] In order to solve the above problems, the present invention provides an image processing method for removing mosaics, comprising:

[0010] Obtain an original image; perform white balancing on the original image to obtain a white balanced image; calculate the brightness component of the white balanced image; detect dense texture areas in the white balanced image; and use the dense texture areas in the white balanced image to replace the corresponding areas in the brightness component to obtain the replaced brightness component.

[0011] Optionally, the step of detecting the dense texture area in the white balance image includes: extracting dense texture features of the white balance image to obtain a texture distribution image; and obtaining the dense texture area of ​​the white balance image according to the texture distribution image.

[0012] Optionally, the step of extracting dense texture features of the white balance image includes: performing spatial domain convolution on the white balance image using a preset detection convolution kernel to extract the dense texture features of the white balance image.

[0013] Optionally, the step of detecting dense texture areas in the white balance image further includes: before extracting dense texture features of the white balance image to obtain a texture distribution image, designing the detection convolution kernel.

[0014] Optionally, the step of designing the detection convolution kernel includes: designing an initial convolution kernel according to a preset spatial frequency; and adjusting the weights of different pixel positions of the initial convolution kernel to obtain the detection convolution kernel.

[0015] Optionally, the step of adjusting the weights of different pixel positions of the initial convolution kernel includes: giving higher weights to pixel positions close to the center of the initial convolution kernel; and giving lower weights to pixel positions far from the center of the initial convolution kernel.

[0016] Optionally, in the step of adjusting the weights of different pixel positions of the initial convolution kernel, the weights of different pixel positions of the initial convolution kernel are adjusted by multiplying the initial convolution kernel and the adjusted Gaussian function pixel by pixel.

[0017] Optionally, in the step of multiplying each pixel position of the initial convolution kernel by an adjusted Gaussian function, the standard deviation of the Gaussian function in the adjusted Gaussian function is set so that the ratio of the numerical value of the pixel position at the edge of the detection convolution kernel to the numerical value of the pixel position at the center is 10% to 30%.

[0018] Optionally, in the step of extracting dense texture features of the white balance image using a preset detection convolution kernel, the size of the detection convolution kernel is greater than or equal to 3.

[0019] Optionally, the step of extracting dense texture features of the white balance image to obtain a texture distribution image includes: extracting horizontal texture features of the white balance image to obtain a horizontal texture distribution image, the horizontal texture distribution image includes a horizontal dense texture area and a mosaic part; extracting vertical texture features of the white balance image to obtain a vertical texture distribution image, the vertical texture distribution image includes a vertical dense texture area and a mosaic part; the step of obtaining the dense texture area of ​​the white balance image includes: performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image to obtain the dense texture area of ​​the white balance image, the dense texture area includes: a horizontal dense texture area and a vertical dense texture area.

[0020] Optionally, the step of obtaining the dense texture area of ​​the white balance image also includes: after performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image, performing an open operation on the result of the XOR operation; and obtaining the dense texture area of ​​the white balance image based on the result after the open operation.

[0021] Optionally, the step of obtaining the dense texture area of ​​the white balance image also includes: before performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image, marking the horizontal dense texture area and mosaic part in the horizontal texture distribution image and the vertical dense texture area and mosaic part in the vertical texture distribution image.

[0022] Optionally, in the step of marking the horizontal dense texture areas and mosaic parts in the horizontal texture distribution image and the vertical dense texture areas and mosaic parts in the vertical texture distribution image, morphological operations are performed on the horizontal texture distribution image and the vertical texture distribution image, respectively, to mark the horizontal dense texture areas and mosaic parts in the horizontal texture distribution image and the vertical dense texture areas and mosaic parts in the vertical texture distribution image.

[0023] Optionally, the step of performing morphological operations on the horizontal texture distribution image and the vertical texture distribution image respectively includes: binarizing the horizontal texture distribution image and the vertical texture distribution image respectively to mark the horizontal dense texture areas and mosaic parts in the horizontal texture distribution image and the vertical dense texture areas and mosaic parts in the vertical texture distribution image.

[0024] Optionally, the step of binarizing the horizontal texture distribution image and the vertical texture distribution image respectively includes: determining a binarization threshold for binarizing the horizontal texture distribution image according to a histogram of the horizontal texture distribution image; and determining a binarization threshold for binarizing the vertical texture distribution image according to a histogram of the vertical texture distribution image.

[0025] Optionally, the step of performing morphological operations on the horizontal texture distribution image and the vertical texture distribution image respectively further includes: before binarizing the horizontal texture distribution image and the vertical texture distribution image respectively, bridging gaps in the horizontal texture distribution image and the vertical texture distribution image respectively.

[0026] Optionally, in the step of respectively bridging the gaps in the horizontal texture distribution image and the vertical texture distribution image, the gaps are bridged by performing morphological grayscale closing operations on the horizontal texture distribution image and the vertical texture distribution image respectively.

[0027] Optionally, the step of detecting dense texture areas in the white balance image also includes: constructing a replacement mask based on the dense texture areas of the white balance image; and in the step of replacing the corresponding area in the brightness component, using the replacement mask to replace the corresponding area in the brightness component.

[0028] Optionally, the step of calculating the brightness component of the white balance image includes: performing spatial domain convolution on the white balance image to calculate the brightness component of the white balance image.

[0029] Optionally, the method further includes: obtaining all complete color channels according to the replaced brightness components in combination with the white balance image; and synthesizing the color image after mosaic removal according to all the complete color channels.

[0030] Correspondingly, the present invention also provides an image processing device, comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, and when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0031] In addition, the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0032] Compared with the prior art, the technical solution of the present invention has the following advantages:

[0033] In the technical solution of the present invention, the original image is white balanced to obtain a white balanced image; the brightness component of the white balanced image is calculated; and the dense texture area in the white balanced image is used to replace the corresponding area in the brightness component to obtain the replaced brightness component. Since the dense texture area in the white balanced image has a higher resolution and is clearer, the dense texture area in the replaced brightness component has a higher resolution and is clearer, and the image quality is higher.

[0034] In an optional technical solution of the present invention, before the horizontal texture distribution image and the vertical texture distribution image are binarized respectively, the gaps in the horizontal texture distribution image and the vertical texture distribution image are bridged respectively. The bridging of the gaps in the horizontal texture distribution image and the vertical texture distribution image can effectively improve the marking accuracy of dense texture areas and mosaic parts.

[0035] In an optional technical solution of the present invention, all complete color channels are obtained based on the replaced brightness component and the white balance image; and the color image after mosaic removal is synthesized based on all the complete color channels. Since the replaced brightness component has a higher resolution and is clearer in the dense texture area, the complete color channel obtained based on the replaced brightness component has a higher resolution and is clearer, and has less pseudo-color, so that a higher quality image can be synthesized. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the structure of a Bayer-type RGGB pixel array;

[0037] Figure 2 It is a schematic diagram of the structure of a Bayer-type RGBW pixel array;

[0038] Figure 3 It is a flow chart of an embodiment of an image processing method for removing mosaics according to the present invention;

[0039] Figure 4 yes Figure 3 A white-balanced image obtained by performing white balancing on the original image in the image processing method for removing mosaics;

[0040] Figure 5 yes Figure 3 A schematic flow chart of the step of detecting a dense texture area in the white balance image in the embodiment of the image processing method for removing mosaics;

[0041] Figure 6 yes Figure 5 A schematic flow chart of the step of extracting dense texture features of the white balance image to obtain a texture distribution image in the embodiment of the image processing method for removing mosaics;

[0042] Figure 7 yes Figure 5 A schematic flow chart of the steps of designing the detection convolution kernel in the embodiment of the image processing method for removing mosaics;

[0043] Figure 8 yes Figure 5 A schematic flow chart of the step of obtaining the dense texture area of ​​the white balance image in the embodiment of the image processing method for removing mosaics;

[0044] Fig. 9 yes Figure 8 A schematic flow chart of the steps of performing morphological operations on the horizontal texture distribution image and the vertical texture distribution image respectively in the embodiment of the image processing method for removing mosaics;

[0045] Fig.10 yes Fig. 9 The histogram of the horizontal texture distribution image after the gap is bridged in the embodiment of the image processing method shown,

[0046] Fig.11 yes Fig. 9 A histogram of a vertical texture distribution image after the gap is bridged in the embodiment of the image processing method shown;

[0047] Fig.12 yes Fig. 9 The binary horizontal image obtained in the embodiment of the image processing method shown;

[0048] Fig.13 yes Fig. 9 The binarized vertical image obtained in the embodiment of the image processing method shown;

[0049] Fig.14 yes Figure 8 The result of performing an XOR operation on the binary horizontal image and the binary vertical image in the embodiment of the image processing method shown;

[0050] Fig.15 Yes Fig.14 The result of the XOR operation shown is the result of the open operation;

[0051] Fig.16 yes Figure 3 A schematic diagram of the R position, B position, G position and W position in the step of obtaining all complete color channels in combination with the white balance image according to the replaced brightness component in the embodiment of the image processing method shown;

[0052] Fig.17 yes Fig.16 A schematic diagram of a slowly varying R sampling image in an embodiment of the image processing method shown;

[0053] Fig.18Yes Yes Fig.16 The schematic diagram of interpolating a slowly varying R sampling image by using a mean filtering method in an embodiment of the image processing method shown is. DETAILED DESCRIPTION

[0054] As can be seen from the background technology, the existing image processing methods for removing mosaics have the problems of low image resolution and many false colors. Now, combined with an image processing method for removing mosaics, the reasons for the low image resolution and many false colors are analyzed:

[0055] After adding W pixels, the proportion of each pixel in the image captured by the image sensor of the RGBW pixel array is only 25%, such as Figure 2 Therefore, in the process of converting the mosaic image into a complete RGBW image, due to the low sampling rate, the obtained complete RGBW image is likely to have a low resolution, especially in areas with dense textures, the image becomes blurred.

[0056] On the other hand, the interpolation accuracy of existing image processing methods is low, and more pseudo colors are prone to appear, especially for black and white gray areas in the image, such as text on book pages, documents on printed paper, targets in snow scenes, clothing with black and white gray textures, etc., pseudo colors will be more eye-catching, causing image quality to deteriorate. One way to improve the pseudo color problem is to add a low-pass filter in front of the image sensor. However, the addition of a low-pass filter, on the one hand, increases the cost of the imaging device, and on the other hand, further worsens the problem of image blur.

[0057] In order to solve the technical problem, the present invention provides an image processing method for removing mosaics, comprising:

[0058] Obtain an original image; perform white balancing on the original image to obtain a white balanced image; calculate the brightness component of the white balanced image; detect dense texture areas in the white balanced image; and use the dense texture areas in the white balanced image to replace the corresponding areas in the brightness component to obtain the replaced brightness component.

[0059] In the technical solution of the present invention, the original image is white balanced to obtain a white balanced image; the brightness component of the white balanced image is calculated; and the dense texture area in the white balanced image is used to replace the corresponding area in the brightness component to obtain the replaced brightness component. Since the dense texture area of ​​black, white and gray in the white balanced image has a higher resolution and is clearer, the dense texture area in the replaced brightness component has a higher resolution and is clearer, and the image quality is higher.

[0060] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0061] refer to Figure 3 , shows a flow chart of an embodiment of an image processing method for removing mosaics according to the present invention.

[0062] The image processing method comprises: executing step S110 to obtain an original image; executing step S120 to perform white balance on the original image to obtain a white balanced image f WB Execute step S130 to calculate the white balance image f WB Execute step S140 to detect the white balance image f WB Execute step S150, using the white balance image f WB The dense texture area in the image is replaced by replacing the corresponding area in the brightness component to obtain the replaced brightness component.

[0063] Since the white balanced image f WB The dense texture area in the image has a higher resolution and is clearer, so in the replaced brightness component, the dense texture area has a higher resolution and is clearer, and the image quality is higher.

[0064] First, execute step S110 to obtain the original image, denoted as f cfa , that is, the original image f cfa .

[0065] In some embodiments of the present invention, the original image f cfa This is achieved by using a Bayer-type RGBW pixel array (such as Figure 2 The Bayer-type RGBW pixel array is composed of color filters that only transmit red light (R), green light (G), blue light (B), and transparent light (W). Therefore, the original image f cfa is an RGBW mosaic image. The original image f cfa In the graph, there is only one pixel value (R pixel value, G pixel value, B pixel value or W pixel value) at each pixel position.

[0066] Next, step S120 is performed to process the original image f cfa Perform white balancing to obtain a white balanced image f WB .

[0067] Specifically, the original image f is processed in the Bayer domain cfa Perform white balance. The Bayer domain refers to the mosaic image that has not been demosaiced, that is, the original mosaic image captured by the image sensor.

[0068] For the original image f cfa Perform white balance to obtain the white balance image f after white balance WBThe white balance refers to making the white balance image f WB In the white balance image, the average values ​​of the R pixels, G pixels, B pixels, and W pixels in the black, white, and gray regions are equal, so the white balance image f WB The black, white and gray areas no longer have mosaic effects, but are clear grayscale images without mosaics; the color areas still have mosaics. cfa In the step of performing white balance, the original image f may be calibrated using existing techniques. cfa Perform white balance to obtain the white balance image f WB . Specifically, Figure 4 As shown, the white balance image f WB In the image, there is no mosaic in the black, white, and gray areas, but there is still mosaic in the color area. Therefore, the black, white, and gray areas are the white balance image f WB The non-mosaic part and the colored area are the white balance image f WB The mosaic part, that is, the white balance image f WB It is divided into two parts: mosaic part and non-mosaic part.

[0069] Get the white balance image f WB Then, step S130 is performed to calculate the white balance image f WB The brightness component luma.

[0070] In some embodiments of the present invention, step S130 is performed to calculate the white balance image f WB The step of obtaining the brightness component luma comprises: WB Perform spatial domain convolution to calculate the white balance image f WB Specifically, the existing technology can be used to process the white balance image f WB Perform spatial domain convolution to calculate the white balance image f WB For example, the convolution template used for the spatial domain convolution described in Chinese invention patent CN201910840627.0 can be used to perform spatial domain convolution to obtain the white balance image f WB Specifically, the brightness component luma is a grayscale image and no longer has mosaics.

[0071] Continue to refer Figure 3 Then, step S140 is performed to detect the white balance image f WB Densely textured areas in the

[0072] Combined with reference Figure 5 , showing Figure 3In the embodiment of the image processing method for removing mosaics, the white balance image f is detected. WB Schematic diagram of the steps in creating dense texture areas in .

[0073] In some embodiments of the present invention, step S140 is performed to detect the white balance image f WB The steps of extracting the dense texture area in the white balance image f WB The dense texture features are used to obtain a texture distribution image; then, step S142 is performed to obtain the white balance image f according to the texture distribution image. WB densely textured areas.

[0074] Specifically, textures are directional, so the white balance image f WB The texture-dense areas in the non-mosaic part are directional, while the mosaics in the mosaic part are non-directional. Figure 6 , execute step S141 to extract the white balance image f WB The step of obtaining a texture distribution image by using dense texture features comprises: executing step S141a, extracting the white balance image f WB The horizontal texture features are used to obtain a horizontal texture distribution image, wherein the horizontal texture distribution image includes a horizontal dense texture area and a mosaic part; executing step S141b, extracting the white balance image f WB vertical texture features to obtain a vertical texture distribution image, wherein the vertical texture distribution image includes a vertical dense texture area and a mosaic part;

[0075] In some embodiments of the present invention, the white balance image f is extracted by spatial domain convolution. WB That is, step S141 is executed to extract the white balance image f WB The step of obtaining dense texture features comprises: detecting the white balance image f by a preset detection convolution kernel WB Perform spatial domain convolution to extract the white balance image f WB dense texture features.

[0076] Specifically, step S141a is performed to extract the white balance image f WB In the step of obtaining a horizontal texture distribution image by using a horizontal texture feature of the image, the white balance image f is calibrated by a preset horizontal detection convolution kernel. WB Perform spatial domain convolution to extract the white balance image f WB Execute step S141b to extract the white balance image f WBIn the step of obtaining a vertical texture distribution image by using a vertical texture feature of the white balance image f WB Perform spatial domain convolution to extract the white balance image f WB Vertical dense texture features

[0077] Therefore, if Figure 5 As shown, in some embodiments of the present invention, step S140 is performed to detect the white balance image f WB The step of extracting the dense texture area in the white balance image f WB Before obtaining the texture distribution image by using dense texture features, step S143 is performed to design the detection convolution kernel.

[0078] Specifically, executing step S143, the step of designing the detection convolution kernel includes: designing the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy. The horizontal detection convolution kernel hGx is used to extract the white balance image f WB The vertical detection convolution kernel hGy is used to extract the white balance image f WB Dense texture features with alternating light and dark in the vertical direction.

[0079] It can be seen from the spatial arrangement of Bayer-type RGBW pixels that the spatial repetition period of the pixels in the horizontal and vertical directions is also 2 pixels / period, and the corresponding spatial frequency is 0.5 cycle / pixel, so the highest spatial frequency of dense texture that can be recorded by the Bayer-type RGBW image sensor is 0.5 cycle / pixel. Since the higher the spatial frequency of the dense texture area, the more serious the resolution of the corresponding area of ​​its brightness component decreases, in order to solve this problem, in some embodiments of the present invention, the dense texture feature refers to a texture feature with a corresponding spatial frequency of 0.5 cycle / pixel and its nearby frequencies, and its texture width and texture interval are 1 pixel width or slightly larger than 1 pixel width, showing the characteristics of alternating light and dark, and its spatial repetition period is 2 pixels / period, or slightly lower than 2 pixels / period.

[0080] In some embodiments of the present invention, step S141 is performed to extract the white balance image f using a preset detection convolution kernel. WBIn the step of detecting dense texture features, the size of the detection convolution kernel is an odd number greater than or equal to 3, and the specific size can be determined by experiments. The smaller the detection convolution kernel, the smaller the amount of calculation, but the larger the frequency broadening, which is prone to misjudgment, for example, it is easy to misjudge a single isolated line as a texture-dense area; the larger the detection convolution kernel, the narrower the frequency broadening, the higher the frequency detection accuracy, and the less likely it is to make a misjudgment, but the amount of calculation is greater. In some embodiments of the present invention, the sizes of the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy are both equal to 7, 9 or 11.

[0081] Specifically, Figure 7 As shown, executing step S143, the step of designing the detection convolution kernel includes: first executing step S143a, designing an initial convolution kernel according to a preset spatial frequency; then executing step S143b, adjusting the weights of different pixel positions of the initial convolution kernel to obtain the detection convolution kernel.

[0082] Since the vertical detection convolution kernel hGy and the horizontal detection convolution kernel hGx are both used to extract dense texture features of alternating light and dark, that is, to extract periodically changing components, in some embodiments of the present invention, step S143a is executed, and in the step of designing an initial convolution kernel according to a preset spatial frequency, the initial convolution kernel is implemented by a cosine function based on the preset spatial frequency, so that the detection convolution kernel has the function of extracting the preset spatial frequency component.

[0083] Specifically, based on the preset spatial frequency, the step of implementing the initial convolution kernel through a cosine function includes: based on the preset spatial frequency, implementing the horizontal initial convolution kernel hGx0 and the vertical initial convolution kernel hGy0 through a cosine function.

[0084] Among them, the value of each pixel position in the horizontal initial convolution kernel hGx0 can be expressed as: hGx0(n) = cos(2·n·π·f), f represents the preset spatial frequency, and in this embodiment, f=0.5 cycle / pixel is set, and n represents the pixel position in the horizontal initial convolution kernel hGx0. In some embodiments of the present invention, the size of the horizontal detection convolution kernel hGx is equal to 9, that is, the size of the horizontal initial convolution kernel hGx0 is equal to 9, then n=-4, -3, ..., 0, ..., 3, 4, where n=0 represents the pixel position of the center of the horizontal initial convolution kernel hGx0. Since a convolution kernel with a finite size will produce a certain frequency broadening, in some embodiments of the present invention, the convolution kernel size is set to 9, which can ensure that the texture features of the preset spatial frequency f=0.5 cycle / pixel and its nearby frequencies are detected.

[0085] Therefore, the horizontal initial convolution kernel hGx0 is:

[0086] hGx0=[1 -1 1 -1 1 -1 1 -1 1]

[0087] Continue to refer Figure 7 After designing the initial convolution kernel, execute step S143b to adjust the weights of different pixel positions of the initial convolution kernel to obtain the detection convolution kernel.

[0088] In some embodiments of the present invention, step S143b is performed to adjust the weights of different pixel positions of the initial convolution kernel to obtain the detection convolution kernel: a higher weight is given to the pixel position close to the center of the initial convolution kernel; a lower weight is given to the pixel position far from the center of the initial convolution kernel. Increasing the weight of the pixel position close to the center and reducing the weight of the pixel position far from the center can reduce the ringing effect to improve the accuracy of the extracted dense texture features.

[0089] Specifically, execute step S143b, in the step of adjusting the weights of different pixel positions of the initial convolution kernel, the weights of different pixel positions of the initial convolution kernel are adjusted by multiplying the initial convolution kernel and the adjusted Gaussian function Gau pixel by pixel, that is, the vertical initial convolution kernel hGy0 is multiplied by the adjusted Gaussian function Gau pixel by pixel to adjust the weights of different pixel positions of the vertical initial convolution kernel, so as to obtain the vertical detection convolution kernel hGy; the horizontal initial convolution kernel hGx0 is multiplied by the adjusted Gaussian function Gau pixel by pixel to adjust the weights of different pixel positions of the horizontal initial convolution kernel hGx0, so as to obtain the horizontal detection convolution kernel hGx.

[0090] It should be noted that, in the step of multiplying each pixel position of the initial convolution kernel by the adjustment Gaussian function Gau, the setting of the Gaussian function standard deviation σ in the adjustment Gaussian function Gau makes the ratio of the numerical value of the pixel position at the edge of the detection convolution kernel to the numerical value of the pixel position at the center be 10% to 30%. If the ratio is too small, the numerical value of the edge position of the detection convolution kernel has almost no effect, resulting in a waste of computing resources; if the ratio is too large, the adjustment Gaussian function Gau is too gentle, and loses its beneficial effect of reducing the ringing effect.

[0091] In some embodiments of the present invention, the standard deviation of the Gaussian function is σ=2.2. The discrete adjusted Gaussian function Gau is expressed as:

[0092] Gau=[0.035 0.072 0.12 0.164 0.181 0.164 0.12 0.072 0.035]

[0093] Therefore, the horizontal initial convolution kernel hGx0 is multiplied pixel by pixel by the adjusted Gaussian function Gau to obtain the horizontal detection convolution kernel hGx; the horizontal detection convolution kernel hGx is transposed to obtain the vertical detection convolution kernel hGy, hGy = hGx T .

[0094] Specifically, the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy are respectively:

[0095] hGx=[0.035 -0.072 0.12 -0.164 0.181 -0.164 0.12 -0.072 0.035]

[0096]

[0097] Extract the white balance image f WB Specifically, the detection convolution kernel is used to detect the white balance image f WB Perform spatial domain convolution and then take the absolute value or square to extract dense texture features, that is, use the horizontal detection convolution kernel hGx and the vertical detection convolution kernel hGy to respectively detect the white balance image f WB Perform spatial domain convolution and then take the absolute value or square to extract the white balance image f WB The horizontal texture features in the white balance image f WB Vertical texture features in .

[0098] Among them, the horizontal detection convolution kernel hGx is used to detect the white balance image f WB Perform spatial domain convolution and then take the absolute value or square to extract the white balance image f WB The horizontal texture features in the image are used to obtain the horizontal texture distribution image fx; the vertical detection convolution kernel hGy is used to detect the white balance image f WB Perform spatial domain convolution and then take the absolute value or square to extract the white balance image f WB The vertical texture features in the image are used to obtain the vertical texture distribution image fy.

[0099] Table 1 shows the areas included in the horizontal texture distribution image fx and the vertical texture distribution image fy.

[0100]

[0101] As shown in Table 1, it can be seen that the horizontal texture distribution image fx contains horizontal dense texture areas (i.e., dense texture areas with alternating light and dark in the horizontal direction) and mosaic areas, but does not contain vertical dense texture areas; the vertical texture distribution image fy contains vertical dense texture areas (i.e., dense texture areas with alternating light and dark in the vertical direction) and mosaic areas, but does not contain horizontal dense texture areas.

[0102] like Figure 8 As shown, in some embodiments of the present invention, step S142 is performed to obtain the white balance image f WB The step of identifying the dense texture area includes: executing step S142a, performing an XOR operation on the horizontal texture distribution image fx and the vertical texture distribution image fy to obtain the white balance image f WB The dense texture area includes: a horizontal dense texture area and a vertical dense texture area.

[0103] Specifically, step S142 is performed to obtain the white balance image f WB The step of identifying dense texture areas of the horizontal texture distribution image fx and the vertical texture distribution image fy further includes: executing step S142a, before performing an XOR operation on the horizontal texture distribution image fx and the vertical texture distribution image fy, executing step S142b to mark the horizontal dense texture area and mosaic part in the horizontal texture distribution image fx and the vertical dense texture area and mosaic part in the vertical texture distribution image fy. Before performing the XOR operation, marking different areas and parts can effectively ensure the accuracy of the XOR operation.

[0104] In which, different areas and parts in the horizontal texture distribution image fx and the vertical texture distribution image fy are marked by morphological operations, that is, in the step of executing step S142b, in which the horizontal dense texture areas and mosaic parts in the horizontal texture distribution image fx and the vertical dense texture areas and mosaic parts in the vertical texture distribution image fy are marked, morphological operations are performed on the horizontal texture distribution image fx and the vertical texture distribution image fy, respectively, to mark the horizontal dense texture areas and mosaic parts in the horizontal texture distribution image fx and the vertical dense texture areas and mosaic parts in the vertical texture distribution image fy.

[0105] like Fig. 9As shown, the step of performing morphological operations on the horizontal texture distribution image fx and the vertical texture distribution image fy includes: executing step S142bi, binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy to indicate the horizontal dense texture area and mosaic part in the horizontal texture distribution image fx and the vertical dense texture area and mosaic part in the vertical texture distribution image fy. The horizontal texture distribution image fx and the vertical texture distribution image fy are binarized to obtain a binarized horizontal image and a binarized vertical image.

[0106] Specifically, a binarization threshold is set, that is, a binarization threshold thr_x for binarizing the horizontal texture distribution image fx and a binarization threshold thr_y for binarizing the vertical texture distribution image fy; the horizontal texture distribution image fx and the vertical texture distribution image fy are binarized, that is, the pixel values ​​less than or equal to the binarization threshold thr_x in the horizontal texture distribution image fx are set to 0, and the pixel values ​​greater than the binarization threshold thr_x are set to 1; the pixel values ​​less than or equal to the binarization threshold thr_y in the vertical texture distribution image fy are set to 0, and the pixel values ​​greater than the binarization threshold thr_y are set to 1.

[0107] It should be noted that in order to improve the accuracy of morphological operations and ensure the accuracy of marking different areas and parts, in some embodiments of the present invention, step S142b is executed, and the step of performing morphological operations on the horizontal texture distribution image fx and the vertical texture distribution image fy respectively also includes: before binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy respectively, step S142bii is executed to fill the gaps in the horizontal texture distribution image fx and the vertical texture distribution image fy respectively.

[0108] Specifically, executing step S142bii, in the step of respectively closing the gaps in the horizontal texture distribution image fx and the vertical texture distribution image fy, the gaps are closed by performing morphological grayscale closing operations on the horizontal texture distribution image fx and the vertical texture distribution image fy respectively.

[0109] In some embodiments of the present invention, a 3×3 structure element b is used to perform a morphological grayscale closing operation on the horizontal texture distribution image fx and the vertical texture distribution image fy to fill the gap:

[0110] fx close =fx·b

[0111] fy close =fy·b

[0112] Among them, · represents the morphological grayscale closing operation; fx close Represents the horizontal texture distribution image after bridging the gap; fy close This represents the vertical texture distribution image after the gap is closed.

[0113] Therefore, in the step S142bi of binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy, the horizontal texture distribution image fx after the gap is bridged is binarized. close And the vertical texture distribution image fy after bridging the gap close Binarization is performed separately to obtain a binary horizontal image gx and a binary vertical image gy.

[0114] Specifically, the horizontal texture distribution image fx after the gap is bridged close The pixel values ​​less than or equal to the binarization threshold thr_x are set to 0, and the pixel values ​​greater than the binarization threshold thr_x are set to 1; the vertical texture distribution image fy after the gap is bridged close The pixel values ​​less than or equal to the binarization threshold thr_y are set to 0, and the pixel values ​​greater than the binarization threshold thr_y are set to 1, that is:

[0115]

[0116]

[0117] Where gx represents the binary horizontal image, gy represents the binary vertical image, and (x, y) is any pixel coordinate.

[0118] In some embodiments of the present invention, the binarization threshold is set by a histogram, that is, step S142bi is executed, and the steps of binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy respectively include: determining the binarization threshold thr_x for binarizing the horizontal texture distribution image according to the histogram of the horizontal texture distribution image fx; and determining the binarization threshold thr_y for binarizing the vertical texture distribution image fy according to the histogram of the vertical texture distribution image fy.

[0119] It should be noted that, in some embodiments of the present invention, before binarization, the gaps in the image are bridged respectively. Therefore, in the step of binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy respectively, the horizontal texture distribution image fx after the gap is bridged is binarized. close And the vertical texture distribution image fy after bridging the gap closeThe binarization is performed respectively to obtain a binarized horizontal image gx and a binarized vertical image gy; therefore, in the step of setting the binarization threshold by the histogram, the binarization threshold is set by the histogram of the image after the gap is bridged, that is, in the step of binarizing the horizontal texture distribution image fx and the vertical texture distribution image fy respectively, the horizontal texture distribution image fx after the gap is bridged is used to obtain the binarized horizontal image gx and the binarized vertical image gy. close The histogram determines the horizontal texture distribution image fx after bridging the gap close The binarization threshold thr_x for binarization; the vertical texture distribution image fy after bridging the gap close The histogram of the vertical texture distribution image fy after the gap is bridged is determined close The binarization threshold thr_y for binarization.

[0120] Specifically, refer to Fig.10 and Fig.11 ,in Fig.10 for Fig. 9 The horizontal texture distribution image fx after the gap is bridged in the embodiment of the image processing method shown in FIG. close The histogram of Fig.11 for Fig. 9 The vertical texture distribution image fy after the gap is bridged in the embodiment of the image processing method shown in FIG. close Histogram of .

[0121] Specifically, in the histogram, the horizontal axis represents the normalized pixel value; the vertical axis represents the number of pixels corresponding to the pixel value. Fig.10 As shown, the highest peak on the left represents the black and dark gray background. Excluding the highest peak on the left, setting the binarization threshold thr_x=0.025 is a more appropriate choice; Fig.11 As shown, the highest peak on the left represents the black and dark gray background. Excluding the highest peak on the left and setting the binarization threshold thr_y=0.02 is a more appropriate choice.

[0122] It should be noted that the method of determining the binarization threshold value through the histogram is only an example. In other embodiments of the present invention, the empirical parameter may also be determined through experiments, that is, the binarization threshold value may be determined through experiments.

[0123] Combined with reference Fig.12 and Fig.13 ,in Fig.12 express Fig. 9 The binary horizontal image obtained in the embodiment of the image processing method shown; Fig.13 express Fig. 9 The binarized vertical image obtained in the embodiment of the image processing method shown.

[0124] like Fig.12As shown, in the binarized horizontal image gx after binarization, white is the value 1, indicating that the white balance image f WB The horizontal dense texture area and the mosaic part, and the other black areas are value 0; in the binarized vertical image gy after binarization, the white is value 1, indicating that the white balance image f WB The vertical dense texture area and mosaic part in the middle, and the other black areas are value 0.

[0125] Continue to refer Figure 8 After marking different areas and parts, step S142a is executed to perform an XOR operation on the horizontal texture distribution image fx and the vertical texture distribution image fy.

[0126] It should be noted that, as mentioned above, the horizontal texture distribution image fx and the vertical texture distribution image fy are only marked to ensure the accuracy of the XOR operation. Therefore, in the step of performing an XOR operation on the horizontal texture distribution image fx and the vertical texture distribution image fy, an XOR operation is performed on the marked horizontal texture distribution image fx and the vertical texture distribution image fy.

[0127] In addition, in some embodiments of the present invention, the horizontal texture distribution image fx and the vertical texture distribution image fy are binarized to obtain the binarized horizontal image gx and the binarized vertical image gy, so as to achieve marking. Therefore, the step of performing an XOR operation on the marked horizontal texture distribution image fx and the vertical texture distribution image fy includes: performing an XOR operation on the binarized horizontal image gx and the binarized vertical image gy.

[0128] Referring to Table 2, the regions represented by the binarized horizontal image gx and the binarized vertical image gy are shown:

[0129]

[0130] From the above table, we can see that by performing an XOR operation on the binary horizontal image gx and the binary vertical image gy, we can obtain a binary image that only contains horizontal dense texture areas and vertical dense texture areas and does not contain mosaic areas. Let it be mask, then:

[0131] mask = xor(gx,gy)

[0132] Among them, xor(gx,gy) represents the XOR operation of the binary horizontal image gx and the binary vertical image gy, and mask represents the result of the XOR operation of the binary horizontal image gx and the binary vertical image gy. The result mask is a binary image with the same size as the binary horizontal image gx and the binary vertical image gy. Each pixel value of the result mask is equal to the XOR operation of the pixel value of the same coordinate of the binary horizontal image gx and the binary vertical image gy.

[0133] It should be noted that, in some embodiments of the present invention, step S142 is performed to obtain the white balance image f WB The step of selecting the dense texture area further includes: executing step S142a, performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image, and then performing an open operation on the result of the XOR operation; and obtaining the white balance image f according to the result of the open operation. WB densely textured areas.

[0134] like Fig.14 and Fig.15 As shown, Fig.14 The result mask of the XOR operation of the binary horizontal image gx and the binary vertical image gy is shown; Fig.15 The result of opening the result mask is shown. Fig.14 and Fig.15 It can be seen that performing an opening operation on the result of the XOR operation can effectively eliminate small dots to improve the accuracy of the dense texture area obtained. Specifically, the white area in the mask of the result of the opening operation is the white balance image f WB densely textured areas.

[0135] Continue to refer Figure 5 In order to implement the subsequent replacement operation, in some embodiments of the present invention, step S140 is performed to detect the white balance image f WB The step of: performing step S144, according to the white balance image f WB The dense texture area of ​​​​the ,construction of the replacement mask.

[0136] Specifically, the white area in the result mask after the open operation is the white balance image f WB Therefore, according to the result mask of the open operation, the replacement mask MASK is constructed, that is, the white area in the replacement mask MASK is also the white balance image f WB densely textured areas.

[0137] Continue to refer Figure 3 , obtain the white balance image f WBAfter the dense texture area is obtained, step S150 is performed, using the white balance image f WB The dense texture area in the luminance component luma is replaced to obtain the replaced luminance component luma. Y .

[0138] Specifically, executing step S150, the step of replacing the area corresponding to the brightness component luma includes: replacing the area corresponding to the brightness component luma with a replacement mask MASK. The white area (i.e., the pixel with a value of 1) in the replacement mask MASK is replaced with the white balance image f WB To replace the brightness component luma, the formula is:

[0139]

[0140] Among them luma Y is the replaced brightness component, (x, y) is any pixel coordinate. In the dense texture area, the replaced brightness component luma Y It has higher resolution and is clearer than the luminance component luma before replacement.

[0141] In the white balance image f WB The texture-dense area of ​​the non-mosaic part, the white-balanced image f WB Than the white balance image f WB The brightness component luma has a higher resolution, so we convert the white balance image f WB This part of the luminance component luma is replaced by the white balance image f WB while maintaining the white balance image f WB The rest of the luminance component luma remains unchanged, so that the replaced white balance image f WB The brightness component luma Y In the black, white, and gray areas, the white balance image f WB The luma texture of the brightness component is clearer and more accurate, and the demosaiced RGB image has less pseudo color.

[0142] Continue to refer Figure 3 , to obtain the replaced brightness component luma with higher resolution in the texture-dense area Y Afterwards, the image processing method further includes: executing step S160, according to the replaced brightness component luma Y , combined with the white balance image f WB , obtain all complete color channels; execute step S170, and synthesize the color image after removing the mosaic according to all the complete color channels.

[0143] In some embodiments of the present invention, the replaced luminance component luma is used. Y The differences with other color channels are interpolated along the spatially slowly varying characteristics to obtain a channel image of all complete color channels.

[0144] Specifically, the original image f cfa is acquired by an image sensor using a Bayer-type RGBW pixel array, so the original image f cfa and the white balance image f WB They all include R channel, G channel, B channel and W channel, 4 channels. Therefore, the replaced brightness component luma Y Channel and white balance image f WB The difference between the R channel, G channel, B channel and W channel varies slowly along the space, and each channel is interpolated to obtain a complete channel image of the R channel, a complete channel image of the G channel, a complete channel image of the B channel and a complete channel image of the W channel. The obtained R channel, G channel, B channel and W channel are respectively expressed as:

[0145]

[0146] Among them, R Y , G Y , B Y , W Y They represent the complete R channel image, the complete G channel image, the complete B channel image, and the complete W channel image respectively; the following takes the R channel as an example for explanation.

[0147] Specifically, sampleR(img) means only retaining the value of the image img at position R, and the other positions are all pixel positions to be interpolated. The result of sampleR(img) is an image of the same size as the image img, that is, the result of sampleR(img) is an image with the same number of rows and columns as the image img.

[0148] Among them, Fig.16 As shown, the R position refers to the white balance image f WB Similarly, the G position refers to the same position as the R pixel in the white balance image f WB The same position as the G pixel in the white balance image f WB The W position refers to the same position as the B pixel in the white balance image f WB The W pixels in the same position.

[0149] In addition, (f WB -lumaY) represents the white balance image fWB And the replaced brightness component luma Y The channel images of the channels are subtracted pixel by pixel, (f WB -luma Y ) is a slowly varying image, and the slowly varying image is the same as the white balance image f WB The images of the same size, namely the slowly varying image and the white balance image f WB Have the same number of rows and columns.

[0150] like Fig.17 As shown, sampleR(f WB -luma Y ) represents a slowly varying R sampling image, wherein the slowly varying R sampling image sampleR(f WB -luma Y ) means only keep the slowly varying image (f WB -luma Y ) at position R, and the other positions are the pixel positions to be interpolated (such as Fig.17 The shaded area is shown in the figure). Fig.17 R in the figure represents the white balance image f WB The R pixel in the image is replaced by the luminance component luma. Y The pixel in the channel image of the channel.

[0151] In addition, interpolar[] means to interpolate the slowly varying sampled image in brackets [] to obtain a complete slowly varying channel image, that is, interpolar[sampleR(f WB -luma Y )] represents the slowly varying R sampling image sampleR(f WB -luma Y ) is interpolated, that is, a slowly varying R channel image pixel value mR is set at each pixel position to obtain a complete slowly varying R channel image.

[0152] In some embodiments of the present invention, interpolar[sampleR(f WB -luma Y )], at least one interpolation method selected from bilinear interpolation, bicubic interpolation, mean filtering, and median filtering may be used to interpolate to obtain a complete slowly varying channel image. In other embodiments of the present invention, other interpolation methods may also be used to interpolate.

[0153] In this embodiment, the interpolar [sampleR (f WB -luma Y)] steps to respectively obtain a complete slowly varying R channel image, a complete slowly varying G channel image, a complete slowly varying B channel image and a complete slowly varying W channel image.

[0154] Specifically, Fig.18 As shown, the filter window size of the mean filter is 7×7 pixels (such as Fig.18 As shown in the black box in the figure), then interpolar[sampleR(f WB -luma Y )] will slowly change R sampling image sampleR(f WB -luma Y ) The average pixel value of the pixels at position R within the filter window is set to the interpolated value mR at the center of the filter window:

[0155]

[0156] Among them, K R represents the slowly varying R sampled image sampleR(f WB -luma Y ) The total number of pixels at the R position contained in the current filtering window, specifically, KR = 12; (R i -Li) represents the slowly varying R sampling image sampleR(f WB -luma Y ) is the i-th (RL) pixel in the current filter window.

[0157] Make the filter window traverse the slowly varying R sampling image sampleR(f WB -luma Y ) at each pixel position, in the slowly varying R sampling image sampleR(f WB -luma Y The average pixel value of each pixel position of ) is set to the interpolated value mR of the center position of the filtering window, that is, the interpolar [sampleR (f CFA，Y -luma Y )]; then interpolar[sampleR(f CFA，Y -luma Y )] and the replaced brightness component luma Y The channel images of the channel are added pixel by pixel to obtain the complete channel image R of the R channel. Y .

[0158] A similar method can be used to obtain the complete G channel channel image G Y , the complete B channel channel image B Yand the full W channel image W Y .

[0159] After obtaining the channel images of all complete color channels, the channel images of all color channels are merged to obtain the color image after mosaic removal, that is, the channel image R of the complete R channel is merged. Y , the channel image G of the complete G channel Y , the complete B channel channel image B Y and the full W channel image W Y , you can get a color image RGBW with complete RGBW channels y .

[0160] Since the replaced luminance component luma Y The channel image of the channel has a higher resolution and is clearer in the dense black, white, and gray line areas than the channel image of the brightness component luma before replacement; Y The full RGBW channel color image RGBW is obtained based on the channel image of the channel Y , with higher resolution, clearer, and less pseudo-color. Among them, the color image RGBW of the complete RGBW channel y It can be displayed on a display with RGBW pixels, or the RGB channels can be displayed on a display with only RGB pixels.

[0161] Correspondingly, the present invention also provides an image processing device, specifically comprising: a memory and a processor, wherein the memory stores a computer program that can be run on the processor, wherein when the processor executes the program, the processor executes the steps of the image processing method of the present invention.

[0162] Specifically, when the processor executes the program, the processor executes the steps of the image processing method of the present invention. Therefore, the specific technical solution of the processor refers to the embodiment of the aforementioned image processing method, and the present invention will not be repeated here.

[0163] In addition, the present invention also provides a computer-readable storage medium, which is a non-volatile storage medium or a non-transient storage medium, on which computer instructions are stored, and when the computer instructions are executed, the steps of the image processing method of the present invention are executed.

[0164] Specifically, the computer instructions stored on the readable storage medium execute the steps of the image processing method of the present invention when they are executed. Therefore, the specific technical solution of the readable storage medium refers to the embodiment of the aforementioned image processing method, and the present invention will not be repeated here.

[0165] In summary, the original image is white balanced to obtain a white balanced image; the brightness component of the white balanced image is calculated; and the dense texture area in the white balanced image is used to replace the corresponding area in the brightness component to obtain the replaced brightness component. Since the dense texture area in the white balanced image has a higher resolution and is clearer, the dense texture area in the obtained brightness component has a higher resolution and is clearer, and the image quality is higher.

[0166] Moreover, before binarizing the horizontal texture distribution image and the vertical texture distribution image, the gaps in the horizontal texture distribution image and the vertical texture distribution image are respectively bridged. Bridging the gaps in the horizontal texture distribution image and the vertical texture distribution image can effectively improve the marking accuracy of dense texture areas and mosaic parts.

[0167] In addition, all complete color channels are obtained based on the replaced brightness component and the white balance image; and the color image after mosaic removal is synthesized based on all the complete color channels. Since the replaced brightness component has a higher resolution and is clearer, the complete color channel obtained based on the replaced brightness component has a higher resolution, is clearer, and is more accurate, thereby being able to synthesize a higher quality image.

[0168] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the scope defined by the claims.

Claims

1. A method for removing mosaics from an image, characterized in that: include: Obtaining an original image, wherein the original image is a mosaic image; Performing white balancing on the original image to obtain a white balanced image; Calculating a brightness component of the white-balanced image; Detecting a dense texture area in the white balance image, the dense texture area comprising: a horizontal dense texture area and a vertical dense texture area, the step of detecting the dense texture area in the white balance image further comprising: constructing a replacement mask according to the dense texture area of ​​the white balance image; Using the dense texture area in the white balance image, the corresponding area in the brightness component is replaced to obtain the replaced brightness component. In the step of replacing the corresponding area in the brightness component, the brightness component is replaced by the white balance image according to the replacement mask. The formula is: ， Among them, luma Y is the brightness component after replacement, luma is the brightness component, f WB is the white balance image, MASK is the replacement mask, and (x, y) is any pixel coordinate; According to the replaced brightness component, combined with the white-balanced image, all complete color channels are obtained; Based on all the complete color channels, a de-mosaiced color image is synthesized.

2. The image processing method according to claim 1, characterized in that: The step of detecting dense texture areas in the white balance image comprises: Extracting dense texture features of the white balance image to obtain a texture distribution image; A dense texture area of ​​the white balance image is obtained according to the texture distribution image.

3. The image processing method according to claim 2, characterized in that: The step of extracting dense texture features of the white balance image includes: performing spatial domain convolution on the white balance image through a preset detection convolution kernel to extract the dense texture features of the white balance image.

4. The image processing method according to claim 3, characterized in that: The step of detecting the dense texture area in the white balance image also includes: before extracting the dense texture features of the white balance image to obtain the texture distribution image, designing the detection convolution kernel.

5. The image processing method according to claim 4, characterized in that: The steps of designing the detection convolution kernel include: Design the initial convolution kernel according to the preset spatial frequency; The weights of different pixel positions of the initial convolution kernel are adjusted to obtain the detection convolution kernel.

6. The image processing method according to claim 5, characterized in that: The step of adjusting the weights of different pixel positions of the initial convolution kernel includes: Giving higher weights to pixel positions close to the center of the initial convolution kernel; Lower weights are given to pixel positions that are far from the center of the initial convolution kernel.

7. The image processing method according to claim 5 or 6, characterized in that: In the step of adjusting the weights of different pixel positions of the initial convolution kernel, the weights of different pixel positions of the initial convolution kernel are adjusted by multiplying the initial convolution kernel and the adjusted Gaussian function pixel by pixel.

8. The image processing method according to claim 7, characterized in that: In the step of multiplying each pixel position of the initial convolution kernel by the adjusted Gaussian function, the standard deviation of the Gaussian function in the adjusted Gaussian function is set so that the ratio of the numerical value of the pixel position at the edge of the detection convolution kernel to the numerical value of the pixel position at the center is 10% to 30%.

9. The image processing method according to claim 3, characterized in that: In the step of extracting dense texture features of the white balance image using a preset detection convolution kernel, the size of the detection convolution kernel is greater than or equal to 3.

10. The image processing method according to claim 2, characterized in that: The step of extracting dense texture features of the white balance image to obtain a texture distribution image comprises: Extracting horizontal texture features of the white balance image to obtain a horizontal texture distribution image, wherein the horizontal texture distribution image includes a horizontal dense texture area and a mosaic part; Extracting vertical texture features of the white balance image to obtain a vertical texture distribution image, wherein the vertical texture distribution image includes a vertical dense texture area and a mosaic part; The step of obtaining the dense texture area of ​​the white balance image includes: performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image to obtain the dense texture area of ​​the white balance image.

11. The image processing method according to claim 10, characterized in that: The step of obtaining the dense texture area of ​​the white balance image further includes: After performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image, performing an opening operation on the result of the XOR operation; According to the result of the opening operation, a dense texture area of ​​the white balance image is obtained.

12. The image processing method according to claim 10, characterized in that: The step of obtaining the dense texture area of ​​the white balance image further includes: Before performing an XOR operation on the horizontal texture distribution image and the vertical texture distribution image, the horizontal dense texture area and the mosaic part in the horizontal texture distribution image and the vertical dense texture area and the mosaic part in the vertical texture distribution image are marked.

13. The image processing method according to claim 12, characterized in that: In the step of marking the horizontal dense texture area and the mosaic part in the horizontal texture distribution image and the vertical dense texture area and the mosaic part in the vertical texture distribution image, Morphological operations are performed on the horizontal texture distribution image and the vertical texture distribution image respectively to mark the horizontal dense texture area and the mosaic part in the horizontal texture distribution image and the vertical dense texture area and the mosaic part in the vertical texture distribution image.

14. The image processing method according to claim 13, characterized in that: The steps of performing morphological operations on the horizontal texture distribution image and the vertical texture distribution image respectively include: The horizontal texture distribution image and the vertical texture distribution image are binarized respectively to indicate the horizontal dense texture area and the mosaic part in the horizontal texture distribution image and the vertical dense texture area and the mosaic part in the vertical texture distribution image.

15. The image processing method according to claim 14, characterized in that: The steps of binarizing the horizontal texture distribution image and the vertical texture distribution image respectively include: Determining a binarization threshold for binarizing the horizontal texture distribution image according to a histogram of the horizontal texture distribution image; A binarization threshold for binarizing the vertical texture distribution image is determined according to the histogram of the vertical texture distribution image.

16. The image processing method according to claim 14, characterized in that: The step of performing morphological operations on the horizontal texture distribution image and the vertical texture distribution image respectively further comprises: Before binarizing the horizontal texture distribution image and the vertical texture distribution image, gaps in the horizontal texture distribution image and the vertical texture distribution image are bridged.

17. The image processing method according to claim 16, characterized in that: In the steps of respectively bridging the gaps in the horizontal texture distribution image and the vertical texture distribution image, The gaps are bridged by performing morphological grayscale closing operations on the horizontal texture distribution image and the vertical texture distribution image respectively.

18. The image processing method according to claim 1, characterized in that: The step of calculating the brightness component of the white balance image includes: performing spatial domain convolution on the white balance image to calculate the brightness component of the white balance image.

19. An image processing device, characterized in that: include: A memory and a processor, wherein the memory stores a computer program that can be run on the processor, and wherein when the processor executes the program, the processor executes the steps of the image processing method according to any one of claims 1 to 18.

20. A computer-readable storage medium, wherein the computer-readable storage medium is a non-volatile storage medium or a non-transient storage medium, and a computer instruction is stored thereon, characterized in that: When the computer instructions are executed, the steps of the image processing method according to any one of claims 1 to 18 are executed.

Images