A color image moiré removal method based on color vector processing

Abstract

The present invention discloses a color image de-moiré method based on color vector processing, and relates to the field of image algorithms. The color image de-moiré method based on color vector processing comprises the following steps: converting an RGB image into a grayscale image, and performing moiré detection on the grayscale image based on an edge detection algorithm to obtain a moiré region; judging the cause of the color cast problem in the moiré region based on the channel processing result of the moiré region; eliminating the corresponding moiré region and performing color balance according to the spectrum diagram after Fourier transformation; and processing the channel spectrum diagram using an inverse Fourier transformation method and performing channel fusion to obtain an image with the moiré eliminated. Compared with the prior art, the present invention can run on low-configuration equipment, introduces Fourier transformation on the basis of low cost, and adopts the idea of ​​proportional weighting, so that the moiré removal speed is faster and the final effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of image algorithms, and in particular to a color image moiré removal method based on color vector processing. Background Art

[0002] Moiré is a high-frequency interference stripe that appears on the photosensitive element of a digital camera or scanner. It is a high-frequency irregular stripe that makes the image appear colorful. When the spatial frequency of the pixels of the photosensitive element is close to the spatial frequency of the stripes in the image, a new wavy interference pattern may be generated, the so-called moiré. The grid-like texture of the sensor forms such a pattern. When the thin strips in the pattern intersect the structure of the sensor at a small angle, this effect will also produce obvious interference in the image. Moiré is a manifestation of the beat principle. It is the superposition of two equal-amplitude sine waves with similar frequencies. The amplitude of the composite signal will change according to the difference between the two frequencies.

[0003] At present, the main methods for eliminating moiré are:

[0004] 1. Increase the sampling frequency to more than twice the highest signal frequency. This method can effectively avoid moiré. However, increasing the sampling frequency places higher requirements on the equipment and is more expensive.

[0005] 2. Introduce a low-pass filter or increase the parameters of the low-pass filter. The low-pass filter can limit the bandwidth of the signal so that the sampling frequency reaches more than twice the highest signal frequency. In theory, this is feasible, but it is impossible to do in practice. Because the filter cannot completely filter out the signal above the Nyquist frequency, there will be some interference in the end.

[0006] Currently, no effective solution has been proposed for the problems in the related technologies. Summary of the invention

[0007] In view of the problems in the related art, the present invention proposes a color image moiré removal method based on color vector processing to overcome the above technical problems existing in the existing related art.

[0008] To this end, the specific technical solution adopted by the present invention is as follows:

[0009] A color image de-moiré method based on color vector processing, the color image de-moiré method based on color vector processing comprises the following steps:

[0010] S1. Convert the RGB image into a grayscale image, and perform moiré detection on the grayscale image based on an edge detection algorithm to obtain a moiré area.

[0011] S2. Based on the channel processing result of the moiré area, determine the cause of the color cast problem in the moiré area.

[0012] S3. According to the spectrum diagram after Fourier transformation, the corresponding moiré area is eliminated and the color is balanced.

[0013] S4. Use the inverse Fourier transform method to process the channel spectrum and perform channel fusion to obtain an image with moiré removed.

[0014] Furthermore, based on the edge detection algorithm, when performing moiré detection on the grayscale image, the Canny operator is used to perform edge detection on the grayscale image to obtain the moiré region.

[0015] Furthermore, when converting an RGB image into a grayscale image, the RGB image is converted into a YUV image, and the Y channel is a grayscale image.

[0016] Further, based on the channel processing result of the moiré area, determining the cause of the color cast problem in the moiré area includes the following steps:

[0017] S21, dividing the RGB image into three channels: R, G, and B;

[0018] S22, respectively calculating the mean of the brightness values ​​of all pixels in the moiré regions of the three channels R, G, and B;

[0019] S23. Compare the three mean values ​​to determine whether the moiré region has color cast.

[0020] Furthermore, according to the spectrum diagram after Fourier transformation, eliminating the corresponding moiré area and performing color balancing includes the following steps:

[0021] S31, dividing the RGB image based on resolution;

[0022] S32, processing the channels of the RGB image based on Fourier transform, obtaining a frequency spectrum of the channels, and dividing the channels into several regions;

[0023] S33, calculating weights according to the spectrum graph itself;

[0024] S34, calculating weight according to color balance;

[0025] S35, eliminating the moiré region according to the frequency spectrum obtained by the weight processing in S33 and S34.

[0026] Furthermore, after the RGB image is divided based on the resolution, a corresponding template is set for the corresponding resolution.

[0027] Furthermore, calculating the weights based on the spectrum graph itself includes the following steps:

[0028] S331, comparing the grayscale values ​​of the high brightness area and the surrounding area;

[0029] S332, for each area in the spectrum graph, divide the range into a range including the entire brightness area and a part of the surrounding area;

[0030] S333, calculating the mean of the brightness area and the mean of the surrounding area, and the ratio of the mean of the surrounding area to the mean of the brightness area, and multiplying the grayscale values ​​of all pixels in the brightness area by the ratio to obtain the first part of the weight.

[0031] Further, calculating the weight according to the color balance includes the following steps:

[0032] S341, calculating the mean of the grayscale values ​​of the moiré regions of the three channels;

[0033] S342. For the three means, weight calculation is performed according to the proportion of each mean.

[0034] Furthermore, when eliminating the corresponding moiré area and performing color balancing, weights are assigned to the spectrograms of the three channels, and high-brightness areas of the spectrogram except for the central area are eliminated.

[0035] Furthermore, after calculating the weights according to the color balance, the first part of the weights is weighted using the θ values ​​corresponding to the three channels;

[0036] The divided regions are spliced ​​together to obtain the frequency spectrum of the corresponding channels.

[0037] The beneficial effects of the present invention are:

[0038] (1) The present invention provides a color image moiré removal method based on color vector processing. Compared with the prior art, the present invention can be run on low-configuration devices. On the basis of low cost, the Fourier transform is introduced, and the idea of ​​proportional weight is adopted to remove moiré faster and improve the final effect. The present invention requires low cost, and the final output image has obvious moiré removal effect and the processing time is extremely short.

[0039] (2) The present invention directly processes the color image by performing Fourier transform on the three channels of the color image to obtain the spectrum of the three channels, processes the spectrum of the three channels, then performs inverse Fourier transform on the three channels to obtain the images of the three channels, and finally fuses the three channels to obtain the final image. Since most of the current shooting devices are 720p, 1080p, 2k, and 4k, and the moiré is related to the spatial frequency of the CCD / CMOS pixels of the photosensitive element of the shooting device, the position of the moiré corresponding to the spectrum in the image of the same resolution is determined, so corresponding templates can be provided for the above four resolution devices, which can be faster when removing the moiré. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 is a flow chart of a method for removing moiré from a color image based on color vector processing according to an embodiment of the present invention;

[0042] Figure 2 It is a simplified example of a color image moiré removal method based on color vector processing according to an embodiment of the present invention;

[0043] Figure 3 are the coordinates of the template and brightness area corresponding to each resolution according to the embodiment of the present invention. DETAILED DESCRIPTION

[0044] To further illustrate each embodiment, the present invention provides drawings, which are part of the disclosure of the present invention and are mainly used to illustrate the embodiments and can be used in conjunction with the relevant descriptions in the specification to explain the operating principles of the embodiments. With reference to these contents, ordinary technicians in the field should be able to understand other possible implementations and advantages of the present invention. The components in the figures are not drawn to scale, and similar component symbols are generally used to represent similar components.

[0045] According to an embodiment of the present invention, a color image moiré removal method based on color vector processing is provided.

[0046] The present invention is further described with reference to the accompanying drawings and specific embodiments. Figure 1-3 As shown, according to a color image de-moiré method based on color vector processing according to an embodiment of the present invention, the color image de-moiré method based on color vector processing comprises the following steps:

[0047] S1. Convert the RGB image into a grayscale image, and perform moiré detection on the grayscale image based on an edge detection algorithm to obtain a moiré area.

[0048] In one embodiment, when performing moiré detection on a grayscale image based on an edge detection algorithm, the Canny operator is used to perform edge detection on the grayscale image to obtain a moiré area, that is, the edge area and the moiré area of ​​the image are separated, and the edge area of ​​the image is screened out. Finally, the moiré area of ​​the image can be obtained, and then the moiré area of ​​the image is located.

[0049] In one embodiment, when converting an RGB image into a grayscale image, the RGB image is converted into a YUV image, and the Y channel is a grayscale image. The calculation formula is:

[0050] Y = 0.299R + 0.587G + 0.114B;

[0051] Where R is the red channel in the image, G is the green channel in the image, and B is the blue channel in the image.

[0052] According to the causes of moiré, the high-frequency area of ​​the spectrum graph contains the edge area and moiré area of ​​the image. Since there is a difference in direction between the moiré and the edge area, the Canny operator is used to detect the edge of the image, separate the edge area and the moiré area of ​​the image, and then process the moiré area. The Canny operator is a standard algorithm widely used in edge detection. Its goal is to find an optimal edge detection solution or find the location with the strongest change in grayscale intensity in an image. The optimal edge detection is mainly evaluated by three criteria: low error rate, high localization, and minimum response.

[0053] S2. Based on the channel processing result of the moiré area, determine the cause of the color cast problem in the moiré area; in order to restore the color of the moiré area to the correct color, the color cast problem in the moiré area needs to be processed.

[0054] In one embodiment, based on the channel processing result of the moiré region, determining the cause of the color cast problem in the moiré region includes the following steps:

[0055] S21. Divide the RGB image into three channels: R, G, and B.

[0056] S22, respectively calculate the mean values ​​of the brightness values ​​of all pixels in the moiré regions of the three channels R, G, and B (i.e., the mean grayscale values).

[0057] S23. Compare the three mean values ​​to determine whether the moiré region has color cast.

[0058] S3. Eliminate the corresponding moiré region and balance the color according to the spectrum after Fourier transformation. That is, perform Fourier transformation on the three channels of the color image, respectively, obtain the spectrum of the three channels, and process the spectrum of the three channels: eliminate the moiré region, and balance the color while eliminating the moiré.

[0059] Since the sampling frequencies corresponding to the current mainstream image resolutions (720p, 1080p, 2k, 4k) are basically fixed, and the spatial frequency of the CCD / CMOS pixels of the photosensitive elements of the shooting equipment is relatively fixed, the frequency of the moiré pattern generated is relatively fixed, and the position of the high-brightness area that appears in the spectrum is also basically determined. Therefore, when processing the image spectrum, it is classified according to the resolution, and the corresponding resolution has a corresponding template.

[0060] The method for processing the spectrum graphs of the three channels is as follows: weights are assigned to the spectrum graphs of the three channels, and all high-brightness areas except the central area of ​​the spectrum graph are eliminated, so that the high-brightness area of ​​the spectrum graph obtained in the end is only the central area. Figure 2 This is a simplified example, 1 represents the bright part and 0 represents the dark part.

[0061] In one embodiment, according to the spectrum diagram after Fourier transformation, eliminating the corresponding moiré area and performing color balancing includes the following steps:

[0062] S31. Divide the RGB image based on resolution.

[0063] S32. Process the channels of the RGB image based on Fourier transform to obtain a frequency spectrum of the channels and divide the channels into several regions.

[0064] S33. Calculate the weight according to the spectrum graph itself, that is, convert the value of the brightness area into a value close to that of the surrounding area according to the comparison of the grayscale values ​​of the high brightness area and the surrounding area.

[0065] S34, calculating weights according to color balance, that is, calculating weights according to the proportion of the brightness mean obtained in S22.

[0066] S35, eliminating the moiré region according to the frequency spectrum obtained by the weight processing in S33 and S34.

[0067] In one embodiment, after the RGB image is divided based on resolution, a corresponding template is set for each resolution.

[0068] In one embodiment, calculating the weights based on the spectrum graph itself includes the following steps:

[0069] S331. Compare the grayscale values ​​of the high-brightness area and the surrounding area.

[0070] S332, converting the value of the brightness area into a value close to that of the surrounding area: for each area in the spectrum diagram, a range including the entire brightness area and part of the surrounding area is divided.

[0071] S333. Calculate the mean of the brightness area and the mean of the surrounding area, as well as the ratio of the mean of the surrounding area to the mean of the brightness area, and multiply the grayscale value L of all pixels in the brightness area by the ratio to obtain the first part of the weight.

[0072] In one embodiment, calculating weights according to color balance includes the following steps:

[0073] S341, calculating the mean of the grayscale values ​​of the moiré regions of the three channels.

[0074] S342. For the three means, weight calculation is performed according to the proportion of each mean.

[0075] In one embodiment, when eliminating the corresponding moiré region and performing color balancing, weights are assigned to the spectrograms of the three channels, and high-brightness regions of the spectrogram except for the central region are eliminated.

[0076] In one embodiment, after calculating the weights according to the color balance, the first part of the weights is weighted by the θ values ​​corresponding to the three channels; and the divided regions are spliced ​​to obtain the frequency spectrum of the corresponding channels.

[0077] S4. Use the inverse Fourier transform method to process the channel spectrum and perform channel fusion to obtain an image with moiré removed.

[0078] Taking a 720p image as an example, assuming that the spectrum diagrams of the three channels of the image have four high-brightness areas (except the central area), then only the four high-brightness areas need to be processed.

[0079] First, since the spectrum graph is centrally symmetrical, the spectrum graph can be divided into four areas, marked as area 1, area 2, area 3, and area 4 from left to right and from top to bottom. It is only necessary to assign weights to area 1 and area 2.

[0080] Secondly, the calculation of weights: The calculation of weights is mainly divided into two parts. The first part is to calculate the weights based on the spectrum graph itself, and the second part is to calculate the weights based on color balance.

[0081] The first part of the weight: According to the gray value of the high brightness area and the surrounding area, the value of the brightness area is converted into a value close to the surrounding area. The specific calculation method is: divide an area, including the entire brightness area of ​​area 1 and part of the surrounding area, and calculate the mean value D of the brightness area 11 and the mean value D of the surrounding area 12 The ratio of the two means is Multiply the grayscale value L of all pixels in the brightness area by Right now In the final area 1, there is no high brightness area other than the center of the original spectrum. Similarly, the above operation can be performed on area 2 to obtain the same result. Areas 3 and 4 are centrally symmetrical with areas 1 and 2, respectively, and the final result can be obtained without calculation.

[0082] The second part of the weight: In the second step, the gray value mean D of the moiré area of ​​the three channels in the RGB color mode has been calculated R , D G , D B , calculated based on these three means. The three means are calculated according to their proportion weights, that is, the R channel weight G channel weight B channel weight

[0083] Processing method: In the first part of the weight calculation, the gray value in the high brightness area has been calculated to obtain Then use the second part of the weight to calculate, Further weighting, that is The corresponding θ value of each channel. Then the four areas are spliced ​​together to get the final spectrum.

[0084] Finally, the inverse Fourier transform is used to process the spectrum graphs of the three channels respectively to obtain the images of the three channels, and then the three channels are fused to obtain the final image.

[0085] Inverse Fourier transform formula:

[0086]

[0087] Wherein, F(u, v) represents the value of the frequency domain signal at (u, v); f(x, y) represents the value of the input two-dimensional discrete signal at the position (x, y); M and N are the number of pixels of the input signal in the x and y directions respectively; j is an imaginary unit; and They are the normalized values ​​of the corresponding frequency components in the x and y directions respectively.

[0088] The above operation on the spectrum graph mainly processes part of the area, multiplying the gray value of the part of the area by a value, so that a matrix can be generated. The size of this matrix is ​​the same as the resolution of the processed image. The values ​​in the matrix are mainly divided into two parts: one part is the value of the corresponding position of the matrix corresponding to the part of the spectrum graph that needs to be processed, which is The other part is the area where the spectrum graph is not processed, and the value of the corresponding position in the corresponding matrix is ​​1. The templates corresponding to images of different resolutions are also different. According to the above method based on 720p, the weight matrix corresponding to other resolutions can be calculated, and the moiré pattern in the image of the corresponding resolution can be quickly processed through this matrix.

[0089] The above operation is performed on an RGB image with a resolution of 720p and moiré to obtain a new image. The moiré is obviously not visible on the new image, and the processing time is extremely short.

[0090] In summary, the present invention provides a color image moiré removal method based on color vector processing. Compared with the prior art, the present invention can be run on low-configuration devices. On the basis of low cost, Fourier transform is introduced, and the idea of ​​proportional weight is adopted to remove moiré faster and improve the final effect. The present invention requires low cost, and the final output image moiré removal effect is obvious, and the processing time is extremely short. The present invention directly processes the color image, obtains the spectrum of the three channels by Fourier transforming the three channels of the color image, processes the spectrum of the three channels, and then inverse Fourier transforms the three channels to obtain the images of the three channels, and finally fuses the three channels to obtain the final image. Since most of the current shooting devices are 720p, 1080p, 2k, and 4k, and the moiré is related to the spatial frequency of the CCD / CMOS pixel of the photosensitive element of the shooting device, the position of the moiré corresponding to the spectrum in the image of the same resolution is determined, so corresponding templates can be provided for the above four resolution devices, which can be faster when removing moiré.

[0091] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A color image moiré removal method based on color vector processing, characterized in that: The color image de-moiré method based on color vector processing comprises the following steps: S1. Convert the RGB image into a grayscale image, and perform moiré detection on the grayscale image based on an edge detection algorithm to obtain a moiré area. S2. based on the channel processing result of the moiré area, determining the cause of the color cast problem in the moiré area; S3, according to the spectrum after Fourier transformation, eliminate the corresponding moiré area and balance the color; S4, using the inverse Fourier transform method to process the channel spectrum and perform channel fusion to obtain an image with moiré removed; The determining the cause of the color cast problem in the moiré region based on the channel processing result of the moiré region comprises the following steps: S21, dividing the RGB image into three channels: R, G, and B; S22, respectively calculating the mean of the brightness values ​​of all pixels in the moiré regions of the three channels R, G, and B; S23, comparing the sizes of the three means to determine whether the moiré region has color cast; The method of eliminating the corresponding moiré region and performing color balancing according to the spectrum diagram after Fourier transformation comprises the following steps: S31, dividing the RGB image based on resolution; S32, processing the channels of the RGB image based on Fourier transform, obtaining a frequency spectrum of the channels, and dividing the channels into several regions; S33, calculating weights according to the spectrum graph itself; S34, calculating weight according to color balance; S35, eliminating the moiré region according to the spectrum obtained by weight processing in S33 and S34; After the RGB image is divided based on the resolution, a corresponding template is set for each resolution; The calculation of weights according to the spectrum graph itself comprises the following steps: S331, comparing the grayscale values ​​of the high brightness area and the surrounding area; S332, for each area in the spectrum graph, divide the range into a range including the entire brightness area and a part of the surrounding area; S333, calculating the mean of the brightness area and the mean of the surrounding area, and the ratio of the mean of the surrounding area to the mean of the brightness area, and multiplying the grayscale values ​​of all pixels in the brightness area by the ratio to obtain the first part of the weight; Calculating weights according to color balance includes the following steps: S341, calculating the mean of the grayscale values ​​of the moiré regions of the three channels; S342. For the three means, weight calculation is performed according to the proportion of each mean; When eliminating the corresponding moiré area and performing color balancing, weights are assigned to the spectrum graphs of the three channels, and high-brightness areas of the spectrum graph except the central area are eliminated; After calculating the weight according to the color balance, the three channels are used to Value, weight the first part; The divided regions are spliced ​​together to obtain the frequency spectrum of the corresponding channels.

2. The method for removing moiré from a color image based on color vector processing according to claim 1, characterized in that: When performing moiré detection on a grayscale image based on an edge detection algorithm, a Canny operator is used to perform edge detection on the grayscale image to obtain a moiré region.

3. The method for removing moiré from a color image based on color vector processing according to claim 2, characterized in that: When converting the RGB image into a grayscale image, the RGB image is converted into a YUV image, and the Y channel is a grayscale image.

Images