Image processing device and image processing method

Abstract

In an image processing device according to the present disclosure: a 3DNR circuit has a first noise extraction unit, a motion detection unit that detects a motion signal of image data, a first coefficient generation unit that generates a first coefficient, a first noise removal unit that removes noise using a signal obtained by multiplying the extracted noise by the first coefficient, and a dispersion calculation unit that calculates a dispersion value representing the frequency distribution of the luminance of each frame of the image data; a 2DNR circuit has a second noise extraction unit that extracts noise from the image data that was subjected to the first noise removal process, a second coefficient generation unit that generates a second coefficient, and a second noise removal unit that removes noise using a signal obtained by multiplying the extracted noise by the second coefficient; the first coefficient generation unit sets the inclination of the first coefficient with respect to the motion signal on the basis of the dispersion value; and the second coefficient generation unit sets the inclination of the second coefficient with respect to the motion signal on the basis of the dispersion value.

Description

Image processing device and image processing method

[0001] The present disclosure relates to an image processing device and an image processing method, and more particularly to an image processing device and an image processing method suitable for efficiently removing noise from image data.

[0002] Image processing devices are required to efficiently remove noise from image data. Patent Document 1 discloses an image processing device that removes noise from image data.

[0003] The image processing device disclosed in Patent Document 1 has an image synchronization unit that synchronizes frame images from an image input unit with frame images from a frame storage unit, a two-dimensional noise removal unit that performs noise removal on the frame images in accordance with a two-dimensional noise removal technique, a motion detection unit that detects movement of the frame images, a frame cyclic coefficient determination unit that determines a frame cyclic coefficient, a three-dimensional noise removal unit that performs noise removal on the frame images in accordance with a three-dimensional noise removal technique, a combination ratio determination unit that determines a combination ratio between the two-dimensional noise-removed image and the three-dimensional noise-removed image, and a two-dimensional / three-dimensional combination unit that combines the two-dimensional noise-removed image and the three-dimensional noise-removed image.

[0004] Here, the image processing device disclosed in Patent Document 1 suppresses image blurring (afterimages) that can occur when noise removal is performed according to a three-dimensional noise removal technique by increasing the ratio of noise removal according to a two-dimensional noise removal technique the greater the image movement, while appropriately removing noise by increasing the ratio of noise removal according to a three-dimensional noise removal technique the smaller the image movement.

[0005] Japanese Patent Application Laid-Open No. 2005-150903

[0006] However, the related art does not disclose or suggest a specific method for removing noise from image data with different contrasts. Therefore, even if the related art can properly remove noise from high-contrast image data, it may not be able to properly remove noise from low-contrast image data. In other words, the related art still has a problem in that it cannot efficiently remove noise from image data.

[0007] The present disclosure has been made in consideration of the above points, and has an object to provide an image processing device and an image processing method that can efficiently remove noise from image data.

[0008] An image processing device according to the present disclosure includes a 3DNR (3 Dimensional Noise Reduction) circuit that performs a first noise removal process on image data, and a 2DNR (2 Dimensional Noise Reduction) circuit that performs a second noise removal process on the image data that has been subjected to the first noise removal process, the 3DNR circuit having a first noise extraction unit that extracts noise from the image data from a difference between frames of the image data, a motion detection unit that detects a motion signal that represents motion between frames of the image data, a first coefficient generation unit that generates a first coefficient that indicates a smaller value as the motion represented by the motion signal increases, a first noise removal unit that removes noise from the image data by subtracting a first noise removal signal that is generated by multiplying the noise extracted by the first noise extraction unit by the first coefficient from the image data, and a variance calculation unit that calculates a variance value that represents a frequency distribution of luminance in each frame of the image data, and the 2DNR circuit extracts a variance value that represents a frequency distribution of luminance in each frame of the image data from each frame of the image data that has been subjected to the first noise removal process. a second noise extraction unit that extracts noise from the image data; a second coefficient generation unit that generates a second coefficient that indicates a larger value as the motion represented by the motion signal increases; and a second noise removal unit that removes noise from the image data that has been subjected to the first noise removal process by subtracting a second noise removal signal generated by multiplying the noise extracted by the second noise extraction unit by the second coefficient from the image data, wherein the first coefficient generation unit is configured to set a gradient of the first coefficient with respect to the motion signal based on a contrast of the image data that corresponds to the calculated variance value, and the second coefficient generation unit is configured to set a gradient of the second coefficient with respect to the motion signal based on a contrast of the image data that corresponds to the calculated variance value.

[0009] The image processing method according to the present disclosure is an image processing method for an image processing device including a 3DNR (3 Dimensional Noise Reduction) circuit that performs a first noise removal process on image data, and a 2DNR (2 Dimensional Noise Reduction) circuit that performs a second noise removal process on the image data that has been subjected to the first noise removal process, wherein the 3DNR circuit extracts a first noise that is noise in the image data from a difference between frames of the image data, detects a motion signal that represents motion between the frames of the image data, calculates a variance value that represents a frequency distribution of luminance in each frame of the image data, sets a slope of a first coefficient with respect to the motion signal based on the contrast of the image data that corresponds to the calculated variance value, and then generates the first coefficient that indicates a smaller value as the motion represented by the motion signal increases, and performs a first noise removal signal generated by multiplying the first noise by the first coefficient, The noise of the image data is removed by subtracting the second noise from the image data that has been subjected to the first noise removal process, and in the 2DNR circuit, a second noise, which is the noise of the image data, is extracted from each frame of the image data that has been subjected to the first noise removal process, and the slope of the second coefficient with respect to the motion signal is set based on the contrast of the image data that corresponds to the calculated variance value, and the second coefficient is generated so that the larger the motion represented by the motion signal, the larger the value of the second coefficient is, and the second noise removal signal generated by multiplying the second noise by the second coefficient is subtracted from the image data that has been subjected to the first noise removal process, thereby removing the noise of the image data.

[0010] According to the present disclosure, it is possible to provide an image processing device and an image processing method that can efficiently remove noise from image data.

[0011] 1 is a block diagram showing an example configuration of an image processing device according to the present disclosure; FIG. 2 is a block diagram showing an example configuration of a 3DNR circuit provided in the image processing device according to the present disclosure; FIG. 3 is a block diagram showing an example configuration of a 2DNR circuit provided in the image processing device according to the present disclosure; FIG. 4 is a diagram showing the relationship between a motion signal mv and a cyclic coefficient k3d in a 3DNR circuit; FIG. 5 is a diagram showing the relationship between a motion signal mv and a multiplication coefficient k2d in a 2DNR circuit; FIG. 6 is a diagram showing an example of a frequency distribution of luminance of each frame of image data; FIG. 7 is a diagram showing another example of a frequency distribution of luminance of each frame of image data; FIG. 8 is a diagram for explaining a method of setting the slopes of coefficients k3d and k2d with respect to the motion signal mv; and FIG. 9 is a flowchart showing the operation of an image processing device according to the present disclosure.

[0012] 1 is a block diagram showing an example of the configuration of an image processing device 1 according to the present disclosure. The image processing device 1 is a device that suppresses noise in image data Din and outputs the image data Dout. A specific description will be given below.

[0013] 1, the image processing device 1 includes a 3DNR circuit 11 and a 2DNR circuit 12. DNR is an abbreviation for Dimensional Noise Reduction.

[0014] The 3DNR circuit 11 reduces noise contained in the image data Din and outputs image data Dmid. In other words, the 3DNR circuit 11 outputs image data Dmid obtained by performing a first noise removal process on the image data Din. The 2DNR circuit 12 reduces noise contained in the image data Dmid and outputs image data Dout. In other words, the 2DNR circuit 12 outputs image data Dout obtained by performing a second noise removal process on the image data Dmid that has been subjected to the first noise removal process.

[0015] (Details of the 3DNR Circuit 11) The 3DNR circuit 11 is a circuit that extracts and reduces noise contained in the image data Din by analyzing the difference between consecutive frames in the image data Din. The 3DNR circuit 11 is more suited to reducing noise in image areas of the image data Din with small movements than in image areas with large movements where afterimages and the like are more likely to occur.

[0016] Fig. 2 is a block diagram showing a specific example configuration of the 3DNR circuit 11. As shown in Fig. 2, the 3DNR circuit 11 includes an addition / subtraction circuit 111, a frame memory 112, an addition / subtraction circuit 113, a noise extraction unit 114, a motion detection unit 115, a cyclic coefficient generation unit 116, a multiplication circuit 117, and a variance calculation unit 118.

[0017] The adder-subtractor circuit 111 subtracts a signal s14 representing the noise component to be removed from the image data Din and outputs image data Dmid that has been subjected to noise removal processing. The frame memory 112 outputs image data Din_d, which is the image data Dmid delayed by one frame.

[0018] The addition / subtraction circuit 113 subtracts the image data Din_d from the image data Din to output a signal s12 representing the difference between the image data Din_d and the image data Din (i.e., the difference between frames). The noise extraction unit 114 extracts noise components contained in the signal s12 output from the addition / subtraction circuit 113 and outputs it as a signal s13. The noise extraction unit 114 is a first noise extraction unit, and in other words, extracts noise from the difference between the image data Din_d and the image data Din.

[0019] The motion detection unit 115 detects a motion signal mv that represents changes (motion) between frames of the image data Din. Specifically, the motion detection unit 115 detects a motion signal mv that represents changes (motion) in the image from image data Din_d to image data Din. The motion signal mv exhibits a larger value in an area where the image changes more significantly from image data Din_d to image data Din, and a smaller value in an area where the image changes less significantly from image data Din_d to image data Din.

[0020] The cyclic coefficient generation unit 116 is a first coefficient generation unit that generates a cyclic coefficient k3d according to the motion signal mv. The cyclic coefficient k3d is set in a range greater than or equal to 0 and less than 1. Here, the cyclic coefficient generation unit 116 reduces the cyclic coefficient k3d as the motion signal mv increases so as to reduce the noise removal effect in the 3DNR circuit 11 in order to prevent the occurrence of afterimages, and increases the cyclic coefficient k3d as the motion signal mv decreases so as to increase the noise removal effect in the 3DNR circuit 11. The cyclic coefficient k3d is sometimes referred to as a first coefficient.

[0021] The multiplication circuit 117 multiplies the signal s13 representing the noise component by the cyclic coefficient k3d to output a signal s14 representing the noise component to be removed. As already described, the addition / subtraction circuit 111 subtracts the signal s14 representing the noise component to be removed from the image data Din to output image data Dmid that has been subjected to noise removal processing. Therefore, the addition / subtraction circuit 111 and the multiplication circuit 117 form a first noise removal unit. The first noise removal unit removes noise from the image data by subtracting from the image data a first noise removal signal generated by multiplying the noise extracted by the noise extraction unit 114 (first noise extraction unit) by the cyclic coefficient k3d (first coefficient).

[0022] The variance calculation unit 118 calculates a variance value v1 that represents the frequency distribution of luminance for each frame of the image data Din. The variance value v1 will be described in detail later. Hereinafter, the signal that represents the variance value v1 will also be simply referred to as the variance value v1.

[0023] The 3DNR circuit 11 outputs image data Dmid, a motion signal mv, and a variance value v1. The image data Dmid is image data that has been subjected to the first noise removal process.

[0024] (Details of the 2DNR Circuit 12) The 2DNR circuit 12 is a circuit that extracts and reduces noise contained in the image data Dmid by analyzing each frame of the image data Dmid. The 2DNR circuit 12 is suitable for reducing noise in image areas of the image data Dmid that have large movements.

[0025] Fig. 3 is a block diagram showing a specific configuration example of the 2DNR circuit 12. As shown in Fig. 3, the 2DNR circuit 12 includes a HPF (High Pass Filter) 121, a limiter 122, a coefficient generation unit 123, a multiplication circuit 124, and an addition / subtraction circuit 125.

[0026] The HPF 121 passes high-frequency components contained in the image data Dmid. The limiter 122 limits the amplitude of the signal s21, which is the high-frequency component that has passed through the HPF 121, to a predetermined range and outputs the signal s22. In other words, the HPF 121 and the limiter 122 function as a second noise extraction unit that extracts the signal s22 representing minute high-frequency noise components contained in the image data Dmid. In other words, the second noise extraction unit extracts noise from the image data from each frame of the image data that has been subjected to the first noise removal process.

[0027] The coefficient generation unit 123 is a second coefficient generation unit that generates a multiplication coefficient k2d according to the motion signal mv. The multiplication coefficient k2d is set in the range of 0 or greater and less than 1. Here, the coefficient generation unit 123 increases the multiplication coefficient k2d so that the noise removal effect in the 2DNR circuit 12 increases as the motion signal mv increases. In other words, the coefficient generation unit 123 generates a multiplication coefficient k2d that increases as the motion represented by the motion signal mv increases. The multiplication coefficient k2d is sometimes referred to as a second coefficient. The coefficient generation unit 123 decreases the multiplication coefficient k2d so that the noise removal effect in the 2DNR circuit 12 decreases as the motion signal mv decreases.

[0028] The multiplication circuit 124 multiplies a signal s22 representing a noise component by a multiplication coefficient k2d, and outputs a signal s23 representing the noise component to be removed. The addition / subtraction circuit 125 subtracts the signal s23 representing the noise component to be removed from the data Dmid, and outputs image data Dout that has been subjected to noise removal processing. Therefore, the multiplication circuit 124 and the addition / subtraction circuit 125 form a second noise removal unit. The second noise removal unit removes noise from the image data that has been subjected to the first noise removal processing by subtracting a second noise removal signal generated by multiplying the noise extracted by the second noise extraction unit by a multiplication coefficient k2d (second coefficient) from the image data.

[0029] In this way, the image processing device 1 according to the present disclosure controls the weight (degree) of the noise removal processing on the image data Din by each of the 3DNR circuit 11 and the 2DNR circuit 12 depending on the magnitude of the motion between frames of the image data Din (i.e., the motion signal mv).

[0030] For example, the image processing device 1 according to the present disclosure reduces the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 and increases the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 for image regions with greater motion in the image data Din (i.e., image regions with greater values ​​for the motion signal mv). Also, the image processing device 1 according to the present disclosure increases the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 and decreases the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 for image regions with less motion in the image data Din (i.e., image regions with smaller values ​​for the motion signal mv).

[0031] As a result, the image processing device 1 according to the present disclosure can reduce the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 for image regions of the image data Din with large motion to suppress the occurrence of afterimages, while increasing the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 to sufficiently remove noise that was not completely removed by the 3DNR circuit 11. Furthermore, the image processing device 1 according to the present disclosure can reduce the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 for image regions of the image data Din with small motion to prevent a decrease in resolution, while increasing the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 to sufficiently remove noise. In other words, the image processing device 1 according to the present disclosure can efficiently remove noise from the image data Din while preventing the occurrence of afterimages, etc.

[0032] Furthermore, the image processing device 1 according to the present disclosure switches the slopes of the recursive coefficient k3d and the multiplication coefficient k2d for the motion signal mv depending on the detection sensitivity of the motion signal mv. Here, the detection sensitivity of the motion signal mv refers to the ease of detecting changes (motion) between frames of the image data Din. The detection sensitivity of the motion signal mv increases as the contrast of the image increases, and decreases as the contrast of the image decreases. This will be described below with reference to FIGS. 4 and 5.

[0033] FIG. 4 is a diagram showing the relationship between the motion signal mv and the cyclic coefficient k3d in the 3DNR circuit 11. FIG. 5 is a diagram showing the relationship between the motion signal mv and the multiplication coefficient k2d in the 2DNR circuit 12. As shown in FIGS. 4 and 5 , the lower the contrast of the image data Din and the lower the detection sensitivity of the motion signal mv, the larger the absolute values ​​of the slopes of the cyclic coefficient k3d and the multiplication coefficient k2d relative to the motion signal mv are set to. As a result, even if the motion of the subject is large but the motion signal mv is small due to the influence of low contrast in the image data Din, the cyclic coefficient k3d becomes small and the multiplication coefficient k2d becomes large, thereby suppressing the generation of afterimages by the 3DNR circuit 11.

[0034] Generally, when shooting in a low-light environment such as at night, the amount of noise generated in the camera sensor increases. Therefore, the lower the light intensity of the shooting environment (specifically, the lower the average luminance of the image), the greater the effect of the noise removal process by the 3DNR circuit 11 or the 2DNR circuit 12. Therefore, it is preferable to set the maximum value k3dmax of the cyclic coefficient k3d and the maximum value k2dmax of the multiplication coefficient k2d to large values ​​(see FIGS. 4 and 5).

[0035] In other words, the cyclic coefficient generation unit 116 (first coefficient generation unit) increases the maximum value k3dmax of the cyclic coefficient k3d (first coefficient) as the average luminance of the image data decreases, and decreases the maximum value of the cyclic coefficient k3d (first coefficient) as the average luminance of the image data increases. Also, the coefficient generation unit 123 (second coefficient generation unit) increases the maximum value k2dmax of the multiplication coefficient k2d (second coefficient) as the average luminance of the image data decreases, and decreases the maximum value k2dmax of the multiplication coefficient k2d (second coefficient) as the average luminance of the image data increases.

[0036] As described above, the detection sensitivity of the motion signal mv increases as the contrast of the image increases, and decreases as the contrast of the image decreases. Therefore, the detection sensitivity of the motion signal mv can be determined, for example, by the variance v1 that represents the frequency distribution of the luminance of each frame of the image data Din, which corresponds to the contrast of the image data Din.

[0037] 6 and 7 are diagrams showing examples of the frequency distribution of luminance for each frame of image data. In the examples of FIGS. 6 and 7, the frequency distribution of luminance for each frame of image data is displayed as a histogram. As shown in FIG. 6, the larger the variance value v1, the higher the image contrast, and therefore the higher the detection sensitivity of the motion signal mv. In this case, the absolute values ​​of the slopes of the cyclic coefficient k3d and the multiplication coefficient k2d for the motion signal mv are set to smaller values. In contrast, as shown in FIG. 7, the smaller the variance value v1, the lower the image contrast, and therefore the lower the detection sensitivity of the motion signal mv. In this case, the absolute values ​​of the slopes of the cyclic coefficient k3d and the multiplication coefficient k2d for the motion signal mv are set to larger values.

[0038] Specifically, in the 3DNR circuit 11, the cyclic coefficient generation unit 116 sets the absolute value of the slope of the cyclic coefficient k3d with respect to the motion signal mv to be larger when the variance value v1 is less than a predetermined first threshold th1 than when the variance value v1 is equal to or greater than the first threshold th1. Note that the cyclic coefficient generation unit 116 is not limited to such an operation, and may be configured to generate cyclic coefficients k3d such that the absolute value of the slope of the cyclic coefficient k3d with respect to the motion signal mv becomes smaller as the variance value v1 becomes larger, and to generate cyclic coefficients k3d such that the absolute value of the slope of the cyclic coefficient k3d with respect to the motion signal mv becomes larger as the variance value v1 becomes smaller.

[0039] Furthermore, in the 2DNR circuit 12, the coefficient generation unit 123 sets the absolute value of the slope of the multiplication coefficient k2d with respect to the motion signal mv to be larger when the variance value v1 is less than a predetermined first threshold th1 than when the variance value v1 is equal to or greater than the first threshold th1. Note that the coefficient generation unit 123 is not limited to such an operation, and may be configured to generate a multiplication coefficient k2d such that the absolute value of the slope of the multiplication coefficient k2d with respect to the motion signal mv becomes smaller the larger the variance value v1, and to generate a multiplication coefficient k2d such that the absolute value of the slope of the multiplication coefficient k2d with respect to the motion signal mv becomes larger the smaller the variance value v1.

[0040] In other words, when the contrast of the image is less than the first threshold th1, the cyclic coefficient generation unit 116 (first coefficient generation unit) increases the absolute value of the slope of the cyclic coefficient k3d (first coefficient) with respect to the motion signal mv compared to when the contrast is equal to or greater than the first threshold th1, and the coefficient generation unit 123 (second coefficient generation unit) increases the absolute value of the slope of the multiplication coefficient k2d (second coefficient) with respect to the motion signal mv when the contrast of the image is less than the first threshold th1 compared to when the contrast is equal to or greater than the first threshold th1.

[0041] The motion detection unit 115 detects the motion signal mv using, for example, a luminance signal representing the luminance of each frame of image data. However, color signals representing the color of each frame of image data may also be used. For example, when shooting in a low-light environment, if only the luminance signal is used, the contrast is low and the detection sensitivity of the motion signal mv is low. However, if a color signal is used in addition to the luminance signal, the difference between the hue of the subject image and the hue of the background image increases the amount of motion detection and increases the detection sensitivity of the motion signal mv. Therefore, when the luminance signal and color signal are used to detect the motion signal mv, the absolute values ​​of the slopes of the recursive coefficient k3d and the multiplication coefficient k2d relative to the motion signal mv are adjusted to be smaller than when only the luminance signal is used.

[0042] In other words, when the motion detection unit 115 detects a motion signal mv from a luminance signal representing the luminance of each frame of image data and a color signal representing the color of each frame of the image data, the cyclic coefficient generation unit 116 (first coefficient generation unit) reduces the absolute value of the slope of the cyclic coefficient k3d (first coefficient) for the motion signal mv compared to when the motion signal mv is detected from the luminance signal alone; and when the motion detection unit 115 detects a motion signal mv from a luminance signal representing the luminance of each frame of image data and a color signal representing the color of each frame of the image data, the coefficient generation unit 123 (second coefficient generation unit) reduces the absolute value of the slope of the multiplication coefficient k2d (second coefficient) for the motion signal compared to when the motion signal mv is detected from the luminance signal alone.

[0043] Furthermore, when the luminance signal and the color signal are used to detect the motion signal mv, the first threshold th1 is set to a smaller value than when only the luminance signal is used. In the example of Fig. 8, when only the luminance signal is used to detect the motion signal mv, the first threshold th1 is set to a value B, whereas when the luminance signal and the color signal are used to detect the motion signal mv, the first threshold th1 is set to a value A (<B) (see Fig. 8).

[0044] For example, the cyclic coefficient generation unit 116 switches the slope of the cyclic coefficient k3d for the motion signal mv based on a switching signal SW that indicates one of the L level and the H level (e.g., the L level) when only the luminance signal is used and indicates the other of the L level and the H level (e.g., the H level) when the luminance signal and the color signal are used. Also, the coefficient generation unit 123 switches the slope of the multiplication coefficient k2d for the motion signal mv based on a switching signal SW that indicates one of the L level and the H level (e.g., the L level) when only the luminance signal is used and indicates the other of the L level and the H level (e.g., the H level) when the luminance signal and the color signal are used.

[0045] More specifically, when a color signal is used to detect the motion signal mv, the following two methods are used.

[0046] In the first technique, motion detection is performed using each of the color difference signals Cb and Cr in the same manner as when the luminance signal Y is used, and then the motion detection results using the luminance signal Y and the color difference signals Cb and Cr are added together. Note that when the input signals are R, G, B signals, the R, G, B signals are converted into Y, Cb, Cr signals before motion detection.

[0047] In the second technique, motion detection is performed using each of the R, G, and B signals in the same manner as when the luminance signal Y is used, and then the motion detection results using each of the R, G, and B signals are added together. When the input signals are Y, Cb, and Cr signals, the Y, Cb, and Cr signals are converted into R, G, and B signals before motion detection.

[0048] 9 is a flowchart showing the operation of the image processing device 1 according to the present disclosure. The basic operation of each functional block of the image processing device 1 has already been described, and therefore will be omitted as appropriate.

[0049] As shown in FIG. 9 , the image processing device 1 first calculates the average luminance of the input image data Din (step S101). Then, the image processing device 1 calculates the maximum value k3dmax of the recursive coefficient k3d and the maximum value k2dmax of the multiplication coefficient k2d corresponding to the calculated average luminance (step S102). For example, if the average luminance of 8-bit image data is Yave, then k3dmax = k2dmax = 1 - Yave / 256. Then, the image processing device 1 calculates a variance v1 representing the luminance distribution of each frame of the image data Din (step S103). Then, the image processing device 1 determines whether only the luminance signal is used to detect the motion signal mv, or whether both the luminance signal and the color signal are used (step S104). Whether only the luminance signal or both the luminance signal and the color signal are used to detect the motion signal mv may be determined by a user instruction or automatically based on the shooting environment.

[0050] In the process of step S104, for example, when not only the luminance signal but also the color signal is used to detect the motion signal mv (YES in step S104), the image processing device 1 sets the first threshold th1, which is used as a criterion for determining whether the variance value v1 is large, to a low value (step S105). Referring to Fig. 8, the image processing device 1 sets the first threshold th1 to a low value, threshold A.

[0051] Here, for example, if the variance value v1 is equal to or greater than the threshold A (YES in step S106), the image processing device 1 sets the absolute value of the slope of each of the cyclic coefficient k3d and the multiplication coefficient k2d for the motion signal mv to L0 (step S107), as shown in Figures 4 and 5. On the other hand, if the variance value v1 is less than the threshold A (NO in step S106), the image processing device 1 sets the absolute value of the slope of each of the cyclic coefficient k3d and the multiplication coefficient k2d for the motion signal mv to H0, which is greater than L0, as shown in Figures 4 and 5 (step S108).

[0052] In the process of step S104, if only the luminance signal is used to detect the motion signal mv (NO in step S104), the image processing device 1 sets a high value to the first threshold th1 used as a criterion for determining whether the variance value v1 is large (step S109). Referring to Fig. 8, the image processing device 1 sets the first threshold th1 to a high value, that is, threshold B.

[0053] Here, for example, if the variance value v1 is equal to or greater than the threshold B (YES in step S110), the image processing device 1 sets the absolute value of the slope of each of the cyclic coefficient k3d and the multiplication coefficient k2d with respect to the motion signal mv to L1 (step S111), as shown in Figures 4 and 5, where L1 > L0. On the other hand, if the variance value v1 is less than the threshold B (NO in step S110), the image processing device 1 sets the absolute value of the slope of each of the cyclic coefficient k3d and the multiplication coefficient k2d with respect to the motion signal mv to H1, which is greater than L1, as shown in Figures 4 and 5, where H1 > H0.

[0054] Thereafter, if the image data Din of the next frame is input (YES in step S113), the image processing device 1 repeats the processing of steps S101 to S113, and if the image data Din of the next frame is not input (NO in step S113), the image processing device 1 terminates image processing.

[0055] As described above, the image processing device 1 according to the present disclosure can reduce the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 for image regions of the image data Din with large motion to suppress the occurrence of afterimages, while increasing the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 to sufficiently remove noise that was not completely removed by the 3DNR circuit 11. Furthermore, the image processing device 1 according to the present disclosure can reduce the weight (multiplication coefficient k2d) of the noise removal process by the 2DNR circuit 12 for image regions of the image data Din with small motion to prevent a decrease in resolution, while increasing the weight (cyclic coefficient k3d) of the noise removal process by the 3DNR circuit 11 to sufficiently remove noise. In other words, the image processing device 1 according to the present disclosure can efficiently remove noise from the image data Din while preventing the occurrence of afterimages, etc.

[0056] Furthermore, the image processing device 1 according to the present disclosure switches the slopes of the recursive coefficient k3d and the multiplication coefficient k2d for the motion signal mv depending on the detection sensitivity of the motion signal mv, thereby enabling the image processing device 1 according to the present disclosure to efficiently remove noise from the image data Din regardless of the contrast of the image data Din.

[0057] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the present disclosure.

[0058] This application claims priority based on Japanese Patent Application No. 2024-097163, filed on June 17, 2024, the disclosure of which is incorporated herein in its entirety by reference.

[0059] The present disclosure can be suitably applied to an image processing device mounted on an image display device or the like.

[0060] REFERENCE SIGNS LIST 1 Image processing device 11 3DNR circuit 12 2DNR circuit 111 Addition / subtraction circuit 112 Frame memory 113 Addition / subtraction circuit 114 Noise extraction unit 115 Motion detection unit 116 Recursive coefficient generation unit 117 Multiplication circuit 118 Variance calculation unit 121 HPF 122 Limiter 123 Coefficient generation unit 124 Multiplication circuit 125 Addition / subtraction circuit

Claims

1. A system comprising: a 3DNR (3 Dimensional Noise Reduction) circuit that performs a first noise removal process on image data; and a 2DNR (2 Dimensional Noise Reduction) circuit that performs a second noise removal process on the image data that has been subjected to the first noise removal process, wherein the 3DNR circuit comprises: a first noise extraction unit that extracts noise from the image data from a difference between frames of the image data; a motion detection unit that detects a motion signal that represents motion between frames of the image data; a first coefficient generation unit that generates a first coefficient that indicates a smaller value as the motion represented by the motion signal increases; a first noise removal unit that removes noise from the image data by subtracting from the image data a first noise removal signal that is generated by multiplying the noise extracted by the first noise extraction unit by the first coefficient; and a variance calculation unit that calculates a variance value that represents the frequency distribution of luminance for each frame of the image data, wherein the 2DNR circuit comprises: a second noise extraction unit that extracts noise from each frame of the image data that has been subjected to the first noise removal process; an image processing device comprising: a second coefficient generation unit that generates a second coefficient that indicates a larger value as the motion represented by the motion signal increases; and a second noise removal unit that removes noise from the image data that has been subjected to the first noise removal process by subtracting a second noise removal signal generated by multiplying the noise extracted by the second noise extraction unit by the second coefficient from the image data, wherein the first coefficient generation unit is configured to set a slope of the first coefficient with respect to the motion signal based on a contrast of the image data that corresponds to the calculated variance value, and the second coefficient generation unit is configured to set a slope of the second coefficient with respect to the motion signal based on the contrast of the image data that corresponds to the calculated variance value.

2. The image processing device of claim 1, wherein the first coefficient generation unit increases the absolute value of the slope of the first coefficient with respect to the motion signal when the contrast is less than a first threshold value compared to when the contrast is equal to or greater than the first threshold value, and the second coefficient generation unit increases the absolute value of the slope of the second coefficient with respect to the motion signal when the contrast is less than the first threshold value compared to when the contrast is equal to or greater than the first threshold value.

3. The image processing device according to claim 1 or 2, wherein the first coefficient generation unit, when the motion detection unit detects the motion signal from a luminance signal representing the luminance of each frame of the image data and a color signal representing the color of each frame of the image data, reduces the absolute value of the slope of the first coefficient with respect to the motion signal compared to when the motion signal is detected from the luminance signal alone; and the second coefficient generation unit, when the motion detection unit detects the motion signal from a luminance signal representing the luminance of each frame of the image data and a color signal representing the color of each frame of the image data, reduces the absolute value of the slope of the second coefficient with respect to the motion signal compared to when the motion signal is detected from the luminance signal alone.

4. The image processing device of claim 1, wherein the first coefficient generation unit increases the maximum value of the first coefficient as the average luminance of the image data decreases, and decreases the maximum value of the first coefficient as the average luminance of the image data increases; and the second coefficient generation unit increases the maximum value of the second coefficient as the average luminance of the image data decreases, and decreases the maximum value of the second coefficient as the average luminance of the image data increases.

5. An image processing method for an image processing device comprising: a 3DNR (3 Dimensional Noise Reduction) circuit that performs a first noise removal process on image data; and a 2DNR (2 Dimensional Noise Reduction) circuit that performs a second noise removal process on the image data that has been subjected to the first noise removal process, wherein in the 3DNR circuit: extracts a first noise that is noise in the image data from a difference between frames of the image data; detects a motion signal that represents movement between frames of the image data; calculates a variance value that represents a frequency distribution of luminance in each frame of the image data; sets a slope of a first coefficient with respect to the motion signal based on the contrast of the image data that corresponds to the calculated variance value, and generates the first coefficient that indicates a smaller value as the movement represented by the motion signal increases; removes noise from the image data by subtracting a first noise removal signal generated by multiplying the first noise by the first coefficient from the image data; and in the 2DNR circuit: an image processing method comprising: extracting second noise, which is noise in the image data, from each frame of the image data that has been subjected to the first noise removal process; setting a slope of a second coefficient with respect to the motion signal based on the contrast of the image data that corresponds to the calculated variance value; generating the second coefficient, which indicates a larger value as the motion represented by the motion signal increases; and subtracting a second noise removal signal generated by multiplying the second noise by the second coefficient from the image data that has been subjected to the first noise removal process, thereby removing noise from the image data.

Images