Noise reduction and band limiting device and program

Abstract

To perform noise elimination and bandwidth limitation while suppressing deterioration of image quality.SOLUTION: A noise elimination and bandwidth limitation apparatus 1 comprises: a spatio-temporal frequency decomposition unit 11 for performing frequency decomposition on a moving image in the spatio-temporal direction and generating spatio-temporal frequency-resolved components; a static-or-moving area discriminating unit 12 for discriminating whether each spatial phase position is a static area or a moving area by comparing the spatio-temporal frequency-resolved component with a threshold; a degeneration processing unit 14 for degenerating a component value of a temporal high-frequency band to approximately 0 at the spatial phase position determined as the static area by the static-or-moving area determination unit 12 to generate a degenerated spatio-temporal frequency-resolved component; and a spatio-temporal frequency reconstruction unit 15 for performing frequency reconstruction in the spatio-temporal direction on the degenerated spatio-temporal frequency-resolved components.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a noise elimination and band limiting device and program. [Background technology]

[0002] A known noise removal technique is BM3D noise removal, which is characterized by weighted averaging of similar image signals between consecutive frame images of a video sequence using 3D block matching (BM) (see, for example, Non-Patent Document 1). Another known technique is noise removal using wavelet shrinkage processing in the spatial direction (see, for example, Non-Patent Document 2). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] K. Dabov, A. Foi, and V. Katkovnik, “Image denoising by sparse 3-D transform-domain collaborative filtering,” IEEE Transactions on Image Processing, vol. 16, no. 8, pp. 2080-2095, 2007. [Non-patent document 2] DL Donoho, IM Johnstone, G. Kerkyacharian, and D. Picard, “Wavelet shrinkage: asymptopia?,” Journal of the Royal Statistical Society, vol. 57, no. 2, pp. 301-471, 1995. Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional techniques, when simultaneously processing noise reduction and band limiting, do not take into account both spatiotemporal frequency information and static and dynamic information, which can lead to a problem of image quality degradation. For example, in the signal components of moving areas in moving images, the power of the temporal high-frequency band decreases according to the speed of motion. Therefore, when the band of such moving images is further suppressed, there is a risk of significant degradation of subjective image quality because the temporal sampling frequency of many existing video formats is significantly lower than the limit of human perception.

[0005] In view of the above circumstances, an object of the present invention is to provide a noise reduction and band limitation device and program that can perform noise reduction and band limitation while suppressing deterioration of image quality. [Means for solving the problem]

[0006] In order to solve the above problems, a noise elimination and band limitation device according to one embodiment performs noise elimination and band limitation on an input moving image, and includes: a spatiotemporal frequency decomposition unit that performs frequency decomposition on the moving image in the spatiotemporal direction to generate spatiotemporal frequency resolved components; a still / motion region discrimination unit that discriminates whether the spatiotemporal frequency resolved components are a still region or a moving region for each spatial phase position by comparing the spatiotemporal frequency resolved components with a threshold; a degeneration processing unit that degenerates component values ​​of a temporal high frequency band to approximately zero at spatial phase positions that have been discriminated as still regions by the still / motion region discrimination unit to generate degenerated spatiotemporal frequency resolved components; and a spatiotemporal frequency reconstruction unit that performs frequency reconstruction on the degenerated spatiotemporal frequency resolved components in the spatiotemporal direction. The still / motion region discriminator discriminates a spatial phase position as a still region when a temporal high frequency band component is equal to or less than a first threshold and a temporal low frequency / spatial high frequency band component is equal to or more than a second threshold at the spatial phase position, and discriminates a spatial phase position other than the still region as a moving region. do.

[0008] Furthermore, in one embodiment, the present invention further includes a noise power calculation unit that calculates an average value of power within the highest spatiotemporal frequency band of the spatiotemporal frequency decomposition components as average noise power, and the degeneration processing unit may degenerate the component value of the temporal high-frequency band to approximately 0 when the power of the temporal high-frequency band is less than the average noise power at a spatial phase position that has been determined to be a moving region by the still / motion region determination unit.

[0009] Furthermore, in one embodiment, the degeneration processing unit performs the following at a spatial phase position determined as a moving region by the still / moving region determining unit: The aforementioned If the power of the temporal low frequency and spatial high frequency bands is less than the average noise power, the component values ​​of the temporal low frequency and spatial high frequency bands may be degenerated to approximately 0, and if the power is equal to or greater than the average noise power, the component values ​​of the temporal low frequency and spatial high frequency bands may be degenerated by a predetermined ratio.

[0010] Furthermore, in one embodiment, the degeneration processing unit performs the following at a spatial phase position determined as a still region by the still / motion region determining unit: The aforementioned If the power of the temporal low frequency and spatial high frequency bands is less than the average noise power, the component values ​​of the temporal low frequency and spatial high frequency bands may be degenerated to approximately 0, and if the power is equal to or greater than the average noise power, the component values ​​of the temporal low frequency and spatial high frequency bands may be degenerated by a predetermined ratio.

[0011] Furthermore, in one embodiment, the noise power calculation unit may obtain the maximum value of power within the temporal highest frequency band as the maximum noise power, and the degeneration processing unit may perform degeneration more weakly as the component value of the temporal high frequency band becomes larger when the power of the temporal high frequency band is equal to or greater than the average noise power and less than the maximum noise power at a spatial phase position determined to be a motion region by the still / motion region determination unit.

[0012] Furthermore, in one embodiment, at a spatial phase position determined to be a motion region by the still / motion region determination unit, when the power of the temporal low frequency / spatial high frequency band is equal to or greater than the average noise power and less than the maximum noise power, the larger the component value of the temporal low frequency / spatial high frequency band, the weaker the degeneration may be.

[0013] Furthermore, in one embodiment, at a spatial phase position determined to be a still region by the still / motion region determination unit, when the power of the temporal low frequency / spatial high frequency band is equal to or greater than the average noise power and less than the maximum noise power, the larger the component value of the temporal low frequency / spatial high frequency band, the weaker the degeneration may be.

[0014] Furthermore, in one embodiment, the spatiotemporal frequency decomposition unit may perform wavelet packet decomposition on the video image in the spatiotemporal direction.

[0015] In order to solve the above problem, a program according to one embodiment causes a computer to function as the noise elimination and band limiting device. [Effects of the Invention]

[0016] According to the present invention, it is possible to perform noise removal and band limitation while suppressing deterioration of image quality. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a block diagram showing an example of the configuration of a noise elimination and band limiting device according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing the application of first-order wavelet packet decomposition in the spatiotemporal direction to an image frame. [Figure 3] FIG. 10 is a diagram showing frequency bands divided by first-order wavelet packet decomposition in the spatiotemporal direction. [Figure 4] 3 is a diagram illustrating the processing of a noise power calculation unit in the noise reduction and band limiting device according to the first embodiment. FIG. [Figure 5] 4 is a flowchart showing an example of the operation of the noise reduction and band limiting device according to the first embodiment. [Figure 6] FIG. 10 is a block diagram showing an example of the configuration of a noise elimination and band limiting device according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0018] Hereinafter, the embodiments will be described in detail with reference to the drawings.

[0019] First Embodiment Fig. 1 is a block diagram showing an example of the configuration of a noise elimination and band limitation device according to the first embodiment. The noise elimination and band limitation device 1 shown in Fig. 1 includes a spatio-temporal frequency decomposition unit 11, a still / motion region discrimination unit 12, a noise power calculation unit 13, a degeneration processing unit 14, and a spatio-temporal frequency reconstruction unit 15. The noise elimination and band limitation device 1 is a device that simultaneously performs noise elimination and band limitation on a video signal.

[0020] 1 constitute a control and arithmetic circuit (controller). The control and arithmetic circuit may be constituted by dedicated hardware such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field-Programmable Gate Array), or may be constituted by a processor, or may be constituted by including both.

[0021] The spatiotemporal frequency decomposition unit 11 performs frequency decomposition on the input moving image I(t) in the spatiotemporal direction to generate spatiotemporal frequency decomposition components in which frequency bands are decomposed in the spatiotemporal direction. The spatiotemporal frequency decomposition unit 11 then outputs the generated spatiotemporal frequency decomposition components to the still / motion region discrimination unit 12 and the degeneration processing unit 14. The spatiotemporal frequency decomposition unit 11 also outputs at least the spatiotemporal highest frequency band component of the spatiotemporal frequency decomposition components to the noise power calculation unit 13.

[0022] For example, the spatiotemporal frequency decomposition unit 11 performs wavelet packet decomposition processing in the spatiotemporal direction on video frames of a length determined by considering the decomposition order and the wavelet filter tap length (for example, 2 in the case of a Haar wavelet) from the image frame I(t0) at the time position t0 to be processed. The wavelet packet decomposition decomposes the input image into octaves on the low-frequency band side and the high-frequency band side. In this case, the overall size in the spatial and temporal directions is the same as the size of the image frames of the input video.

[0023] Figure 2 shows the application of first-order wavelet packet decomposition to the image frame I(t0) at time position t0 in the input video sequence. In this case, the frequency band of the input image is divided into the time direction (temporal frequency f t direction) into a temporal low frequency band L and a temporal high frequency band H, each of which is further divided into a spatial direction (horizontal frequency f h Directional and vertical frequency f v direction).

[0024] Figure 3 shows the frequency bands separated by first-order wavelet packet decomposition in the spatiotemporal direction. The temporal low-frequency band L is divided into four parts: the spatiotemporal lowest frequency band LL(L), the temporal low-frequency, horizontal low-frequency, vertical high-frequency band LH(L), the temporal low-frequency, horizontal high-frequency, vertical low-frequency band HL(L), and the temporal low-frequency, horizontal high-frequency, vertical high-frequency band HH(L). Here, the bands LH(L), HL(L), and HH(L) within the temporal low-frequency band L, excluding the spatiotemporal lowest frequency band LL(L), are referred to as the temporal low-frequency, spatial high-frequency band H(L). Here, (L) refers to the temporal low-frequency band, and (H) refers to the temporal high-frequency band. Similarly, the temporal high frequency band H is divided into four parts: a temporal high frequency, horizontal low frequency, vertical low frequency band LL(H), a temporal high frequency, horizontal low frequency, vertical high frequency band LH(H), a temporal high frequency, horizontal high frequency, vertical low frequency band HL(H), and a spatiotemporal highest frequency band HH(H). Note that the number of decomposition orders is not limited to one. When the spatiotemporal frequency decomposition unit 11 performs n-order wavelet packet decomposition on the input image in the spatiotemporal direction, the frequency band of the input image is divided into two in the temporal direction. n It is divided into 4 in the spatial direction. n It is divided.

[0025] The still-motion region discriminator 12 compares the spatiotemporal frequency decomposition components input from the spatiotemporal frequency decomposition unit 11 with a threshold value to discriminate between a still region and a moving region for each spatial phase position of the image frame I(t0) to be processed, and generates still-motion region information indicating the discrimination result. The still-motion region discriminator 12 then outputs the generated still-motion region information to the reduction processor 14.

[0026] Generally, in a still region, the power of the temporal high frequency band H is approximately 0. On the other hand, in a moving region, power tends to decrease except in the lowest spatiotemporal frequency band LL(L) due to motion blur, so the power of the temporal low frequency / spatial high frequency band H(L) is likely to be smaller than in a still region. Utilizing this characteristic, the still / moving region discriminator 12 determines whether the power of the temporal high frequency band H at an arbitrary spatial phase position exceeds the first threshold t H The power of the temporal low frequency and spatial high frequency band H(L) is equal to or less than the second threshold t H(L) If the above conditions are met, the spatial phase position is determined to be a still region, and spatial phase positions other than the still region are determined to be moving regions.The still / moving region determination unit 12 then outputs the results of the determination process performed on all spatial phase positions as still / moving region information.

[0027] First threshold t H and the second threshold t H(L) may be a predetermined value or may be dynamically determined by any method. H and the second threshold t H(L) Alternatively, the still / motion region discriminator 12 may dynamically determine the first threshold t based on the power in the lowest spatiotemporal frequency band LL(L). H is set to 1% of the maximum power at each element position in the spatiotemporal lowest frequency band LL(L), and the second threshold t H(L) may be set to 15% of the maximum value of the power at each element position in the lowest spatiotemporal frequency band LL(L).

[0028] The noise elimination / band-limiting device 1 does not need to include the noise power calculation unit 13. In this case, the degeneration processing unit 14 generates degenerated spatio-temporal frequency decomposition components by degenerating the component values ​​of the temporal high-frequency band H to approximately 0 (less than a predetermined threshold) at spatial phase positions determined to be still regions by the still / motion region determination unit 12. The degeneration processing unit 14 does not perform degeneration or limits the amount of degeneration at other spatial phase positions. The degeneration processing unit 14 then outputs the degenerated spatio-temporal frequency decomposition components to the spatio-temporal frequency reconstruction unit 15. When such processing is performed, degradation of image quality can be suppressed by not performing or limiting degeneration in moving regions, and noise elimination and band limitation can be performed on the temporal high-frequency band components of still regions.

[0029] In the following, a case where the noise elimination and band limiting device 1 includes a noise power calculation unit 13 will be described.

[0030] FIG. 4 is a diagram illustrating the processing of the noise power calculation unit 13. The noise power calculation unit 13 receives the spatiotemporal highest frequency band HH(H) from the spatiotemporal frequency decomposition unit 11. As shown in FIG. 4, the noise power calculation unit 13 calculates noise power using the components in the spatiotemporal highest frequency band HH(H), and in this embodiment, calculates the average value of the power of all components in the spatiotemporal highest frequency band HH(H) as the average noise power n1. The noise power calculation unit 13 then outputs the calculated average noise power n1 to the degeneration processing unit 14. Note that in this embodiment, the absolute value of the component value is used as the power.

[0031] The degeneration processing unit 14 performs degeneration processing on the spatio-temporal frequency decomposition components input from the spatio-temporal frequency decomposition unit 11 based on the still / motion region information input from the still / motion region discrimination unit 12 and the average noise power n1 input from the noise power calculation unit 13, to generate degenerated spatio-temporal frequency decomposition components. Then, the degeneration processing unit 14 outputs the degenerated spatio-temporal frequency decomposition components to the spatio-temporal frequency reconstruction unit 15. ... For example, the degeneration processing unit 14 performs degeneration processing on the spatio-temporal frequency decomposition components based on the horizontal band-limiting frequency f hs is half the horizontal sampling frequency, and the vertical band limit frequency f vsis half the vertical sampling frequency, and the band limit frequency in the time direction is f ts is set to half the sampling frequency in the time direction, and the bandwidth limiting amount v s is set to 0.1 (corresponding to an attenuation of 20 dB). The degeneration processing unit 14 may store these values ​​in advance, or may obtain them from the outside.

[0032] Generally, random noise components such as thermal noise have almost constant power in all spatiotemporal frequency bands. Also, in the signal components of still regions in a moving image, the power in the high-frequency temporal band is almost zero. Therefore, noise components are dominant in the high-frequency temporal band of the still region. Therefore, the degeneration processing unit 14 degenerates the time frequency f t is the band limit frequency f ts The component values ​​of the time high frequency band H are degenerated to approximately zero.

[0033] The degeneration processing unit 14 also degenerates the time frequency f t is the band limit frequency f ts Less than horizontal frequency f h is the band limit frequency f hs and vertical frequency f v is the band limit frequency f vs The component value x of the temporal low frequency / spatial high frequency band H(L) that is equal to or greater than this value is degenerated to y in accordance with equation (1). That is, for a spatial phase position determined to be a still region, the degeneration processing unit 14 degenerates the component value x of the temporal low frequency / spatial high frequency band H(L) to approximately 0 if the power of the temporal low frequency / spatial high frequency band H(L) is less than the average noise power n1, and degenerates the component value x of the temporal low frequency / spatial high frequency band H(L) by a predetermined ratio if the power of the temporal low frequency / spatial high frequency band H(L) is equal to or greater than the average noise power n1.

[0034]

number

[0035] On the other hand, in the signal components of the motion region in the motion image, the power of the temporal high frequency band decreases according to the motion speed. Therefore, if the temporal high frequency band of the motion image is further suppressed, there is a risk of significant deterioration in subjective image quality, since the sampling frequency in the time direction of many existing motion image formats is significantly lower than the limit of human perception. Therefore, the degeneration processing unit 14 degenerates the temporal frequency f t is the band limit frequency f ts The component value x of the temporal high-frequency band H that is equal to or greater than this value is degenerated according to equation (2). That is, at a spatial phase position determined to be a moving region, if the power of the temporal high-frequency band H is less than the average noise power n1, the degeneration processing unit 14 degenerates the component value x of the temporal high-frequency band H to approximately 0, and if the power of the temporal high-frequency band H is equal to or greater than the average noise power n1, the degeneration processing unit 14 does not degenerate the component value x of the temporal high-frequency band H.

[0036]

number

[0037] In addition, the degeneration processing unit 14 degenerates the time frequency f t is the band limit frequency f ts Less than horizontal frequency f h is the band limit frequency f hs and vertical frequency f v is the band limit frequency f vs The component value x of the temporal low frequency / spatial high frequency band H(L) that is equal to or greater than this value is degenerated in accordance with equation (1). That is, at a spatial phase position determined to be a motion region, the degeneration processing unit 14 degenerates the component value x of the temporal low frequency / spatial high frequency band H(L) to approximately 0 if the power of the temporal low frequency / spatial high frequency band H(L) is less than the average noise power n1, and degenerates the component value x of the temporal low frequency / spatial high frequency band H(L) by a predetermined ratio if the power of the temporal low frequency / spatial high frequency band H(L) is equal to or greater than the average noise power n1.

[0038] In both still and moving regions, the lowest spatiotemporal frequency band LL(L) is not degenerated. If x is a negative value in equations (1) and (2), y is set to a negative value. The degeneration result y in the above example is summarized in Table 1.

[0039] [Table 1]

[0040] The spatiotemporal frequency reconstruction unit 15 performs frequency reconstruction (wavelet packet reconstruction) in the spatiotemporal direction on the condensed spatiotemporal frequency decomposition components input from the condensation processing unit 14, generates a moving image that has been subjected to noise elimination and band limitation, and outputs it to the outside of the noise elimination and band limitation device 1.

[0041] Next, the operation of the noise elimination and band limiting device 1 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing an example of the operation of the noise elimination and band limiting device 1.

[0042] In step S101, the spatiotemporal frequency decomposition unit 11 performs frequency decomposition on the video image in the spatiotemporal direction to generate spatiotemporal frequency decomposition components.

[0043] In step S102, the still / motion region discriminator 12 compares the spatiotemporal frequency decomposition components with a threshold value to discriminate whether the region is a still region or a motion region for each spatial phase position.

[0044] In step S103, the noise power calculation unit 13 calculates the average value of the power in the highest spatiotemporal frequency band of the spatiotemporal frequency decomposition components as the average noise power.

[0045] In step S104, the degeneration processing unit 14 performs degeneration of the spatiotemporal frequency decomposition components excluding the spatiotemporal lowest frequency band LL(L) in accordance with Table 1 above.

[0046] In step S105, the spatio-temporal frequency reconstruction unit 15 performs frequency reconstruction in the spatio-temporal direction on the degenerated spatio-temporal frequency decomposition components to generate a moving image that has been subjected to noise removal and band limitation.

[0047] As described above, the noise reduction and band-limiting device 1 according to this embodiment performs frequency decomposition on video in the spatiotemporal direction and discriminates between still and moving regions. Then, at spatial phase positions discriminated as still regions, the temporal high-frequency band components are degenerated to approximately zero. In other words, the noise reduction and band-limiting device 1 performs degeneration processing taking into account both spatiotemporal frequency information and still and moving region information, making it possible to perform noise reduction and band-limiting of video with high overall image quality. Furthermore, by using the noise reduction and band-limiting device 1 in the pre-stage of video encoding, it is possible to reduce the difficulty of encoding and improve the encoded image quality.

[0048] Furthermore, the noise elimination and band limiting device 1 may determine, at any spatial phase position, a still region when the temporal high-frequency band component is equal to or less than a first threshold and the temporal low-frequency and spatial high-frequency band component is equal to or greater than a second threshold, and determine spatial phase positions other than the still region as moving regions. This makes it possible to determine whether a region is still or moving with high accuracy.

[0049] Alternatively, the noise elimination / bandwidth limiting device 1 may calculate the average value of the power in the spatiotemporal highest frequency band as the average noise power n1, and perform the degeneration process based on a comparison between the spatiotemporal frequency decomposition components and the average noise power n1. This makes it possible to determine whether the spatiotemporal frequency decomposition components are noise components, and to appropriately eliminate noise.

[0050] <Second embodiment> Next, a noise elimination and band limiting device according to a second embodiment will be described. FIG. 6 is a block diagram showing an example of the configuration of the noise elimination and band limiting device according to the second embodiment. The noise elimination and band limiting device 2 shown in FIG. 6 includes a spatiotemporal frequency decomposition unit 11, a still / motion region discrimination unit 12, a noise power calculation unit 13′, a degeneration processing unit 14′, and a spatiotemporal frequency reconstruction unit 15. The noise elimination and band limiting device 2 according to the second embodiment differs from the noise elimination and band limiting device 1 according to the first embodiment in the processing of the noise power calculation unit 13′ and the degeneration processing unit 14′. Since the other processing is the same, a description thereof will be omitted.

[0051] The noise power calculation unit 13' receives the spatiotemporal highest frequency band HH(H) from the spatiotemporal frequency decomposition unit 11, calculates the average value of the power of all components in the spatiotemporal highest frequency band HH(H) as average noise power n1, and calculates the maximum value of the power in the spatiotemporal highest frequency band HH(H) as maximum noise power n2. The noise power calculation unit 13' then outputs the calculated average noise power n1 and maximum noise power n2 to the degeneration processing unit 14'.

[0052] The degeneration processing unit 14' performs degeneration processing on the spatiotemporal frequency resolved components input from the spatiotemporal frequency decomposition unit 11 based on the still / motion region information input from the still / motion region discrimination unit 12 and the average noise power n1 and maximum noise power n2 input from the noise power calculation unit 13' to generate degenerated spatiotemporal frequency resolved components. The degeneration processing unit 14' then outputs the degenerated spatiotemporal frequency resolved components to the spatiotemporal frequency reconstruction unit 15.

[0053] The concept of the process of the erosion processing unit 14' is the same as that of the erosion processing unit 14 of the first embodiment. That is, the erosion processing unit 14' erodes the time frequency f t is the band limit frequency f ts The component values ​​of the above-mentioned temporal high-frequency band H are degenerated to approximately 0. Furthermore, in order to perform the degeneration process based on the average noise power n1 and the maximum noise power n2, equation (3) is used instead of equation (1), and equation (4) is used instead of equation (2).

[0054]

number

[0055] That is, as shown in the example of equation (3), at a spatial phase position determined to be a still region by the still / motion region determination unit 12, the degeneration processing unit 14′ degenerates the component value x of the temporal low-frequency / spatial high-frequency band H(L) to approximately 0 when the power of the band H(L) is less than the average noise power n1; when the power of the band H(L) is equal to or greater than the average noise power n1 and less than the maximum noise power n2, the degeneration processing unit 14′ degenerates the component value x of the band H(L) more weakly as the band H(L) becomes larger; and when the power of the band H(L) is equal to or greater than the maximum noise power n2, the degenerates the component value x of the band H(L) by a predetermined ratio.

[0056] Furthermore, at a spatial phase position determined to be a moving region by the still / moving region determination unit 12, as exemplified in equation (4), the degeneration processing unit 14′ degenerates the component value x of the temporal high-frequency band H to approximately 0 when the power of the temporal high-frequency band H is less than the average noise power n1, degenerates the component value x of the temporal high-frequency band H more weakly the larger it is when the power of the temporal high-frequency band H is equal to or greater than the average noise power n1 and less than the maximum noise power n2, and does not degenerate the component value x of the temporal high-frequency band H when the power of the temporal high-frequency band H is equal to or greater than the maximum noise power n2.

[0057] Furthermore, at a spatial phase position determined to be a moving region by the still / motion region determination unit 12, as exemplified in equation (3), the degeneration processing unit 14′ degenerates the component value x of the temporal low-frequency / spatial high-frequency band H(L) to approximately 0 when the power of the band H(L) is less than the average noise power n1; when the power of the band H(L) is equal to or greater than the average noise power n1 and less than the maximum noise power n2, the degeneration processing unit 14′ degenerates the component value x of the band H(L) more weakly as the band H(L) becomes larger; and when the power of the band H(L) is equal to or greater than the maximum noise power n2, the degenerates the component value x of the band H(L) by a predetermined ratio.

[0058] The degeneration result y in the above example is summarized in Table 2.

[0059] [Table 2]

[0060] As described above, the noise elimination / bandwidth limiting device 2 according to this embodiment calculates the average value and maximum value of the power in the highest spatiotemporal frequency band as the average noise power n1 and the maximum noise power n2, respectively, and performs degeneration processing based on a comparison between the spatiotemporal frequency decomposition component and the average noise power n1 and the maximum noise power n2. This not only achieves the effects of the noise elimination / bandwidth limiting device 1, but also makes it possible to control the amount of degeneration in stages and further suppress deterioration in image quality.

[0061] <Program> A computer capable of executing program instructions can be used to function as the above-described noise reduction and band limiting devices 1 and 2. Here, the computer may be a general-purpose computer, a dedicated computer, a workstation, a PC (Personal Computer), an electronic notepad, etc. The program instructions may be program code, code segments, etc. for performing the necessary tasks.

[0062] The computer includes a processor, a storage unit, an input unit, an output unit, and a communication interface. The processor may be a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), a DSP (Digital Signal Processor), an SoC (System on a Chip), or the like, and may be configured with multiple processors of the same or different types. The processor reads and executes programs from the storage unit to control the above components and perform various arithmetic processing. Note that at least a portion of these processing contents may be implemented by hardware. The input unit is an input interface that accepts user input operations and acquires information based on the user operations, such as a pointing device, keyboard, or mouse. The output unit is an output interface that outputs information, such as a display or speaker. The communication interface is an interface for communicating with external devices.

[0063] The program may be recorded on a computer-readable recording medium. Using such a recording medium, the program can be installed on a computer. Here, the recording medium on which the program is recorded may be a non-transitory recording medium. The non-transitory recording medium is not particularly limited, and may be, for example, a CD-ROM, a DVD-ROM, or a USB (Universal Serial Bus) memory. Furthermore, the program may be downloaded from an external device via a network.

[0064] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications or alterations can be made without departing from the scope of the claims. For example, it is possible to integrate multiple building blocks shown in the block diagrams of the embodiments, or to divide one building block. [Explanation of symbols]

[0065] 1,2 Noise reduction and band limiting device 11. Space-time frequency decomposition section 12 Static motion area discrimination unit 13,13' Noise power calculation section 14,14' Degeneration processing section 15. Space-time frequency reconstruction unit

Claims

1. A noise elimination and band limitation device that performs noise elimination and band limitation on an input moving image, a spatio-temporal frequency decomposition unit that performs frequency decomposition on the video in the spatio-temporal direction to generate spatio-temporal frequency decomposition components; a still / dynamic region discrimination unit that discriminates whether the region is a still region or a dynamic region for each spatial phase position by comparing the spatiotemporal frequency decomposition component with a threshold; a degeneration processing unit that degenerates component values ​​of a temporal high frequency band to approximately 0 at a spatial phase position determined to be a still region by the still / motion region determining unit, thereby generating degenerated spatio-temporal frequency decomposition components; a space-time frequency reconstruction unit that performs frequency reconstruction in the space-time direction on the degenerated space-time frequency decomposition components; Equipped with The still / motion region discrimination unit discriminates a spatial phase position as a still region when, at any spatial phase position, the temporal high-frequency band component is below a first threshold and the temporal low-frequency / spatial high-frequency band component is above a second threshold, and discriminates spatial phase positions other than the still region as motion regions.

2. a noise power calculation unit that calculates an average value of power in a highest frequency band of the spatiotemporal frequency decomposition components as an average noise power; 2. The noise removal and band limiting device according to claim 1, wherein the degeneration processing unit degenerates the component values ​​of the temporal high-frequency band to approximately 0 when the power of the temporal high-frequency band is less than the average noise power at a spatial phase position determined to be a motion region by the still / motion region determination unit.

3. 3. The noise removal and band limiting device according to claim 2, wherein the degeneration processing unit degenerates the component values ​​of the temporal low-frequency and spatial high-frequency bands to approximately 0 when the power of the temporal low-frequency and spatial high-frequency bands is less than the average noise power at a spatial phase position determined to be a motion region by the still / motion region determination unit, and degenerates the component values ​​of the temporal low-frequency and spatial high-frequency bands by a predetermined ratio when the power of the temporal low-frequency and spatial high-frequency bands is equal to or greater than the average noise power.

4. 4. The noise removal and band limiting device according to claim 2, wherein the degeneration processing unit degenerates the component values ​​of the temporal low-frequency and spatial high-frequency bands to approximately 0 when the power of the temporal low-frequency and spatial high-frequency bands is less than the average noise power at a spatial phase position determined to be a still region by the still / motion region determination unit, and degenerates the component values ​​of the temporal low-frequency and spatial high-frequency bands by a predetermined ratio when the power of the temporal low-frequency and spatial high-frequency bands is equal to or greater than the average noise power.

5. the noise power calculation unit calculates a maximum value of power within the spatiotemporal highest frequency band as a maximum noise power; 5. The noise removal and band limiting device according to claim 2, wherein the degeneration processing unit performs degeneration more weakly as the component value of the temporal high-frequency band increases when the power of the temporal high-frequency band is equal to or greater than the average noise power and less than the maximum noise power at a spatial phase position determined to be a motion region by the still / motion region determination unit.

6. 6. The noise removal and band limiting device according to claim 5, wherein the degeneration processing unit performs degeneration more weakly as the component values ​​of the temporal low frequency and spatial high frequency bands become larger when the power of the temporal low frequency and spatial high frequency bands is equal to or greater than the average noise power and less than the maximum noise power at a spatial phase position determined to be a moving region by the still / motion region determination unit.

7. 7. The noise removal and band limiting device according to claim 5, wherein the degeneration processing unit performs degeneration more weakly as the component values ​​of the temporal low frequency and spatial high frequency bands become larger when the power of the temporal low frequency and spatial high frequency bands is equal to or greater than the average noise power and less than the maximum noise power at a spatial phase position determined to be a still region by the still / motion region determination unit.

8. The noise reduction and band limiting device according to claim 1 , wherein the spatiotemporal frequency decomposition unit performs wavelet packet decomposition on the video image in the spatiotemporal direction.

9. A program for causing a computer to function as the noise elimination and band limiting device according to any one of claims 1 to 8.

Images