Image processing apparatus and image processing method

Abstract

To provide an image processing device and an image processing method, capable of outputting an image that brings out an original texture, with respect to a video including a film grain, without degrading a feature of the film grain.SOLUTION: The image processing device is an image processing device for performing image correction on an input image, and includes a noise amount measurement unit, a film grain detection unit, a control unit, and an image correction unit. The noise amount measurement unit solves for a noise amount of the image. The film grain detection unit determines whether film grain is included in the image. The control unit determines a correction amount for image processing for the image on the basis of determination results of the noise amount measurement unit and the film grain detection unit. The image correction unit performs image processing on the basis of the correction amount determined in the control unit.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Embodiments of the present invention relate to an image processing apparatus and an image processing method.

Background Art

[0002] Conventionally, there has been known a technique for removing noise included in video input by noise reduction processing so that noise-free video can be viewed.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in film content, etc., there are cases where the producer intentionally uses film grain (the texture of old movie film), which is a kind of noise, as the flavor of the video, and in such videos, it is required not to apply noise reduction too strongly.

[0005] The problem to be solved by the present invention is to provide an image processing apparatus and an image processing method capable of outputting an image that makes use of the original flavor without degrading the characteristics of film grain for a video including film grain.

Means for Solving the Problems

[0006] The image processing apparatus according to the embodiment is an image processing apparatus that corrects an input image, and includes a noise amount detection unit, a film grain detection unit, a control unit, and an image correction unit. The noise amount detection unit obtains the noise amount of the image. The film grain detection unit determines whether the image contains film grain. The control unit determines the correction amount of the image processing for the image based on the determination results of the noise amount detection unit and the film grain detection unit. The image correction unit performs image processing based on the correction amount determined by the control unit. The film grain detection unit detects the size and amount of film grain particles. The control unit changes the correction amount of the image processing according to the size and amount degree of the film grain particles.

Brief Description of the Drawings

[0007]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Embodiments for Carrying Out the Invention

[0008] FIG. 1 is a diagram showing an example of the hardware configuration of a television apparatus 10 according to an embodiment. The television apparatus 10 is an image processing apparatus that executes image processing on an image input by broadcast or the like. Then, the television apparatus 10 displays the image on which the image processing has been executed.

[0009] As shown in FIG. 1, the television apparatus 10 includes an antenna 101, input terminals 102a to 102c, a tuner 103, a demodulator 104, a demultiplexer 105, an A / D (analog / digital) converter 106, a selector 107, a signal processing unit 108, a speaker 109, a display panel 110, an operation unit 111, a light receiving unit 112, an IP communication unit 113, a CPU (Central Processing Unit) 114, a memory 115, a storage 116, a microphone 117, and an audio I / F (interface) 118.

[0010] The antenna 101 receives a broadcast signal of digital broadcasting and supplies the received broadcast signal to the tuner 103 via the input terminal 102a.

[0011] The tuner 103 selects a broadcast signal of a desired channel from the broadcast signal supplied from the antenna 101 and supplies the selected broadcast signal to the demodulator 104.

[0012] The demodulator 104 demodulates the broadcast signal supplied from the tuner 103 and supplies the demodulated broadcast signal to the demultiplexer 105.

[0013] The demultiplexer 105 separates the broadcast signal supplied from the demodulator 104 to generate a video signal and an audio signal, and supplies the generated video signal and audio signal to the selector 107.

[0014] The selector 107 selects one from a plurality of signals supplied from the demultiplexer 105, the A / D converter 106, and the input terminal 102c, and supplies the selected one signal to the signal processing unit 108.

[0015] The signal processing unit 108 performs predetermined signal processing on the video signal supplied from the selector 107 and supplies the processed video signal to the display panel 110. Also, the signal processing unit 108 performs predetermined signal processing on the audio signal supplied from the selector 107 and supplies the processed audio signal to the speaker 109. The signal processing unit 108 includes the image processing circuit 1000 shown in FIG. 2.

[0016] The speaker 109 outputs sound or various sounds based on the audio signal supplied from the signal processing unit 108. Also, the speaker 109 changes the volume of the output sound or various sounds based on the control by the CPU 114.

[0017] The display panel 110 as the display unit displays video such as still images and moving images, other images, and character information, etc. based on the video signal supplied from the signal processing unit 108 or based on the control by the CPU 114.

[0018] The input terminal 102b receives analog signals such as video signals and audio signals input from the outside. Also, the input terminal 102c receives digital signals such as video signals and audio signals input from the outside. For example, the input terminal 102c can receive digital signals from a recorder or the like equipped with a drive device for driving a recording medium for recording and playback such as a BD (Blu-ray (registered trademark) Disc).

[0019] The A / D converter 106 supplies the digital signal generated by performing A / D conversion on the analog signal supplied from the input terminal 102b to the selector 107.

[0020] The operation unit 111 receives the operation input of the user.

[0021] The light receiving unit 112 receives infrared rays from the remote controller 119.

[0022] The IP communication unit 113 is a communication interface for performing IP (Internet Protocol) communication via the network 40.

[0023] The CPU 114 controls the entire television apparatus 10.

[0024] The memory 115 is a ROM that stores various computer programs executed by the CPU 114, a RAM that provides a working area for the CPU 114, and the like.

[0025] The storage 116 is an HDD (Hard Disk Drive), an SSD (Solid State Drive), or the like. The storage 116 records, for example, the signal selected by the selector 107 as recording data.

[0026] The microphone 117 as a voice input unit acquires the voice uttered by the user and sends it to the audio I / F 118.

[0027] The audio I / F 118 converts the voice acquired by the microphone 117 from analog to digital and sends it to the CPU 114 as a voice signal.

[0028] Next, the image processing circuit 1000 included in the signal processing unit 108 will be described.

[0029] FIG. 2 is a block diagram showing an example of the circuit configuration of the image processing circuit 1000. As shown in FIG. 2, the image processing circuit 1000 includes a noise amount detection unit 1001, a film grain detection unit 1002, a control unit 1003, and a high image quality processing unit 1004 that functions as an image correction unit.

[0030] The noise amount detection unit 1001 obtains the degree of noise for the input image input to the image processing circuit 1000.

[0031] The film grain detection unit 1002 determines the presence, absence, strength, and weakness of film grain, which is a type of noise, for the input image input to the image processing circuit 1000.

[0032] The high - quality processing unit 1004 performs high - quality processing on the input image input to the image processing circuit 1000. The high - quality processing unit 1004 includes a noise reduction processing unit 1005 that functions as a noise removal unit and a sharpness processing unit 1006 that functions as a sharpness enhancement unit.

[0033] When there is a large amount of noise detected by the noise amount detection unit 1001, the control unit 1003 strengthens the noise reduction for the noise reduction processing unit 1005 of the high - quality processing unit 1004.

[0034] Here, the determination of the presence, absence, and strength of film grain in the film grain detection unit 1002 will be described.

[0035] Figure 3 is a diagram showing the determination of the presence, absence, and strength of film grain in the film grain detection unit 1002. As shown in Figure 3, the film grain detection unit 1002 analyzes the input image input to the image processing circuit 1000 and determines the degree of film grain contained therein. The film grain detection unit 1002 classifies videos with film grain and videos without film grain in advance by machine learning. More specifically, the film grain detection unit 1002 performs feature extraction such as film grain extraction from the image by a CNN (convolutional neural network), and then fully connects and uses an activation function.

[0036] Based on the determination result of the presence, absence, and strength of film grain, the film grain detection unit 1002 outputs the strength F(x, y, z) of the film grain according to the output value shown in Figure 3, for example, as the following formula. F(x,y,z)=αx + βy + γz

[0037] Note that the film grain measurement method may be a combination of any of the following. · Signal - to - noise ratio · Size of film grain particles (average, maximum) · Difference from surrounding pixels · Difference between frames · Frame rate of the input video

[0038] In the present embodiment, the film grain detection unit 1002 determines the strength of the film grain based on the size and amount of the film grain particles. FIG. 4 is a diagram showing an example of determining the strength of the film grain. As shown in FIGS. 4(a) and 4(b), when the ratio of the film grain occupying per unit area is large (when the amount of film grain per unit area is large, or when the particle size is large even if the amount of film grain per unit area is not so large), the film grain detection unit 1002 determines that the strength of the film grain is strong. On the other hand, as shown in FIG. 4(c), when the ratio of the film grain occupying per unit area is small (when the amount of film grain per unit area is small, or when the particle size is small even if the amount of film grain per unit area is not so large), the film grain detection unit 1002 determines that the strength of the film grain is weak.

[0039] FIG. 5 is a flowchart showing the processing flow of determining the presence, strength, and weakness of film grain in the film grain detection unit 1002. As shown in FIG. 5, the film grain detection unit 1002 determines the presence or absence of film grain (step S1).

[0040] If it is determined that there is film grain (Yes in step S1), the film grain detection unit 1002 measures the noise amount of the film grain (step S2) and determines the strength and weakness of the film grain (step S3).

[0041] If it is determined that there is no film grain (No in step S1), the film grain detection unit 1002 assumes that there is no film grain (step S4).

[0042] Next, the processing in the noise reduction processing unit 1005 at the time of film grain detection will be described.

[0043] Here, FIG. 6 is a diagram showing the processing in the noise reduction processing unit 1005. As described above, generally, when there is a large amount of noise detected by the noise amount detection unit 1001, the control unit 1003 executes control to strengthen the noise reduction for the noise reduction processing unit 1005. However, when detecting film grain, as shown in FIG. 6, the control unit 1003 sets an offset value and performs control to reduce the noise reduction correction amount for the noise reduction processing unit 1005. This is to output an image with the flavor of film grain remaining.

[0044] Next, the processing in the sharpness processing unit 1006 when detecting film grain will be described.

[0045] Here, FIG. 7 is a diagram showing the processing in the sharpness processing unit 1006. When film grain is detected by the film grain detection unit 1002, the control unit 1003 sets an offset value and changes the degree of processing in the sharpness processing unit 1006. Also, the control unit 1003 determines the amount of change (offset value) in the processing in the sharpness processing unit 1006 according to the strength of the film grain detected by the film grain detection unit 1002. That is, as shown in FIG. 7, when detecting film grain, the control unit 1003 controls the sharpness processing unit 1006 to reduce the sharpness correction amount. This is to prevent emphasizing the remaining noise for the amount by which the noise reduction processing is weakened.

[0046] Here, FIG. 8 is a diagram showing the relationship between the offset value and the strength of film grain. As shown in FIG. 8, the control unit 1003 controls to increase the offset value as the strength of the film grain increases, and to reduce the noise reduction correction amount and the sharpness correction amount.

[0047] Thus, according to the image processing apparatus and the image processing method of the present embodiment, by performing image processing based on the correction amount of the image processing determined based on the determination result of the presence, strength, and weakness of film grain, it is possible to output an image that brings out the original flavor without degrading the characteristics of the film grain for a video including film grain. Also, for a video not including film grain, it is possible to output an image with sufficiently removed noise and sharpened details.

[0048] Although the embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. This embodiment and its modifications are included in the scope and gist of the invention, and are also included in the invention described in the claims and its equivalent scope.

Explanation of Reference Numerals

[0049] 10 Image processing apparatus 1001 Noise amount detection unit 1002 Film grain detection unit 1003 Control unit 1004 Image correction unit 1005 Noise removal unit 1006 Sharpening unit

Claims

1. An image processing apparatus that corrects an input image, comprising: a noise amount detection unit that obtains the amount of noise in the image; a film grain detection unit that determines whether the image contains film grain; a control unit that determines a correction amount for image processing on the image based on the determination results of the noise amount detection unit and the film grain detection unit; an image correction unit that performs image processing based on the correction amount determined by the control unit; wherein the film grain detection unit detects the size and amount of film grain particles; the control unit changes the correction amount of the image processing according to the degree of the size and amount of the film grain particles. An image processing apparatus characterized by the above.

2. The image correction unit includes a noise removal unit that removes noise in the image; a sharpness enhancement unit that enhances the contour or texture of the image; wherein the amount of noise to be removed or the degree of enhancement is changed according to the correction amount determined by the control unit. The image processing apparatus according to claim 1, characterized by the above.

3. An image processing method in an image processing apparatus that corrects an input image, comprising: a noise amount detection step of obtaining the amount of noise in the image; a film grain detection step of determining whether the image contains film grain; a control step of determining a correction amount for image processing on the image based on the determination results of the noise amount detection step and the film grain detection step; an image correction step of performing image processing based on the correction amount determined by the control step; wherein the film grain detection step detects the size and amount of film grain particles; the control step changes the correction amount of the image processing according to the degree of the size and amount of the film grain particles. An image processing method characterized by the above.

Images