Image processing apparatus and image processing method
Patent Information
- Application Number
- CN202480002335.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2023-03-15
- Filing Date
- 2024-03-06
- Publication Date
- 2024-12-10
AI Technical Summary
When the existing technology processes videos containing film grains, the noise reduction effect is too strong, resulting in the original film grain characteristics being degraded and unable to be effectively retained.
An image processing device is designed, which includes a noise amount detection part, a film grain detection part and a control part. By detecting the noise amount and film grain intensity in the image, the correction amount of the image processing is determined to ensure that the film grain characteristics are retained while maintaining Image correction.
It achieves the output of retaining the original characteristics of videos containing film grains, avoiding excessive degradation of noise reduction processing, and at the same time fully denoising videos that do not contain film grains and outputs sharp details.
Smart Images
Figure CN119111070A_ABST
Abstract
Description
Image processing device and image processing method
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to Japanese patent application No. 2023-040463, filed with the Japan Patent Office on March 15, 2023, and entitled “Image Processing Device and Image Processing Method,” the entire contents of which are incorporated herein by reference. Technical Field
[0003] Embodiments of the present application relate to an image processing device and an image processing method. Background Art
[0004] Conventionally, there is known a technique for removing noise contained in video input by performing noise reduction processing, thereby enabling viewing of noise-free video.
[0005] Prior art literature
[0006] Patent Literature
[0007] Patent Document 1: Japanese Patent Application Laid-Open No. 2008-35439
[0008] Summary of the Invention
[0009] However, in film contents, producers sometimes intentionally use film grain, a type of noise (the texture of old film), as the original flavor of the image. In such images, the noise reduction effect is not required to be too strong.
[0010] The problem to be solved by the present application is to provide an image processing device and an image processing method that can output an image that retains the original flavor without degrading the characteristics of the film grains, for a video containing film grains.
[0011] An image processing device according to an embodiment corrects an input image. The image processing device includes a noise detection unit, a film grain detection unit, a control unit, and an image correction unit. The noise detection unit determines the noise level of the image. The film grain detection unit determines whether the image contains film grain. The control unit determines the correction amount for image processing based on the determination results of the noise detection unit and the film grain detection unit. The image correction unit performs image processing based on the correction amount determined by the control unit. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] 1 is a diagram showing an example of a hardware configuration of a television device according to an embodiment;
[0013] 2 is a block diagram showing an example of a circuit configuration of an image processing circuit;
[0014] FIG3 is a diagram showing the determination of the presence and intensity of film grains in a film grain detection unit;
[0015] FIG4 is a diagram showing an example of determining the intensity of film grain;
[0016] FIG5 is a flowchart showing the process of determining whether there is film grain and intensity;
[0017] FIG6 is a diagram showing the processing in the noise reduction processing unit;
[0018] FIG7 is a diagram showing processing in a sharpening processing section;
[0019] FIG8 is a graph showing the relationship between the offset value and the intensity of film grain.
[0020] DESCRIPTION OF REFERENCE NUMERALS 10 image processing apparatus, 1001 noise amount detection section, 1002 film grain detection section, 1003 control section, 1004 image correction section, 1005 noise reduction processing section, 1006 sharpening processing section DETAILED DESCRIPTION
[0021] 1 is a diagram showing an example of a hardware configuration of a television set 10 according to an embodiment. The television set 10 is an image processing device that performs image processing on images input from broadcasts, etc. The television set 10 then displays the processed images.
[0022] As shown in Figure 1, the television device 10 has an antenna 101, input terminals 102a~102c, a tuner 103, a demodulator 104, a demultiplexer 105, an A / D (analog / digital) converter 106, a selector 107, a signal processing unit 108, a speaker 109, a display panel 110, an operation unit 111, a light receiving unit 112, an IP communication unit 113, a CPU (Central Processing Unit) 114, a memory 115, a storage 116, a microphone 117 and an audio I / F (interface) 118.
[0023] The antenna 101 receives a digital broadcast signal and supplies the received broadcast signal to the tuner 103 via the input terminal 102 a .
[0024] The tuner 103 selects a broadcast signal of a desired channel from the broadcast signals supplied from the antenna 101 , and supplies the selected broadcast signal to the demodulator 104 .
[0025] The demodulator 104 demodulates the broadcast signal supplied from the tuner 103 and supplies the demodulated broadcast signal to the demultiplexer 105 .
[0026] The demultiplexer 105 separates the broadcast signal supplied from the demodulator 104 to generate a video signal and an audio signal, and supplies the generated video signal and audio signal to the selector 107 .
[0027] The selector 107 selects one signal from the plurality of signals supplied from the demultiplexer 105 , the A / D converter 106 , and the input terminal 102 c , and supplies the selected signal to the signal processing unit 108 .
[0028] The signal processing unit 108 performs predetermined signal processing on the video signal supplied from the selector 107 and supplies the processed video signal to the display panel 110. Furthermore, the signal processing unit 108 performs predetermined signal processing on the audio signal supplied from the selector 107 and supplies the processed audio signal to the speaker 109. The signal processing unit 108 includes the image processing circuit 1000 shown in FIG. 2 .
[0029] The speaker 109 outputs speech or various sounds based on the speech signal supplied from the signal processing unit 108. Furthermore, the speaker 109 changes the volume of the output speech or various sounds based on the control of the CPU 114.
[0030] The display panel 110 as a display unit displays images such as still images and moving images, other images, and text information based on the video signal supplied from the signal processing unit 108 or under the control of the CPU 114 .
[0031] Input terminal 102b receives externally input analog signals such as video and audio signals. Furthermore, input terminal 102c receives externally input digital signals such as video and audio signals. For example, input terminal 102c can receive digital signals from a video recorder equipped with a drive that drives a recording and playback storage medium such as a BD (Blu-ray Disc) to record and play back video.
[0032] The A / D converter 106 supplies a digital signal generated by performing A / D conversion on the analog signal supplied from the input terminal 102 b to the selector 107 .
[0033] The operation unit 111 receives operation input from the user.
[0034] The light receiving unit 112 receives infrared rays from the remote controller 119 .
[0035] The IP communication unit 113 is a communication interface that performs IP (Internet Protocol) communication via the network 40 .
[0036] The CPU 114 controls the entire television device 10 .
[0037] The memory 115 includes a ROM that stores various computer programs executed by the CPU 114 , a RAM that provides a work area for the CPU 114 , and the like.
[0038] The memory 116 is a HDD (Hard Disk Drive) or an SSD (Solid State Drive), etc. The memory 116 records the signal selected by the selector 107 as video data, for example.
[0039] The microphone 117 , which serves as a voice input unit, acquires the voice uttered by the user and transmits it to the audio I / F 118 .
[0040] The audio I / F 118 converts the sound acquired by the microphone 117 into an analog / digital signal and transmits the signal to the CPU 114 as a sound signal.
[0041] Next, the image processing circuit 1000 included in the signal processing unit 108 will be described.
[0042] 2 is a block diagram showing an example of the circuit configuration of the image processing circuit 1000. As shown in FIG2, the image processing circuit 1000 includes a noise amount detection unit 1001, a film grain detection unit 1002, a control unit 1003, and a high-quality image processing unit 1004 functioning as an image correction unit.
[0043] The noise amount detection unit 1001 obtains the degree of noise in the input image input to the image processing circuit 1000 .
[0044] The film grain detection unit 1002 determines whether or not film grain, which is a type of noise, exists in the input image input to the image processing circuit 1000 and whether the intensity of the noise is determined.
[0045] The image quality improvement unit 1004 performs image quality improvement processing on the input image input to the image processing circuit 1000. The image quality improvement unit 1004 includes a noise reduction unit 1005 functioning as a noise removal unit and a sharpening unit 1006 functioning as a sharpening unit.
[0046] When the noise detected by the noise amount detection unit 1001 is large, the control unit 1003 causes the noise reduction processing unit 1005 of the high image quality processing unit 1004 to enhance the noise reduction.
[0047] Here, the determination of the presence and intensity of film grains in the film grain detection unit 1002 will be described.
[0048] Figure 3 illustrates the process of determining the presence and intensity of film grain in the film grain detection unit 1002. As shown in Figure 3, the film grain detection unit 1002 analyzes the input image to the image processing circuit 1000 to determine the degree of film grain. Using machine learning, the film grain detection unit 1002 pre-classifies images as containing film grain and those not containing film grain. More specifically, the film grain detection unit 1002 extracts features from the image, such as film grain, using a CNN (convolutional neural network) and then combines all features to form an activation function.
[0049] The film grain detection unit 1002 outputs the film grain intensity F(x, y, z) based on the output value as shown in FIG3 detected based on the determination result of the presence and intensity of film grain, for example, as follows:
[0050] F(x,y,z)=αx+βy+γz
[0051] It should be noted that the method for measuring film grain can be a combination of any of the following methods.
[0052] SN ratio
[0053] The size of film grain particles (average, maximum)
[0054] Difference with surrounding pixels
[0055] Difference between frames
[0056] Input video frame rate
[0057] In this embodiment, the film grain detection unit 1002 determines the intensity of film grain based on the size and number of film grain particles. FIG4 illustrates an example of determining film grain intensity. As shown in FIG4(a) and (b), the film grain detection unit 1002 determines that the film grain intensity is strong when the film grain ratio per unit area is large (e.g., when the amount of film grain per unit area is large, or when the amount of film grain per unit area is small but the particle size is large). On the other hand, as shown in FIG4(c), the film grain detection unit 1002 determines that the film grain intensity is weak when the film grain ratio per unit area is small (e.g., when the amount of film grain per unit area is small, or when the amount of film grain per unit area is small but the particle size is small).
[0058] Fig. 5 is a flowchart showing the flow of processing for determining the presence or absence and intensity of film grain in the film grain detection unit 1002. As shown in Fig. 5, the film grain detection unit 1002 determines the presence or absence of film grain (step S1).
[0059] When it is determined that film grain exists (YES in step S1 ), the film grain detection unit 1002 measures the noise amount of the film grain (step S2 ) and determines the strength of the film grain (step S3 ).
[0060] When it is determined that there is no film grain (No in step S1 ), the film grain detection unit 1002 sets the setting to no film grain (step S4 ).
[0061] Next, the processing in the noise reduction processing unit 1005 during film grain detection will be described.
[0062] FIG6 illustrates the processing performed by the noise reduction processing unit 1005. As previously mentioned, the control unit 1003 generally controls the noise reduction processing unit 1005 to increase noise reduction when the noise level detection unit 1001 detects a high level of noise. However, when detecting film grain, the control unit 1003 sets an offset value, as shown in FIG6 , to control the noise reduction processing unit 1005 to decrease the amount of noise reduction correction. This is done to output a video that retains the original appeal of film grain.
[0063] Next, the processing in the sharpening processing unit 1006 during film grain detection will be described.
[0064] FIG7 illustrates the processing performed by the sharpening process section 1006. When film grain is detected by the film grain detection section 1002, the control section 1003 sets an offset value to change the degree of processing performed by the sharpening process section 1006. Furthermore, the control section 1003 determines the amount of change (offset value) in the sharpening process section 1006 based on the intensity of the film grain detected by the film grain detection section 1002. Specifically, as shown in FIG7 , the control section 1003 controls the sharpening process section 1006 to reduce the amount of sharpening correction when film grain is detected. This is done to de-emphasize any remaining noise by reducing the amount of noise reduction processing.
[0065] Here, Fig. 8 is a graph showing the relationship between the offset value and the intensity of film grain. As shown in Fig. 8, the control unit 1003 controls such that the larger the intensity of film grain, the larger the offset value and the smaller the noise reduction correction amount and the sharpening correction amount.
[0066] As described above, the image processing apparatus and method of this embodiment perform image processing based on an image processing correction amount determined based on the determination of the presence and intensity of film grain. This allows for outputting an image that retains the original quality of film grain without degrading the characteristics of the film grain, even for images containing film grain. Furthermore, for images that do not contain film grain, noise can be adequately removed, resulting in an image with sharpened details.
[0067] Although embodiments of the present application have been described, these embodiments are merely provided as examples and are not intended to limit the scope of the application. These new embodiments may be implemented in various other ways, and various omissions, substitutions, and modifications may be made without departing from the spirit of the invention. Such embodiments or modifications are included within the scope or spirit of the invention and are within the scope equivalent to the invention described in the claims.
Claims
1. An image processing device, which performs image correction on an input image, wherein: The image processing device comprises: a noise amount detection unit for obtaining the noise amount of the image; a film grain detection unit, which determines whether the image contains film grains; a control section that determines a correction amount of image processing for the image based on determination results of the noise amount detection section and the film grain detection section; as well as The image correction unit performs image processing based on the correction amount determined by the control unit.
2. The image processing device according to claim 1, wherein: The film grain detection unit detects the size and amount of film grain particles. The control section changes the correction amount of the image processing according to the size and amount of the film grains.
3. The image processing device according to claim 1 or 2, wherein: The image correction unit comprises: a noise removal unit for removing noise from the image; and The sharpening part, which emphasizes the outline or texture of the image, The amount of the noise to be removed or the degree of emphasis is changed according to the correction amount determined by the control unit.
4. An image processing method, which is an image processing method in an image processing device for performing image correction on an input image, wherein: The image processing method comprises: a noise amount detection step, in which the noise amount of the image is obtained; a film grain detection step, in which it is determined whether the image contains film grains; a control step of determining a correction amount of image processing for the image based on the determination results of the noise amount detection step and the film grain detection step; and An image correction step is performed in which image processing is performed based on the correction amount determined in the control step.