Image processing method and image processing device using the same

Abstract

The present invention provides an image processing method, comprising the following steps. First, a block size value is obtained. Then, according to the block size value, a first screen area is divided into a plurality of first blocks. Then, according to the block size value, a second screen area is divided into a plurality of second blocks. Then, according to the first pixel information of each first block and the second pixel information of each second block, a noise reduction strength array is obtained. Then, according to the noise reduction strength array, the first screen and the second screen, an output screen is obtained.

Description

Technical Field

[0001] The present invention relates to a processing method and a processing device using the same, and in particular to an image processing method and an image processing device using the same. Background Art

[0002] The captured image is usually affected by the changes in the environment, the camera itself, and the motion of the subject, resulting in pixel information differences from the previous frame, where the pixel information differences may be caused by noise or the motion of the subject. In order to obtain a clearer image, the captured image is usually subjected to noise reduction processing. However, when the noise reduction effect is strong, the pixel information corresponding to the motion of the subject may be noise-reduced together, resulting in a more blurred image after noise reduction; when the noise reduction effect is weak, the noise may not be effectively reduced. Therefore, the industry in this field believes that it is necessary to study a new image processing method and an image processing device using the same to improve the aforementioned known problems. Summary of the invention

[0003] The present invention relates to an image processing method and an image processing device using the same, which can improve the above-mentioned existing problems.

[0004] An embodiment of the present invention provides an image processing method. The image processing method includes the following steps: obtaining a block size value; dividing a first screen area into a plurality of first blocks according to the block size value; dividing a second screen area into a plurality of second blocks according to the block size value; obtaining a noise reduction strength array according to first pixel information of each first block and second pixel information of each second block; obtaining an output screen according to the noise reduction strength array, the first screen and the second screen.

[0005] Another embodiment of the present invention provides an image processing device. The image processing device includes a block size value acquirer and a noise reducer. The block size value acquirer is used to acquire a block size value. The noise reducer is used to: divide a first picture area into a plurality of first blocks according to the block size value; divide a second picture area into a plurality of second blocks according to the block size value; acquire a noise reduction strength array according to a first pixel information of each first block and a second pixel information of each second block; and acquire an output picture according to the noise reduction strength array, the first picture and the second picture.

[0006] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 FIG. 4 is a functional block diagram of an image processing device according to an embodiment of the present invention.

[0008] Figure 2 for Figure 1 Flow chart of an image processing method of an image processing device.

[0009] Figure 3 for Figure 1 FIG. 4 is a block diagram of image processing functions of an image processing device.

[0010] Figure 4 for Figure 3 Schematic diagram of the first screen.

[0011] Figure 5 FIG. 4 is a functional block diagram of an image processing device according to another embodiment of the present invention. DETAILED DESCRIPTION

[0012] The present invention is described in detail below with reference to the accompanying drawings and specific embodiments, but is not intended to limit the present invention.

[0013] Please refer to Figures 1 to 4 , Figure 1 is a functional block diagram of an image processing apparatus 100 according to an embodiment of the present invention. Figure 2 for Figure 1 A flowchart of an image processing method of the image processing device 100, Figure 3 for Figure 1 The image processing function block diagram of the image processing apparatus 100 is shown in FIG. Figure 4 for Figure 3 The first screen F P Schematic diagram of .

[0014] The image processing device 100 includes a block size value acquirer 110 and a noise reducer 120. The image processing device 100 is, for example, a camera, a notebook computer, a desktop computer, a tablet computer, a communication device (eg, a mobile phone) or any other electronic device capable of performing image processing.

[0015] The block size value acquirer 110 and the noise reducer 120 are, for example, physical circuits formed by semiconductor processes, such as semiconductor chips, semiconductor packages, etc. In addition, the block size value acquirer 110 and the noise reducer 120 can be integrated into a single component, or the block size value acquirer 110 and / or the noise reducer 120 can be integrated into a processor or a controller.

[0016] In one embodiment, the block size value acquirer 110 is used to: (a) acquire a block size value BA. The noise reducer 120 is used to: (b) transform the first frame F into a P The area is divided into a plurality of first blocks A P,n×m (c) According to the block size value BA, the second frame FC The area is divided into multiple second blocks A C,n×m (d) according to each first block A P,n×m The first pixel information B P,n×m and a second pixel information B of each second block C,n×m , obtain a noise reduction strength array TNR; and (e) according to the noise reduction strength array TNR, the first picture F P And the second screen F C , obtain an output screen F O Thus, compared with the known method of performing noise reduction on each pixel, the embodiment of the present invention divides the picture area into multiple blocks and processes them individually, which can improve the noise reduction capability and enhance the noise reduction effect. The larger the block size value BA, the higher the noise reduction capability.

[0017] The following Figure 2 The process of the image processing method (or, noise reduction method) is further illustrated by examples.

[0018] In step S110, the block size value acquirer 110 acquires the block size value BA. For example, the block size value acquirer 110 may use the following formula (1) to acquire the block size value BA. In formula (1), α represents an adjustment value of a noise floor value (Noise Floor), which can be any real number equal to or greater than 1, σ represents a noise intensity value, and d C Represents a noise judgment critical value.

[0019] BA ≥ (α × σ) / d c (1)

[0020] The noise intensity value σ is, for example, a standard deviation, and its value can be seen from the first frame F P And the second screen F C The noise intensity value σ may be obtained by measuring the characteristics of the image processing device 100. However, as long as the noise intensity value σ can be obtained, the embodiment of the present invention does not limit the method of measuring the characteristics of the image processing device 100.

[0021] Noise judgment critical value d C For example, the change of the pixel information of the picture belongs to (or is biased towards) the threshold value of noise (Noise) or pixel dynamic change, where the "pixel dynamic change" is caused by the motion of the photographed object (Motion). In one embodiment, the noise judgment threshold value d C It can be the pixel information difference value. C For example, when the first screen F P The pixel brightness of the second picture F CWhen the difference in pixel brightness of the first frame exceeds 16 levels (for example, the grayscale value is 255 levels in total), it means that the change in the pixel information of the second frame may be caused by the dynamic change of the pixel; P The pixel brightness of the second picture F C When the difference in pixel brightness does not exceed 16 steps, it indicates that the change in pixel information of the second picture may be caused by noise. The embodiment of the present invention does not limit the noise judgment threshold value d C The value of may depend on the subject and / or shooting environment.

[0022] Adjustment value α, noise intensity value σ and noise judgment threshold value d C It can be obtained or set in advance before executing the image processing process. When the shooting environment and / or the camera remain unchanged, the adjustment value α, the noise intensity value σ and / or the noise judgment threshold value d C Can remain unchanged.

[0023] In step S120, the noise reducer 120 transforms the first frame F into P The area is divided into N×M first blocks A P,n×m , where subscript n is a positive integer between 1 and N, and subscript m is a positive integer between 1 and M. Each first block A P,n×m The resolution is BA×BA.

[0024] like Figure 4 As shown, BA is equal to 8 (ie, each first block A P,n×m The size (or resolution) is 8×8) and the first frame F P The resolution is 1920×1080. For example, the noise reducer 120 determines the first block A according to the following equations (2) and (3): P,n×m The number of, where R and Q represent the first picture F P The resolution (R×Q) is obtained by N being equal to 240 (calculation formula: 1920 / 8=240), and M being equal to 135 (calculation formula: 1080 / 8=135).

[0025] N=R / BA (2)

[0026] M=Q / BA (3)

[0027] In step S130, the noise reducer 120 obtains each first block A P,n×m The first average brightness value of a plurality of first pixels (eg, BA pixels) is obtained, and the first average brightness value is used as the first pixel information B P,n×m In other words, each first block A P,n×m With the first pixel information B P,n×m Represents the entire first block A P,n×m Block information.

[0028] In step S140, the noise reducer 120 converts the second frame F into C The area is divided into N×M second blocks A C,n×m BA is equal to 8 (ie, each second block A C,n×m The size (or resolution) of the second screen is 8×8) and the second screen F C For example, the resolution of the second block A is 1920×1080. The noise reducer 120 determines the second block A according to the above formulas (2) and (3). C,n×m The number of the second screen F C The resolution is the same as R×Q.

[0029] First Screen F P For example, the previous frame, and the second frame F C For example, it is the current frame. The previous frame may be a frame that has been subjected to noise reduction processing or a frame that has not been subjected to noise reduction processing. The image processing method for the previous and next frames of the present invention belongs to the time domain noise reduction method.

[0030] In step S150, the noise reducer 120 obtains each second block A C,n×m The second average brightness value of the plurality of second pixels is obtained, and the second average brightness value is used as the second pixel information B C,n×m In other words, each second block A C,n×m With the second pixel information B C,n×m Represents the entire second block A C,n×m Block information.

[0031] In step S160, the noise reducer 120 may P,n×m The first pixel information B P,n×m And each second block A C,n×m The second pixel information B C,n×m , obtain the noise reduction strength array TNR.

[0032] For example, if Figure 3 As shown, the noise reducer 120 obtains each first block A P,n×m The first pixel information B P,n×m The corresponding second block A C,n×m The second pixel information B C,n×m The one-pixel information difference value d n×m , and use these pixel information difference values ​​d n×m A plurality of noise reduction strength values ​​T as a noise reduction strength array TNR r×q , wherein the subscript r is a positive integer between 1 and R, and the subscript q is a positive integer between 1 and Q.

[0033] In one embodiment, if Figure 3 As shown, the noise reducer 120 can calculate the difference value d of each pixel information. n×m Perform low-pass filtering and / or signal enhancement processing to obtain pixel information difference value d' n×m For low-pass filtering, the noise reducer 120 can filter out the pixel information difference value d n×m The noise (low frequency) part belongs to the noise (low frequency) part. In terms of signal enhancement processing, the noise reducer 120 can enhance the pixel information difference value d n×m The signal part is the part other than the noise. After processing, the noise reducer 120 generates N×M pixel information difference values ​​d' n×m .

[0034] In one embodiment, if Figure 3 As shown, the noise reducer 120 can calculate the pixel information difference value d' n×m The enlargement process is performed to obtain the same image as the first image F P Or the second screen F C The noise reduction strength array TNR of the same size (or resolution). For example, if the resolution is 1920×1080, the noise reducer 120 is based on N×M d' n×m At least one of the above, obtain R×Q noise reduction strength values ​​T r×q , and with R×Q noise reduction strength values ​​T r×q Composed of the noise reduction strength array TNR.

[0035] In step S170, the noise reducer 120 generates a noise reduction signal according to the noise reduction strength array TNR, the first frame F P And the second screen F C , obtain an output screen F O For example, the noise reducer 120 obtains the output image F according to the following formula (4): O .

[0036] B O,r×q =(1-T r×q )×B P,r×q +(T r×q ×B C,r×q ) (4)

[0037] In formula (4), T r×q represents the r×qth noise reduction strength value of the noise reduction strength array TNR, B P,r×q Indicates the first screen F P The pixel information of the r×qth pixel, and B C,r×q Indicates the second screen F C The pixel information of the r×qth pixel. Noise reduction strength value T r×q , pixel information B P,r×q and pixel information BC,r×q is known information, so the noise reducer 120 can obtain the output image F according to formula (4) O The pixel information B of the r×qth pixel O,r×q .

[0038] Noise reduction intensity value T r×q Any real number between 0 and 1. From formula (4), we can see that when the noise reduction intensity value T r×q The higher the value, the higher the possibility (degree) of dynamic change of pixel information. O The pixel information B of the r×qth pixel O,r×q The closer to the second frame F C The pixel information B of r×q pixels C,r×q (i.e., the lower the noise reduction degree); on the contrary, the lower the possibility (degree) of dynamic change of pixel information, the lower the output image F O The pixel information B of the r×qth pixel O,r×q The closer to the first screen F P The pixel information B of r×q pixels P,r×q (i.e., the greater the degree of noise reduction).

[0039] When all (R×Q) pixel information B is obtained O,r×q After that, the output screen F is completed O .

[0040] Then, the noise reducer 120 outputs the picture F O Output to a display (not shown) to display the output screen F O Alternatively, the noise reducer 120 or the controller outputs the picture F O Replace the second screen F C When the image processing apparatus 100 processes another frame (eg, the next frame or the previously processed output frame F O ) when performing noise reduction processing, Figure 2 The first screen F P Can output screen F O (after noise reduction) is substituted, and the second picture F C The other frame can be substituted. In another embodiment, Figure 2 The first screen F P You can use the second screen F C (without noise reduction) is substituted, and Figure 2 The second screen F C Substitute the other frame.

[0041] The image processing device 100 can process a stream image according to the above process, which includes multiple frames, wherein the noise reduction method of the current frame of any two adjacent frames can be substituted into Figure 2 The second screen F C , and the previous frame can be substituted into Figure 2 The first screen F P , wherein the previous frame may be a frame that has undergone the same noise reduction processing or a frame that has not undergone the noise reduction processing.

[0042] Please refer to Figure 5 , which is a functional block diagram of an image processing apparatus 100 according to another embodiment of the present invention. The image processing apparatus 100 can obtain a plurality of noise reduction strength arrays TNR according to a plurality of different block size values ​​BA. ,t , proceed to the second screen F C Noise reduction processing. Figure 5 Each functional block in the virtual box corresponds to Figure 3 The functional blocks within the virtual box R1.

[0043] For example, if Figure 5 As shown, the image processing device 100 performs the same image processing flow as the above with the block size value BA equal to 1, and obtains the corresponding noise reduction strength array TNR ,1 The image processing device 100 performs the same image processing flow as above with the block size value BA equal to 2, and obtains the corresponding noise reduction strength array TNR ,2 The image processing device 100 performs the same image processing flow as above with the block size BA equal to t, and obtains the corresponding noise reduction strength array TNR ,t t is, for example, any positive integer. The noise reducer 120 may be configured based on a plurality of noise reduction strength arrays TNR ,t Get a comprehensive noise reduction strength array TNR ,T For example, the noise reducer 120 converts the plurality of noise reduction strength arrays TNR ,t Add together to obtain a comprehensive noise reduction strength array TNR ,T Then, the noise reducer 120 uses the above formula (4) according to the comprehensive noise reduction strength array TNR ,T , First Screen F P And the second screen F C , get the output screen F O In another embodiment, the integrated noise reduction strength array TNR ,T The method of obtaining is not limited to multiple noise reduction strength arrays TNR ,t The addition of multiple noise reduction intensity arrays TNR can also be performed by other mathematical operations (such as addition, subtraction, multiplication, division, average or a combination thereof) or equation operations. ,t , and obtain the comprehensive noise reduction strength array TNR ,T In addition, the embodiment of the present invention does not limit the noise reduction strength array TNR ,tThe total number of , which can be any positive integer equal to or greater than 2. In addition, Figure 5 The block size BA is not limited by the above formula (1). In other words, Figure 5 Each block size value BA can be greater than, equal to, or less than ((α×σ) / d C ) 2 The numerical value of .

[0044] Of course, the present invention may have many other embodiments. Without departing from the spirit and essence of the present invention, those skilled in the art may make various corresponding changes and modifications based on the present invention, but these corresponding changes and modifications should all fall within the scope of protection of the claims attached to the present invention.

Claims

1. An image processing method, characterized in that: include: Get a block size value; According to the block size value, a first frame area is divided into a plurality of first blocks; Dividing a second frame area into a plurality of second blocks according to the block size value; Obtaining a noise reduction strength array according to a first pixel information of each of the first blocks and a second pixel information of each of the second blocks; According to different values ​​of the block size, a plurality of noise reduction strength arrays are obtained; Obtaining a comprehensive noise reduction strength array of the plurality of noise reduction strength arrays; as well as An output frame is obtained according to the integrated noise reduction strength array, the first frame and the second frame.

2. The image processing method according to claim 1, characterized in that: The block size value satisfies the following formula: BA≥(α×σ) / d c ; Where BA is the block size, α is the adjustment value of the noise floor, σ is the noise intensity, and d C is a noise judgment critical value.

3. The image processing method according to claim 1, characterized in that: The image processing method further comprises: Obtaining a first average brightness value of a plurality of first pixels of each of the first blocks, and using the first average brightness value as the first pixel information; and Obtaining a second average brightness value of a plurality of second pixels of each of the second blocks, and using the second average brightness value as the second pixel information; The step of obtaining the noise reduction strength array includes: A pixel information difference value between the first pixel information of each first block and the second pixel information of the corresponding second block is obtained, and the pixel information difference values ​​are used as a plurality of noise reduction strength values ​​of the noise reduction strength array.

4. The image processing method according to claim 3, characterized in that: The image processing method further comprises: A low-pass filter is performed on the pixel information difference values.

5. The image processing method according to claim 3, characterized in that: The image processing method further comprises: A signal enhancement process is performed on the pixel information difference values.

6. The image processing method according to claim 3, characterized in that: The image processing method further comprises: The pixel information difference values ​​are amplified, and the amplified pixel information difference values ​​are used as the multiple noise reduction strength values ​​of the noise reduction strength array.

7. An image processing device, characterized in that: include: A block size value acquirer, used to: acquire a block size value; as well as A noise reducer, for: According to the block size value, a first frame area is divided into a plurality of first blocks; Dividing a second frame area into a plurality of second blocks according to the block size value; Obtaining a noise reduction strength array according to a first pixel information of each of the first blocks and a second pixel information of each of the second blocks; According to different values ​​of the block size, a plurality of noise reduction strength arrays are obtained; Obtaining a comprehensive noise reduction strength array of the plurality of noise reduction strength arrays; as well as An output frame is obtained according to the integrated noise reduction strength array, the first frame and the second frame area.

8. The image processing device according to claim 7, wherein: The block size value satisfies the following formula: BA≥(α×σ) / d c ; Where BA is the block size, α is the adjustment value of the noise floor, σ is the noise intensity, and d C is a noise judgment critical value.

9. The image processing device according to claim 7, wherein: The Denoiser is also used to: Obtaining a first average brightness value of a plurality of first pixels of each of the first blocks, and using the first average brightness value as the first pixel information; as well as Obtaining a second average brightness value of a plurality of second pixels of each of the second blocks, and using the second average brightness value as the second pixel information; A pixel information difference value between the first pixel information of each first block and the second pixel information of the corresponding second block is obtained, and the pixel information difference values ​​are used as a plurality of noise reduction strength values ​​of the noise reduction strength array.

10. The image processing device according to claim 9, wherein: The Denoiser is also used to: A low-pass filter is performed on the pixel information difference values.

11. The image processing device according to claim 9, wherein: The Denoiser is also used to: A signal enhancement process is performed on the pixel information difference values.

12. The image processing device according to claim 9, wherein: The Denoiser is also used to: The pixel information difference values ​​are amplified, and the amplified pixel information difference values ​​are used as the multiple noise reduction strength values ​​of the noise reduction strength array.

Images