Image processing method and device, terminal and readable storage medium

Abstract

The present application provides an image processing method, device, terminal and readable storage medium. The image processing method comprises: a first step: obtaining an initial image, the initial image comprises a plurality of blocks of the same size, each block comprising at least one pixel. A second step: taking any one of the blocks as a target block, obtaining weight values ​​of a plurality of adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image and the noise level of the target block, the adjacent blocks being blocks surrounding and connected to the target block, and the size of the extended image being the same as the size of the initial image. A third step: performing noise reduction processing on the target block according to a plurality of weight values. Traverse all blocks in the initial image, take each block as a target block once, and repeat the second and third steps to obtain a noise-reduced image.

Description

Technical Field

[0001] The present application relates to the field of electronic technology, and more specifically, to an image processing method, an image processing device, a terminal, and a non-volatile computer-readable storage medium. Background Art

[0002] In the Image Signal Process (ISP) system, the central area of ​​the convex lens has a much greater light-gathering ability than its edge area, which results in stronger light in the central area of ​​the sensor than in the edge area. Therefore, in the ISP system, there is a Lens Shading Correction (LSC) module to increase the brightness of the image corners so that the brightness of the entire image is almost the same. However, after the corners are brightened by the LSC module, the noise will be more obvious.

[0003] The current optimization method for corner noise is to adjust the pixel noise reduction strength by calculating the distance between the pixel and the center of the image. This method can effectively suppress the noise at the corners, but this method also has the following problems: it is necessary to set the noise reduction / sharpening strength at different distances to suppress the corner noise. However, the ideal threshold value will be different in different scenes or under different shooting conditions. Therefore, this method requires debugging of many scenes to obtain a relatively ideal threshold value. When collecting data, some special scenes will be encountered, such as some scenes with large brightness flashes, in which the noise at the corners will be extremely obvious. The parameters obtained through debugging of a few specific scenes are difficult to adapt to all scenes, resulting in excessive or weak noise reduction / sharpening in some special scenes. Summary of the invention

[0004] Embodiments of the present application provide an image processing method, an image processing device, a terminal, and a non-volatile computer-readable storage medium.

[0005] The image processing method of the embodiment of the present application includes: a first step: obtaining an initial image, wherein the initial image includes a plurality of blocks of the same size, each of which includes at least one pixel. A second step: taking any one of the blocks as a target block, and obtaining weight values ​​of a plurality of adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image and the noise level of the target block, wherein the adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as that of the initial image. A third step: performing noise reduction processing on the target block according to the plurality of weight values. Traverse all the blocks in the initial image, take each of the blocks as the target block once, and repeat the second and third steps to obtain a noise-reduced image.

[0006] The image processing device of the embodiment of the present application includes a noise reduction module. The noise reduction module includes a first acquisition unit, a second acquisition unit, a noise reduction unit and a third acquisition unit. The first acquisition unit is used to perform the first step: acquiring an initial image, the initial image includes a plurality of blocks of the same size, each of which includes at least one pixel. The second acquisition unit is used to perform the second step: taking any one of the blocks as a target block, and obtaining the weight values ​​of a plurality of adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image and the noise level of the target block, the adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as the size of the initial image. The noise reduction unit is used to perform the third step: performing noise reduction processing on the target block according to the plurality of weight values. The third acquisition unit is used to traverse all the blocks in the initial image, take each of the blocks as the target block once, and repeat the second step and the third step to obtain a noise-reduced image.

[0007] The terminal of the embodiment of the present application includes one or more processors, a memory and one or more programs. Among them, one or more of the programs are stored in the memory and executed by one or more of the processors, and the program includes instructions for executing the image processing method described in the embodiment of the present application. The image processing method includes: a first step: obtaining an initial image, the initial image includes a plurality of blocks of the same size, each of which includes at least one pixel. A second step: taking any one of the blocks as a target block, obtaining the weight values ​​of a plurality of adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image and the noise level of the target block, the adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as the size of the initial image. A third step: performing noise reduction processing on the target block according to the plurality of weight values. Traverse all the blocks in the initial image, take each of the blocks as the target block once, and repeat the second step and the third step to obtain a noise-reduced image.

[0008] The non-volatile computer-readable storage medium of the embodiment of the present application includes a computer program. When the computer program is executed by one or more processors, the processor performs the following image processing method: Step 1: Acquire an initial image, wherein the initial image includes multiple blocks of the same size, each of which includes at least one pixel. Step 2: Take any one of the blocks as a target block, and obtain the weight values ​​of multiple adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image and the noise level of the target block. The adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as that of the initial image. Step 3: Perform noise reduction processing on the target block according to the multiple weight values. Traverse all the blocks in the initial image, take each block as the target block once, and repeat the second and third steps to obtain a denoised image.

[0009] In the image processing method, image processing device, terminal and non-volatile computer-readable storage medium of the present application, the target block is subjected to noise reduction processing according to the gain data corresponding to the target block in the extended image, the noise level of the target block and the weight values ​​of multiple adjacent blocks of the target block, and all blocks in the initial image are traversed to perform noise reduction processing on each block in the initial image to obtain a noise-reduced image. Among them, the gain data in the extended image corresponds to the block in the initial image, that is, each block in the initial image can find the corresponding gain data in the extended image, so as to effectively regulate the noise reduction intensity of different blocks in the initial image, thereby more specifically suppressing the noise of brightening the dark area, better protecting the details in the initial image, and avoiding the phenomenon of uneven noise reduction in the central area and the corner area of ​​the noise-reduced image.

[0010] Additional aspects and advantages of the embodiments of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the embodiments of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0012] Figure 1 is a flowchart of an image processing method of certain embodiments of the present application;

[0013] Figure 2 is a schematic diagram of the structure of an image processing device in some embodiments of the present application;

[0014] Figure 3 is a schematic diagram of the structure of a terminal in some implementation modes of the present application;

[0015] Figure 4 is a schematic diagram of an initial image and an expanded image of an image processing method in certain embodiments of the present application;

[0016] Figure 5 is a flowchart of an image processing method of certain embodiments of the present application;

[0017] Figure 6 It is a schematic diagram of the principle of obtaining a first ratio of weight values ​​of adjacent blocks in the image processing method in some embodiments of the present application;

[0018] Figures 7 and 8 is a flowchart of an image processing method of certain embodiments of the present application;

[0019] Fig. 9 It is a schematic diagram of the principle of performing noise reduction processing on a target block in an initial image to obtain a noise-reduced image by an image processing method in certain embodiments of the present application;

[0020] Figures 10 to 12 is a flowchart of an image processing method of certain embodiments of the present application;

[0021] Fig.13 It is a schematic diagram of the principle of sharpening the detail layer in the denoised image by the image processing method of certain embodiments of the present application;

[0022] Fig.14 is a flowchart of an image processing method of certain embodiments of the present application;

[0023] Fig.15 It is a schematic diagram of the principle of obtaining an extended image according to GR channel data and GB channel data in an image processing method in certain embodiments of the present application;

[0024] Fig.16 is a flowchart of an image processing method of certain embodiments of the present application;

[0025] Fig.17 is an expanded diagram in the image processing method in certain embodiments of the present application;

[0026] Fig.18 It is a connection diagram of a non-volatile computer-readable storage medium and a processor in certain embodiments of the present application. DETAILED DESCRIPTION

[0027] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals represent the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the embodiments of the present application, and cannot be understood as limiting the embodiments of the present application.

[0028] See also Figure 1 and Figure 4 , the embodiment of the present application provides an image processing method, the image processing method comprising:

[0029] 01: First step: obtaining an initial image P0, the initial image P0 includes a plurality of blocks of the same size, each block including at least one pixel;

[0030] 02: Step 2: Take any one of the blocks as the target block, and obtain the weight values ​​of multiple adjacent blocks around the target block according to the gain data corresponding to the target block in the extended graph A and the noise level of the target block. The adjacent blocks are blocks surrounding and connected to the target block. The size of the extended graph is the same as that of the initial image P0.

[0031] 03: The third step: performing noise reduction processing on the target block according to multiple weight values; and

[0032] 04: Traverse all blocks in the initial image P0, take each block as a target block, and repeat the second and third steps to obtain the denoised image P1 ( Figure 6 shown).

[0033] Please combine Figure 2The present application also provides an image processing device 10, the image processing device 10 includes a noise reduction module 11, the noise reduction module 11 includes a first acquisition unit 111, a second acquisition unit 113, a noise reduction unit 115 and a third acquisition unit 117. The first acquisition unit 111 is used to execute the method in 01, that is, the first acquisition unit 111 is used to execute the first step: acquire an initial image, the initial image P0 includes multiple blocks of the same size, and each block includes at least one pixel. The second acquisition unit 113 is used to execute the method in 02, that is, the second acquisition unit 113 is used to execute the second step: take any one of the blocks as a target block, and acquire weight values ​​of multiple adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image A and the noise level of the target block. The adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image A is the same as that of the initial image P0. The denoising unit 115 is used to execute the method in 03, that is, the denoising unit 115 is used to execute the third step: denoising the target block according to multiple weight values. The third acquisition unit 117 is used to execute the method in 04, that is, the third acquisition unit 117 is used to traverse all blocks in the initial image P0, take each block as a target block, and repeat the second step and the third step to obtain the denoised image P1.

[0034] Please combine Figure 3 The present application also provides a terminal 100, which includes one or more processors 30, a memory 50 and one or more programs. Among them, one or more programs are stored in the memory 50 and executed by one or more processors 30, and the program includes instructions for executing the image processing method of the embodiment of the present application. That is, when one or more processors 30 execute the program, the processor 30 can implement the methods in 01, 02, 03 and 04. That is, one or more processors 30 are used to perform: the first step: obtain the initial image P0, the initial image P0 includes a plurality of blocks of the same size, and each block includes at least one pixel. The second step: take any one of the blocks as the target block, and obtain the weight values ​​of the plurality of adjacent blocks around the target block according to the gain data corresponding to the target block in the extended image A and the noise level of the target block. The adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image A is the same as the size of the initial image P0. The third step: perform noise reduction processing on the target block according to the plurality of weight values. All blocks in the initial image P0 are traversed, each block is used as a target block, and the second step and the third step are repeatedly performed to obtain a denoised image P1.

[0035] Specifically, the terminal 100 may include but is not limited to a mobile phone, a laptop computer, a smart TV, a tablet computer, a smart watch, or a computer. The image processing device 10 may be an integration of functional modules integrated in the terminal 100. This application only takes the case where the terminal 100 is a mobile phone as an example for explanation. When the terminal 100 is other types of devices, the situation is similar to that of a mobile phone and will not be described in detail.

[0036] In the image signal processing (ISP) system, since the central area of ​​the convex lens has a much greater light-gathering ability than its edge area, the light in the central area of ​​the sensor is stronger than that in the edge area. Therefore, in the ISP system, there is a lens shading correction (LSC) module to improve the brightness of the image corners so that the brightness of the whole image is almost the same. However, after the corner area is brightened by the LSC module, the noise will be more obvious. The current optimization method for corner noise is to calculate the distance between the pixel point and the center point of the image, thereby adjusting the pixel point noise reduction intensity. This method can effectively suppress the corner noise, but this method also has the following problems: it is necessary to set the noise reduction / sharpening intensity at different distances to suppress the corner noise. However, the ideal threshold value will be different in different scenes or different shooting conditions. Therefore, this method requires many scenes to be debugged to obtain a relatively ideal threshold value. When collecting data, some special scenes will be encountered, such as some scenes with large brightness flickering, and the corner noise will be abnormally obvious. The parameters obtained through debugging in a few specific scenes are difficult to adapt to all scenes, resulting in excessive or weak noise reduction / sharpening in some special scenes.

[0037] In the image processing method, image processing device 10, terminal 100 and non-volatile computer-readable storage medium 200 of the present application, the target block is subjected to noise reduction processing according to the gain data corresponding to the target block in the extended image A, the noise level of the target block and the weight values ​​of multiple adjacent blocks of the target block, and all blocks in the initial image P0 are traversed to perform noise reduction processing on each block in the initial image P0 to obtain the noise reduction image P1. Among them, the gain data in the extended image A corresponds to the blocks in the initial image A, that is, each block in the initial image P0 can find the corresponding gain data in the extended image A, so that the noise reduction intensity of different blocks in the initial image P0 can be effectively adjusted, so as to suppress the noise of brightening the dark area more specifically, better protect the details in the initial image P0, and avoid the phenomenon of uneven noise reduction in the central area and the corner area of ​​the noise reduction image P1.

[0038] In method 01, the first acquisition unit 111 or the processor 30 acquires an initial image P0, wherein the initial image P0 includes a plurality of blocks of the same size, and each block includes at least one pixel. After the first acquisition unit 111 or the processor 30 acquires the initial image P0, the initial image P0 may be divided into a plurality of blocks of the same size. For example, the initial image P0 may be divided into 3×3 blocks, 4×4 blocks, 5×5 blocks, or 6×6 blocks, without limitation. Specifically, the number of blocks may be set according to the distribution of bright and dark areas in the initial image P0 and the size of the dark areas, so that the multiple weight values ​​acquired by the second acquisition unit 113 or the processor 30 can suppress the noise in the dark areas more specifically. For example, Figure 4 As shown, the initial image P0 includes 16 blocks, and the 16 blocks are P0(0,0), P0(0,1), P0(0,2), P0(0,3), P0(1,0), P0(1,1), P0(1,2), P0(1,3), P0(2,0), P0(2,1), P0(2,2), P0(2,3), P0(3,0), P0(3,1), P0(3,2), P0(3,3), and the blocks include one pixel, two pixels, three pixels, four pixels, or more than four pixels.

[0039] In method 02, any block in the initial image P0 is used as the target block, and the weight values ​​of multiple adjacent blocks of the target block are obtained according to the gain data corresponding to the target block in the extended image A and the noise level of the target block. The adjacent blocks are blocks surrounding and connected to the target block. Figure 4 As shown, the target block is block P0(0,0) in the initial image P0, and the adjacent blocks of the target block P0(0,0) are block P0(0,1), block P0(1,1), and block P0(1,0). When the target block is block P0(1,1) in the initial image P0, the multiple adjacent blocks of the target block P0(1,1) are block P0(0,0), block P0(0,1), block P0(0,2), block P0(1,2), block P0(2,2), block P0(2,1), block P0(2,0), and block P0(1,0). When the target block is block P0(2,0), the multiple adjacent blocks of the target block P0(2,0) are block P0(1,0), block P0(1,1), block P0(2,1), block P0(3,1) and block P0(3,0). The extended image A includes the gain data corresponding to each block in the initial image P0, such as Figure 4As shown, A(0, 0), A(0, 1), ..., A(3, 3) in the expanded image A are gain data respectively. A(0, 0) in the expanded image A is the gain data corresponding to the block P0(0, 0) in the initial image P0, A(0, 1) in the expanded image A is the gain data corresponding to the block P0(0, 1) in the initial image P0, ..., A(3, 3) in the expanded image A is the gain data corresponding to the block P0(3, 3) in the initial image P0. When the target block is block P0(0,0), the second acquisition unit 113 or the processor 30 acquires the weight value of the adjacent block P0(0,1) according to the gain data A(0,1) in the extended graph A and the noise level of the target block P0(0,0), acquires the weight value of the adjacent block P0(1,1) according to the gain data A(1,1) in the extended graph A and the noise level of the target block P0(0,0), and acquires the weight value of the adjacent block P0(1,0) according to the gain data A(1,0) in the extended graph A and the noise level of the target block P0(0,0).

[0040] In method 03, the noise reduction unit 115 or the processor 30 performs noise reduction processing on the target block according to the weight values ​​of multiple adjacent blocks of the target block. Figure 4 As shown, when the target block is block P0(0,0), the noise reduction unit 115 or the processor 30 performs noise reduction processing on the target block P0(0,0) according to the weight value of the adjacent block P0(0,1), the weight value of the adjacent block P0(1,1) and the weight value of the adjacent block P0(1,0).

[0041] In method 04, all blocks in the initial image P0 are traversed to treat each block as a target block for noise reduction. The specific noise reduction process is the same as the implementation process of the aforementioned methods 02 and 03, which will not be repeated here. Since all blocks in the initial image P0 are used as target blocks, and each target block is subjected to noise reduction according to the weight values ​​of its corresponding multiple adjacent blocks, and the weight value of each adjacent block is related to the noise level of the target block and the gain data corresponding to the target block, the noise reduction unit 115 or the processor 30 effectively regulates the noise reduction intensity of different blocks in the initial image, thereby suppressing the noise of brightening the dark area in a more targeted manner, better protecting the details in the initial image, and avoiding the phenomenon of uneven noise reduction in the central area and the corner area of ​​the denoised image.

[0042] See also Figure 4 and Figure 5 In some embodiments, 02: obtaining weight values ​​of multiple adjacent blocks around the target block according to the gain data corresponding to the block in the extended graph A and the noise level of the target block, including:

[0043] 021: Obtain a first ratio of the adjacent block according to the pixel values ​​of all pixels in the target block and the pixel values ​​of corresponding pixels in the adjacent block;

[0044] 023: Obtain a second ratio according to the first ratio, the noise level and the gain data corresponding to the target block in the extended graph A; and

[0045] 025: Obtain the weight value corresponding to the adjacent block according to the first ratio and the second ratio.

[0046] Please combine Figure 2 , the second acquisition unit 113 is further used to execute the methods in 021, 023 and 025. That is, the second acquisition unit 113 is further used to: acquire a first ratio of the adjacent block according to the pixel values ​​of all pixels in the target block and the pixel values ​​of the corresponding pixels in the adjacent block. Acquire a second ratio according to the first ratio, the noise level and the gain data corresponding to the target block in the extended graph A. Acquire a weight value corresponding to the adjacent block according to the first ratio and the second ratio.

[0047] Please combine Figure 3 , the processor 30 is further used to execute the methods in 021, 023 and 025. That is, the processor 30 is further used to: obtain a first ratio of the adjacent block according to the pixel values ​​of all pixels in the target block and the pixel values ​​of the corresponding pixels in the adjacent block. Obtain a second ratio according to the first ratio, the noise level and the gain data corresponding to the target block in the expanded graph A. Obtain a weight value corresponding to the adjacent block according to the first ratio and the second ratio.

[0048] In method 021, when the second acquisition unit 113 or the processor 30 calculates the weight value of the adjacent block in the target block, the first ratio of the weight value of the adjacent block is obtained according to the pixel values ​​of all the pixels in the target block and the adjacent block. For example, when each block in the initial image P0 includes one pixel, the second acquisition unit 113 or the processor 30 calculates the first ratio of the adjacent block according to the pixel points in the target block and the pixel points in the adjacent block. For example, please combine Figure 4 and Figure 6When the block in the initial image P0 includes four pixels, the second acquisition unit 113 or the processor 30 calculates a first ratio in the weight value of the adjacent block according to the pixel values ​​of all pixels in the target block and the pixel values ​​of all pixels in the adjacent block. For example, when the target block is block P0 (0, 0) and the second acquisition unit 113 or the processor 30 acquires the first ratio of the weight value of the adjacent block P0 (0, 1), the pixel values ​​of the four pixels in the target block P0 (0, 0) are pa, pb, pc and pd, respectively, and the pixel values ​​of the four pixels in the adjacent block P0 (0, 1) are pe, pf, pg and ph, respectively. The second acquisition unit 113 or the processor 30 acquires the first ratio of the adjacent block P0 (0, 1) according to the pixel values ​​pa, pb, pc, pd in the target block P0 (0, 0) and the pixel values ​​pe, pf, pg, ph in the adjacent block P0 (0, 1) to make full use of the pixel values ​​of the pixels in the adjacent blocks, so that the weight value obtained by the first ratio can more specifically filter the noise in the target block P0 (0, 0). In method 023, the second acquisition unit 113 or the processor 30 acquires the second ratio of the adjacent block according to the first ratio obtained in method 021, the noise level of the target block, and the gain data corresponding to the target block in the extended graph A. For example, the first ratio can be acquired according to the difference between the total pixel value of all pixels in the target block and the total pixel value of all pixels in the adjacent block. Alternatively, the first ratio is acquired according to the sum of the difference between the pixel values ​​of all pixels in the target block and the pixel values ​​of all pixels in the adjacent block. Finally, the method in 025 is executed: the weight value of the adjacent block is acquired according to the first ratio and the second ratio. When the second acquisition unit 113 or the processor 30 acquires the weight values ​​of other adjacent blocks of the target block, method 021, method 023 and method 025 are executed in sequence.

[0049] In summary, when the second acquisition unit 113 or the processor 30 obtains the weight value of the adjacent block of the target block, the pixel values ​​of all pixels in the target block, the pixel values ​​of all pixels in the adjacent block, the noise level of the target block, and the gain data corresponding to the target block in the expanded figure A are combined to calculate the weight value of the adjacent block, and the weight values ​​corresponding to multiple adjacent blocks are used to perform targeted noise reduction on the target block to effectively protect the details in the target block.

[0050] See also Figure 7 In some embodiments, 021: obtaining a first ratio of the adjacent block according to the pixel values ​​of all pixels in the target block and the pixel values ​​of corresponding pixels in the adjacent block, including:

[0051] 0211: Obtaining a difference value of each pixel point according to the pixel value of each pixel point in the target block and the pixel value of the pixel point at the corresponding position in the adjacent block; and

[0052] 0213: Accumulate the absolute values ​​of the difference values ​​corresponding to all the pixels to obtain a first ratio of the adjacent blocks.

[0053] Please combine Figure 2 , the second acquisition unit 113 is further used to execute the methods in 0211 and 0213. That is, the second acquisition unit 113 is further used to: obtain the difference value of each pixel point according to the pixel value of each pixel point in the target block and the pixel value of the pixel point at the corresponding position in the adjacent block. Accumulate the absolute values ​​of the difference values ​​corresponding to all the pixels to obtain the first ratio of the adjacent blocks.

[0054] Please combine Figure 3 , the processor 30 is further used to execute the methods in 0211 and 0213. That is, the processor 30 is further used to: obtain the difference value of each pixel point according to the pixel value of each pixel point in the target block and the pixel value of the pixel point at the corresponding position in the adjacent block. Accumulate the absolute values ​​of the difference values ​​corresponding to all the pixels to obtain the first ratio of the adjacent blocks.

[0055] Please combine Figure 6 In one embodiment, if the target block includes four pixels, the second acquisition unit 113 or the processor 30 obtains the difference value of each pixel according to the pixel value of each pixel in the target block and the pixel value of the pixel at the corresponding position in the adjacent block. Among them, the pixel at the upper left corner of the target block corresponds to the pixel at the upper left corner of the adjacent block, the pixel at the upper right corner of the target block corresponds to the pixel at the upper right corner of the adjacent block, the pixel at the lower left corner of the target block corresponds to the pixel at the lower left corner of the adjacent block, and the pixel at the lower right corner of the target block corresponds to the pixel at the lower right corner of the adjacent block. For example, the pixel with a pixel value of pa in the target block P0 (0, 1) corresponds to the pixel with a pixel value of pe in the adjacent block P0 (0, 1). When the target block is P0 (0, 0), and the second acquisition unit 113 or the processor 30 acquires the first ratio in the weight value of the adjacent block P0 (0, 1), the second acquisition unit 113 or the processor 30 calculates the difference between each pixel value in the two blocks (the target block and the adjacent block) in sequence, and the four difference values ​​of the adjacent block P0 (0, 1) are pa-pe, pb-pf, pc-pg, and pd-ph, respectively. Then, the absolute values ​​of the four difference values ​​are added to obtain the first ratio in the weight value of the adjacent block P0 (0, 1), which is |pa-pe|+|pb-pf|+|pc-pg|+|pd-ph|.

[0056] In summary, when the second acquisition unit 113 or the processor 30 acquires the weight value of the adjacent block of the target block according to methods 021, 023, 025, 0211 and 0213, it can be acquired according to the following formula (1).

[0057]

[0058] Wherein, Weight in formula (1) represents the weight value of the adjacent block, diff represents the first ratio of the weight values ​​of the adjacent blocks, and diff+noiselevel*lut (x,y) represents the second ratio of the weight values ​​of the adjacent blocks, and noiselevel represents the noise level of the target block, which is a known value determined by other modules in the image processing device 10 and is a positive number. lut (x,y) Indicates that the target block P0(x, y) corresponds to the gain data A(x, y) in the extended graph A. From formula (1), it can be seen that when the gain data corresponding to the target block in the extended graph A is larger, the weight value of the adjacent block of the target block is larger. Figure 4 and Figure 6 For example, when the target block is P0(0,0), the first ratio of the weight value of the adjacent block P0(0,1) is diff=|pa-pe|+|pb-pf|+|pc-pg|+|pd-ph|, and the second ratio is diff+noiselevel*A(0,0), then the weight value of the adjacent block P0(0,1) is

[0059] See also Figure 8 In some embodiments, 03: performing noise reduction processing on the target block according to multiple weight values, including:

[0060] 031: Obtain a third ratio according to the pixel values ​​of the adjacent blocks and the weight values ​​corresponding to the adjacent blocks respectively;

[0061] 033: Accumulate the multiple weight values ​​corresponding to the adjacent blocks to obtain a fourth ratio; and

[0062] 035: Obtain the pixel value of the target block according to the third ratio and the fourth ratio to serve as the pixel value of the block of the denoised image corresponding to the target block.

[0063] Please combine Figure 2, the noise reduction unit 115 is also used to execute the methods in 031, 033 and 035. That is, the noise reduction unit 115 is also used to: obtain a third ratio according to the pixel values ​​of the adjacent blocks and the weight values ​​corresponding to the adjacent blocks. Accumulate multiple weight values ​​corresponding to the adjacent blocks to obtain a fourth ratio. Obtain the pixel value of the target block according to the third ratio and the fourth ratio as the pixel value of the block corresponding to the target block in the denoised image.

[0064] Please combine Figure 3 , the processor 30 is also used to execute the methods in 031, 033 and 035. That is, the processor 30 is also used to: obtain a third ratio according to the pixel values ​​of the adjacent blocks and the weight values ​​corresponding to the adjacent blocks. Accumulate multiple weight values ​​corresponding to the adjacent blocks to obtain a fourth ratio. Obtain the pixel value of the target block according to the third ratio and the fourth ratio as the pixel value of the block corresponding to the target block in the denoised image.

[0065] After the second acquisition unit 113 or the processor 30 obtains the multiple weight values ​​corresponding to the adjacent blocks of the target block, the noise reduction unit 115 or the processor 30 obtains the third ratio according to the multiple weight values ​​and the pixel values ​​of the adjacent blocks, and accumulates the multiple weight values ​​corresponding to the adjacent blocks to obtain the fourth ratio. The noise reduction unit 115 combines the third ratio and the fourth ratio to obtain the pixel value of the target block after noise reduction, and uses the pixel value as the pixel value of the block corresponding to the target block in the noise reduction image. Among them, the pixel value of the block, the pixel value of the adjacent block, and the pixel value of the target block all represent the sum of the pixel values ​​of all pixels in the block. Specifically, the noise reduction unit or the processor 30 obtains the pixel value of the block corresponding to the target block in the noise reduction image according to formula (2).

[0066]

[0067] Among them, Pathch de-noise Represents the pixel value of the target block after noise reduction, ∑ i (Patch i *Weight i ) represents the third ratio, represents the fourth ratio, and n represents the number of adjacent blocks corresponding to the target block. For example, Figure 4 As shown, when block P0(0,0) is the target block, the number of its corresponding adjacent blocks is 3, then n=3. When block P0(1,0) is the target block, the number of its corresponding adjacent blocks is 5, then n=5. When block P0(1,1) is the target block, the number of its corresponding adjacent blocks is 8, then n=8. i Represents the sum of the pixel values ​​of all pixels in the i-th adjacent block. iRepresents the weight value of the i-th adjacent block.

[0068] Please combine Figure 4 and Fig. 9 When the target block is block P0(0,0), the sum of the pixel values ​​of all pixels in the target block without noise reduction is recorded as Patch noise , the pixel value of the adjacent block P0(0,1) is Patch1, and the weight value of the adjacent block P0(0,1) is Weight1; the pixel value of the adjacent block P0(1,1) is Patch2, and the weight value of the adjacent block P0(1,1) is Weight2; the pixel value of the adjacent block P0(1,0) is Patch3, and the weight value of the adjacent block P0(1,0) is Weight3. The pixel value of the target block after denoising obtained by the denoising unit 115 or the processor 30 according to formula (2) is That is, the pixel value of the block in the denoised image P1 corresponding to the block P0(0,0) in the initial image P0 (i.e., P1(0,0)) is Pathch de-noise .

[0069] If the target block P0(0,1) includes four pixels, the pixel value distribution of each pixel is as follows: Figure 6 As shown, that is, Pathch noise =pa+pb+pc+pd. In one embodiment, the pixel value of each pixel in the block P1(0,0) in the denoised image P1 is compared with the pixel value of the pixel at the corresponding position in the initial image P0 and the pixel value of the target block after denoising (i.e., Pathch de-noise ). The pixel value of the pixel at the upper left corner of the block P1(0,0) in the denoised image P1 is The pixel value of the pixel at the upper right corner of the block P1(0,0) in the denoised image P1 The pixel value of the pixel at the lower left corner of the block P1(0,0) in the denoised image P1 The pixel value of the pixel at the lower right corner of the block P1(0,0) in the denoised image P1

[0070] See also Fig.10 In some embodiments, 03: performing noise reduction processing on the target block according to multiple weight values, including

[0071] 037: Perform noise reduction processing on the three YUV channels or the three RGB channels in the target block according to multiple weight values.

[0072] Please combine Figure 2The noise reduction unit 115 is also used to execute the method in 037. That is, the noise reduction unit 115 is also used to: perform noise reduction processing on the three YUV channels or the three RGB channels in the target block according to multiple weight values.

[0073] Please combine Figure 3 The processor 30 is further configured to execute the method in 037. That is, the processor 30 is further configured to: perform noise reduction processing on the three YUV channels or the three RGB channels in the target block according to the multiple weight values.

[0074] In some embodiments, when the noise reduction unit 115 or the processor 30 performs noise reduction processing on the target block according to multiple weight values, the noise reduction processing can be performed on the three channels of RGB in the target block. Alternatively, the noise reduction unit 115 or the processor 30 performs noise reduction processing on the brightness noise on the Y channel in the target block to suppress the brightness noise in the target block. Alternatively, the noise reduction unit 115 or the processor 30 performs noise reduction processing on the color noise on the UV channels in the target block to suppress the color noise in the target block. Alternatively, the noise reduction unit 115 or the processor 30 performs joint noise reduction on the three channels of YUV. Among them, the specific noise reduction process is the same as the noise reduction process in methods 031, 033 and 035, and will not be repeated here.

[0075] See also Fig. 9 and Fig.11 In some embodiments, the image processing method further comprises:

[0076] 05: Sharpening the denoised image P1 according to the denoised image P1 and the gain data.

[0077] Please combine Figure 2 The image processing device 10 further includes a sharpening module 13. The sharpening module 13 is used to execute the method in 05. That is, the sharpening module 13 is used to perform sharpening processing on the denoised image P1 according to the denoised image P1 and the gain data.

[0078] Please combine Figure 3 The processor 30 is further configured to execute the method in 05. That is, the processor 30 is further configured to perform a sharpening process on the denoised image P1 according to the denoised image P1 and the gain data.

[0079] Specifically, after the third acquisition unit 117 or the processor 30 obtains the denoised image P1, the sharpening module 13 performs sharpening processing on the denoised image P1 according to the gain data to further improve the sharpness of a part of the denoised image P1 and improve the image quality.

[0080] See also Fig.12 and Fig.13 In some embodiments, 05: sharpening the denoised image P1 according to the denoised image P1 and the gain data, including:

[0081] 051: Obtain the detail layer D of the denoised image P1;

[0082] 053: Sharpening the denoised image P1 according to the detail layer D and the gain data corresponding to the detail layer D in the extended image A.

[0083] Please combine Figure 2 , the sharpening module 13 is also used to execute the methods in 051 and 053. That is, the sharpening module 13 is also used to: obtain the detail layer D of the denoised image P1; and perform sharpening processing on the denoised image P1 according to the detail layer D and the gain data corresponding to the detail layer D in the extended image A.

[0084] Please combine Figure 3 The processor 30 is further used to execute the methods in 051 and 053. That is, the processor 30 is further used to: obtain the detail layer D of the denoised image P1; and perform sharpening processing on the denoised image P1 according to the detail layer D and the gain data corresponding to the detail layer in the extended image A.

[0085] The denoised image P1 can be decomposed into a bottom layer and a detail layer, wherein the bottom layer contains low-frequency information of the denoised image P1, and the detail layer contains high-frequency information of the denoised image P1. When the sharpening module 13 obtains the bottom layer of the denoised image P1, it can use low-pass filtering (such as mean filtering), Gaussian filtering or guided filtering to filter the denoised image P1 to extract the bottom layer of the denoised image P1. After extracting the bottom layer, the sharpening module 13 uses the denoised image P1 to subtract the bottom layer to obtain the detail layer of the denoised image P1.

[0086] like Fig.13 As shown, in one example, the detail layer D in the denoised image P1 is shown as the shaded portion in the denoised image P1, and the gain data corresponding to the detail layer D in the extended image A is A(2,0)+A(3,0). The sharpening module 13 or the processor 30 performs a sharpening process on the detail layer D to obtain a sharpened detail layer D`, D`=D*(A(2,0)+A(3,0)), wherein the multiplication of the detail layer D and the gain data (A(2,0)+A(3,0)) refers to the process of sharpening the pixel values ​​in the detail layer D using the gain data (A(2,0)+A(3,0)). Since the bottom layer of the denoised image P1 is extracted in method 051, the sharpening module 13 or the processor 30 adds the bottom layer and the sharpened detail layer D` to obtain the sharpened image P2.

[0087] See also Figure 4 and Fig.14 In some embodiments, the image processing method further comprises:

[0088] 06: Perform lens shading correction on the initial image P0 to obtain an intermediate image;

[0089] 07: Obtain the GR channel data and GB channel data of the intermediate image (not shown) in the RAW domain;

[0090] 08: Obtain the expanded graph A based on the data of the GR channel and the GB channel.

[0091] Please combine Figure 2 The image processing device 10 further includes an acquisition module 17, which is used to execute the methods in 06, 07 and 08. That is, the acquisition module 17 is used to: perform lens shading correction processing on the initial image P0 to obtain an intermediate image. Obtain the GR channel data and the GB channel data of the intermediate image in the RAW domain. Obtain the extended image A according to the GR channel data and the GB channel data.

[0092] Please combine Figure 3 The processor 30 is also used to execute the methods in 06, 07 and 08. That is, the processor 30 is used to: perform lens shading correction processing on the initial image P0 to obtain an intermediate image. Obtain the GR channel data and the GB channel data of the intermediate image in the RAW domain. Obtain the extended image A according to the GR channel data and the GB channel data.

[0093] Specifically, after the first acquisition unit 111 acquires the initial image P0, due to the manufacturing process and optical characteristics of the camera module, the initial image acquired by the first acquisition unit 111 often has inconsistent color and brightness from the center to the surroundings. The acquisition module 17 performs lens shading correction processing on the initial image P0 so that the color and brightness of the center of the intermediate image are consistent with those of the surroundings. Fig.15 , and then obtain the GR channel data and GB channel data of the intermediate image in the RAW domain. Among them, the data size of the GR channel represents the gain data of the GR channel pixels (that is, the brightening multiple of the pixels), and the data size of the GB channel represents the gain data of the GB channel pixels. The size of the initial image P0 is 3072*4096, and the GR channel and GB channel obtained by the acquisition module 17 are both matrices of size 13*17. Finally, the acquisition module 17 obtains the extended image A based on the data of the GR channel and the data of the GB channel.

[0094] See also Fig.15 and Fig.16 In some embodiments, 08: obtaining an extended graph A according to the data of the GR channel and the data of the GB channel, comprising:

[0095] 081: Calculate the average value of the GR channel data and the GB channel data at the same position to obtain the data at the corresponding position in the G channel;

[0096] 083: Interpolate the G channel to obtain the expanded image A.

[0097] Please combine Figure 2 , the acquisition module 17 is also used to execute the methods in 081 and 083. That is, the acquisition module 17 is also used to: calculate the average value of the data of the GR channel and the data of the GB channel at the same position to obtain the data of the corresponding position in the G channel; and perform interpolation processing on the G channel to obtain the extended image A.

[0098] Please combine Figure 3 The processor 30 is also used to execute the methods in 081 and 083. That is, the processor 30 is also used to: calculate the average value of the data of the GR channel and the data of the GB channel at the same position to obtain the data of the corresponding position in the G channel; and perform interpolation processing on the G channel to obtain the expanded image A.

[0099] Specifically, when the acquisition module 17 acquires the data in the G channel, it averages the data in the GR channel and the GB channel, that is, the data in the G channel G(x, y) = (GR(x, y) + GR(x, y)) * 0.5, wherein the value range of x is [0, 12], and the value range of y is [0, 16]. For example, G(0, 0) = (GR(0, 0) + GR(0, 0)) * 0.5, G(0, 4) = (GR(0, 4) + GR(0, 4)) * 0.5, ..., G(16, 12) = (GR(16, 12) + GR(16, 12)) * 0.5. Then, the data in the G channel is interpolated to obtain an extended image A of the same size as the initial image P0.

[0100] Specifically, in method 083, the acquisition module 17 may perform interpolation processing on the G channel based on a bilinear interpolation method or a bicubic interpolation method. In one example, the acquisition module 17 performs interpolation processing on the G channel based on a bilinear interpolation method. Fig.15As shown, 255 pixels are sequentially inserted between two adjacent data in the horizontal direction of the G channel, and 255 pixels are sequentially inserted between two adjacent data in the vertical direction of the G channel to generate an extended image A with the same size as the initial image P0 (3072*4096). For example, the first data in the first row of the G channel is G(0,0), and the second data in the first row is G(0,1). Then 255 pixels are horizontally inserted between the two data. If the difference between the two data is recorded as C1, then C1=G(0,1)-G(0,0), the first pixel inserted between G(0,0) and G(0,1) is G(0,0)+C1*1 / 256, the second pixel inserted between G(0,0) and G(0,1) is G(0,0)+C1*2 / 256, ..., the 255th pixel is G(0,0)+C1*256 / 256. Then insert 255 pixels between the second data G(0,1) and the third data G(0,2). If the difference between the two data is recorded as C2, then C2=G(0,2)-G(0,1). The first pixel inserted between G(0,1) and G(0,2) is G(0,1)+C2*1 / 256, the second pixel inserted between G(0,1) and G(0,2) is G(0,1)+C2*2 / 256, and the 255th pixel is G(0,1)+C2*256 / 256. By analogy, insert 255 pixels between two adjacent data in each row of data in the G channel to obtain a matrix of size 3072*17. Similarly, 255 pixels are inserted between two adjacent data in each column of data in the G channel. For example, 255 pixels are inserted between the first data G(0,0) in the first column and the second data G(1,0) in the first column. Then, the first pixel inserted is G(0,0)+(G(1,0)-G(0,0))*1 / 256. Finally, the acquisition module 17 obtains an extended image A of a size of 3072*4096.

[0101] It should be noted that Fig.15 The gain data A(0,0) in the extended graph A includes at least one G channel data, that is, when the block in the initial image P0 includes one pixel, A(0,0)=G(0,0) in the extended graph A. When the block in the initial image P0 includes more than two pixels, A(0,0) in the extended graph A includes multiple G channel data after interpolation.

[0102] Please combine Fig.17 , Fig.17 The acquisition module 17 has an expanded image A obtained by acquiring a certain initial image. The central area of ​​the expanded image A is very dark, and the brightness of the expanded image A becomes higher as it moves toward the corner area, indicating that the pixels in the corresponding area are brightened by a larger multiple.

[0103] In summary, in the image processing method, image processing device 10, terminal 100 and non-volatile computer-readable storage medium 200 of the present application, the target block is subjected to noise reduction processing according to the gain data corresponding to the target block in the extended image A, the noise level of the target block and the weight values ​​of multiple adjacent blocks of the target block, and all blocks in the initial image P0 are traversed to perform noise reduction processing on each block in the initial image P0 to obtain the noise reduction image P1. Among them, the gain data in the extended image A corresponds to the blocks in the initial image A, that is, each block in the initial image P0 can find the corresponding gain data in the extended image A, so as to effectively regulate the noise reduction intensity of different blocks in the initial image P0, thereby more specifically suppressing the noise of brightening the dark area, better protecting the details in the initial image P0, and avoiding the phenomenon of uneven noise reduction in the central area and the corner area of ​​the noise reduction image P1.

[0104] See also Fig.18 , the embodiment of the present application further provides a non-volatile computer-readable storage medium 200 including a computer program 201. When the computer program 201 is executed by one or more processors 30, the processor 30 executes the image processing methods in 01, 02, 03, 04, 04, 05, 06, 07, 08, 021, 023, 025, 0211, 0213, 031, 033, 035, 037, 051, 053, 081 and 083.

[0105] In the description of this specification, the descriptions with reference to the terms "certain embodiments", "in an example", "exemplarily", etc., mean that the specific features, structures, materials or characteristics described in conjunction with the embodiments or examples are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0106] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0107] Although the embodiments of the present application have been shown and described above, it can be understood that the above embodiments are exemplary and cannot be understood as limitations on the present application. Ordinary technicians in this field can change, modify, replace and modify the above embodiments within the scope of the present application.

Claims

1. An image processing method, characterized in that: include: The first step: obtaining an initial image, wherein the initial image includes a plurality of blocks of the same size, and each block includes at least one pixel; Step 2: taking any one of the blocks as a target block, obtaining weight values ​​of a plurality of adjacent blocks around the target block according to gain data corresponding to the target block in the extended image and the noise level of the target block, wherein both the gain data and the noise level are positively correlated with the weight value, the adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as that of the initial image; Step three: performing noise reduction processing on the target block according to the plurality of weight values; and Traversing all the blocks in the initial image, taking each block as the target block once, and repeatedly performing the second step and the third step to obtain a denoised image; The image processing method further comprises: Performing lens shading correction processing on the initial image to obtain an intermediate image; Acquire the GR channel data and the GB channel data of the intermediate image in the RAW domain; and The extended map is obtained according to the data of the GR channel and the data of the GB channel.

2. The image processing method according to claim 1, characterized in that: The step of obtaining weight values ​​of a plurality of adjacent blocks around the target block according to the gain data corresponding to the block in the expanded graph and the noise level of the target block comprises: Obtaining a first ratio of the adjacent block according to pixel values ​​of all pixels in the target block and pixel values ​​of corresponding pixels in the adjacent block; Acquire a second ratio according to the first ratio, the noise level, and the gain data corresponding to the target block in the extended graph; and The weight value corresponding to the adjacent block is obtained according to the first ratio and the second ratio.

3. The image processing method according to claim 2, characterized in that: The step of obtaining a first ratio of the adjacent block according to pixel values ​​of all pixels in the target block and pixel values ​​of corresponding pixels in the adjacent block includes: Obtaining a difference value of each pixel point according to a pixel value of each pixel point in the target block and a pixel value of a pixel point at a corresponding position in the adjacent block; and The absolute values ​​of the difference values ​​corresponding to all the pixel points are accumulated to obtain the first ratio of the adjacent block.

4. The image processing method according to claim 1, characterized in that: The performing noise reduction processing on the target block according to the plurality of weight values ​​comprises: Acquire a third ratio according to the pixel values ​​of the adjacent blocks and the weight values ​​respectively corresponding to the adjacent blocks; Accumulating the plurality of weight values ​​respectively corresponding to the adjacent blocks to obtain a fourth ratio; and The pixel value of the target block is acquired according to the third ratio and the fourth ratio to serve as the pixel value of the block of the denoised image corresponding to the target block.

5. The image processing method according to claim 1, characterized in that: The performing noise reduction processing on the target block according to the plurality of weight values ​​comprises: The YUV three channels or the RGB three channels in the target block are subjected to noise reduction processing according to the multiple weight values.

6. The image processing method according to claim 1, characterized in that: The image processing method further comprises: The denoised image is sharpened according to the denoised image and the gain data.

7. The image processing method according to claim 1, characterized in that: The sharpening process of the denoised image according to the denoised image and the gain data comprises: Acquiring a detail layer of the denoised image; and The denoised image is sharpened according to the detail layer and the gain data corresponding to the detail layer in the extended image.

8. The image processing method according to claim 1, characterized in that: The acquiring the extended map according to the data of the GR channel and the data of the GB channel comprises: averaging the data of the GR channel and the data of the GB channel at the same position to obtain data at a corresponding position in the G channel; and An interpolation process is performed on the G channel to obtain the expanded image.

9. An image processing device, characterized in that: Including noise reduction module, the noise reduction module includes: A first acquisition unit is used to perform a first step: acquiring an initial image, wherein the initial image includes a plurality of blocks of the same size, and each block includes at least one pixel; a second acquisition unit, configured to execute the second step: taking any one of the blocks as a target block, and acquiring weight values ​​of a plurality of adjacent blocks around the target block according to gain data corresponding to the target block in the extended image and a noise level of the target block, wherein both the gain data and the noise level are positively correlated with the weight value, the adjacent blocks are blocks surrounding and connected to the target block, and the size of the extended image is the same as that of the initial image; a noise reduction unit, configured to execute the third step: performing noise reduction processing on the target block according to the plurality of weight values; and A third acquisition unit is used to traverse all the blocks in the initial image, take each block as the target block once, and repeatedly perform the second step and the third step to obtain a denoised image; The image processing device further includes an acquisition module, which is used to: Performing lens shading correction processing on the initial image to obtain an intermediate image; Acquire the GR channel data and the GB channel data of the intermediate image in the RAW domain; and The extended map is obtained according to the data of the GR channel and the data of the GB channel.

10. A terminal, characterized in that: The terminal comprises: One or more processors, memory; and One or more programs, wherein one or more of the programs are stored in the memory and executed by one or more of the processors, and the programs include instructions for executing the image processing method according to any one of claims 1 to 8.

11. A non-volatile computer-readable storage medium storing a computer program, which, when executed by one or more processors, implements the image processing method according to any one of claims 1 to 8.

Images