Compressed image noise processing method and circuit system thereof

Abstract

The present invention relates to a method for processing compressed image noise and a circuit system thereof. In the method, the circuit system receives an image compressed in a compression format, decompresses the image, obtains the pixel value of each pixel, calculates the difference in pixel values ​​between adjacent pixels, then divides a certain range of the image into multiple regions, obtains the maximum and minimum pixel values ​​in each region, and compares the maximum difference values ​​of each region to obtain the maximum difference value within the range. Based on this maximum difference value, it is determined whether the corresponding range, such as a flat area or a high-frequency area in the image, should be subjected to noise removal. The difference can also be adjusted using the weighted average brightness of pixels near the current pixel, and a noise removal process is performed only on this range, ultimately outputting an image with reduced noise generated during compression.

Description

Technical Field

[0001] The present invention relates to an image noise processing method, in particular to a subsequent processing method for noise generated by image compression, and a related circuit system. Background Art

[0002] Currently, widely used image compression technologies include JPG, JPEG, MJPEG, and H.26x. These are mainly block-based compression methods. MPEG-4 is a compression coding standard developed by the Moving Picture Experts Group (MPEG) for audio and video. MPEG-4 has the advantages of high compression ratio, space saving, and excellent image quality.

[0003] The MPEG-4 specification proposes methods for deblocking and de-de-noising. Deblocking filters pixels at the edges of image blocks to identify edges, a computationally intensive method. De-de-noising requires first determining the maximum and minimum values ​​of the entire image block. Then, using a threshold, it determines whether pixels within the block require de-de-noising. If so, a smoothing operation is performed on the pixels.

[0004] The common image compression technologies mentioned above are primarily block-based. The quantization step in the compression process loses some high-frequency information, resulting in noise near high frequencies after decompression. The smoothing algorithms used in these familiar compression technologies can blur image details. Furthermore, since edges are high-frequency features, performing first-order or second-order operations on horizontal, vertical, and 45-degree angles requires significant computational effort and storage space, resulting in high hardware costs. Summary of the Invention

[0005] To address the shortcomings of known compression techniques, such as blurring details through smoothing operations and increasing hardware costs through edge operations, the present invention proposes a compressed image noise processing method and circuit system. This method calculates and detects image content, using a threshold to determine which portions of the image need to be processed and which need not. Filtering is then applied only to the portions that need to be processed, mitigating noise caused by image compression. This reduces computational requirements, thereby lowering hardware costs and avoiding the drawback of blurred details.

[0006] According to an embodiment, the circuit system executing the compressed image noise processing method is, for example, an image processor in an electronic device, used to process an image obtained from a video camera of the electronic device.

[0007] In a circuit system in which a processing circuit executes the compressed image noise processing method, an image compressed in a compression format is first received. After decompressing the image, the pixel value of each pixel in the image can be obtained. Then, the difference in pixel values ​​of adjacent pixels can be calculated to obtain the maximum and minimum pixel values ​​of multiple regions in a range of the image. After comparing the maximum difference values ​​of each region, a maximum difference value in this range can be obtained. This maximum difference value is the basis for the method to determine whether to execute the noise removal process for the corresponding range.

[0008] Furthermore, if the range is an area where a denoising process is required, the denoising process is performed; if the range is an area where a denoising process is not required, the denoising process is not performed; and finally an output image with noise generated during compression is obtained.

[0009] Furthermore, in the compressed image noise processing method, a lower threshold and an upper threshold may be set, and the lower threshold and the upper threshold are compared with the maximum difference value of the range, so that the image can be divided into a flat area, a general texture area and a high-frequency area, or at least one of them, wherein the flat area and the high-frequency area are the areas where the denoising process needs to be performed.

[0010] Furthermore, a weighted average brightness can be calculated using the pixel values ​​of each pixel in the image and the pixel values ​​of pixels near the pixel, and then the average brightness of each range can be calculated using the weighted average brightness of each pixel. A brightness adjustment curve is introduced, and the brightness coefficient corresponding to the average brightness of each range is obtained to adjust the maximum difference value in each range.

[0011] In order to further understand the features and technical contents of the present invention, please refer to the following detailed description and drawings of the present invention. However, the drawings provided are only for reference and illustration and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 A circuit block diagram of an embodiment of a display circuit system;

[0013] Figure 2 A flow chart showing an embodiment of a method for processing noise in a compressed image;

[0014] Figure 3 A schematic diagram showing an example of calculating the maximum difference value within a certain range in a method for processing noise in a compressed image;

[0015] Figure 4 A schematic diagram showing an example of weights of pixel values ​​near a current pixel in a method for processing noise in a compressed image; and

[0016] Figure 5 A schematic diagram showing an example of a brightness adjustment curve in a compressed image noise processing method. DETAILED DESCRIPTION

[0017] The following is an explanation of the embodiments of the present invention through specific embodiments. Those skilled in the art can understand the advantages and effects of the present invention from the contents disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and the details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. In addition, it is stated in advance that the drawings of the present invention are only simple schematic illustrations and are not depicted according to actual dimensions. The following embodiments will further explain the relevant technical contents of the present invention in detail, but the disclosed contents are not intended to limit the scope of protection of the present invention.

[0018] It should be understood that although terms such as "first," "second," and "third" may be used herein to describe various components or signals, these components or signals should not be limited by these terms. These terms are primarily used to distinguish one component from another, or one signal from another. In addition, the term "or" as used herein may include any one or more combinations of the associated listed items, as appropriate.

[0019] The specification discloses a compressed image noise processing method and circuit system thereof, which are applied for the purpose of image compression. The main concept of compressed image processing is to calculate and detect the image content of the input image during the compression process, and then use a threshold to determine whether filtering processing is required to reduce the noise caused by image compression. Therefore, the image can be divided into parts to be processed and parts not to be processed, thereby reducing computing requirements.

[0020] The circuit system for realizing the compressed image noise processing method can refer to Figure 1 The circuit block diagram of an embodiment of a circuit system is shown. The compressed image noise processing method is applicable to any dynamic or static image that requires compression and decompression, and can be implemented in the circuit system 10 shown in the figure. The circuit system can, for example, be an image processor in a video camera (or device) on an electronic device (such as a mobile phone or laptop computer), including a processing circuit (such as a CPU or microprocessor) implemented in hardware or software with a circuit combination to process images captured by the video camera, execute the compressed image noise processing method, and implement various functions, such as the decompression unit 101, difference calculation unit 103, average brightness calculation unit 105, difference adjustment unit 107, and noise removal unit 109, which are distinguished by function in the figure.

[0021] As shown, circuit system 10 receives an input image 11, which can be a static image or a dynamic image input frame by frame. Input image 11 is image data in a specific compression format (such as JPG, JPEG, MJPEG, and H.26x). After being input into circuit system 10, a decompression unit 101, implemented in hardware or software with a processing circuit, first decompresses the image according to the compression format to obtain the pixel value of each pixel in the image. A difference calculation unit 103 then calculates the pixel value difference between adjacent pixels. In one embodiment, the image is divided into multiple ranges, each of which can be divided into multiple regions. The maximum and minimum pixel values ​​in each region are found, and the maximum difference value of each region is calculated. Ultimately, the maximum difference value within a specific range of the entire image is determined by comparing the maximum difference values ​​of each region in the image. The average brightness calculation unit 105 then calculates a weighted average brightness based on the pixel values ​​near the current pixel according to the weighted value, so that the difference adjustment unit 107 can adjust the previously calculated difference value according to the brightness information near the current pixel. The difference value is mainly adjusted according to the brightness curve to highlight the dark parts of the image and adjust the difference between bright and dark parts to be consistent.

[0022] At the end of the process, based on the maximum difference values ​​in one or more ranges, a determination can be made as to whether denoising should be performed on a range within the image. If the range requires denoising, the denoising process is performed; if the range does not require denoising, the denoising process is not performed and the image within the range is directly output. Ultimately, an output image is obtained that partially reduces the noise generated during compression. In another embodiment, the average pixel value of each region within each range of the image can be determined, and after pairwise comparison, the maximum difference value of each range can be determined. This maximum difference value can also be used to determine whether denoising should be performed on the range.

[0023] In one embodiment, two thresholds can be used to divide the image into several regions, including regions where noise removal (e.g., moiré noise) is to be performed by the noise removal unit 109, and regions where noise removal is not to be performed. In regions where noise removal is to be performed, flat areas can be smoothed and noise in high-frequency areas can be reduced. The above process does not specifically process image edges, i.e., there is no need to determine edge directions. This reduces computational requirements, ultimately resulting in an output image 12 with reduced noise generated during compression.

[0024] Figure 2 A flow chart illustrating an embodiment of a compressed image noise processing method that can be implemented in a specific circuit system.

[0025] The process begins with step S201, where an input image compressed according to a specific format is received. This can be a static image or a continuous image input frame by frame. To mitigate noise introduced by image compression, the image is decompressed. Then, in step S203, image processing within the circuit system calculates pixel values ​​and calculates the difference between the current pixel and its neighboring pixels.

[0026] The method for calculating pixel differences can be referred to Figure 3 A diagram illustrates an example of calculating the maximum difference value within a certain range in this compressed image noise processing method. This example shows a 5x5 image. Each pixel in the image, when processed, is the current pixel 30, marked with an X. The diagram shows four regions surrounding the current pixel 30 (the number of regions is not limited to that shown in this diagram). Each region is numbered i, with i = 1 to 4. i = 1, 2 indicates a pixel in both regions 1 and 2; i = 1, 3 indicates a pixel in both regions 1 and 3; i = 3, 4 indicates a pixel in both regions 3 and 4; and i = 2, 4 indicates a pixel in both regions 2 and 4.

[0027] In step S203, the maximum pixel value (M) and the minimum pixel value (m) within a specific range of the entire image are obtained, and the difference between the two is the maximum difference. Considering the regions, the maximum pixel value M[i] and the minimum pixel value m[i] are obtained in each region, and the difference between the two (M[i]-m[i]) is the maximum difference value of the region (in this example, i = 1 to 4). Then, as shown in equation 1, i.e., step S205, the maximum difference value of the multiple regions within a range (M[i]-m[i]) can be calculated, that is, the maximum difference value (MD) within the range (e.g., a 5x5 image).

[0028] Equation 1:

[0029]

[0030] According to the embodiments, pixel values ​​near the edge of a specific object in an image have significant differences. Therefore, the region with the maximum difference value (MD) within a certain range of the image can be determined as an edge. Therefore, the compressed image noise processing method proposed in this specification does not directly determine the direction of the edge in the image, nor does it require edge filtering, effectively reducing computational requirements.

[0031] When the method finds the maximum pixel value (M) and the minimum pixel value (m) from each partition, the maximum pixel value (M[i]) of each partition is subtracted from the minimum pixel value (m[i]). The largest of the differences can be used as the maximum difference value. This maximum difference value can reflect the image characteristics of the image (or multiple ranges in the image). For example, the portion with the maximum difference value may be an edge, so the edge can be determined without determining the edge direction. In this way, the maximum difference value can be used to determine whether a certain range is a flat area, a high-frequency area, or a region that requires noise removal, or a region that generally does not require noise removal. After determining the maximum difference value of each range in the image, the method can use the maximum difference value of each range to determine whether the range requires noise removal. In an embodiment, a lower threshold and an upper threshold can be set. By comparing the image with the two thresholds based on the maximum difference value of each range, the image can be divided into three types of regions: a flat area, a general texture area, and a high-frequency area, which can be at least one of them.

[0032] Before performing the noise removal step, in one embodiment, the method can adjust the difference value based on the brightness information around the current pixel. This is because many details in the dark areas (lower brightness) of the image may be ignored during decompression. Therefore, adjusting the difference value by weighted average brightness avoids excessive neglect of dark details. For example, the dark areas in the image indicate weak or insufficient light, but during the compression process, the noise is not easily highlighted. In this case, the difference value of the dark areas at high frequencies may be smaller than the difference value of the bright areas. Therefore, it is necessary to adjust the difference value by adjusting the curve to reduce the occurrence of this situation. Therefore, the difference between the two should be balanced in the image. For example, the difference between pixels in the bright areas of the image is large, while the difference between pixels in the dark areas is small. By adjusting, the difference between the large differences can be reduced, while the difference between the small differences can be increased. Therefore, the weighted average brightness proposed in the embodiment can effectively handle the situation where the difference value of the dark areas at high frequencies in the image is still smaller than the difference value of the bright areas.

[0033] In step S207, the weighted average brightness is calculated using the pixel values ​​near the current pixel X. In this way, the calculated difference value can be adjusted according to the brightness of the pixels near the current pixel. Figure 4 The diagram shows an example of weighting of pixel values ​​near the current pixel in the noise processing method for compressed images. As shown in the figure, the current pixel 40 is the main pixel, and its brightness value is set with a weight of 4. The brightness values ​​of the surrounding pixels are set with different weights according to their distance, indicating that the surrounding pixels have different brightness effects on the current pixel 40 according to their distance. The weighted average brightness of each pixel (each time for the current pixel) is calculated. In this case, the pixel value of each pixel in the image (the current pixel) and the pixel values ​​of the pixels near this pixel are applied as follows: Figure 4The displayed weights are used to calculate a weighted average brightness. Therefore, the weighted average brightness of each pixel in the image can be obtained, and the difference value is adjusted based on this.

[0034] In step S209, the weighted average brightness of each pixel obtained above can be used to further calculate the average brightness of each range, which is used to adjust the difference value of each area (M[i]-m[i]) so as to adjust the maximum difference value of each range. According to one embodiment, the method of adjusting the difference value by brightness can be adopted as follows: Figure 5 Schematic diagram of an example brightness adjustment curve in the compressed image noise processing method shown.

[0035] Figure 5 A calculated brightness curve (e.g., logarithmic calculation) is displayed. The horizontal axis represents the weighted average brightness of each range obtained in step S207. In this example, it is divided into 8 segments (the actual implementation does not limit the specific number of segments), and each segment corresponds to a brightness coefficient represented by the vertical axis. Based on this brightness adjustment curve formed according to the characteristics of the input image, the average brightness calculated based on the weighted average brightness of a specific range in the image can be used to correspond to the position of the point on the curve to obtain the brightness coefficient. If it falls in the middle of the segment, the brightness coefficient can be obtained by interpolation, and then the maximum difference value (MD) is adjusted based on this brightness coefficient. As can be seen from the legend, when the average brightness is lower, it corresponds to a larger brightness coefficient, and when the average brightness is higher, it corresponds to a lower brightness coefficient. For the adjustment of the maximum difference value, refer to Equation 2.

[0036] Equation 2:

[0037] MD′=MD×bright coef , where: 0≤brigh coef ≤1

[0038] MD is the maximum difference value for each range obtained in step S205 according to Equation 1. When the brightness coefficient is determined by the brightness adjustment curve, the difference value is minimized when the average brightness is higher, while the difference value is amplified when the average brightness is lower, indicating darkness. The goal is to bring the difference values ​​of the entire image to a consistent level. In this example, the adjusted maximum difference value (MD') is obtained using Equation 2. However, in the main embodiment of the method, steps S207 and S209 can be omitted.

[0039] After the above difference value adjustment (or in certain embodiments, the difference value does not necessarily need to be adjusted based on brightness), in step S211, the maximum difference value within a specific range of the image is divided into several regions based on several thresholds provided by the system. The maximum difference value used in step S211 can be the maximum difference value (MD') after brightness adjustment, but the unadjusted maximum difference value (MD) is still applicable.

[0040] For example, the system is designed with two thresholds: MD_THD0 (lower threshold) and MD_THD1 (upper threshold), which divide all maximum difference values ​​in the image into three regions, as shown in Equation 3.

[0041] Equation 3:

[0042]

[0043] When the maximum difference value (MD) (or the maximum difference value (MD') adjusted based on brightness information) is less than MD_THD0 (lower threshold), it indicates that the maximum difference value is relatively small, indicating that the area corresponding to the current pixel is a flat region, and subsequent noise removal can be performed on this area. When the maximum difference value (MD) is between MD_THD0 (lower threshold) and MD_THD1 (upper threshold), this area has specific details (texture) and does not require noise removal. When the maximum difference value (MD or MD') is greater than MD_THD1 (upper threshold), it indicates that the maximum difference value is relatively large, indicating that the area corresponding to the current pixel has high-frequency image features, and subsequent noise removal should be performed on this area. In other words, in addition to the areas that do not require noise removal, the flat region and the high-frequency region are the areas that require noise removal.

[0044] Furthermore, after the area where noise removal is required is obtained in step S211, the weight (W) of each pixel in this area is determined according to the thresholds (EPF_THD0, EPF_THD1) provided by the system. i，j ), which is to set a weight according to the characteristics between the current pixel and the nearby pixels before the noise removal step to strengthen the processing of each area. For example, a smaller weight, or even 0, is given to the one with a large difference, and a larger weight, or even the maximum value, is given to the one with a small difference. In the process of this differentiation processing, the flat area can be made flatter, and the noise can be effectively removed in the high-frequency area, such as step S213.

[0045] In this stage, the difference D between each pixel (current pixel) and the nearby pixels can be calculated. i，j Compared with the two thresholds (EPF_THD1, EPF_THD0), the weights (W i，jAccording to the embodiment shown in equation 4, when the difference between each pixel and its neighboring pixels is small (D i，j <EPF_THD0), give the maximum weight (W i，j =maxW); when the difference between each pixel and its neighboring pixels is large, but not too large, give the weight (((D i，j -EPF_THD0)×maxW) / (EPF_THD1-EPF_THD0)); when the difference between each pixel and its neighboring pixels is large (D i，j >EPF_THD1), that is, assign a weight of 0 (W i，j =0).

[0046] Equation 4:

[0047]

[0048] Then, as in step S215, according to the weight of each pixel in the image (W i，j ), the pixel value of each pixel (P x，y ) performs noise removal, especially for the parts judged as flat areas and high-frequency areas where noise removal procedures need to be performed, as shown in Equation 5.

[0049] Equation 5:

[0050]

[0051] where ∑ i，j P x+i，y+j ×W i，j Indicates the current pixel value (P x，y ) each pixel value (P x+i，y+j ) multiplied by its corresponding weight (W i，j ), and P x，y ×(totalW-∑ i，j W i，j ) is not enough, then multiply the current pixel by the total weight (totalW) minus the sum of all pixel weights (Σ i，j W i，j ) to compensate, and then divided by the total weight, the pixel value after noise removal (P′ x，y ).

[0052] It is worth mentioning that, using the compressed image noise processing method, after the pixel values ​​are obtained, the difference values ​​between the pixels represent the characteristics of the image, including determining the flat area and the high-frequency area, calculating the weighted average brightness, and then adjusting the difference with the brightness information. One of the main purposes is to make the difference almost the same so that the difference of the entire image can have a unified difference standard. However, the edges of the image are not processed, so there is no need to determine the edge direction, thereby reducing the need for calculation, but the details are not processed, the details can be retained, and the noise in the bright part can be effectively processed. It can solve the mosquito noise, ripple noise, etc. generated by the well-known image compression method. The overall technical means is to filter the pixels that need to be processed.

[0053] In summary, according to the embodiments of the compressed image noise processing method and circuit system described above, the circuit system, such as an image processing circuit in an electronic device with a video camera, uses difference values ​​to determine image characteristics in order to mitigate noise caused by image compression. A threshold is then used to determine which pixels require processing and which do not. De-noising is then performed only on the pixels that require processing. This effectively reduces computational overhead and saves hardware costs.

[0054] The contents disclosed above are only preferred feasible embodiments of the present invention and are not intended to limit the scope of the present invention. Therefore, any equivalent technical changes made using the contents of the present invention description and drawings are included in the scope of the present invention.

[0055] Description of Reference Numerals

[0056] 10: Circuit system

[0057] 11: Input image

[0058] 12: Output image

[0059] 101: Decompression unit

[0060] 103: Difference calculation unit

[0061] 105: Average brightness calculation unit

[0062] 107: Difference adjustment unit

[0063] 109: Noise removal unit

[0064] 30: Current pixel

[0065] 40: Current pixel

[0066] Steps S201 to S215: Compressed Image Noise Processing Flow

Claims

1. A method for processing compressed image noise, applied to a circuit system, comprising: receiving an image compressed in a compression format; Decompressing the image to obtain a pixel value of each pixel in the image; Calculating pixel value differences, dividing a range of the image into a plurality of regions, obtaining a maximum pixel value and a minimum pixel value in each region, and comparing maximum difference values ​​of each region to obtain a maximum difference value in the range; as well as determining whether to perform the step of removing noise in the range according to the maximum difference value in the range; If the range is an area where a denoising process is required, a denoising process is performed; if the range is an area where a denoising process is not required, the denoising process is not performed; and finally, an output image with noise generated during compression reduced is obtained. In which, a lower threshold and an upper threshold are provided, and the lower threshold and the upper threshold are compared with the maximum difference value in the range to divide the image into a flat area, a general texture area and a high-frequency area, or at least one of them, wherein the flat area and the high-frequency area are the areas where the denoising process needs to be performed.

2. The method for processing compressed image noise according to claim 1, wherein: A weighted average brightness is calculated using the pixel value of each pixel in the image and the pixel values ​​of pixels near the pixel. The average brightness of each range is calculated using the weighted average brightness of each pixel to adjust the maximum difference value in each range.

3. The method for processing compressed image noise according to claim 2, wherein: A brightness adjustment curve is introduced, and a brightness coefficient is obtained according to the maximum difference value of each range, so as to adjust the maximum difference value of each range.

4. The method for processing compressed image noise according to claim 3, wherein: A lower threshold and an upper threshold are set, and the lower threshold and the upper threshold are compared with the adjusted maximum difference value in the range to divide the image into a flat area, a general texture area and a high-frequency area, or at least one of them, wherein the flat area and the high-frequency area are the areas where the denoising process needs to be performed.

5. The method for processing compressed image noise according to any one of claims 1 to 4, characterized in that: When the area where the denoising process needs to be performed is obtained, the difference D between each pixel in this area and the nearby pixels is used. i，j Compare the two thresholds EPF_THD0 and EPF_THD1 to determine the weight W of each pixel i,j , the equation for obtaining the weight of each pixel is as follows:

6. The method for processing compressed image noise according to claim 5, wherein: According to the weight W of each pixel i,j , for each pixel value P x,y Execute the denoising procedure to obtain the pixel value P' after removing the noise x，y The equation is as follows: Among them, ∑ i，j P x+i，y+j ×W i，j Represents the current pixel value P x,y Each pixel value P nearby x+i，y+j Multiply by its corresponding weight W i，j , and P x，y ×(totalW-∑ i，j W i，j ) is not enough, then multiply the current pixel by the total weight totalW minus the sum of all pixel weights ∑ i，j W i，j To compensate, and then divided by the total weight, the pixel value P' after noise removal can be obtained x，y .

7. A circuit system comprising: A processing circuit executes a compressed image noise processing method, the method comprising: receiving an image compressed in a compression format; Decompressing the image to obtain a pixel value of each pixel in the image; Calculating pixel value differences, dividing a range of the image into a plurality of regions, obtaining a maximum pixel value and a minimum pixel value in each region, and comparing the maximum difference values ​​of each region to obtain a maximum difference value in the range; and determining whether to perform the step of removing noise in the range according to the maximum difference value in the range; If the range is an area where a denoising process is required, a denoising process is performed; if the range is an area where a denoising process is not required, the denoising process is not performed; and finally, an output image with noise generated during compression reduced is obtained. In the compressed image noise processing method executed by the processing circuit, a lower threshold and an upper threshold are provided, and the lower threshold and the upper threshold are compared with the maximum difference value in the range to divide the image into a flat area, a general texture area and a high-frequency area, or at least one of them, wherein the flat area and the high-frequency area are areas where the denoising process needs to be performed.

8. The circuit system according to claim 7, wherein: When the area where the denoising process needs to be performed is obtained, the difference D between each pixel in this area and the nearby pixels is used. i，j Compare the two thresholds EPF_THD0 and EPF_THD1 to determine the weight W of each pixel i，j , the equation for obtaining the weight of each pixel is as follows:

Images