Image processing methods, systems, and non-transitory computer-readable storage media
By generating low-bit luminance histograms and extracting local histograms from the original luminance histograms, the problem that existing technologies cannot be applied to images with multiple luminance ranges is solved, resulting in a reduction in hardware area and computational load, and making histogram equalization applicable to images with multiple luminance ranges.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- REALTEK SEMICON CORP
- Filing Date
- 2021-12-20
- Publication Date
- 2026-05-26
AI Technical Summary
Existing histogram equalization techniques are not suitable for images with a single brightness range or more than two brightness ranges, and require a large amount of computation.
By generating a low-bit luminance histogram, a local original luminance histogram is extracted from the original luminance histogram using the low-bit range of the target pixel, and a histogram equalization procedure is performed based on the original bit depth to reduce the bit depth, thereby reducing the computational load and making it applicable to images with multiple luminance ranges.
It reduces hardware area and power consumption, decreases computational load, and is suitable for images with multiple brightness ranges, avoiding the need for additional low-pass filter circuit design and computation.
Smart Images
Figure CN116309200B_ABST
Abstract
Description
Technical Field
[0001] This disclosure generally relates to an image processing technique. In particular, it relates to an image processing method, a system, and a non-transitory computer-readable storage medium. Background Technology
[0002] With the development of technology, various image processing techniques have been developed. Among these techniques, histogram equalization can be used to enhance the contrast of an image, thereby improving its visibility. However, some existing histogram equalization techniques are not suitable for images with a single brightness range or more than two brightness ranges, or require a large amount of computation. Summary of the Invention
[0003] Some embodiments of this disclosure relate to an image processing method. The image processing method includes the following operations: generating a sliding window for a target pixel among a plurality of pixels in image data using a processor; generating an original brightness histogram of the sliding window based on an original bit depth using the processor; generating a low-bit brightness histogram of the sliding window based on a low bit depth using the processor; determining a target low-bit range from the low-bit histogram based on the target pixel using the processor; extracting a local original brightness histogram from the original brightness histogram based on the target low-bit range using the processor; and performing a histogram equalization procedure on the local original brightness histogram based on the original bit depth using the processor to generate a final brightness of the target pixel.
[0004] Some embodiments of this disclosure relate to an image processing system. The image processing system includes a memory and a processor. The memory is used to store image data. The processor is used to perform the following operations: generating a sliding window for a target pixel among a plurality of pixels in the image data; generating an original brightness histogram of the sliding window based on an original bit depth; generating a low-bit brightness histogram of the sliding window based on a low bit depth; determining a target low-bit range from the low-bit histogram based on the target pixel; extracting a local original brightness histogram from the original brightness histogram based on the target low-bit range; and performing a histogram equalization procedure on the local original brightness histogram based on the original bit depth to generate a final brightness of the target pixel.
[0005] Some embodiments of this disclosure relate to a non-transitory computer-readable storage medium. The non-transitory computer-readable storage medium is used to store one or more computer programs containing multiple instructions. A processor is used to execute these instructions. When the processor executes these instructions, the processor performs the following operations: generating a sliding window for a target pixel among a plurality of pixels in an image data; generating an original brightness histogram of the sliding window based on an original bit depth; generating a low-bit brightness histogram of the sliding window based on a low bit depth; determining a target low-bit range from the low-bit histogram based on the target pixel; extracting a local original brightness histogram from the original brightness histogram based on the target low-bit range; and performing a histogram equalization procedure on the local original brightness histogram based on the original bit depth to generate a final brightness of the target pixel.
[0006] In summary, this disclosure first generates a low-bit luminance histogram, and then extracts a local original luminance histogram from the original luminance histogram using the target low-bit range where the target pixel is located. Finally, a histogram equalization process is performed on the local original luminance histogram based on the original bit depth. Therefore, this disclosure is applicable to images with more than two luminance ranges and can reduce hardware area, power consumption, or computational load. Furthermore, since low-pass filtering is also applied during the generation of the low-bit luminance histogram in this disclosure, no additional low-pass filtering circuitry or computation is required. Attached Figure Description
[0007] To make the above and other objects, features, advantages and embodiments of this disclosure more apparent and understandable, the accompanying drawings are described below:
[0008] Figure 1 This is a schematic diagram of an image processing system illustrated in accordance with some embodiments of the present disclosure;
[0009] Figure 2 This is a flowchart illustrating an image processing method according to some embodiments of the present disclosure;
[0010] Figure 3 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of one operation;
[0011] Figure 4 This is a schematic diagram illustrating an original luminance histogram and a low-bit luminance histogram according to some embodiments of the present disclosure;
[0012] Figure 5 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of one operation;
[0013] Figure 6The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of one operation;
[0014] Figure 7 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of one operation; and
[0015] Figure 8 This is a schematic diagram of an image processing system illustrated according to some embodiments of the present disclosure. Detailed Implementation
[0016] The term "coupled" as used in this article can also refer to "electrical coupling," and the term "connection" can also refer to "electrical connection." "Coupled" and "connection" can also refer to two or more components cooperating or interacting with each other.
[0017] refer to Figure 1 . Figure 1 This is a schematic diagram of an image processing system 100 illustrated according to some embodiments of the present disclosure. Figure 1 For example, image processing system 100 includes memory 110, memory 120, processor 140, and display 160. Memory 110 is coupled to memory 120. Processor 140 is coupled to memory 120 and display 160.
[0018] In some embodiments, memory 110 may be implemented using dynamic random access memory (DRAM). Memory 120 may be implemented using static random access memory (SRAM). Processor 140 may be implemented using digital logic circuitry. For example, processor 140 may be digital logic circuitry designed using register-transfer level (RTL). That is, these embodiments are primarily implemented in digital hardware. Display 160 may be implemented using a display panel or a touch display panel.
[0019] Also refer to Figure 1 as well as Figure 2 . Figure 2 This is a flowchart illustrating an image processing method 200 according to some embodiments of the present disclosure.
[0020] by Figure 2 For example, the image processing method 200 includes operations S210, S220, S230, S240, S250, S260 and S270.
[0021] In some embodiments, the image processing method 200 can be applied to Figure 1 The image processing system 100 is described herein, but this disclosure is not limited thereto. For ease of understanding, the following will be used in conjunction with... Figure 1 Let's have a discussion.
[0022] Please refer to the above. Figure 3 . Figure 3 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of operation S210 is provided. In operation S210, processor 140 generates sliding windows W1-W4 for target pixels P1-P4 among multiple pixels in image data IMG, where pixel P1 is located in the first column and first row, pixel P2 is located in the first column and second row, pixel P3 is located in the first column and third row, and pixel P4 is located in the second column and first row. In some embodiments, memory 110 is used to store the entire image data IMG, while memory 120 is used to temporarily store regional image data. For example, if sliding window W1 is to be generated, the image data within the range of sliding window W1 is transferred from memory 110 to memory 120 and temporarily stored in memory 120 for processing by processor 140. Since the image data within the range of sliding window W1 is stored in memory 120 column by column (line), memory 120 can be used as a line register.
[0023] by Figure 3 For example, firstly, the processor 140 can determine a sliding window W1 based on the target pixel P1, where the target pixel P1 is located at the exact center of the sliding window W1. The sliding window W1 can be rectangular. The length or width of the sliding window W1 can be designed according to actual needs. The brightness values of the portion of the sliding window W1 that exceeds the image data IMG can be compensated for using methods such as repetition, zero padding, or mapping.
[0024] Next, the processor 140 can determine sliding windows W2-W4 based on the target pixels P2-P4, where the target pixels P2-P4 are located at the exact center of the sliding windows W2-W4. The brightness values of the portion of the sliding windows W2-W4 that exceed the image data IMG can also be compensated for by repetition, zero padding, or mapping.
[0025] For simplicity and ease of understanding, Figure 3 Taking only four target pixels P1-P4 in the image data IMG as an example, other pixels have similar content. Applying the above operation to all pixels in the image data IMG will produce a sliding window for each pixel.
[0026] The following explanation will use the target pixel P1 and its corresponding sliding window W1 as an example; other pixels have similar content.
[0027] refer to Figure 4 . Figure 4 These are schematic diagrams of the original luminance histogram 400A and the low-bit luminance histogram 400B, drawn according to some embodiments of the present disclosure.
[0028] In operation S220, processor 140 generates the original luminance histogram 400A of sliding window W1 based on the original bit depth. In some embodiments, the original bit depth may be, for example, 8 bits, but this disclosure is not limited thereto. That is, if the original bit depth is 8 bits, the total number of original luminance values for each pixel is 256 (2^35 - 10^35) 8 The values are 0-255. The processor 140 can generate an original luminance histogram 400A based on the original luminance values (original bit depth) of all pixels in the sliding window W1.
[0029] In operation S230, processor 140 generates a low-bit luminance histogram 400B of sliding window W1 based on the low bit depth. In some embodiments, the low bit depth is less than the original bit depth. For example, the low bit depth may be 4 bits, but this disclosure is not limited thereto. That is, if the low bit depth is 4 bits, the low bit luminance value of each pixel will always be reduced to level 16 (2). 4 The values are 0-15. In some embodiments, the processor 140 can divide the original luminance value of each pixel by 16 and perform an unconditional discarding procedure to obtain the low-bit luminance value of each pixel. However, based on the above principle, any integer between 48 and 63 as the original luminance value will result in a calculated low-bit luminance value of 4. Next, the processor 140 can generate a low-bit luminance histogram 400B based on the low-bit luminance values (low-bit depth) of all pixels in the sliding window W1. The operation of S230 can be regarded as a "downsampling" process. That is, compared to the original luminance histogram 400A, the changes in the lines in the low-bit luminance histogram 400B are less drastic.
[0030] refer to Figure 5 . Figure 5 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of operation S240. In operation S240, processor 140 distinguishes the low-bit histogram 400B.
[0031] In some embodiments, the processor 140 may find two boundary points A and B, and a region boundary point C corresponding to at least one region minimum value, from the low-bit histogram 400B. Then, the processor 140 may divide the low-bit histogram 400B into a low-bit range DV1 and a low-bit range DV2 based on the brightness values (e.g., grayscale values 0 and 15) corresponding to the two boundary points A and B, and the brightness value (e.g., grayscale value 6) corresponding to the region boundary point C. Low-bit range DV1 is the range between boundary point A and region boundary point C. Low-bit range DV2 is the range between region boundary point C and boundary point B.
[0032] In some embodiments, the vertical axis value N of the region boundary point C must simultaneously satisfy the following two conditions:
[0033] N≤(N-1) and N≤(N+1)...Condition 1
[0034] N≠(N-1) and N≠(N+1)...Condition 2
[0035] Where N is the vertical axis value (number of occurrences) of the region boundary point C (i.e., N pixels have the brightness value of the region boundary point C), (N+1) is the vertical axis value (number of occurrences) of the right-hand point of the region boundary point C, and (N-1) is the vertical axis value (number of occurrences) of the left-hand point of the region boundary point C.
[0036] refer to Figure 6 . Figure 6 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of the operation of S250.
[0037] In operation S250, processor 140 determines the target low-bit range DV1 from the low-bit histogram 400B based on the target pixel P1. Figure 6 For example, since the low bit brightness value of the target pixel P1 is located in the low bit range DV1, the processor 140 determines the low bit range DV1 as the target low bit range.
[0038] refer to Figure 7 . Figure 7 The correspondence is illustrated according to some embodiments of this disclosure. Figure 2 A schematic diagram of operation S260.
[0039] In operation S260, processor 140 extracts a local original luminance histogram 400A' from the original luminance histogram 400A based on the target low-bit range DV1. In some embodiments, processor 140 may first align (map) the low-bit histogram 400B with the original luminance histogram 400A. Figure 7For example, based on the low bit ranges DV1 and DV2, the original luminance histogram 400A can also be divided into original bit ranges DV1' and DV2', where original bit range DV1' corresponds to low bit range DV1, and original bit range DV2' corresponds to low bit range DV2. Next, the processor 140 extracts the corresponding original bit range DV1' from the original luminance histogram 400A based on the target low bit range DV1 determined in operation S250, to form a local original luminance histogram 400A'. That is, the local original luminance histogram 400A' only includes the original bit range DV1', while removing the original bit range DV2'. For example, the vertical axis values (number of occurrences) of the original bit range DV2' portion can all be set to 0.
[0040] In operation S270, processor 140 performs a histogram equalization procedure on the local original luminance histogram 400A' based on the original bit depth to generate the final luminance of target pixel P1. Since the local original luminance histogram 400A' still corresponds to the original bit depth (e.g., 8-bit depth / 256 levels), processor 140 performs a histogram equalization procedure on the local original luminance histogram 400A' based on the original bit depth to generate the final luminance of target pixel P1.
[0041] The detailed operation of the histogram equalization procedure will be described in the following paragraphs.
[0042] First, the processor 140 executes a cumulative distribution function (CDF) program on the local original brightness histogram 400A' to calculate the cumulative distribution function values corresponding to all gray values in the local original brightness histogram 400A'.
[0043] Next, the processor 140 will obtain the equalized grayscale value according to the following formula (1):
[0044]
[0045] Where L is the order corresponding to the original bit depth (taking an original bit depth of 8 as an example, L is 256), cdf min The minimum value among all non-zero values of the cumulative distribution function (CDF) is given by CDF. max The cumulative distribution function is the maximum value among all non-zero values. cdf(n) is a target gray value (before equalization) in the local original brightness histogram 400A', and n is the index value. newHist(n) is the new gray value after equalization of this target gray value. The round function performs a rounding procedure on the value to make the final value an integer value.
[0046] In this disclosure, since the histogram equalization procedure is ultimately performed on the local original brightness histogram 400A' with the original bit depth, a high grayscale resolution range can still be obtained.
[0047] Some related techniques (e.g., Otsu's binarization algorithm) require a large number of recursive operations and can only divide the histogram into two ranges. This does not produce good results for images with a single brightness range or more than two brightness ranges.
[0048] In some other related techniques, the minimum value of all regions is directly scanned from the original brightness histogram with the original bit depth. However, as the original bit depth increases, the amount of scanning or computation required for the original brightness histogram also increases significantly. These factors are detrimental to the image processing process.
[0049] Compared to the aforementioned related technologies, in this disclosure, the processor 140 first generates a low-bit luminance histogram, and uses the target low-bit range DV1 where the target pixel P1 is located to extract a local original luminance histogram 400A' from the original luminance histogram 400A. Then, a histogram equalization procedure is performed on the local original luminance histogram 400A' based on the original bit depth. Since the low-bit range in this disclosure is not limited to only two (although...) Figure 5 (The example uses two low bit ranges DV1-DV2), therefore this disclosure is applicable to images with more than two brightness ranges.
[0050] Furthermore, as previously mentioned, the process of reducing the bit depth from the original bit depth to a lower bit depth in operation S230 of this disclosure can be considered a "downsampling" process. Compared to finding the region minimum on the original luminance histogram 400A, finding the region boundary point C corresponding to the region minimum on the lower bit luminance histogram 400B requires less computation. Accordingly, the processor 140 of this disclosure can have a smaller hardware area (e.g., reducing the number of comparators, multiplexers, adders, and multipliers required for finding the region minimum and performing low-pass filtering) and lower power consumption. Additionally, since the lower bit luminance histogram 400B is smoother, irrelevant noise interference can be avoided.
[0051] Furthermore, since the "downsampling" process also has the effect of low-pass filtering, the processor 140 does not need to design an additional low-pass filter circuit or perform additional low-pass filter operations.
[0052] Refer again Figure 7 In some embodiments, during the process of aligning (mapping) the low-bit histogram 400B with the original luminance histogram 400A in the above-described operation S260, the processor 140 performs a boundary adjustment procedure based more on the original bit depth.
[0053] by Figure 7 For example, since the low-bit range DV1 in the low-bit histogram 400B is used to capture the local original luminance histogram 400A', the processor 140 can perform a boundary adjustment procedure on the region boundary point C (low-bit luminance value 6) to the right of the low-bit range DV1.
[0054] As mentioned earlier, the low-bit luminance value is obtained by dividing the original luminance value by 16 and performing an unconditional truncation procedure. Equivalently, one low-bit luminance value (e.g., 6) corresponds to multiple original luminance values (e.g., any integer between 96 and 111). In this case, if the low-bit luminance value (e.g., 6) at region boundary point C is directly multiplied by 16 to deduce the corresponding grayscale value in the original luminance histogram 400A, only the grayscale value 96 will be obtained. Grayscale values from 97 to 111 will be omitted.
[0055] Based on the above problems, processor 140 can execute the boundary adjustment procedure according to the following formula (2):
[0056] S2 = S1 × 2 (N2-N1) +(2 (N2-N1) -1)...Formula (2)
[0057] Where S1 is the gray value of a boundary point in the low-bit histogram 400B, N2 is the original bit depth, N1 is the low bit depth, and S2 is the corresponding gray value in the original brightness histogram 400A.
[0058] by Figure 7 For example, the region boundary point C has a gray value of 6 (i.e., S1), the original bit depth is, for example, 8 (i.e., S2), and the low bit depth is, for example, 4 (i.e., N1). Based on the above formula (2), P2 can be calculated to be 111. That is, the region boundary point C in the low bit histogram 400B needs to be aligned with the position of the gray value 111 in the original brightness histogram 400A. In this way, gray values of 97-111 can be included to extract the appropriate original bit range DV1' from the original brightness histogram 400A.
[0059] The description of the image processing method 200 above includes exemplary operations, but these operations of the image processing method 200 need not be performed in the order shown. The order of these operations of the image processing method 200 may be changed, or these operations may be performed simultaneously, partially simultaneously, or partially omitted where appropriate, all within the spirit and scope of the embodiments of this disclosure.
[0060] refer to Figure 8 . Figure 8This is a schematic diagram illustrating an image processing system 800 according to some embodiments of the present disclosure. In some embodiments, the image processing method 200 may be applied to... Figure 8 Image processing system 800.
[0061] by Figure 8 For example, the image processing system 800 includes a memory 810, a memory 820, a processor 840, and a display 860. The processor 840 is coupled to the memory 810, the memory 820, and the display 860.
[0062] In some embodiments, memory 810 may be implemented using dynamic random access memory to store the aforementioned image data IMG. Memory 820 may be implemented using a non-transitory computer-readable storage medium and used to store one or more computer programs CP containing multiple instructions. Processor 840 may be implemented using a central processor or a microprocessor. When the computer program CP is executed by processor 840, the aforementioned image processing method 200 is executed. That is, these embodiments primarily implement the image processing method 200 in software. The display 860 may be implemented using a display panel or a touch display panel.
[0063] In embodiments primarily implemented in software, it can also be applied to images with more than two brightness ranges, and can achieve the effects of reducing computational load, increasing computational speed, and eliminating the need for additional low-pass filtering operations.
[0064] In summary, this disclosure first generates a low-bit luminance histogram, and then extracts a local original luminance histogram from the original luminance histogram using the target low-bit range where the target pixel is located. Finally, a histogram equalization process is performed on the local original luminance histogram based on the original bit depth. Therefore, this disclosure is applicable to images with more than two luminance ranges and can reduce hardware area, power consumption, or computational load. Furthermore, since low-pass filtering is also applied during the generation of the low-bit luminance histogram in this disclosure, no additional low-pass filtering circuitry or computation is required.
[0065] Although this disclosure has been described above with reference to embodiments, it is not intended to limit this disclosure. Any person skilled in the art may make various modifications and alterations without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be determined according to the scope of the claims.
[0066] [Symbol Explanation]
[0067] 100, 800: Image processing system
[0068] 110, 120, 810, 820: Memory
[0069] 140, 840: Processors
[0070] 160, 860: Monitor
[0071] 200: Image Processing Methods
[0072] 400A: Original brightness histogram
[0073] 400B: Low Bit Brightness Histogram
[0074] 400A': Local original brightness histogram
[0075] S210, S220, S230, S240, S250, S260, S270: Operation
[0076] IMG: Image Data
[0077] P1, P2, P3, P4: pixels
[0078] W1, W2, W3, W4: Sliding windows
[0079] A, B: Boundary points
[0080] C: Region boundary point
[0081] DV1, DV2: Low bit range
[0082] DV1', DV2': Original bit range
Claims
1. An image processing method, comprising: Generating a sliding window for a target pixel among multiple pixels in an image data by a processor; Generating an original brightness histogram of the sliding window by the processor according to an original bit depth; Generating a low-bit brightness histogram of the sliding window by the processor according to a low bit depth; Determining a target low-bit range from the low-bit brightness histogram by the processor according to the target pixel; Extracting a local original brightness histogram from the original brightness histogram by the processor according to the target low-bit range; And Performing a histogram equalization process on the local original brightness histogram by the processor according to the original bit depth to generate a final brightness of the target pixel.
2. The image processing method according to claim 1, wherein the low bit depth is less than the original bit depth.
3. The image processing method according to claim 1, wherein determining the target low-bit range from the low-bit brightness histogram by the processor according to the target pixel further comprises: Determining multiple boundary points and a region boundary point corresponding to at least one region minimum by the processor according to the low-bit brightness histogram; Determining multiple low-bit ranges by the processor according to the boundary points and the region boundary point; and Determining the target low-bit range from the multiple low-bit ranges by the processor according to a low-bit brightness value of the target pixel.
4. The image processing method according to claim 1, further comprising: Performing a boundary adjustment process by the processor according to the original bit depth.
5. The image processing method according to claim 4, wherein the processor performs the boundary adjustment process according to the following formula: where S1 is a gray value of a boundary point in the low-bit brightness histogram, N2 is the original bit depth, N1 is the low bit depth, and S2 is a corresponding gray value in the original brightness histogram.
6. An image processing system, comprising: A memory for storing an image data; and A processor for performing the following operations: Generating a sliding window for a target pixel among multiple pixels in the image data; Generating an original brightness histogram of the sliding window according to an original bit depth; Generating a low-bit brightness histogram of the sliding window according to a low bit depth; Determining a target low-bit range from the low-bit brightness histogram according to the target pixel; Extracting a local original brightness histogram from the original brightness histogram according to the target low-bit range; And Performing a histogram equalization process on the local original brightness histogram according to the original bit depth to generate a final brightness of the target pixel.
7. The image processing system according to claim 6, wherein the processor is a digital logic circuit designed using register transfer level, and the memory is a line register.
8. The image processing system according to claim 6, wherein the low bit depth is less than the original bit depth.
9. A non-transitory computer-readable storage medium for storing one or more computer programs containing a plurality of instructions, a processor for executing the instructions, and when the processor executes the instructions, the processor performs the following operations: Generating a sliding window for a target pixel among a plurality of pixels in an image data; Generating an original luminance histogram of the sliding window according to an original bit depth; Generating a low-bit luminance histogram of the sliding window according to a low bit depth; Determining a target low-bit range from the low-bit luminance histogram according to the target pixel; Generating a local original luminance histogram according to the target low-bit range; And Performing a histogram equalization process on the local original luminance histogram according to the original bit depth to generate a final luminance of the target pixel.
10. The non-transitory computer-readable storage medium according to claim 9, wherein the low bit depth is less than the original bit depth.