A processing method for image enhancement

Abstract

This invention provides a processing method for image enhancement, comprising the following steps: first, receiving an input image; converting the input image to a color space that separates luminance and color; then, generating a first target luminance histogram based on the original luminance information of the input image; next, performing a first pixel reconfiguration procedure based on the first target luminance histogram to generate a first intermediate image; then, generating a second target luminance histogram based on the first intermediate image; next, performing a second pixel reconfiguration procedure based on the second target luminance histogram to generate a second intermediate image; and finally, combining the second intermediate image with the original color information and performing inversion color matching processing to output an enhanced image with natural visual effects and clear details from limited single image information.

Description

Technical Field

[0001] This invention relates to an image processing method, and more particularly to an image enhancement processing method. Prior Technology

[0002] In digital imaging technology, High Dynamic Range (HDR) images aim to present a wider range of light and dark areas than standard digital images, thus more closely resembling the real-world scenes perceived by the human visual system. Traditional consumer digital cameras capture Low Dynamic Range (LDR) images, such as JPEG or BMP formats, where brightness is typically quantized to 8 bits, representing only 256 brightness levels. This limitation results in overexposure of bright details or loss of dark details in scenes containing both highlights and deep shadows, failing to fully reproduce the rich tonal gradations of a real scene.

[0003] To address this issue, early mainstream HDR technology employed a multiple exposure fusion strategy. This method requires capturing multiple LDR images of the same scene from a fixed position with different exposure values, and then using algorithms to fuse these images into a single HDR image. While this technique achieves excellent results, its application scenarios are severely limited. If the subject includes moving objects (such as pedestrians or vehicles) or the scene itself exhibits dynamic changes (such as drifting clouds or swaying leaves), the synthesized HDR image will produce unnatural flaws, and the shooting process is quite cumbersome.

[0004] In recent years, research has gradually shifted towards techniques for generating HDR images from single LDR images. These techniques eliminate the need for multiple exposures, significantly improving ease of use. However, existing single-image enhancement methods, such as simple global histogram equalization, while improving overall contrast, often sacrifice local detail or introduce color distortion and increased noise after processing. How to recover rich highlights and shadows from limited single LDR image information and generate visually natural and detailed HDR images remains a pressing technical challenge in this field. Summary of the Invention

[0005] The main objective of this invention is to provide a processing method for image enhancement. This method includes the following steps: first, receiving an input image; converting the input image to a color space that separates luminance and color; then, generating a first target luminance histogram based on the original luminance information of the input image; next, performing a first pixel reconfiguration procedure based on the first target luminance histogram to generate a first intermediate image; then, generating a second target luminance histogram based on the first intermediate image; then, performing a second pixel reconfiguration procedure based on the second target luminance histogram to generate a second intermediate image; finally, combining the second intermediate image with the original color information and performing inversion color matching processing to output an enhanced image with natural visual effects and clear details from limited single image information. This effectively solves the problem in the prior art that when generating a high dynamic range (HDR) image from a single low dynamic range (LDR) image, local details are easily lost and the processing flow is complex.

[0006] To achieve the above objectives, the present invention provides a processing method for image enhancement, comprising: Step S1: providing an input image, the input image comprising a plurality of pixels; Step S2: converting the color space of the input image from a red primary color-green primary color-blue primary color (RGB) color space to a hue-saturation-luminance (HSI) color space to separate an original luminance information and an original color information of the input image; Step S3: generating a first target luminance histogram based on the original luminance information, wherein step S3 further comprises: Step S3-1: expanding the dynamic range of the original luminance histogram of the original luminance information to [0, 1023] using a bicubic interpolation algorithm to generate an expanded histogram, wherein the expanded histogram contains a plurality of empty intervals with a pixel count of 0; Step S3-2: applying the bicubic interpolation algorithm to upsample and downsample the image corresponding to the expanded histogram. Step S3-2: Downsample processing is performed to fill in the gaps where multiple pixels in the expanded histogram have a value of 0; Step S4: A first pixel reconfiguration procedure is executed using the Connected Component algorithm. Based on the spatial proximity and brightness similarity of multiple pixels in the original brightness histogram, multiple pixels in the original brightness histogram are reconfigured to generate a first intermediate image, corresponding to the first target brightness histogram; Step S5: The intermediate image histogram is extended using the Cumulative Distribution Function statistical method based on the intermediate image histogram of the first intermediate image to generate a second target brightness histogram; Step S6: Using the first intermediate image as the source, the Connected Component algorithm is used to... The Component algorithm executes a second pixel reconfiguration procedure, corresponding to the second target brightness histogram, reconfiguring a plurality of pixels in the intermediate image histogram to generate a second intermediate image; and step S7: converting the color space of the second intermediate image from the hue-saturation-brightness color space to the red primary color-green primary color-blue primary color space, and combining the second intermediate image with the original color information, and processing it using the Reverse Tone Mapping algorithm to generate an enhanced image.

[0007] In a preferred embodiment of the present invention, in step S4, when the pixel requirement of the first target brightness histogram is greater than the total number of pixels in the currently processed original brightness histogram, the entire source pixel cluster is first configured with the target brightness value. For the insufficient number, supplementary pixels are searched in the candidate pixel cluster, which consists of other pixels that have not yet been configured and whose brightness values ​​are within a preset pane of the source pixel cluster.

[0008] In a preferred embodiment of the present invention, when the candidate pixel cluster consists of other pixels that have not yet been configured and whose brightness values ​​are within the preset pane of the source pixel cluster, the selection of supplementary pixels is further evaluated by a cost function, and the pixel with the lowest cost value is selected as the supplementary pixel.

[0009] In a preferred embodiment of the present invention, in step S4, when the pixel requirement of the first target brightness histogram is exactly equal to the total number of pixels of the original brightness histogram currently being processed, all pixels of the source pixel cluster are directly configured to the target brightness value.

[0010] In a preferred embodiment of the present invention, in step S4, when the pixel requirement of the first target brightness histogram is less than the total number of pixels in the currently processed original brightness histogram, some pixels need to be discarded from the source pixel cluster. The principle of discarding is to first calculate the geometric center of the cluster as a representative point, then calculate the spatial distance of each pixel in the cluster to this representative point, and prioritize discarding the pixels with the farthest distance until the number of remaining pixels meets the requirement.

[0011] In a preferred embodiment of the present invention, steps S5 and S6 may be repeated multiple times.

[0012] In a preferred embodiment of the present invention, the first pixel reconfiguration procedure in step S4 further includes: when there is no pixel cluster in the pixel set corresponding to a specific original brightness value of a specific original brightness histogram, and the pixel demand for a target brightness value in the first target brightness histogram is greater than 0, firstly, it is determined whether there are multiple non-clustered pixels in the pixel set corresponding to the specific original brightness value; when the pixel demand is less than or equal to the total number of the multiple non-clustered pixels, the average brightness value in the neighborhood range of each non-clustered pixel is calculated, and according to the average brightness value, the non-clustered pixels with lower average brightness values ​​are preferentially reconfigured to the target brightness value; and when the pixel demand is greater than the total number of the multiple non-clustered pixels, all the non-clustered pixels are reconfigured to the target brightness value, the pixel demand is updated, and then pixels are searched from the pixel set corresponding to the original brightness value of another original brightness histogram for supplementary configuration.

[0013] In a preferred embodiment of the present invention, the first pixel reconfiguration procedure in step S4 further includes: when there is at least one pixel cluster in the pixel set corresponding to a specific original brightness value of a specific original brightness histogram, and the pixel requirement of a target brightness value in the first target brightness histogram is equal to 1, firstly determining whether there is at least one non-clustered pixel in the pixel set corresponding to the specific original brightness value; if there is at least one non-clustered pixel, then randomly selecting one of them to reconfigure to the target brightness value to maintain the integrity of the at least one pixel cluster; and if there is no non-clustered pixel, then finding a pixel from the pixel set corresponding to an original brightness value of another original brightness histogram for configuration. Simple Explanation of the Diagram

[0014] Figure 1 is a flowchart of the processing method of the present invention. Implementation

[0015] The structure and technical features of the present invention are described in detail below with reference to the illustrations. The illustrations are only used to illustrate the structural relationships and related functions of the present invention. Therefore, the dimensions of each component in the illustrations are not drawn to actual scale and are not intended to limit the present invention.

[0016] Referring to Figure 1, the present invention provides a processing method for image enhancement, comprising:

[0017] Step S1: Provide an input image 10 to a computer, embedded system, or image processing chip for processing, but not limited thereto. The input image 10 contains a plurality of pixels. The input image 10 contains JPEG or BMP format images taken by a standard digital camera, but not limited thereto. The input image 10 is a low dynamic range (LDR) image.

[0018] Step S2: Convert the color space of the input image 10 from the red primary color-green primary color-blue primary color (RGB) color space to the hue-saturation-luminance (HSI) color space to separate the original luminance information 11 and the original color information 12 of the input image 10. The brightness and color of the image are processed independently to avoid color distortion. The subsequent histogram processing and pixel reconfiguration will mainly target the original luminance information 11, while the original color information 12 will be temporarily stored and used in the subsequent step S7.

[0019] Step S3: Based on the original brightness information 11, a first target brightness histogram 20 is generated. This step aims to create an initial, improved target histogram to guide the first pixel reconfiguration.

[0020] Step S3 further includes: Step S3-1: Expanding the dynamic range of the original brightness histogram 111 of the original brightness information 11 to [0, 1023] using a bicubic interpolation algorithm to generate an expanded histogram 112, wherein the expanded histogram 112 contains a plurality of empty intervals with a pixel value of 0; Step S3-2: Applying the bicubic interpolation algorithm to perform image upsampling and image downsampling on the image corresponding to the expanded histogram 112 to fill the plurality of empty intervals with a pixel value of 0 in the expanded histogram 112. This process utilizes the bicubic interpolation algorithm to generate new brightness values ​​between pixels, thereby effectively filling the empty intervals generated in the histogram in step S3-1, forming a smoothly distributed expanded histogram 112; and Step S3-3: Applying a Butterworth filter. The Butterworth filter adjusts the pixel distribution of the expanded histogram 112 after processing in step S3-2 to generate the first target brightness histogram 20. The Butterworth filter is used to shape the histogram, mainly to suppress the pixel saturation phenomenon that may occur at both ends (extremely dark and extremely bright areas) of the expanded histogram 112, and to soften the overall distribution. For example, the peak area of ​​the processed first target brightness histogram 20 is retained, while the beginning and end ends are moderately suppressed. In addition, the parameters of the Butterworth filter can be adjusted according to the image characteristics.

[0021] Step S4: A first pixel reconfiguration procedure is executed using the Connected Component algorithm. Based on the spatial proximity and brightness similarity of a plurality of pixels in the original brightness histogram 111, corresponding to the first target brightness histogram 20, the plurality of pixels in the original brightness histogram 111 are reconfigured to generate a first intermediate image 30. Step S4 mainly involves layering the input image 10 according to brightness values. Each layer is a binary image containing only pixels with brightness values. The Connected Component algorithm is applied to each layer to identify source pixel clusters composed of spatially adjacent pixels with the same brightness value. Each brightness value of the first target brightness histogram 20 is traversed sequentially (e.g., from low brightness to high brightness), and pixels are moved from the input image 10 according to their pixel requirements. This moving operation considers the following three situations:

[0022] Scenario 1 (Demand > Supply): When the pixel demand for the target brightness value (e.g., the first target brightness histogram 20) is greater than the total number of pixels in the currently processed source pixel cluster (e.g., the original brightness histogram 111), the entire source pixel cluster is first allocated to the target brightness value. For any shortfall, supplementary pixels are searched in the candidate pixel cluster, which consists of other pixels that have not yet been allocated and whose brightness values ​​are within the preset pane of the source pixel cluster. When the candidate pixel cluster consists of other pixels that have not yet been allocated and whose brightness values ​​are within the preset pane of the source pixel cluster, the selection of supplementary pixels is further evaluated using a cost function, prioritizing the selection of pixels with the lowest cost value (i.e., representing close spatial distance and high brightness similarity) for supplementation.

[0023] Scenario 2 (Demand = Supply): When the pixel demand of the target brightness value (such as the first target brightness histogram 20) is exactly equal to the total number of pixels in the current source pixel cluster (such as the original brightness histogram 111), then all pixels of the source pixel cluster are directly and uniformly allocated to the target brightness value.

[0024] Scenario 3 (Demand < Supply): When the pixel demand for the target brightness value (e.g., the first target brightness histogram 20) is less than the total number of pixels in the currently processed source pixel cluster (e.g., the original brightness histogram 111), some pixels need to be discarded from the source pixel cluster. The principle of discarding is to first calculate the geometric center of the cluster as a representative point, then calculate the spatial distance of each pixel in the cluster to this representative point, and prioritize discarding the pixels with the farthest distance, until the number of remaining pixels meets the demand.

[0025] Step S5: Based on the intermediate image histogram 31 of the first intermediate image 30, the intermediate image histogram 31 is extended using the cumulative distribution function statistical method to generate a second target brightness histogram 40. The reason for generating the second target brightness histogram 40 is that although the first intermediate image 30 has been optimized, its histogram distribution may still not fill the entire 10-bit dynamic range. Therefore, this step extends the brightness histogram of the first intermediate image 30. Taking the standard point of the histogram (such as the peak point) as the center, the cumulative distribution function curve is shifted and extended to the left and right ends until it touches the boundary of the dynamic range. Based on the extended cumulative distribution function, the second target brightness histogram 40 with a wider distribution can be obtained.

[0026] Step S6: Using the first intermediate image 30 as the source, a second pixel reconfiguration procedure is executed using the Connected Component algorithm. Corresponding to the second target brightness histogram 40, a plurality of pixels in the intermediate image histogram 31 are reconfigured to generate a second intermediate image 50. In some embodiments of the present invention, steps S5 and S6 can be repeated multiple times without limitation. Each time, the histogram is further extended until a preset expansion width is reached or the effect converges.

[0027] Step S7: Convert the color space of the second intermediate image 50 from the hue-saturation-luminance color space to the red primary color-green primary color-blue primary color space, and combine the second intermediate image 50 with the original color information 12, and process it using the reverse tone mapping algorithm to produce an enhanced image 60 with rich details and sharp contrast; wherein the enhanced image 60 is a high dynamic range (HDR) image.

[0028] As can be seen from the above, the processing method of the present invention effectively solves the problem in the prior art where generating a high dynamic range image from a single low dynamic range image often results in the loss of local details due to corresponding processing of the entire video. Instead, it performs multi-stage targeted shaping of the image brightness histogram and executes a fine pixel-level reconfiguration procedure to maximize the preservation and enhancement of local details and textures of the image while expanding the dynamic range. Furthermore, the present invention uses an algorithmic approach that requires only a single low dynamic range image to generate a high dynamic range image, eliminating the need for a large number of low dynamic range images as required by existing technologies, thus saving costs.

[0029] Furthermore, in order to more accurately process the more discrete pixels in the image, the processing method of the present invention, when executing the first pixel reconfiguration procedure (step S4), designs a special processing mechanism, but is not limited thereto. This mechanism corresponds to a specific situation: when processing a set of pixels corresponding to a specific original brightness value of a specific original brightness histogram 111, if the analysis by the connected tube principle algorithm reveals that there is no pixel cluster consisting of two or more adjacent pixels in the set, but the first target brightness histogram 20 still has a pixel requirement for a certain target brightness value, then the following procedure is initiated:

[0030] First, the system will determine whether there are any "non-clustered pixels" (i.e., pixels that exist alone) in the pixel set.

[0031] In one embodiment, if the number of pixels required for the target brightness value is less than or equal to the total number of detected non-clustered pixels, in order to make the configured image visually smoother and avoid abrupt noise, this method calculates the average brightness value within a preset neighborhood range (e.g., a 3x3 pixel range) around each non-clustered pixel. Then, based on the calculated average brightness values, they are sorted from low to high, and non-clustered pixels with darker surrounding environments (i.e., lower average brightness values) are preferentially reconfigured with the target brightness value until the required amount is met. This ensures that the brightness difference between the configured pixel and its new neighborhood is small, reducing the risk of visual discontinuity.

[0032] In another embodiment, if the pixel requirement for the target brightness value is greater than the total number of detected non-clustered pixels, it means that the current pixel set cannot meet the requirement alone. In this case, the system will reconfigure all non-clustered pixels in this set to the target brightness value and deduct the configured number from the total requirement to update the pixel requirement. Subsequently, the process will move to the pixel set corresponding to the original brightness value of another original brightness histogram 111 in the next order to continue searching for suitable pixels to make up for the remaining requirement.

[0033] In addition to the above-described processing method, the processing method of the present invention also considers another special case, but is not limited, when executing the first pixel reconfiguration procedure (step S4), the purpose of which is to protect the existing structural features in the image from being easily destroyed. This case occurs when: at least one pixel cluster has been detected in a pixel set corresponding to a specific original brightness value in a specific original brightness histogram 111, but the pixel requirement of the first target brightness histogram 20 for a certain target brightness value is exactly 1.

[0034] In this context, directly splitting a pixel from a pixel cluster for configuration would disrupt the structural integrity of the cluster, potentially causing breaks in the original lines or textures of the image. Therefore, the processing principle of this invention is to preserve the integrity of the pixel cluster as much as possible.

[0035] Specifically, the system first searches for non-clustered pixels within the set of pixels with the same brightness value. If such a single pixel exists, the system randomly selects one and reconfigures it to the target brightness value. In this way, the requirements of the target histogram are met without affecting the structure of any pixel clusters.

[0036] If all pixels with this brightness value in the pixel set have formed a pixel cluster, meaning no usable non-clustered pixels can be found, in order to avoid disrupting the clusters, the system will skip the current pixel set and instead search for suitable pixels for configuration from the pixel set of the original brightness values ​​corresponding to the next-order original brightness histogram 111. This strategy ensures the flexibility of image configuration and the preservation of the original image features.

[0037] In summary, the advantages of this processing method compared to prior art are as follows:

[0038] This invention, through multi-stage target histogram shaping and a pixel-level reconfiguration process that considers spatial and brightness correlations, successfully generates high-quality HDR images using only a single LDR image, effectively overcoming the shortcomings of previous technologies and possessing significant industrial application value.

[0039] In addition, for special cases in the pixel reconfiguration process, the present invention has also designed a sophisticated processing mechanism, as described in paragraphs

[0028] to

[0036] above. The mechanism addresses the following situations: when the source pixels are discrete, unclustered, non-clustered pixels, the configuration order is determined by evaluating the brightness of their neighborhoods to ensure visual smoothness; or when the target demand is extremely small (e.g., only 1), a strategy of prioritizing the configuration of non-clustered pixels is adopted to avoid disrupting the already formed pixel cluster structure. These mechanisms ensure the robustness and high-quality output of the present invention when processing various types of images.

[0040] The above are merely preferred embodiments of the present invention and are illustrative rather than restrictive. Those skilled in the art will understand that many changes, modifications, and even equivalent alterations can be made within the spirit and scope defined in the claims of the present invention, but all such changes will fall within the protection scope of the present invention.

[0041] 10: Input Image 11: Raw brightness information 111: Original brightness histogram 112: Expanded Histogram 12: Original Color Information 20: First target brightness histogram 30: First Intermediary Image 31: Histogram of Intermediate Images 40: Second target brightness histogram 50: Second Intermediary Image 60: Enhanced Image

Claims

1. A processing method for image enhancement, comprising: Step S1: providing an input image, the input image comprising a plurality of pixels; Step S2: converting the color space of the input image from a red primary color-green primary color-blue primary color (RGB) color space to a hue-saturation-luminance (HSI) color space to separate an original luminance information and an original color information of the input image; Step S3: generating a first target luminance histogram based on the original luminance information, wherein step S3 further comprises: Step S3-1: expanding the dynamic range of the original luminance histogram of the original luminance information to [0, 1023] using a bicubic interpolation algorithm to generate an expanded histogram, wherein the expanded histogram contains a plurality of empty intervals with a pixel count of 0; Step S3-2: applying the bicubic interpolation algorithm to upsample and downsample the image corresponding to the expanded histogram. Step S3-2: Downsample the expanded histogram to fill in the gaps where multiple pixels have a value of 0; Step S4-3: Apply a Butterworth filter to adjust the pixel distribution of the expanded histogram after Step S3-2 to generate the first target brightness histogram; Step S5-6: Use the Connected Component algorithm to perform a first pixel reconfiguration procedure, based on the spatial proximity and brightness similarity of multiple pixels in the original brightness histogram, to reconfigure multiple pixels in the original brightness histogram to generate a first intermediate image; Step S6-7: Extend the intermediate image histogram using the Cumulative Distribution Function (CDF) statistical method based on the intermediate image histogram of the first intermediate image to generate a second target brightness histogram; Step S7-8-9: Using the first intermediate image as the source, apply the Connected Component algorithm to generate a second target brightness histogram. The Component algorithm executes a second pixel reconfiguration procedure, corresponding to the second target brightness histogram, to reconfigure a plurality of pixels in the intermediate image histogram to generate a second intermediate image; and step S7: convert the color space of the second intermediate image from the hue-saturation-brightness color space to the red primary color-green primary color-blue primary color space, and combine the second intermediate image with the original color information, and process it using the Reverse Tone Mapping algorithm to generate an enhanced image.

2. The processing method as described in claim 1, wherein in step S4, when the pixel requirement of the first target brightness histogram is greater than the total number of pixels of the original brightness histogram currently being processed, the entire source pixel cluster is first configured with the target brightness value. For the insufficient number, supplementary pixels are searched in the candidate pixel cluster, which consists of other pixels that have not yet been configured and whose brightness values ​​are within the preset pane of the source pixel cluster.

3. The processing method as described in claim 2, wherein when the candidate pixel cluster consists of other pixels that have not yet been configured and whose brightness values ​​are within the preset pane of the source pixel cluster, the selection of supplementary pixels is further evaluated by a cost function, prioritizing the selection of the pixel with the lowest cost value for supplementation.

4. The processing method as described in claim 1, wherein in step S4, when the pixel requirement of the first target brightness histogram is exactly equal to the total number of pixels of the original brightness histogram currently being processed, all pixels of the source pixel cluster are directly configured as a whole to the target brightness value.

5. The processing method as described in claim 1, wherein in step S4, when the pixel requirement of the first target brightness histogram is less than the total number of pixels of the original brightness histogram currently being processed, some pixels need to be discarded from the source pixel cluster; the principle of discarding is to first calculate the geometric center of the cluster as a representative point, then calculate the spatial distance of each pixel in the cluster to this representative point, and discard the pixels with the farthest distance first, until the number of remaining pixels meets the requirement.

6. The processing method as described in Request 1, wherein step S5 and step S6 may be repeated multiple times.

7. The processing method as described in claim 1, wherein the first pixel reconfiguration procedure in step S4 further comprises: when there is no pixel cluster in the pixel set corresponding to a specific original brightness value of a specific original brightness histogram, and the pixel demand for a target brightness value in the first target brightness histogram is greater than 0, firstly determining whether there are multiple non-clustered pixels in the pixel set corresponding to the specific original brightness value; when the pixel demand is less than or equal to the total number of the multiple non-clustered pixels, calculating the average brightness value within the neighborhood of each non-clustered pixel, and reconfiguring the non-clustered pixel with the lower average brightness value to the target brightness value based on the average brightness value; and when the pixel demand is greater than the total number of the multiple non-clustered pixels, reconfiguring all the non-clustered pixels to the target brightness value, updating the pixel demand, and then finding pixels from the pixel set corresponding to the original brightness value of another original brightness histogram for supplementary configuration.

8. The processing method as described in claim 1, wherein the first pixel reconfiguration procedure in step S4 further comprises: when there is at least one pixel cluster in the pixel set corresponding to a specific original brightness value of a specific original brightness histogram, and the pixel requirement of a target brightness value in the first target brightness histogram is equal to 1, firstly determining whether there is at least one non-clustered pixel in the pixel set corresponding to the specific original brightness value; if there is at least one non-clustered pixel, then randomly selecting one of them to reconfigure to the target brightness value to maintain the integrity of the at least one pixel cluster; and if there is no non-clustered pixel, then finding a pixel from the pixel set corresponding to an original brightness value of another original brightness histogram for configuration.

Images