Image enhancement method and device based on camera response function nonlinear information color block

Abstract

The application provides an image enhancement method and device based on a camera response function nonlinear information color block, relates to the technical field of image processing, and can restore details and colors of non-normal exposure areas in the image to be enhanced by multiple different degrees of exposure of the image to be enhanced, improve the robustness of CRF estimation in the image to be enhanced, especially the robustness of CRF estimation in the image to be enhanced under noise and non-normal exposure conditions. Moreover, the downsampling operation is respectively performed on each exposure image, lossless reduction of each exposure image can be realized, and color blocks with richer color types and changes are obtained. By automatically selecting the camera response function nonlinear information color block, manual intervention can be avoided, noise can be inhibited, and the efficiency and accuracy of color block selection can be improved.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image enhancement method and apparatus based on nonlinear information color blocks of camera response function. Background Technology

[0002] Low-brightness image enhancement is an important research direction in the field of computer vision and has been widely applied in scenarios such as night shooting, security monitoring, and medical image analysis. However, low-brightness images usually have the following problems: information in dark areas is difficult to identify, resulting in blurry image content; images taken under low-light conditions have greater noise, and traditional enhancement algorithms struggle to guarantee detail and naturalness; color deviations are easily introduced during image enhancement, compromising the realism of the original scene.

[0003] The Camera Response Function (CRF) describes the nonlinear relationship between image pixel values ​​and scene radiance. Accurate CRF estimation is crucial for enhancing low-brightness images, as linearization of the CRF allows for better recovery of image brightness and color. However, current CRF-based low-brightness image enhancement methods often rely on direct image traversal or manual selection, deliberately choosing color patches with color edges, color mixing, and significant color differences as CRF nonlinear information patches, which is difficult to achieve. Consequently, the fitted CRF exhibits significant bias, making it difficult to calculate the true CRF.

[0004] Therefore, there is an urgent need to provide an image enhancement method based on nonlinear information color patches of the camera response function. Summary of the Invention

[0005] This invention provides an image enhancement method and apparatus based on nonlinear information color blocks of camera response function, in order to overcome the deficiencies existing in related technologies.

[0006] This invention provides an image enhancement method based on nonlinear information color patches of camera response function, comprising:

[0007] The image to be enhanced is acquired, and the image to be enhanced is exposed to multiple times at different degrees to obtain multiple exposed images;

[0008] Perform downsampling on each exposed image to determine the downsampled brightness image corresponding to each exposed image; in the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value.

[0009] Traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks for the camera response function based on the comprehensive score.

[0010] Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, the camera response function is fitted to obtain the camera response function.

[0011] Based on the camera response function, the image to be enhanced is converted to linear space to obtain a backup image, and the brightness of the backup image is increased to obtain the enhanced target image.

[0012] According to the present invention, an image enhancement method based on nonlinear information color patches of camera response function is provided, wherein performing downsampling operation on each exposure image to determine the downsampled brightness image corresponding to each exposure image includes:

[0013] Based on the superpixel segmentation algorithm, a downsampling operation is performed on each exposed image to determine the pixel region in each exposed image;

[0014] Based on the brightness values ​​of each pixel in each pixel region, determine the representative brightness value of each pixel region;

[0015] The downsampled brightness image is determined by using pixel regions in each exposed image as pixels and the representative brightness value of each pixel region as the pixel value.

[0016] According to the image enhancement method based on nonlinear information color patches of camera response function provided by the present invention, the step of performing downsampling operation on each exposure image to determine the downsampled brightness image corresponding to each exposure image includes:

[0017] Each exposed image is filtered to obtain a filtered image;

[0018] The filtered image is then normalized.

[0019] According to the present invention, an image enhancement method based on nonlinear information color patches of camera response function is provided, wherein traversing each color patch containing a preset number of pixels in each downsampled brightness image further includes:

[0020] For any downsampled brightness image in each downsampled brightness image, if it is impossible to traverse a color block containing the preset number of pixels in any downsampled brightness image, or if the root mean square error of the intensity and / or the intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal is stopped.

[0021] According to the image enhancement method based on nonlinear information color patches of camera response function provided by the present invention, the intensity range coverage of each color patch is determined based on the following steps:

[0022] Determine the number of different intensity value types in each channel of each color patch;

[0023] Calculate the ratio of the number of different intensity value types to the number of maximum intensity value types in each channel of each color block, and determine the intensity range coverage of each color block based on the ratio corresponding to each channel.

[0024] According to the present invention, an image enhancement method based on a nonlinear information color patch of camera response function is provided, wherein the camera response function is obtained by fitting the pixel value of each pixel in the nonlinear information color patch of camera response function and the scene radiance, comprising:

[0025] Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, an objective function is constructed.

[0026] The objective function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated objective function to obtain the camera response function.

[0027] According to the present invention, an image enhancement method based on nonlinear information color patches of camera response function is provided, wherein the step of enhancing the brightness of the backup image to obtain the enhanced target image includes:

[0028] The brightness of the backup image is enhanced based on the gamma correction algorithm to obtain the target image.

[0029] The present invention also provides an image enhancement device based on nonlinear information color patches of camera response function, comprising:

[0030] A multi-scale exposure image acquisition module is used to acquire an image to be enhanced and expose the image to be enhanced multiple times at different degrees to obtain multiple exposure images;

[0031] The downsampling module is used to perform downsampling operations on each exposed image to determine the downsampled brightness image corresponding to each exposed image. In the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value.

[0032] The color block determination module is used to traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks of the camera response function based on the comprehensive score.

[0033] The fitting module is used to fit the camera response function based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance.

[0034] The brightness enhancement module is used to convert the image to be enhanced to a linear space based on the camera response function to obtain a backup image, and then enhance the brightness of the backup image to obtain the enhanced target image.

[0035] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image enhancement method based on nonlinear information color patches of camera response function as described above.

[0036] The present invention also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image enhancement method based on nonlinear information color patches of camera response function as described above.

[0037] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the image enhancement method based on nonlinear information color patches of camera response function as described above.

[0038] The image enhancement method and apparatus based on nonlinear information color patches of the camera response function provided by this invention can restore the details and colors of abnormally exposed areas in the image to be enhanced by subjecting it to multiple exposures of varying degrees, thereby improving the robustness of CRF estimation in the image to be enhanced, especially under noisy and abnormal exposure conditions. Furthermore, by performing downsampling operations on each exposed image, lossless reduction of each image can be achieved to obtain color patches with richer color variety and variation. By calculating the root mean square error of intensity and intensity range coverage of each color patch, a comprehensive score for each color patch can be calculated, allowing for quantitative evaluation of each patch. Using the comprehensive score of each color patch as a basis, nonlinear information color patches of the camera response function are automatically selected, eliminating the need for manual intervention, suppressing noise, and improving the efficiency and accuracy of color patch selection. Furthermore, the camera response function can be accurately fitted using the pixel values ​​of each pixel in the nonlinear information color patch of the camera response function and the scene radiance. Using this camera response function, brightness enhancement of the image to be enhanced yields a target image with restored dark details, restored colors, uniform brightness, no significant color difference, and a clearer, more natural appearance. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in this invention or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on the drawings described below without creative effort.

[0040] Figure 1 This is a flowchart illustrating the image enhancement method based on nonlinear information color blocks of camera response function provided by the present invention.

[0041] Figure 2 This is a schematic diagram of the image enhancement device based on nonlinear information color blocks of camera response function provided by the present invention;

[0042] Figure 3 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0043] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0044] The terms "first" and "second" in the specification and claims of this invention may explicitly or implicitly include one or more of those features. In the description of the invention, unless otherwise stated, "a plurality of" means two or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0045] To address the technical problems existing in current image enhancement methods, such as Figure 1 As shown, this embodiment of the invention provides an image enhancement method based on nonlinear information color patches of the camera response function, the method comprising:

[0046] S1, acquire the image to be enhanced, and expose the image to be enhanced multiple times at different degrees to obtain multiple exposed images;

[0047] S2, perform downsampling operation on each exposed image to determine the downsampled brightness image corresponding to each exposed image; in the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value.

[0048] S3, traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks of the camera response function based on the comprehensive score.

[0049] S4. Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, the camera response function is fitted to obtain the camera response function.

[0050] S5. Based on the camera response function, the image to be enhanced is converted to linear space to obtain a backup image, and the brightness of the backup image is increased to obtain the enhanced target image.

[0051] Specifically, the image enhancement method based on nonlinear information color blocks of camera response function provided in this embodiment of the invention is executed by an image enhancement device based on nonlinear information color blocks of camera response function. The device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, tablet, etc., and no specific limitation is made here.

[0052] First, step S1 is executed to acquire the image to be enhanced. This image can be a color image acquired by an image acquisition device, and it must be a low-brightness image that requires brightness enhancement. The image acquisition device can be a camera, which can be mounted on a robot; no specific limitation is made here.

[0053] Since the image to be enhanced may contain underexposed or overexposed areas, these areas can easily affect the extraction of information from color blocks. To address the problem of lost dark information in the image to be enhanced, the image can be exposed multiple times at different levels to obtain multiple exposure images under different exposure conditions.

[0054] First, you can set multiple exposure scales. , This is a fixed parameter, representing the maximum allowable adjustment factor.

[0055] Then, multiple different adjustment coefficients are selected from the multi-scale exposure range to determine the exposure coefficients. The image to be enhanced is then exposed to different degrees. For example, five different adjustment coefficients can be selected from the multi-scale exposure range to determine the exposure coefficients. The image to be enhanced is then randomly exposed using each exposure coefficient to obtain five exposure images. The image to be enhanced can be regarded as an exposure image with an exposure level of 0 (i.e., an exposure coefficient of 1).

[0056] Each exposure image can satisfy:

[0057] ;

[0058] ;

[0059] in, Let r be the r-th adjustment coefficient, where .when At that time, the multi-scale exposure range is {-2, -1, 0, 1, 2}, that is... The range of values ​​for the exposure factor is given. The value range is {0.25, 0.5, 1, 2, 4}. This refers to the exposure image corresponding to the r-th adjustment factor, i.e., the r-th exposure image. Image to be enhanced.

[0060] By simulating various exposure conditions in the image to be enhanced, multi-scale exposure images are generated to restore details in dark areas and avoid information loss in overexposed areas, thus providing a basis for subsequent color patch selection.

[0061] Then, step S2 is executed to perform a downsampling operation on each exposed image, determining the downsampled brightness image corresponding to each exposed image. In the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value. For any exposed image, the pixel region in any exposed image can include multiple pixels in that exposed image, forming a pixel block. The representative brightness value of the pixel region in any exposed image can be the average brightness value of each pixel point in that pixel region, which can be expressed by the following formula:

[0062] ;

[0063] in, Let be the number of pixels in the j-th pixel region of the r-th exposed image; For pixels Brightness value; Let be the representative brightness value of the j-th superpixel region in the r-th exposure image. This method can improve the robustness of color patch selection and reduce information loss caused by downsampling.

[0064] Each exposure image corresponds to a downsampled brightness image.

[0065] Then, step S3 is executed, which iterates through each color block containing a preset number of pixels in each downsampled brightness image, and calculates the comprehensive score of each color block using the root mean square error of intensity and intensity range coverage.

[0066] Each color patch in the downsampled brightness image can include a preset number of pixels. By constructing a sliding window containing the preset number of pixels, traversal of each color patch can be achieved. Adjacent color patches in the downsampled brightness image may or may not overlap, depending on the step size of the sliding window. The preset number can be represented as S×S. If the step size of the sliding window is less than S, adjacent color patches will overlap; if the step size of the sliding window is equal to S, adjacent color patches will not overlap.

[0067] To preserve richer color and intensity range coverage in the image, the root mean square error of intensity and intensity range coverage are calculated for each color patch traversed. The root mean square error of intensity is used to determine the degree of color non-uniformity within the color patch and can be expressed as:

[0068] ;

[0069] in, The root mean square error of intensity is given, where n is the nth pixel in the color patch. This represents the normalized intensity value of the nth pixel in the color block. It is the mean of the normalized intensity values ​​of all pixels in the color block.

[0070] When calculating the intensity range coverage, we can first determine the number of different intensity value types in each channel of the color patch, and then calculate the ratio of the number of different intensity value types in each channel of each color patch to the number of maximum intensity value types, that is:

[0071] ;

[0072] in, The ratio of the m-th channel is the intensity range coverage of the m-th channel, with a maximum intensity value variety of 255. This represents the number of different intensity value types in the m-th channel. m can take the values ​​r, g, and b, representing the red, green, and blue channels, respectively.

[0073] Subsequently, the intensity range coverage of the color block is determined using the ratios corresponding to each channel. For example, the maximum value among the ratios corresponding to each channel can be selected as the intensity range coverage of the color block. That is:

[0074] ;

[0075] in, The intensity range coverage of the color blocks. The ratio of the red channel represents the intensity range coverage of the red channel. The ratio of green channels represents the coverage rate of the green channel's intensity range. The ratio of the blue channel represents the intensity range coverage of the blue channel.

[0076] By weighting and summing the root mean square error of intensity and intensity range coverage of each color patch, a comprehensive score can be obtained for each color patch. This comprehensive score represents the richness of the nonlinear information of the camera response function (CRF) of the color patch. That is:

[0077] ;

[0078] in, The weights for the root mean square error of the intensity are... The final score is calculated based on the overall score.

[0079] Subsequently, using a comprehensive score, the color patches in each downsampled brightness image can be sorted, and a specified number of color patches with high comprehensive scores can be selected as the camera response function (CRF) nonlinear information color patches. The CRF nonlinear information color patches all possess rich color and brightness information. This specified number can be set as needed and is not specifically limited here.

[0080] Then, step S4 is executed, using the pixel values ​​of each pixel in the CRF nonlinear information color block and the scene radiance to fit the camera response function, which facilitates subsequent image linearization processing.

[0081] CRF describes the non-linear mapping between image pixel intensity (i.e., observed value) and scene radiance (i.e., true value). Due to the physical characteristics of the camera and the limitations of the image sensor, pixel values ​​are typically the result of a non-linear transformation. The mathematical expression of CRF is as follows:

[0082] ;

[0083] Where B is the image pixel value, typically between 0-255 or 0-1; L is the scene radiance. It is a CRF, which is typically a monotonically increasing nonlinear function. This refers to the CRF value under L; This is the noise term, representing random noise during the shooting and light-sensing process.

[0084] Assumption The nonlinear form can be represented by a set of parameterized functions. The formula is as follows:

[0085] ;

[0086] in, represents the fitting parameters for the CRF, where m is the highest order of the polynomial, used to control the complexity of the fitting.

[0087] The fitting objective of CRF is to minimize B and The error between the two terms is the noise term that needs to be minimized. Therefore, the fitting objective of CRF can be defined by minimizing the objective function, which is formulated as follows:

[0088] ;

[0089] Where N is the total number of valid pixels in the color block; a is the fitting parameter of CRF; The scene radiance of the i-th valid pixel in the color block can be estimated by exposure value or other methods; Let be the image pixel value of the i-th valid pixel. for The CRF values ​​under a.

[0090] Finally, step S5 is executed. The goal of low-brightness image enhancement is to restore the true brightness distribution in the image through linearization processing of CRF, while improving image quality through contrast enhancement, detail restoration and denoising techniques. Its core task is to use CRF to convert the image pixel values ​​into the true radiance of the scene.

[0091] Image linearization is the first step in low-brightness enhancement. The fitted CRF is used to perform an inverse transform on the image to be enhanced, yielding a backup image. That is:

[0092] ;

[0093] in, This is the inverse function of CRF, where C is the backup image. This inverse transformation converts the image to be enhanced from a nonlinear space to a linear space, thus better restoring the true brightness distribution.

[0094] Afterward, the brightness of the backup image can be increased to obtain the enhanced target image. The dark details of the target image are significantly restored, while the brightness is uniform and there is no obvious color difference.

[0095] The image enhancement method based on nonlinear information color patches of the camera response function provided in this embodiment of the invention first exposes the image to be enhanced multiple times at different levels. This restores the details and colors of abnormally exposed areas in the image to be enhanced, improving the robustness of CRF estimation in the image to be enhanced, especially under noisy and abnormal exposure conditions. Furthermore, downsampling is performed on each exposed image to achieve lossless reduction of each image, resulting in color patches with richer color variety and variation. By calculating the root mean square error of intensity and intensity range coverage of each color patch, a comprehensive score is calculated for each color patch, allowing for quantitative evaluation. Using the comprehensive score of each color patch as a basis, nonlinear information color patches of the camera response function are automatically selected, eliminating manual intervention, suppressing noise, and improving the efficiency and accuracy of color patch selection. Furthermore, the camera response function can be accurately fitted using the pixel values ​​of each pixel in the nonlinear information color patch of the camera response function and the scene radiance. Using this camera response function, the brightness of the image to be enhanced is increased, resulting in a target image with restored dark details, restored colors, uniform brightness, no significant color difference, and a clearer, more natural appearance.

[0096] Based on the above embodiments, the step of performing a downsampling operation on each exposed image to determine the downsampled brightness image corresponding to each exposed image includes:

[0097] Based on the superpixel segmentation algorithm, a downsampling operation is performed on each exposed image to determine the pixel region in each exposed image;

[0098] Based on the brightness values ​​of each pixel in each pixel region, determine the representative brightness value of each pixel region;

[0099] The downsampled brightness image is determined by using pixel regions in each exposed image as pixels and the representative brightness value of each pixel region as the pixel value.

[0100] Specifically, a superpixel segmentation algorithm can be used to downsample each exposed image to determine the pixel region in each image. Here, the superpixel segmentation algorithm can include graph-based, NCut, Turbopixel, Quick-shift, Graph-cut a, Graph-cut b, and simple linear iterative clustering (SLIC) algorithms, etc. Each pixel region in the exposed image can be considered a superpixel region.

[0101] Subsequently, the representative brightness value of each pixel region is determined using the brightness values ​​of each pixel within that region. The representative brightness value of each pixel region can be the average brightness value of all pixels within that region in each exposed image.

[0102] Subsequently, by using the pixel region in each exposed image as the pixel point and the representative brightness value of each pixel region as the pixel value, a downsampled brightness image can be constructed.

[0103] In this embodiment of the invention, downsampling is performed using a superpixel segmentation algorithm, which can preserve the local color consistency of each exposed image and provide better color information for CRF estimation.

[0104] Based on the above embodiments, the step of performing downsampling operations on each exposed image to determine the downsampled brightness image corresponding to each exposed image includes:

[0105] Each exposed image is filtered to obtain a filtered image;

[0106] The filtered image is then normalized.

[0107] Specifically, noise inevitably exists in real images, affecting the accuracy of the fitted CRF, particularly in terms of nonlinearity. Using a multi-parameter model can easily lead to overfitting to noise. Therefore, filtering each exposure image is necessary. Most noise in real images originates from hardware devices, and both mean filtering and median filtering are effective against this type of noise. However, mean filtering is also inevitably affected by noise. Therefore, in this embodiment of the invention, a median filter is used to filter each exposure image to obtain a filtered image. To improve efficiency, the entire process can be performed with only one filtering operation.

[0108] To facilitate subsequent calculation and selection of high-quality color blocks, normalization processing can be performed on each exposed image before downsampling, converting all pixel intensity values ​​in each exposed image to the range of [0,1].

[0109] Based on the above embodiments, the step of traversing each color patch containing a preset number of pixels in each downsampled brightness image, calculating a comprehensive score for each color patch based on the root mean square error of intensity and intensity range coverage, and further comprising, based on the comprehensive score:

[0110] For any downsampled brightness image in each downsampled brightness image, if it is impossible to traverse a color block containing the preset number of pixels in any downsampled brightness image, or if the root mean square error of the intensity and / or the intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal is stopped.

[0111] Specifically, when traversing each color block containing a preset number of pixels in each downsampled brightness image, there are two traversal stopping conditions: First, for any downsampled brightness image in each downsampled brightness image, if it is impossible to traverse a color block containing a preset number of pixels in any downsampled brightness image, that is, the remaining untraversed pixels in any downsampled brightness image are less than the preset number, or are not arranged in the way required by the preset number of pixels for the color block, such as S rows and S columns; Second, if the root mean square error of the intensity and / or the intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal stops.

[0112] Here, the threshold requirement for the root mean square error of the intensity of the color block means that the root mean square error of the intensity of the color block is greater than the root mean square error threshold, and the threshold requirement for the intensity range coverage of the color block means that the intensity range coverage of the color block is greater than the coverage threshold.

[0113] Here, the root mean square error threshold and coverage threshold can both be set as needed, and no specific limitation is made in this embodiment of the invention.

[0114] It is understandable that the traversal will stop when at least one of the two traversal stopping conditions is met, and then a specified number of color blocks can be selected from the traversed color blocks as nonlinear information color blocks for the camera response function.

[0115] In this embodiment of the invention, two conditions for stopping traversal are given, which can ensure that the size of the traversed color blocks is consistent and guarantee the quality of the traversed color blocks.

[0116] Based on the above embodiments, the step of fitting the camera response function based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance includes:

[0117] Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, an objective function is constructed.

[0118] The objective function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated objective function to obtain the camera response function.

[0119] Specifically, in the process of fitting a CRF, the target function can be constructed by first using the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance. .

[0120] Considering the impact of noise, a robust optimization method is introduced to reduce the interference of noise on CRF fitting. Using the robust optimization algorithm, the objective function is updated, resulting in the updated objective function:

[0121] ;

[0122] in, This is a robust loss function used to reduce the impact of outliers.

[0123] Subsequently, the updated objective function is used to fit the camera response function to obtain the camera response function.

[0124] Based on the above embodiments, the step of enhancing the brightness of the backup image to obtain the enhanced target image includes:

[0125] The brightness of the backup image is enhanced based on the gamma correction algorithm to obtain the target image.

[0126] Specifically, in this embodiment of the invention, a gamma correction algorithm can be used to perform nonlinear transformation on the brightness of the backup image to improve the brightness of dark areas while avoiding overexposure, thereby further improving the brightness and contrast of the backup image.

[0127] Here, the gamma correction algorithm can be expressed as:

[0128] ;

[0129] in, Brightness for the backup image, The brightness of the target image. This is the gamma value, typically taken as 0.4 ≤ ≤0.6.

[0130] Based on the above embodiments, such as Figure 2 As shown, this embodiment of the invention also provides an image enhancement device based on nonlinear information color patches of the camera response function, comprising:

[0131] The multi-scale exposure image acquisition module 21 is used to acquire the image to be enhanced and expose the image to be enhanced multiple times at different degrees to obtain multiple exposure images;

[0132] The downsampling module 22 is used to perform downsampling operations on each exposed image to determine the downsampled brightness image corresponding to each exposed image. In the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value.

[0133] The color block determination module 23 is used to traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks of the camera response function based on the comprehensive score.

[0134] Fitting module 24 is used to fit the camera response function based on the pixel value of each pixel in the nonlinear information color block of the camera response function and the scene radiance.

[0135] The brightness enhancement module 25 is used to convert the image to be enhanced to a linear space based on the camera response function to obtain a backup image, and to enhance the brightness of the backup image to obtain the enhanced target image.

[0136] Based on the above embodiments, the image enhancement device based on nonlinear information color patches of camera response function provided in this embodiment of the invention, wherein the downsampling module is specifically used for:

[0137] Based on the superpixel segmentation algorithm, a downsampling operation is performed on each exposed image to determine the pixel region in each exposed image;

[0138] Based on the brightness values ​​of each pixel in each pixel region, determine the representative brightness value of each pixel region;

[0139] The downsampled brightness image is determined by using pixel regions in each exposed image as pixels and the representative brightness value of each pixel region as the pixel value.

[0140] Based on the above embodiments, the image enhancement device based on nonlinear information color patches of camera response function provided in this embodiment of the invention further includes a preprocessing module, used for:

[0141] Each exposed image is filtered to obtain a filtered image;

[0142] The filtered image is then normalized.

[0143] Based on the above embodiments, the image enhancement device based on nonlinear information color patches of camera response function provided in this embodiment of the invention, wherein the color patch determination module is further configured to:

[0144] For any downsampled brightness image in each downsampled brightness image, if it is impossible to traverse a color block containing the preset number of pixels in any downsampled brightness image, or if the root mean square error of the intensity and / or the intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal is stopped.

[0145] Based on the above embodiments, the image enhancement device based on nonlinear information color patches of camera response function provided in this embodiment of the invention further includes an intensity range coverage determination module, used for:

[0146] Determine the number of different intensity value types in each channel of each color patch;

[0147] Calculate the ratio of the number of different intensity value types to the number of maximum intensity value types in each channel of each color block, and determine the intensity range coverage of each color block based on the ratio corresponding to each channel.

[0148] Based on the above embodiments, the image enhancement device based on nonlinear information color patches of camera response function provided in this embodiment of the invention, wherein the fitting module is used for:

[0149] Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, an objective function is constructed.

[0150] The objective function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated objective function to obtain the camera response function.

[0151] Based on the above embodiments, the image enhancement device based on nonlinear information color blocks of camera response function provided in this embodiment of the invention, wherein the brightness enhancement module is specifically used for:

[0152] The brightness of the backup image is enhanced based on the gamma correction algorithm to obtain the target image.

[0153] Specifically, the functions of each module in the image enhancement device based on nonlinear information color blocks of camera response function provided in this embodiment of the invention correspond one-to-one with the operation flow of each step in the above method-like embodiments, and the achieved effects are also the same. For details, please refer to the above embodiments, and this will not be repeated in this embodiment of the invention.

[0154] Figure 3 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 3As shown, the electronic device may include a processor 310, a communications interface 320, a memory 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other via the communication bus 340. The processor 310 can call logical instructions in the memory 330 to execute the image enhancement method based on nonlinear information color patches of the camera response function provided in the above embodiments.

[0155] Furthermore, the logical instructions in the aforementioned memory 330 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to related technologies, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0156] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a computer-readable storage medium. When the computer program is executed by a processor, the computer is able to perform the image enhancement method based on nonlinear information color patches of the camera response function provided by the above methods. It is understood that the computer-readable storage medium can be a non-transitory computer-readable storage medium or a transient computer-readable storage medium, and no specific limitation is made here.

[0157] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image enhancement methods based on nonlinear information color patches of camera response functions provided by the methods described above.

[0158] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0159] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the parts that contribute to the related technology, can be embodied in the form of software products. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0160] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image enhancement method based on nonlinear information color patches of camera response function, characterized in that, include: The image to be enhanced is acquired, and the image to be enhanced is exposed to multiple times at different degrees to obtain multiple exposed images; Perform downsampling on each exposed image to determine the downsampled brightness image corresponding to each exposed image; in the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value. Traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks for the camera response function based on the comprehensive score. Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, the camera response function is fitted to obtain the camera response function. Based on the camera response function, the image to be enhanced is converted to linear space to obtain a backup image, and the brightness of the backup image is increased to obtain the enhanced target image; The step of performing a downsampling operation on each exposed image to determine the downsampled brightness image corresponding to each exposed image includes: Based on the superpixel segmentation algorithm, a downsampling operation is performed on each exposed image to determine the pixel region in each exposed image; Based on the brightness values ​​of each pixel in each pixel region, determine the representative brightness value of each pixel region; The downsampled brightness image is determined by using pixel regions in each exposed image as pixels and the representative brightness value of each pixel region as the pixel value. The intensity range coverage of each color patch is determined based on the following steps: Determine the number of different intensity value types in each channel of each color patch; Calculate the ratio of the number of different intensity value types in each channel of each color block to the number of maximum intensity value types, and determine the intensity range coverage of each color block based on the ratio corresponding to each channel; The process of fitting the camera response function based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance includes: Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, an objective function is constructed. The objective function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated objective function to obtain the camera response function.

2. The image enhancement method based on nonlinear information color patches of camera response function according to claim 1, characterized in that, The step of performing downsampling operations on each exposed image to determine the corresponding downsampled brightness image for each exposed image includes: Each exposed image is filtered to obtain a filtered image; The filtered image is then normalized.

3. The image enhancement method based on nonlinear information color patches of camera response function according to claim 1, characterized in that, The process of traversing each color block containing a preset number of pixels in each downsampled brightness image also includes: For any downsampled brightness image in each downsampled brightness image, if it is impossible to traverse a color block containing the preset number of pixels in any downsampled brightness image, or if the root mean square error of the intensity and / or the intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal is stopped.

4. The image enhancement method based on nonlinear information color patches of camera response function according to any one of claims 1-3, characterized in that, The step of enhancing the brightness of the backup image to obtain the enhanced target image includes: The brightness of the backup image is enhanced based on the gamma correction algorithm to obtain the target image.

5. An image enhancement device based on nonlinear information color patches of camera response function, characterized in that, include: A multi-scale exposure image acquisition module is used to acquire an image to be enhanced and expose the image to be enhanced multiple times at different degrees to obtain multiple exposure images; The downsampling module is used to perform downsampling operations on each exposed image to determine the downsampled brightness image corresponding to each exposed image. In the downsampled brightness image corresponding to each exposed image, the pixel region in each exposed image is taken as the pixel point, and the representative brightness value of the pixel region in each exposed image is taken as the pixel value. The color block determination module is used to traverse each color block containing a preset number of pixels in each downsampled brightness image, calculate the comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as nonlinear information color blocks of the camera response function based on the comprehensive score. The fitting module is used to fit the camera response function based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance. The brightness enhancement module is used to convert the image to be enhanced to a linear space based on the camera response function to obtain a backup image, and to enhance the brightness of the backup image to obtain the enhanced target image; The downsampling module is specifically used for: Based on the superpixel segmentation algorithm, a downsampling operation is performed on each exposed image to determine the pixel region in each exposed image; Based on the brightness values ​​of each pixel in each pixel region, determine the representative brightness value of each pixel region; The downsampled brightness image is determined by using pixel regions in each exposed image as pixels and the representative brightness value of each pixel region as the pixel value. The intensity range coverage of each color patch is determined based on the following steps: Determine the number of different intensity value types in each channel of each color patch; Calculate the ratio of the number of different intensity value types in each channel of each color block to the number of maximum intensity value types, and determine the intensity range coverage of each color block based on the ratio corresponding to each channel; The fitting module is specifically used for: Based on the pixel values ​​of each pixel in the nonlinear information color block of the camera response function and the scene radiance, an objective function is constructed. The objective function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated objective function to obtain the camera response function.

6. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image enhancement method based on nonlinear information color blocks of camera response function as described in any one of claims 1-4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the image enhancement method based on nonlinear information color blocks of camera response function as described in any one of claims 1-4.

Images