Image enhancement method and device based on camera response function nonlinear information color blocks

Abstract

The invention provides an image enhancement method and device based on a camera response function nonlinear information color block, relates to the technical field of image processing, and aims to perform multiple times of different degrees of exposure on a to-be-enhanced image, restore details and colors of an abnormal exposure area in the to-be-enhanced image, improve the robustness of CRF estimation in the to-be-enhanced image and improve the image enhancement effect. Particularly, the robustness of CRF estimation in the image to be enhanced under noise and abnormal exposure conditions is improved. Moreover, each exposure image is subjected to down-sampling operation, so that lossless reduction of each exposure image can be realized, and color blocks with richer color types and changes can be obtained. By automatically selecting the nonlinear information color blocks of the camera response function, manual intervention can be avoided, noise can be suppressed, and the efficiency and accuracy of color block selection can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, and in particular to an image enhancement method and device based on camera response function nonlinear information color blocks. BACKGROUND

[0002] Low-light image enhancement is an important research direction in the field of computer vision, and has been widely used in night shooting, security monitoring, medical image analysis and other scenes. However, low-light images usually have the following problems: the information in the dark area is difficult to identify, resulting in blurred image content; the shooting image under low light conditions has large noise, and traditional enhancement algorithms are difficult to ensure details and naturalness; color deviation is easily introduced in the image enhancement process, destroying the authenticity of the original scene.

[0003] The camera response function (CRF) describes the nonlinear relationship between the image pixel value and the scene radiance brightness. Accurate estimation of CRF is of great significance for low-light image enhancement, and through the linearization process of CRF, the brightness and color of the image can be better restored. However, the current method for low-light image enhancement based on CRF uses a direct traversal of the image or a manual selection method, deliberately selects color edges, color mixing and large color difference color blocks from the image as CRF nonlinear information color blocks, which is difficult to select. Further, the CRF obtained by fitting has large deviation, and it is difficult to calculate the real CRF.

[0004] Therefore, there is an urgent need to provide an image enhancement method based on camera response function nonlinear information color blocks. SUMMARY

[0005] The present application provides an image enhancement method and device based on camera response function nonlinear information color blocks to solve the defects in the related art.

[0006] The present application provides an image enhancement method based on camera response function nonlinear information color blocks, comprising: Obtain an image to be enhanced, and perform multiple different degrees of exposure on the image to be enhanced to obtain multiple exposure images; Perform downsampling operation on each exposure image respectively to determine the corresponding downsampled brightness image of each exposure image; in the downsampled brightness image corresponding to each exposure image, the pixel region in each exposure image is taken as a pixel point, and the representative brightness value of the pixel region in each exposure image is taken as a pixel value; traverse each color block containing a preset number of pixel points in each down-sampled luminance image, calculate a comprehensive score of each color block based on the root mean square error of intensity and intensity range coverage of each color block, and select a specified number of color blocks from the down-sampled luminance images corresponding to the plurality of exposure images as camera response function nonlinear information color blocks based on the comprehensive score; fit a camera response function based on the pixel values and scene radiance luminance of each pixel in the camera response function nonlinear information color blocks; convert the image to be enhanced to a linear space based on the camera response function to obtain a backup image, and perform luminance enhancement on the backup image to obtain an enhanced target image.

[0007] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the down-sampling operation is performed on each exposure image respectively, and the down-sampled luminance image corresponding to each exposure image is determined, which comprises: performing a down-sampling operation on each exposure image based on a superpixel segmentation algorithm to determine the pixel region in each exposure image; determine the representative luminance value of each pixel region based on the luminance value of each pixel in each pixel region; determine the down-sampled luminance image by taking the pixel region in each exposure image as the pixel point and the representative luminance value of each pixel region as the pixel value.

[0008] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the down-sampling operation is performed on each exposure image respectively, and the down-sampled luminance image corresponding to each exposure image is determined, which comprises: filtering each exposure image to obtain a filtered image; normalizing the filtered image.

[0009] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the traversal of each color block containing a preset number of pixels in each down-sampled luminance image further comprises: for any down-sampled luminance image in each down-sampled luminance image, if the color block containing the preset number of pixels cannot be traversed in the down-sampled luminance image, or the root mean square error of intensity and / or intensity range coverage of the traversed color block does not meet the threshold requirement, stop traversing.

[0010] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the intensity range coverage of each color block is determined based on the following steps: determine the number of different intensity value types in each channel of each color block; The ratio of the number of different intensity value categories in each channel of each color block to the number of maximum intensity value categories is calculated, and the intensity range coverage of each color block is determined based on the corresponding ratio of each channel.

[0011] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the pixel value and the scene radiation brightness of each pixel point in the camera response function nonlinear information color block are used to fit a camera response function, which comprises the following steps: A target function is constructed based on the pixel value and the scene radiation brightness of each pixel point in the camera response function nonlinear information color block. The target function is updated based on a robust optimization algorithm, and the camera response function is fitted based on the updated target function, so as to obtain the camera response function.

[0012] According to the image enhancement method based on the camera response function nonlinear information color block provided by the application, the pixel value and the scene radiation brightness of each pixel point in the camera response function nonlinear information color block are used to fit a camera response function, which comprises the following steps: The target image is obtained by performing brightness enhancement on the backup image based on a gamma correction algorithm.

[0013] The application further provides an image enhancement device based on a camera response function nonlinear information color block, which comprises the following steps: A multi-scale exposure image acquisition module is configured to acquire a to-be-enhanced image, perform multiple different degrees of exposure on the to-be-enhanced image, and obtain multiple exposure images. A downsampling module is configured to perform a downsampling operation on each exposure image to determine a corresponding downsampled brightness image of each exposure image, wherein each pixel region in each exposure image is used as a pixel point, and the representative brightness value of each pixel region in each exposure image is used as a pixel value. A color block determination module is configured to traverse each color block containing a preset number of pixel points in each downsampled brightness image, calculate the comprehensive score of each color block based on the intensity root mean square error and the intensity range coverage of each color block, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as camera response function nonlinear information color blocks based on the comprehensive score. A fitting module is configured to fit a camera response function based on the pixel value and the scene radiation brightness of each pixel point in the camera response function nonlinear information color block. A brightness enhancement module is configured to convert the to-be-enhanced image to a linear space based on the camera response function, obtain a backup image, and perform brightness enhancement on the backup image to obtain an enhanced target image.

[0014] The application further provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the camera response function nonlinear information color block-based image enhancement method according to any one of the above when executing the computer program.

[0015] The application further provides a computer readable storage medium, which stores a computer program, wherein the computer program is executed by a processor to implement the camera response function nonlinear information color block-based image enhancement method according to any one of the above.

[0016] The application further provides a computer program product, which comprises a computer program, wherein the computer program is executed by a processor to implement the camera response function nonlinear information color block-based image enhancement method according to any one of the above.

[0017] The camera response function nonlinear information color block-based image enhancement method and device provided by the application can restore the details and colors of the non-normal exposure area in the image to be enhanced, improve the robustness of CRF estimation in the image to be enhanced, especially the robustness of CRF estimation in the image to be enhanced under the conditions of noise and non-normal exposure, and can realize lossless reduction of each exposure image by performing downsampling operation on each exposure image, so as to obtain color blocks with richer color types and changes. The intensity root mean square error and intensity range coverage of each color block are calculated, and the comprehensive score of each color block is calculated, so that each color block can be quantitatively evaluated. The comprehensive score of each color block is used as a basis to automatically select the camera response function nonlinear information color block, so that the efficiency and accuracy of color block selection can be improved without manual intervention and noise suppression. Furthermore, the pixel values and scene radiance of each pixel point in the camera response function nonlinear information color block can be used to accurately fit the camera response function. The brightness of the image to be enhanced is improved by using the camera response function, so that the target image with restored dark details, restored colors, uniform brightness, no obvious color difference, and more clear and natural can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0018] In order to more clearly illustrate the technical solutions in the application or the related art, the following will briefly introduce the drawings needed to be used in the embodiments or the related art description. Obviously, the drawings in the following description can also be obtained by those skilled in the art without any creative effort on the basis of these drawings.

[0019] Figure 1 is a flowchart of the camera response function nonlinear information color block-based image enhancement method provided by the application; Figure 2It is the structural schematic view of the image enhancement device based on the camera response function nonlinear information color block provided by the application. Figure 3 It is the structural schematic view of the electronic equipment provided by the application. DETAILED DESCRIPTION

[0020] To make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described below in connection with the drawings in the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0021] The terms "first", "second" in the description and claims of the present application can explicitly or implicitly include one or more features. In the description of the invention, unless otherwise specified, the meaning of "multiple" is two or more. In addition, "and / or" in the specification and claims indicates at least one of the connected objects, and the character " / ", generally indicates that the front and rear associated objects are in an "or" relationship.

[0022] For the technical problems existing in the image enhancement method in the prior art, such as Figure 1 As shown in the figure, the present application provides an image enhancement method based on camera response function nonlinear information color block, which comprises the following steps: S1, obtaining a to-be-enhanced image, and performing multiple different degrees of exposure on the to-be-enhanced image to obtain multiple exposure images; S2, performing down-sampling operation on each exposure image respectively to determine the down-sampling luminance image corresponding to each exposure image; in the down-sampling luminance image corresponding to each exposure image, taking the pixel region in each exposure image as a pixel point, and taking the representative luminance value of the pixel region in each exposure image as a pixel value; S3, traversing each color block containing a preset number of pixel points in each down-sampling luminance image, calculating the comprehensive score of each color block based on the root mean square error of intensity and the intensity range coverage rate of each color block, and selecting a specified number of color blocks from the down-sampling luminance images corresponding to the multiple exposure images as camera response function nonlinear information color blocks based on the comprehensive score; S4, fitting to obtain a camera response function based on the pixel value and the scene radiance luminance of each pixel point in the camera response function nonlinear information color block; S5, converting the to-be-enhanced image to a linear space based on the camera response function to obtain a standby image, and performing luminance enhancement on the standby image to obtain an enhanced target image.

[0023] Specifically, the image enhancement method based on the color block of nonlinear information of the camera response function provided in the embodiments of the present application has an execution subject of an image enhancement device based on the color block of nonlinear information of the camera response function. The device can be configured in a computer, which can be a local computer or a cloud computer. The local computer can be a computer, a tablet, etc., which is not specifically limited here.

[0024] Firstly, step S1 is performed to obtain an image to be enhanced. The image to be enhanced can be a color image collected by an image collection device and a low-brightness image requiring brightness enhancement. The image collection device can be a camera, which can be mounted on a robot, which is not specifically limited here.

[0025] Since there can be underexposed or overexposed areas in the image to be enhanced, these areas can easily affect the extraction of information in the color block. To solve the problem of dark information loss in the image to be enhanced, the image to be enhanced can be exposed for multiple times with different degrees to obtain multiple exposure images under different exposure conditions.

[0026] Firstly, a multi-scale exposure range can be set , is a fixed parameter, representing the maximum adjustment coefficient allowed.

[0027] Then, multiple different adjustment coefficients are selected from the multi-scale exposure range to determine the exposure coefficient. The image to be enhanced is exposed with different degrees, for example, 5 different adjustment coefficients can be selected from the multi-scale exposure range to determine the exposure coefficient. The image to be enhanced is randomly exposed by using each exposure coefficient to obtain 5 exposure images. The image to be enhanced can be regarded as an exposure image with an exposure degree of 0 (i.e., the exposure coefficient is 1).

[0028] Each exposure image can satisfy: ; ; Among them, is the rth adjustment coefficient, where When , the multi-scale exposure range is {-2, -1, 0, 1, 2}, i.e., the value range of , and the value range of the exposure coefficient is {0.25, 0.5, 1, 2, 4}. is the exposure image corresponding to the rth adjustment coefficient, i.e., the rth exposure image, is the image to be enhanced.

[0029] A plurality of exposure images of different scales are generated by simulating a plurality of exposure conditions for the to-be-enhanced image, so as to restore dark detail information and avoid information loss in overexposed areas, thereby providing a basis for subsequent color block screening.

[0030] Then, step S2 is performed, and a down-sampled luminance image corresponding to each exposure image is determined by performing a down-sampling operation on each exposure image respectively. In the down-sampled luminance image corresponding to each exposure image, a pixel region in each exposure image is taken as a pixel point, and a representative luminance value of the pixel region in each exposure image is taken as a pixel value. For any exposure image, the pixel region in the any exposure image can include a plurality of pixel points in the any exposure image, forming a pixel block. The representative luminance value of the pixel region in the any exposure image can be a mean value of luminance values of the pixel points in the pixel region, and can be represented by the following formula: ; wherein, is a pixel number in the jth pixel region in the rth exposure image; is a luminance value of the pixel ; is the representative luminance value of the jth superpixel region in the rth exposure image. By this method, the robustness of color block screening can be improved, and information loss caused by down-sampling can be reduced.

[0031] Each exposure image corresponds to a down-sampled luminance image.

[0032] Thereafter, step S3 is performed, each color block containing a preset number of pixel points in each down-sampled luminance image is traversed, and the comprehensive score of each color block is calculated by using the root mean square error of intensity and the intensity range coverage of each color block.

[0033] Each color block in the down-sampled luminance image can include a preset number of pixel points, and the traversal of each color block can be realized by constructing a sliding window containing a preset number of pixel points. Adjacent color blocks in the down-sampled luminance image can overlap or not overlap, which is determined by the step length of the sliding window. The preset number can be represented as SxS. If the step length of the sliding window is less than S, the adjacent color blocks will overlap. If the step length of the sliding window is equal to S, the adjacent color blocks will not overlap.

[0034] In order to retain more rich colors and intensity range coverage in the image, the root mean square error of intensity and the intensity range coverage are calculated for each color block traversed. The root mean square error of intensity is used to calculate the unevenness of colors in the color block, and can be represented as: ; wherein, is the root mean square error of intensity, n is the nth pixel point in the color block, the normalized intensity value of the nth pixel point in the color block, the average of the normalized intensity values of all pixel points in the color block.

[0035] In the calculation of the intensity range coverage, the number of different intensity value categories in each channel of the color block can be determined first, and then the ratio of the number of different intensity value categories in each channel of each color block to the maximum intensity value category number is calculated, that is: wherein, is the ratio of the mth channel, is the intensity range coverage of the mth channel, the maximum intensity value category is 255, is the number of different intensity value categories in the mth channel. m can take values of r, g, b, representing the red channel, the green channel and the blue channel respectively.

[0036] Thereafter, the intensity range coverage of the color block is determined by using the ratio corresponding to each channel. For example, the maximum value can be selected from the ratio corresponding to each channel as the intensity range coverage of the color block. That is: wherein, is the intensity range coverage of the color block, is the ratio of the red channel, is the intensity range coverage of the red channel, is the ratio of the green channel, is the intensity range coverage of the green channel, is the ratio of the blue channel, is the intensity range coverage of the blue channel.

[0037] By weighted summation of the intensity root mean square error and the intensity range coverage of each color block, the comprehensive score of each color block can be obtained, which represents the richness of the camera response function (CRF) nonlinear information of the color block. That is: wherein, is the weight of the intensity root mean square error, is the comprehensive score.

[0038] Thereafter, the color blocks in each down-sampled luminance image can be sorted by using the comprehensive score, and a specified number of color blocks with high comprehensive score can be selected as the camera response function CRF nonlinear information color blocks. The CRF nonlinear information color blocks all have rich color and luminance information. The specified number can be set as needed, which is not specifically limited here.

[0039] Thereafter, step S4 is performed, and the camera response function is fitted by using the pixel values of each pixel point in the CRF nonlinear information color block and the scene radiance luminance, which is convenient for subsequent image linearization processing.​​​

[0040] CRF describes the non-linear mapping between image pixel intensity (i.e. observation) and scene radiance (i.e. ground truth). Due to the physical characteristics of the camera and the limitation of the photosensitive element, the pixel value is usually the result of non-linear transformation. The mathematical expression of CRF is as follows: ; where B is the image pixel value, generally between 0-255 or 0-1; L is the scene radiance; is the CRF, usually a monotonically increasing non-linear function, is the CRF value under L; is the noise term, representing the random noise in the shooting and photosensitive process.

[0041] Assume The non-linear form of The formula of ; where is the fitting parameter of CRF, and m is the highest order of the polynomial, used to control the complexity of the fitting.

[0042] The fitting goal of CRF is to minimize the error between B and , that is, to minimize the noise term. Therefore, the fitting goal of CRF can be defined by minimizing the objective function, whose formula is as follows: ; where N is the total number of valid pixel points in the color block; a is the fitting parameter of CRF; is the scene radiance of the i-th valid pixel point in the color block, which can be estimated by the exposure value or other methods; is the image pixel value of the i-th valid pixel point. is the CRF value under and a.

[0043] Finally, step S5 is executed, and the goal of low-light image enhancement is to restore the real brightness distribution in the image through the linearization processing of CRF, and to improve the image quality through contrast enhancement, detail restoration and denoising technology, and the core task is to convert the image pixel value to the real radiance of the scene using CRF.

[0044] Image linearization is the first step of low-light enhancement, and the inverse transformation of the obtained CRF is used to the image to be enhanced to obtain the standby image. That is: ; where The C is a backup image, and the CRF is an inverse function of the CRF.

[0045] After that, the backup image can be subjected to brightness enhancement, and a target image after enhancement can be obtained.

[0046] The image enhancement method based on the color block of the camera response function nonlinear information provided in the embodiment of the application can first perform multiple exposures of different degrees on the image to be enhanced, can restore the details and colors of the non-normal exposure area in the image to be enhanced, and can improve the robustness of the CRF estimation of the image to be enhanced, especially the robustness of the CRF estimation of the image to be enhanced under the conditions of noise and non-normal exposure. Moreover, the downsampling operation is performed on each exposure image, the exposure image can be reduced without loss, and the color block with more color types and changes can be obtained. The intensity root mean square error and the intensity range coverage of each color block are calculated, the comprehensive score of each color block is calculated, each color block is quantitatively evaluated, the comprehensive score of each color block is taken as a basis, the color block of the camera response function nonlinear information is automatically selected, the artificial intervention is avoided, the noise is suppressed, and the efficiency and accuracy of the color block selection are improved. Furthermore, the pixel value and the scene radiant brightness of each pixel point in the color block of the camera response function nonlinear information are used to accurately fit the camera response function. The brightness of the image to be enhanced is enhanced through the camera response function, and a target image with restored dark details, restored colors, uniform brightness, no obvious color difference, and more clear and natural can be obtained.

[0047] On the basis of the above embodiment, the downsampling operation is performed on each exposure image to determine the corresponding downsampled brightness image of each exposure image, and the method comprises the following steps. The downsampling operation is performed on each exposure image based on a superpixel segmentation algorithm to determine a pixel region in each exposure image. The representative brightness value of each pixel region is determined based on the brightness values of the pixel points in each pixel region. The downsampled brightness image is determined by taking the pixel regions in each exposure image as pixel points and taking the representative brightness values of the pixel regions as pixel values.

[0048] Specifically, the superpixel segmentation algorithm can be used to perform down-sampling operation on each exposure image respectively to determine the pixel region in each exposure image. Here, the superpixel segmentation algorithm can include Graph-based, NCut, Turbopixel, Quick-shift, Graph-cut a, Graph-cut b, and simple linear iterative clustering (SLIC) algorithm, etc. The pixel region in each exposure image can be a region of a superpixel.

[0049] Thereafter, the representative brightness value of each pixel region is determined by using the brightness value of each pixel point in each pixel region. The representative brightness value of each pixel region can be the average brightness value of each pixel point in the pixel region in each exposure image.

[0050] Thereafter, the down-sampled brightness image can be constructed by taking the pixel region in each exposure image as a pixel point and the representative brightness value of each pixel region as a pixel value.

[0051] In the embodiment of the present application, the down-sampling operation by the superpixel segmentation algorithm can preserve the local color consistency of each exposure image and provide better color information for CRF estimation.

[0052] On the basis of the above embodiment, the down-sampling operation on each exposure image to determine the corresponding down-sampled brightness image of each exposure image includes the following steps: filtering each exposure image to obtain a filtered image; normalizing the filtered image.

[0053] Specifically, noise inevitably occurs in a real image, which affects the accuracy of the fitted CRF, especially in terms of nonlinearity. If a multi-parameter model is used, overfitting to noise is likely to occur. Therefore, it is necessary to filter each exposure image. Noise in a real image mostly comes from hardware devices, and mean filtering and median filtering have good effects on such noise. However, mean filtering is also inevitably affected by noise. Therefore, in the embodiment of the present application, a median filter is used to filter each exposure image to obtain a filtered image. To improve efficiency, only one filtering operation can be performed in the entire method process.

[0054] To facilitate subsequent calculation and selection of high-quality color blocks, normalization can be performed on each exposure image before the down-sampling operation to convert all pixel intensity values in each exposure image to the range of [0, 1].

[0055] On the basis of the above-mentioned embodiments, the traversing each color block containing a preset number of pixels in each down-sampled luminance image, based on the root mean square error of intensity and intensity range coverage of each color block, the comprehensive score of each color block is calculated, and based on the comprehensive score, further comprising: For each down-sampled luminance image, if the color block containing the preset number of pixels cannot be traversed in the down-sampled luminance image, or the root mean square error of intensity and / or intensity range coverage of the traversed color block does not meet the threshold requirement, the traversal is stopped.

[0056] Specifically, when traversing each color block containing a preset number of pixels in each down-sampled luminance image, there are two traversal stop conditions: one is that for each down-sampled luminance image, in any down-sampled luminance image, the color block containing the preset number of pixels cannot be traversed, that is, the remaining pixels in the down-sampled luminance image that have not been traversed are less than the preset number, or it is not the arrangement of the preset number of pixels required by the color block, such as S rows and S columns; the second is that the root mean square error of intensity and / or intensity range coverage of the traversed color block does not meet the threshold requirement, then the traversal is stopped.

[0057] Here, the threshold requirement of the root mean square error of intensity of the color block refers to that the root mean square error of intensity of the color block is greater than the root mean square error threshold, and the threshold requirement of the intensity range coverage of the color block refers to that the intensity range coverage of the color block is greater than the coverage threshold.

[0058] Here, the root mean square error threshold and the coverage threshold can be set as needed, which is not specifically limited in the embodiments of the present application.

[0059] It can be understood that when at least one of the above two traversal stop conditions is met, the traversal is stopped, and then a specified number of color blocks are selected from the traversed color blocks as the camera response function nonlinear information color block.

[0060] In the embodiments of the present application, two conditions for stopping traversal are given, which can make the size of the traversed color block consistent and ensure the quality of the traversed color block.

[0061] On the basis of the above-mentioned embodiments, the camera response function is fitted based on the pixel value and scene radiance luminance of each pixel point in the camera response function nonlinear information color block, comprising: Based on the pixel value and scene radiance luminance of each pixel point in the camera response function nonlinear information color block, a target function is constructed; Based on the robust optimization algorithm, the target function is updated, and based on the updated target function, the camera response function is fitted to obtain the camera response function.

[0062] Specifically, in the process of fitting the CRF, the pixel value of each pixel point in the color block of the camera response function nonlinear information and the scene radiance can be used to construct a target function .

[0063] Considering the influence of noise, a robust optimization method is introduced to reduce the interference of noise on the CRF fitting, and the target function is updated by using a robust optimization algorithm to obtain an updated target function, that is, ; Wherein, is a robust loss function for reducing the influence of outliers.

[0064] After that, the camera response function is fitted by using the updated target function to obtain the camera response function.

[0065] On the basis of the above embodiment, the brightness of the backup image is enhanced to obtain an enhanced target image, comprising: Based on the gamma correction algorithm, the brightness of the backup image is enhanced to obtain the target image.

[0066] Specifically, in the embodiment of the application, the gamma correction algorithm can be used to perform nonlinear transformation on the brightness of the backup image to enhance the dark brightness, while avoiding overexposure, so as to further enhance the brightness and contrast of the backup image.

[0067] Here, the gamma correction algorithm can be expressed as: ; Wherein, is the brightness of the backup image, is the brightness of the target image, is the gamma value, usually 0.4≤ ≤0.6.

[0068] On the basis of the above embodiment, as Figure 2 shown, the embodiment of the application further provides an image enhancement device based on a camera response function nonlinear information color block, comprising: A multi-scale exposure image acquisition module 21 is used to acquire a to-be-enhanced image and perform multiple different degrees of exposure on the to-be-enhanced image to obtain a plurality of exposure images. A downsampling module 22 is used to perform a downsampling operation on each exposure image to determine a downsampled brightness image corresponding to each exposure image; in each downsampled brightness image corresponding to each exposure image, a pixel region in each exposure image is used as a pixel point, and a representative brightness value of the pixel region in each exposure image is used as a pixel value. The color block determination module 23 is configured to traverse each color block containing a preset number of pixels in each down-sampled luminance image, calculate a comprehensive score of each color block based on the intensity root mean square error and intensity range coverage of each color block, and select a specified number of color blocks from the down-sampled luminance images corresponding to the plurality of exposure images as camera response function nonlinear information color blocks based on the comprehensive score. The fitting module 24 is configured to fit a camera response function based on the pixel value and scene radiant luminance of each pixel point in the camera response function nonlinear information color block. The luminance enhancement module 25 is configured to convert the image to be enhanced into a linear space based on the camera response function to obtain a backup image, and perform luminance enhancement on the backup image to obtain an enhanced target image.

[0069] Based on the above embodiment, the image enhancement device based on the camera response function nonlinear information color block provided in the embodiment of the present application comprises a down-sampling module. The down-sampling module is configured to perform a down-sampling operation on each exposure image based on a super-pixel segmentation algorithm to determine a pixel region in each exposure image. The down-sampling module is configured to determine a representative luminance value of each pixel region based on the luminance value of each pixel point in each pixel region. The down-sampling module is configured to determine the down-sampled luminance image by taking the pixel region in each exposure image as a pixel point and taking the representative luminance value of each pixel region as a pixel value.

[0070] Based on the above embodiment, the image enhancement device based on the camera response function nonlinear information color block provided in the embodiment of the present application further comprises a preprocessing module. The preprocessing module is configured to filter each exposure image to obtain a filtered image. The preprocessing module is configured to perform normalization processing on the filtered image.

[0071] Based on the above embodiment, the image enhancement device based on the camera response function nonlinear information color block provided in the embodiment of the present application further comprises a color block determination module. For any down-sampled luminance image in the down-sampled luminance images, if a color block containing the preset number of pixels cannot be traversed in the down-sampled luminance image, or the intensity root mean square error and / or intensity range coverage of the traversed color block does not meet the threshold requirement, the traversal is stopped.

[0072] Based on the above embodiment, the image enhancement device based on the camera response function nonlinear information color block provided in the embodiment of the present application further comprises an intensity range coverage determination module. The intensity range coverage determination module is configured to determine the number of different intensity value categories in each channel of each color block. The number of different intensity value categories in each channel of each color block is calculated, and the ratio of the number of different intensity value categories to the number of maximum intensity value categories is calculated.

[0073] Based on the above embodiments, the image enhancement device based on the color block of the camera response function nonlinear information provided in the embodiments of the present application comprises a fitting module, which is configured to: Based on the pixel value and the scene radiance luminance of each pixel point in the color block of the camera response function nonlinear information, a target function is constructed. Based on a robust optimization algorithm, the target function is updated, and based on the updated target function, the camera response function is fitted to obtain the camera response function.

[0074] Based on the above embodiments, the image enhancement device based on the color block of the camera response function nonlinear information provided in the embodiments of the present application comprises a brightness enhancement module, which is configured to: Based on a gamma correction algorithm, the backup image is subjected to brightness enhancement to obtain the target image.

[0075] Specifically, the functions of each module in the image enhancement device based on the color block of the camera response function nonlinear information provided in the embodiments of the present application are one-to-one corresponding to the operation processes of each step in the method embodiment, and the effects achieved are consistent. For details, refer to the above embodiments, and the embodiments of the present application will not be described here.

[0076] Figure 3 An example of a schematic diagram of the physical structure of an electronic device is shown in FIG. Figure 3 As shown, the electronic device can include a processor (Processor) 310, a communications interface (Communications Interface) 320, a memory (Memory) 330, and a communication bus 340, wherein the processor 310, the communications interface 320, and the memory 330 communicate with each other through the communication bus 340. The processor 310 can invoke the logical instructions in the memory 330 to execute the image enhancement method based on the color block of the camera response function nonlinear information provided in the above embodiments.

[0077] In addition, the logic instructions in the memory 330 described above can be implemented in the form of software functional units and sold or used as independent products, and can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the parts that contribute to the related art or the parts of the technical solutions can be embodied in the form of software products. The computer software product is stored in a storage medium, and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0078] In another aspect, the present application also provides a computer program product, which includes a computer program, the computer program can be stored on a computer readable storage medium, and the computer program can be executed by a processor, so that the computer can execute the image enhancement method based on the camera response function nonlinear information color block provided by the above-mentioned methods. It can be understood that the computer readable storage medium can be a non-transitory computer readable storage medium or a transitory computer readable storage medium, which is not limited here.

[0079] In another aspect, the present application also provides a non-transitory computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the image enhancement method based on the camera response function nonlinear information color block provided by the above-mentioned methods.

[0080] The device embodiments described above are only schematic, wherein the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, that is, they can be located in one place, or distributed on multiple network units. Part or all of the modules can be selected according to actual needs to achieve the purpose of the present embodiment. Those skilled in the art can understand and implement it without creative labor.

[0081] Those skilled in the art can clearly understand the technical solutions of the various embodiments from the above description of the embodiments, and the various embodiments can be implemented by means of software with the necessary general hardware platforms, and of course can also be implemented by hardware. Based on such understanding, the above technical solutions, essentially or in other words, the part that contributes to the related art, can be embodied in the form of a software product. The computer software product can be stored in a computer readable storage medium, such as a ROM / RAM, a magnetic disk, an optical disk, and the like, and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0082] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for some technical features therein; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image enhancement method based on camera response function nonlinear information color block, characterized in that, The method comprises: obtaining a to-be-enhanced image, and performing multiple different degrees of exposure on the to-be-enhanced image to obtain multiple exposure images; performing down-sampling operation on each exposure image respectively to determine a down-sampled luminance image corresponding to each exposure image; in the down-sampled luminance image corresponding to each exposure image, a pixel region in each exposure image is taken as a pixel point, and a representative luminance value of the pixel region in each exposure image is taken as a pixel value; traversing each color block containing a preset number of pixel points in each down-sampled luminance image, calculating a comprehensive score of each color block based on a root mean square error of intensity and an intensity range coverage rate of each color block, and selecting a specified number of color blocks from the down-sampled luminance images corresponding to the multiple exposure images as camera response function nonlinear information color blocks based on the comprehensive score; fitting a camera response function based on pixel values and scene radiance luminance of the pixel points in the camera response function nonlinear information color blocks; converting the to-be-enhanced image to a linear space based on the camera response function to obtain a backup image, and performing luminance enhancement on the backup image to obtain an enhanced target image.

2. The image enhancement method based on camera response function nonlinear information color block according to claim 1, characterized in that, The method further comprises: performing down-sampling operation on each exposure image respectively based on a superpixel segmentation algorithm to determine a pixel region in each exposure image; determining a representative luminance value of each pixel region based on luminance values of the pixel points in each pixel region; determining the down-sampled luminance image by taking the pixel region in each exposure image as a pixel point and the representative luminance value of each pixel region as a pixel value.

3. The image enhancement method based on camera response function nonlinear information color block according to claim 1, characterized in that, The method further comprises: performing filtering on each exposure image to obtain a filtered image; performing normalization processing on the filtered image.

4. The image enhancement method based on camera response function nonlinearity information color block according to claim 1, characterized in that, The method further comprises: for any down-sampled luminance image in the down-sampled luminance images, if a color block containing the preset number of pixels cannot be traversed in the down-sampled luminance image, or the root mean square error of intensity and / or the intensity range coverage rate of the traversed color block does not meet a threshold requirement, the traversal is stopped.

5. The method of claim 1, wherein the color block is based on a camera response function nonlinearity information. The intensity range coverage rate of each color block is determined based on the following steps: determining the number of different intensity value categories in each channel of each color block; calculating a ratio of the number of different intensity value categories in each channel of each color block to the number of maximum intensity value categories, and determining the intensity range coverage rate of each color block based on the ratio corresponding to each channel.

6. The method of claim 1, wherein the color block is based on a camera response function nonlinearity. The method further comprises: constructing an objective function based on the pixel values and the scene radiance luminance of the pixel points in the camera response function nonlinear information color blocks; updating the objective function based on a robust optimization algorithm, and fitting the camera response function based on the updated objective function to obtain the camera response function.

7. The image enhancement method based on camera response function nonlinearity information color block according to any one of claims 1-6, characterized in that, The brightness of the backup image is enhanced to obtain an enhanced target image, including: The brightness of the backup image is enhanced based on a gamma correction algorithm to obtain the target image.

8. An image enhancement device based on camera response function nonlinear information color block, characterized in that, Including: A multi-scale exposure image acquisition module is configured to acquire a to-be-enhanced image and perform multiple different exposures on the to-be-enhanced image to obtain multiple exposure images. A downsampling module is configured to perform a downsampling operation on each exposure image to determine a corresponding downsampled brightness image of each exposure image; in each downsampled brightness image, a pixel region in each exposure image is used as a pixel point, and a representative brightness value of the pixel region in each exposure image is used as a pixel value. A color block determination module is configured to traverse each color block containing a preset number of pixel points in each downsampled brightness image, calculate a comprehensive score of each color block based on a root mean square error of intensity and an intensity range coverage rate of each color block, and select a specified number of color blocks from the downsampled brightness images corresponding to the multiple exposure images as camera response function nonlinear information color blocks based on the comprehensive score. A fitting module is configured to fit a camera response function based on pixel values and scene radiance brightness of each pixel point in the camera response function nonlinear information color blocks. A brightness enhancement module is configured to convert the to-be-enhanced image to a linear space based on the camera response function to obtain a backup image, and enhance the brightness of the backup image to obtain an enhanced target image.

9. An electronic device comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that, The processor executes the computer program to implement the image enhancement method based on the camera response function nonlinear information color blocks according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the image enhancement method based on the camera response function nonlinear information color blocks according to any one of claims 1-7.

Images