Image enhancement method and device, electronic equipment, storage medium and program product

Abstract

The invention provides an image enhancement method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: carrying out the N-1 times of image restoration based on a first image pyramid, and obtaining an enhanced image; n is greater than or equal to 1 and less than or equal to N-1, and the method comprises the following steps: performing up-sampling processing on a target image; fusing the target image after up-sampling processing with the nth-layer image to obtain an nth-layer reconstructed image; and based on a preset cutting threshold range, performing cutting processing and normalization processing on the pixel value of each pixel point in the nth layer of reconstructed image to obtain the nth layer of reconstructed image. According to the scheme, through cutting processing and normalization processing, the pixel value of each pixel point in the image subjected to contrast enhancement is within the preset cutting threshold range, and the defect of contrast imbalance during contrast enhancement of the image can be avoided.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and more specifically, to an image enhancement method, apparatus, electronic device, storage medium, and program product. Background Technology

[0002] In common imaging technologies such as mobile phones and video surveillance, multi-scale pyramids are often used to enhance image contrast. However, contrast enhancement algorithms based on multi-scale pyramids often suffer from halo effects due to the mismatch between information at different scales after enhancement.

[0003] Halo is a non-natural phenomenon introduced by algorithmic processing mechanisms, and its manifestations are as follows: Figure 1 As shown in the circled area, the Halo effect typically manifests as unnatural brightness transitions or halos around edges or high-contrast areas in an image, making the image appear over-processed or artificially manipulated, thus reducing its realism. When using contrast enhancement algorithms to improve image contrast, the Halo effect can lead to a contrast imbalance. Summary of the Invention

[0004] The purpose of this application is to provide an image enhancement method, apparatus, electronic device, storage medium, and program product to solve the above-mentioned problems.

[0005] In a first aspect, embodiments of this application provide an image enhancement method, the method comprising:

[0006] Based on the first image pyramid, N-1 image restorations are performed to obtain an enhanced image; wherein, the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image after enhancement of the image to be enhanced; N is the number of layers in the first image pyramid;

[0007] The nth image restoration, 1≤n≤N-1, includes:

[0008] The target image is upsampled; wherein, when n=1, the target image is the zeroth layer image of the first image pyramid, and the zeroth layer image is the image with the smallest image resolution in the first image pyramid; when n>1, the target image is the (n-1)th layer reconstructed image.

[0009] The target image after upsampling is fused with the nth layer image of the first image pyramid to obtain the nth layer reconstructed image;

[0010] Based on a preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are cropped and normalized to obtain the nth layer reconstructed image.

[0011] In the implementation of the above scheme, on the one hand, through cropping and normalization, the pixel values ​​of each pixel in the image after contrast enhancement are all within the preset cropping threshold range, which helps to avoid the defect of contrast imbalance when enhancing the image; on the other hand, compared with the Halo solution in related technologies, the above scheme has lower computational complexity and is easy to implement because it performs cropping and normalization during the iterative process of image restoration using the first image pyramid.

[0012] In one implementation of the first aspect, before performing N-1 image restorations based on the first image pyramid, the method further includes: acquiring a multi-exposure image sequence of the image to be enhanced; wherein the multi-exposure image sequence includes multiple exposure images of the same scene with different brightness; calculating the second image pyramids of at least two exposure images in the multi-exposure image sequence respectively to obtain at least two second image pyramids; and fusing the second image pyramids of at least two exposure images to obtain the first image pyramid of the enhanced image.

[0013] In the implementation of the above scheme, by fusing the second image pyramids of at least two exposed images, the details and texture information of the well-exposed parts of each image can be preserved, thereby obtaining richer details and textures in the final fusion result, which is beneficial to improving the image enhancement effect of the above image enhancement method. On the other hand, images with different exposures usually cover different brightness ranges. By fusing the second image pyramids, the information of these different brightness ranges can be integrated into one image, thereby improving the dynamic range of the final image, so that the image can retain more details in both bright and dark areas, which is beneficial to improving the dynamic range of the enhanced image. Furthermore, by fusing the second image pyramids, the advantages of different exposure images can be combined, avoiding overexposure or underexposure problems in a single exposure image, which is beneficial to improving the overall quality of the enhanced image.

[0014] In one implementation of the first aspect, acquiring the exposure images in the multi-exposure image sequence includes: acquiring multiple exposure images with different exposure times.

[0015] In the implementation of the above scheme, multi-exposure image sequences can be obtained by setting the camera exposure time. Under different exposure times, the camera can capture details in different brightness areas, which is beneficial to obtaining high-quality multi-exposure image sequences, thereby improving and enhancing the image quality.

[0016] In one implementation of the first aspect, obtaining the target image in the multi-exposure image sequence includes: performing linear brightness scaling on the image to be enhanced to obtain multiple exposure images for the same scene with different brightness levels.

[0017] In the implementation process of the above solution, it can be obtained by performing linear brightness scaling on a single-frame image. By adjusting the gain factor, multiple exposure images with different brightness levels can be quickly obtained; on the other hand, linear brightness scaling can also preserve the original details and texture information of the image, which is beneficial to improving the image quality of the enhanced image.

[0018] In one implementation of the first aspect, calculating the second image pyramid of the exposure image includes: obtaining the contrast, saturation, and exposure of each pixel point in the exposure image; calculating the fusion weight of each pixel point based on the contrast, the saturation, and the exposure; performing normalization processing on the fusion weight to obtain a normalized fusion weight; decomposing the normalized fusion weight into a third image pyramid; obtaining a fourth image pyramid of the exposure image based on the third image pyramid; wherein, the zero-layer image of the fourth image pyramid is the zero-layer image of the third image pyramid; the m-layer image of the fourth image pyramid is obtained based on the image difference between the m-layer image of the third image pyramid and the initial upsampled image, 1 < m ≤ N; the initial upsampled image is the upsampled image of the (m - 1)-layer image of the third image pyramid; fusing the fourth image pyramid and the third image pyramid of the exposure image to obtain the second image pyramid of the exposure image.

[0019] In the implementation process of the above solution, by calculating the contrast, saturation, and exposure of pixel points and calculating the fusion weight based on these metrics, it is possible to retain the details and texture information of the image during the fusion process. The reasonable allocation of the fusion weight makes the contribution of each pixel point match its visual importance, so that more details are retained in the enhanced image; on the other hand, by constructing the third image pyramid and the fourth image pyramid, the image can be processed separately at different scales. The multi-scale processing method can capture the features of the image at different resolutions, so that more details and texture information are retained during the fusion process.

[0020] In one implementation of the first aspect, the third image pyramid includes a Gaussian pyramid.

[0021] In the implementation of the above scheme, the third image pyramid can be a Gaussian pyramid. The Gaussian pyramid can perform layer-by-layer downsampling and Gaussian filtering on the normalized fusion weights, thereby generating a multi-scale weight distribution. This is beneficial to reduce information mismatch in the multi-scale fusion process and to alleviate the contrast imbalance defect that occurs when enhancing the contrast of the image. On the other hand, the construction and operation of the Gaussian pyramid has low complexity, which makes the above image enhancement method applicable to more application scenarios and helps to improve the adaptability of the above image enhancement method.

[0022] In one implementation of the first aspect, the second image pyramid includes a Laplace pyramid; the fourth image pyramid includes a Laplace pyramid.

[0023] In the implementation of the above scheme, both the second and fourth image pyramids can adopt the Laplacian pyramid. The Laplacian pyramid can effectively separate image features at different scales. In the image restoration process, it can also restore the original image details by layer-by-layer superposition of residuals, thereby improving the image enhancement effect.

[0024] In one implementation of the first aspect, the first image pyramid includes the Laplace pyramid.

[0025] In the implementation of the above scheme, the first image pyramid can be a Laplacian pyramid. The Laplacian pyramid captures high-frequency detail information of the image layer by layer through residual calculation, which can effectively separate image features at different scales. In the image restoration process, it can also restore the original image details by layer-by-layer superposition of residuals, avoiding information loss caused by multi-scale fusion, thereby improving the image enhancement effect.

[0026] In one implementation of the first aspect, cropping the pixel values ​​of each pixel in the nth layer reconstructed image based on the preset cropping threshold range includes: performing hard cropping on the pixel values ​​of each pixel in the nth layer reconstructed image based on the preset cropping threshold range.

[0027] In the implementation of the above scheme, by performing hard cropping on the pixel values ​​of each pixel in the reconstructed image, pixel values ​​exceeding the preset cropping threshold range are directly limited to the preset cropping threshold range, thereby ensuring that the pixel values ​​of all pixels are within the specified range, which helps to improve the problem of pixel value exceeding the limit. On the other hand, the hard cropping processing method has high computational efficiency, which helps to improve the processing efficiency of the above image enhancement method. Furthermore, the hard cropping processing method does not introduce additional blurring or distortion, which helps to improve the image enhancement effect of the above image enhancement method.

[0028] In one implementation of the first aspect, normalizing the pixel values ​​of each pixel in the nth layer reconstructed image includes: performing linear normalization on the nth layer reconstructed image after cropping.

[0029] In the implementation of the above scheme, by normalizing the reconstructed image after cropping, the relative brightness and contrast relationship of the original pixel values ​​can be preserved, reducing local distortion caused by nonlinear transformation. This makes the enhanced image more in line with the natural effect perceived by the human eye, which is beneficial to improving the image enhancement effect of the above image enhancement method. On the other hand, the computational complexity of linear normalization is low, which is beneficial to improving the processing efficiency of the above image enhancement method. Furthermore, linear normalization can effectively control the range of pixel values, which is beneficial to solving the problem of pixel values ​​going out of bounds and suppressing Halo artifacts, which is beneficial to further improving the image enhancement effect of the above image enhancement method.

[0030] Secondly, embodiments of this application provide an image enhancement apparatus, the apparatus comprising:

[0031] An image restoration module is used to perform N-1 image restorations based on a first image pyramid to obtain an enhanced image; wherein, the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image after enhancement of the image to be enhanced; and N is the number of layers in the first image pyramid.

[0032] The image restoration module performs the nth image restoration, 1≤n≤N-1, including:

[0033] The target image is upsampled; wherein, when n=1, the target image is the zeroth layer image of the first image pyramid, and when n>1, the target image is the (n-1)th layer reconstructed image;

[0034] The upsampled target image is fused with the nth layer image to obtain the nth layer reconstructed image;

[0035] Based on a preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are cropped and normalized to obtain the nth layer reconstructed image.

[0036] Thirdly, embodiments of this application provide an electronic device, including: a processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other through the communication bus; the memory stores computer program instructions that can be executed by the processor, and the computer program instructions are read and executed by the processor to perform the method provided in the first aspect or any possible implementation of the first aspect.

[0037] Fourthly, embodiments of this application provide a computer-readable storage medium storing computer program instructions, which, when read and executed by a processor, perform the method provided in the first aspect or any possible implementation thereof.

[0038] Fifthly, embodiments of this application provide a computer program product, the computer program product including a computer program, which, when executed by a processor, implements the method provided by the first aspect or any possible implementation of the first aspect.

[0039] Other features and advantages of this application will be set forth in the following description and will be apparent in part from the description or may be learned by practicing embodiments of this application. The objectives and other advantages of this application may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0040] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments of this application will be briefly introduced below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation of the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 A schematic diagram of the Halo effect that occurs after the image undergoes contrast enhancement processing using the related technologies provided in the embodiments of this application;

[0042] Figure 2 A schematic diagram illustrating the acquisition process of the first image pyramid and the third image pyramid in the related technologies provided in the embodiments of this application;

[0043] Figure 3 A schematic diagram illustrating the process of image reconstruction using a first image pyramid in the related technology provided in the embodiments of this application;

[0044] Figure 4 A schematic flowchart illustrating the image enhancement method provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram illustrating the out-of-bounds problem that occurs after the image undergoes contrast enhancement processing using the related technologies provided in the embodiments of this application.

[0046] Figure 6 A schematic diagram illustrating the acquisition process of the fourth image pyramid provided in this application embodiment;

[0047] Figure 7A schematic diagram illustrating the process of obtaining a first image pyramid from two target images, provided for an embodiment of this application;

[0048] Figure 8 A schematic diagram comparing the image enhancement effects of image enhancement schemes in related technologies provided in the embodiments of this application and image enhancement methods provided in the embodiments of this application;

[0049] Figure 9 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0050] The technical solutions of the embodiments of this application will now be described with reference to the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solutions of this application, and are therefore merely examples and should not be used to limit the scope of protection of this application.

[0051] This application provides an image enhancement method. During image restoration using a first image pyramid, the method performs cropping and normalization processing on each layer of the reconstructed image. Cropping is used to cut the pixel values ​​of each pixel in the reconstructed image to a preset cropping threshold range, and normalization is used to smooth the cropped image. On one hand, the preset cropping threshold range, through cropping and normalization processing, helps avoid contrast imbalance during image contrast enhancement. On the other hand, compared to the Halo solution in related technologies, the above method, by performing cropping and normalization processing during the iterative process of image restoration using the first image pyramid, has lower computational complexity and is easier to implement.

[0052] Before introducing the above image enhancement methods, let's briefly introduce the process of obtaining the Laplacian pyramid and the process of image reconstruction using the Laplacian pyramid:

[0053] 1. The process of obtaining the Pyramid of Laplace:

[0054] Please see Figure 2 The Laplacian pyramid and Gaussian pyramid can be obtained by iteratively applying the original image f0, with the number of iterations determined by a preset minimum image resolution. Each iteration includes: filtering the input image for the current iteration to obtain the filtered image; calculating the residual between the input image and the filtered image for the current iteration; downsampling the filtered image and using the downsampled image as the input image for the next iteration.

[0055] by Figure 2Taking the images shown as an example: image f0 is the original image, image l0 is the image obtained after filtering image f0, image h0 is the residual image between image f0 and image l0, image f1 is the image obtained after downsampling image l0, image l1 is the image obtained after filtering image f1, and image h1 is the residual image between image f1 and image l1. Image f2 is the image obtained after downsampling image l1. It can be understood that the above filtering can be Gaussian filtering.

[0056] Both the constructed Laplacian pyramid and Gaussian pyramid have three layers, including a zero-layer image, a first-layer image, and a second-layer image. The second-layer image of the Gaussian pyramid is image l0, the first-layer image is image l1, and the zero-layer image is image f2 (the zero-layer image can also be obtained by Gaussian filtering image f2). The second-layer image of the Laplacian pyramid is image h0, the first-layer image is image h1, and the zero-layer image is image f2 (the zero-layer image can also be obtained by Gaussian filtering image f2).

[0057] 2. The process of image reconstruction using the Laplacian pyramid:

[0058] Please see Figure 3 The reconstructed image is obtained by iterating through each layer of the first image pyramid multiple times. The number of iterations is determined by the preset minimum image resolution and the original resolution of the image. Each iteration process includes: upsampling the reconstructed image of the previous layer. When the previous layer is the zeroth layer of the pyramid, the zeroth layer image can be directly upsampled; and fusing (or overlaying) the upsampled image with the current layer image of the Laplacian pyramid to obtain the reconstructed image of the current layer.

[0059] by Figure 3 Taking the image shown as an example: Image f2 is the zeroth layer image of the Laplacian pyramid. First, the zeroth layer image is upsampled to obtain image l1; the first layer image h1 of the Laplacian pyramid is fused with image l1 to obtain the first layer reconstructed image f1; image f1 is upsampled to obtain image l0; the second layer image h0 of the Laplacian pyramid is fused with image l0 to obtain the second layer reconstructed image f0. Thus, the original image restoration is completed.

[0060] In addition, the upsampling mentioned above can be achieved using Bilinear interpolation, and the downsampling can be achieved using Bilinear downsampling.

[0061] Understandably, due to Figure 2 and Figure 3 The image examples shown are images that have been blurred and occluded. Figure 2 and Figure 3 The illustration may differ from the actual processing effect, for example: in Figure 2 In the middle, it is difficult to distinguish the image with the naked eye. With images The difference between them, but the image For the image The image obtained after filtering will be processed in the actual image processing. With images There will be some differences. Therefore, Figure 2 This is merely one reference point regarding the process of obtaining the pyramids. Figure 3 This is merely one reference for the process of using pyramids for image reconstruction. It should be noted that there are generally two ways to describe the layers of an image in a Laplacian pyramid. One way is to describe them in the order they were acquired, with the highest resolution image at the bottom of the pyramid, or layer zero, and the lowest resolution image at the top. The other way is to sort them by image resolution, with the lowest resolution image at the bottom of the pyramid, or layer zero, and the highest resolution image at the top. In the following description, the number of pyramid layers will be described using the second sorting method, that is, by image resolution.

[0062] The image enhancement methods described above are introduced below. Please refer to [link / reference]. Figure 4 This application provides an image enhancement method. This image enhancement method can be applied to electronic devices, which may include physical devices such as servers, PCs, tablets, or smartphones, or virtual devices such as virtual machines or containers. The electronic device can be a single device, a combination of multiple devices, or a cluster of a large number of devices. The above image enhancement method may include:

[0063] Based on the first image pyramid, N-1 image restorations are performed to obtain an enhanced image; where the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image after enhancement of the image to be enhanced; and N is the number of layers in the first image pyramid.

[0064] The above image restoration process refers to the process of restoring a high-resolution image from a low-resolution image of the first image pyramid. The first image restoration is based on the zero-level image and the first-level image of the first image pyramid. The second image restoration is based on the reconstructed image obtained from the first image restoration and the second-level image, and so on, until the N-1th image restoration obtains an enhanced image with a preset image resolution.

[0065] The nth image restoration, 1≤n≤N-1, includes:

[0066] Step S110: Upsample the target image; wherein, when n=1, the target image is the zeroth layer image of the first image pyramid, and the zeroth layer image is the image with the smallest image resolution in the first image pyramid; when n>1, the target image is the (n-1)th layer reconstructed image.

[0067] Step S120: Fuse the upsampled target image with the nth layer image of the first image pyramid to obtain the nth layer reconstructed image;

[0068] Step S130: Based on the preset cropping threshold range, crop and normalize the pixel values ​​of each pixel in the nth layer reconstructed image to obtain the nth layer reconstructed image.

[0069] Optionally, step S130 above involves first performing cropping and then normalization. The cropping process limits the pixel values ​​of the reconstructed image to a preset cropping threshold range. This prevents pixel values ​​from exceeding the preset threshold, thus reducing Halo effects caused by excessive brightness variations. The normalization process readjusts the pixel values ​​of the reconstructed image to the value range [0,1]. This redistributes the image's brightness values, resulting in more uniform brightness changes. Through normalization, Halo effects caused by excessive brightness variations are reduced, leading to more natural brightness transitions in the image. It is understandable that the Halo artifacts produced by the image restoration scheme based on the first image pyramid are due to an imbalance between the intensity of high-frequency and low-frequency components in strong edge regions. Specifically, the intensity of the high-frequency component exceeds the difference between the low-frequency component and its value range; that is, the dynamic range of the high-frequency component is too high. This causes the sum of the upsampled low-frequency and high-frequency components to exceed the dynamic range that the low-frequency component can handle during image restoration based on the first image pyramid. The aforementioned scheme uses cropping and normalization processes during image restoration to scale the image, which originally exceeded the value range [0,1], to the correct value range. Here, the value range [0,1] can be considered as the dynamic range that a typical sRGB image can exhibit.

[0070] In addition to reducing the halo effect caused by excessive brightness variations, the cropping and normalization processes described above can also alleviate the out-of-range problem in the recovered image. The out-of-range problem refers to the pixel values ​​of the enhanced image exceeding their limits (i.e., the RGB pixel values ​​of a point exceed the range of 0-255; generally, for clarity, the value range of the RGB image is normalized to a floating-point number of 0-1). This often manifests as overexposure or decreased color saturation in some areas of the recovered image, such as... Figure 5The two white-framed areas are shown in the diagram. The above solution, through cropping and normalization, effectively prevents the pixel values ​​of the recovered image from exceeding the value range, thus solving the aforementioned out-of-bounds problem.

[0071] In related technologies that rely on solving the Poisson equation for image restoration, it is necessary to first calculate the first-order gradients of the image in the X and Y directions. Then, it is determined whether the gradient signs of the enhanced image and the original image are reversed. If the signs of the gradients in the two images are inconsistent, the signs of the gradients in the enhanced image need to be corrected based on the signs of the gradients in the image before enhancement. After gradient correction, the corrected gradients need to be converted into Hessian matrices for image restoration. This restoration process is generally accelerated by performing an FFT on the corrected X and Y directions, but even when converted to the Fourier transform domain, the computational complexity is at least linear logarithmic O(NlogN). Compared to related technologies that rely on solving the Poisson equation for image restoration, the image enhancement method provided in this application can suppress Halo artifacts through cropping and normalization processes, resulting in lower computational complexity and a linear complexity of O(N).

[0072] The following describes an optional implementation of the cropping process for the fused image in step S130:

[0073] The first implementation method is to perform hard cropping on the pixel values ​​of each pixel in the reconstructed image.

[0074] Optionally, step S130 above, which performs cropping processing on the pixel values ​​of each pixel in the nth layer reconstructed image based on a preset cropping threshold range, includes: performing hard cropping processing on the pixel values ​​of each pixel in the nth layer reconstructed image based on the preset cropping threshold range. An example of this implementation is:

[0075] Let New denote the fused image of the nth layer, and set the preset cropping threshold range as [MinValue, MaxValue].

[0076] First, calculate the maximum and minimum pixel values ​​for each pixel in New:

[0077] M1 = Max(New)

[0078] M2 = Min(New)

[0079] Secondly, the pixel values ​​of the reconstructed image New in the nth layer are corrected:

[0080] M1′=Min(MaxValue,M1)

[0081] M2′=Max(MinValue,M2)

[0082] N1 = Max(M2′, Nrw)

[0083] N2 = Min(M1′, N1)

[0084] When performing pixel-by-pixel processing, N1 and N2 can be used to represent the pixel value of the current pixel after the lower limit cropping process and the pixel value after the lower limit cropping process and the upper limit cropping process, respectively.

[0085] The above hard cropping principle is as follows: if the pixel value of a pixel is less than the minimum value (MinValue) within the preset cropping threshold range, then the pixel value of that pixel is set to MinValue; if the pixel value of a pixel is greater than the maximum value (MaxValue) within the preset cropping threshold range, then the pixel value of that pixel is set to MaxValue. This cropping principle can be expressed as:

[0086]

[0087] Among them, px clipped px represents the pixel value after cropping; px represents the pixel value of a pixel in the reconstructed image.

[0088] The above scheme performs hard cropping on the pixel values ​​of each pixel in the reconstructed image, directly limiting the pixel values ​​in the reconstructed image that exceed the preset cropping threshold range to within the preset cropping threshold range. This ensures that the pixel values ​​of all pixels are within the specified range, which helps to improve the problem of pixel values ​​exceeding the limit. On the other hand, the hard cropping processing method has high computational efficiency, which helps to improve the processing efficiency of the above image enhancement methods. Furthermore, the hard cropping processing method does not introduce additional blur or distortion, which helps to improve the image enhancement effect of the above image enhancement methods.

[0089] The second implementation method is to perform soft cropping on the pixel values ​​of each pixel in the reconstructed image.

[0090] Soft clipping is a smooth clipping method that uses linear interpolation, non-linear mapping, and other techniques to adjust values ​​that are out of range to the target range.

[0091] The fused image after soft cropping can have a smooth transition at the boundaries and can also reduce the probability of boundary artifacts.

[0092] The following describes an optional implementation of the normalization process for the reconstructed image in step S130:

[0093] The first implementation method is to perform linear normalization on the reconstructed image after cropping.

[0094] Optionally, step S130 above, which normalizes the pixel values ​​of each pixel in the nth layer reconstructed image, includes: performing linear normalization on the nth layer reconstructed image after cropping. An example of this implementation is:

[0095] For each pixel in the reconstructed image after cropping, the normalized pixel value is:

[0096]

[0097] Where N is the pixel value after normalization; M1 and M2 are the maximum and minimum pixel values ​​of each pixel in the fused image after cropping, respectively; N2 is used to represent the pixel values ​​after lower and upper bound cropping.

[0098] The above linear normalization method can map the pixel values ​​of the reconstructed image after cropping to the value range [0,1].

[0099] The above-mentioned scheme, by normalizing the reconstructed image after cropping, can preserve the relative brightness and contrast relationships of the original pixel values, reduce local distortion caused by nonlinear transformation, and make the enhanced image more in line with the natural effect perceived by the human eye, thus improving the image enhancement effect of the above image enhancement method. On the other hand, the computational complexity of linear normalization is low, which helps to improve the processing efficiency of the above image enhancement method. Furthermore, linear normalization can effectively control the range of pixel values, which helps to solve the problem of pixel values ​​going out of bounds and suppress Halo artifacts, further improving the image enhancement effect of the above image enhancement method.

[0100] The following explains some of the factors that influence the above-mentioned preset cropping threshold range:

[0101] (1) Suppression effect on Halo artifacts:

[0102] Halo artifacts are a brightness inversion phenomenon caused by the mismatch of brightness information between different levels during multi-scale pyramid fusion. By appropriately controlling the range of pixel values ​​in the reconstructed image, abrupt brightness changes during pyramid level fusion can be reduced, thereby suppressing Halo artifacts. Furthermore, appropriately controlling the pixel value threshold range can preserve more intermediate details, thus reducing edge blurring caused by strictly limiting the pixel value range.

[0103] (2) Balance between contrast preservation and dynamic range compression:

[0104] The dilemma in balancing contrast preservation and dynamic range compression lies in the fact that while an excessively narrow pixel value range can eliminate out-of-bounds pixel values, it forces dynamic range compression, leading to image flattening and reduced contrast. Conversely, an excessively wide pixel value range, while preserving more brightness levels, is less effective at suppressing Halo artifacts. Therefore, it is necessary to appropriately control the pixel value range of pixels in the reconstructed image.

[0105] In summary, this embodiment sets the preset cropping threshold range to [-0.1, 1.1], which means that compared to the normal range of pixel values ​​[0, 1], the upper and lower limits of the preset cropping threshold range are each expanded by 10%. The above-mentioned empirical value for the preset cropping threshold range represents a balance between Halo suppression and image contrast preservation. Furthermore, this empirical value also considers both the visual effect of image enhancement and algorithm efficiency.

[0106] The following describes the method for obtaining the first image pyramid of the enhanced image in the embodiments of this application:

[0107] Optionally, before performing N-1 image restorations based on the first image pyramid, the above image enhancement method further includes:

[0108] Obtain a multi-exposure image sequence of the image to be enhanced; wherein, the multi-exposure image sequence includes multiple exposure images of the same scene with different brightness; calculate the second image pyramid of at least two exposure images in the multi-exposure image sequence to obtain at least two second image pyramids; fuse the second image pyramids of at least two exposure images to obtain the first image pyramid of the enhanced image.

[0109] The above-mentioned method of fusing at least two second image pyramids can be: superimposing at least two second image pyramids.

[0110] The above scheme, by fusing the second image pyramids of at least two exposed images, can preserve the details and texture information of the well-exposed parts of each image, thus obtaining richer details and textures in the final fusion result, which is beneficial to improving the image enhancement effect of the above image enhancement method. On the other hand, images with different exposures usually cover different brightness ranges. By fusing the second image pyramids, the information of these different brightness ranges can be integrated into one image, thereby improving the dynamic range of the final image, so that the image can retain more details in both bright and dark areas, which is beneficial to improving the dynamic range of the enhanced image. Furthermore, by fusing the second image pyramids, the advantages of different exposure images can be combined, avoiding overexposure or underexposure problems in a single exposure image, which is beneficial to improving the overall quality of the enhanced image.

[0111] Optionally, the above-mentioned acquisition of the exposure images in the multi-exposure image sequence includes:

[0112] Acquire multiple images with different exposure times.

[0113] The aforementioned multiple exposure images with different exposure times are typically obtained by adjusting the camera's exposure time. At different exposure times, the camera can capture details in areas of varying brightness, which is beneficial for obtaining a high-quality sequence of multiple exposure images, thereby improving and enhancing the image quality.

[0114] Optionally, the above-mentioned acquisition of the exposure images in the multi-exposure image sequence includes:

[0115] Linear brightness scaling is applied to the image to be enhanced to obtain multiple exposure images of the same scene with different brightness levels.

[0116] Understandably, when enhancing the contrast of a single-frame image, a multi-exposure image sequence can simulate adjusting the camera's exposure time to obtain images with different brightness levels by globally scaling the brightness of each single frame. The adjustment process is as follows:

[0117] I i =Min(1,I×gain) i ); gain i ∈{0.1,0.5,1,2,4,6,8,12……}

[0118] Where I represents the single frame image to be enhanced; gain i `i` is the i-th gain factor used to adjust the brightness of the image; `Min(1,·)` is a cropping operation used to ensure that the pixel value of the image after brightness scaling will not be greater than 1. If a pixel value in `I×gain5` is greater than 1, it is set to 1; i For the gain factor of the i-th gain factor i The exposure image obtained after brightness scaling.

[0119] The above method can be obtained by linearly scaling the brightness of a single frame image. By adjusting the gain factor, multiple exposure images with different brightness can be obtained quickly. On the other hand, linear brightness scaling can also preserve the original details and texture information of the image, which is beneficial to improving the image quality of the enhanced image.

[0120] The following describes how the second image pyramid of the above-mentioned exposed image is calculated:

[0121] Optionally, a second image pyramid of the target image is calculated, including:

[0122] Step S210: Obtain the contrast, saturation, and exposure of each pixel in the exposed image.

[0123] Calculate local contrast pixel by pixel:

[0124]

[0125] Where Contrast is the local contrast of a pixel; LaplacianKernel is the Laplacian filter function; and I′ is the exposed image. This is a convolution operation;

[0126] Calculate saturation pixel by pixel;

[0127] Large differences between the RGB channels can be considered areas of high saturation. Conversely, for overexposed or underexposed areas, the values ​​of the RGB channels are consistent across the same region, indicating low saturation. Therefore, the standard deviation between the three RGB channels can be used as an indicator of saturation.

[0128] In this embodiment, pixels with values ​​around 0.5 can be considered well-exposed and therefore assigned a larger weight, while pixels with values ​​close to 0 and 1 are considered underexposed and overexposed, respectively, and therefore assigned a smaller weight. Exposure is calculated pixel-by-pixel:

[0129] Exposure = e -(I′-0.5)

[0130] Step S220: Calculate the fusion weight of each pixel based on contrast, saturation, and exposure.

[0131] The fusion weights can be calculated as follows:

[0132] weight=Contrast.*Colorful.*Exposure

[0133] Step S230: Normalize the fusion weights to obtain normalized fusion weights.

[0134] The normalized fusion weights can be calculated as follows:

[0135]

[0136] in, The normalized weights for the i-th target image; weight i is the fusion weight for the i-th target image; N is the number of target images.

[0137] Step S240: Decompose the normalized fusion weights into a third image pyramid.

[0138] It can be understood that the third image pyramid can be the third image pyramid of the target weight map, and the target weight map can be obtained by multiplying the normalized fusion weight obtained in the above step S230 by the target image.

[0139] Step S250: Based on the third image pyramid, obtain the fourth image pyramid of the exposure image.

[0140] The process of obtaining the fourth image pyramid is as Figure 6 shown. The zeroth layer image of the fourth image pyramid is the zeroth layer image of the third image pyramid; the mth (1 < m ≤ N) layer image of the fourth image pyramid is obtained based on the image difference between the mth layer image of the third image pyramid and the initial upsampled image; the initial upsampled image is the upsampled image of the (m - 1)th layer image of the third image pyramid.

[0141] Step S260: Fuse the fourth image pyramid and the third image pyramid to obtain the second image pyramid of the exposure image.

[0142] Please refer to Figure 7 . Each layer image of the third image pyramid is a weight map (Weight Map). The fusion method of the fourth image pyramid and the third image pyramid can be: multiply the corresponding layer images of the fourth image pyramid and the third image pyramid to obtain the corresponding layer image of the second image pyramid.

[0143] It can be understood that Figure 7 shows the process of fusing the second image pyramids of two target images. The obtained fusion pyramid is the first image pyramid corresponding to the above enhanced image, and the enhanced image of the image to be enhanced can be restored through the fusion pyramid. The fusion process of the fourth image pyramids and the third image pyramids of multiple target images and the fusion process of the second image pyramids of multiple target images can be described as:

[0144]

[0145] Among them, N is the number of target images, k is the index of the target image; I is the pixel value of the Y channel of the target image; G is the third image pyramid, is the normalized fusion weight, is the third image pyramid; L{I} is the fourth image pyramid, L{R} is the first image pyramid corresponding to the enhanced image; l is the index of the pyramid layer; (i, j) is the index of the pixel point within a certain layer of the pyramid; R is the restored enhanced image.

[0146] It is understandable that the aforementioned fourth image pyramid refers to an image pyramid constructed based on the Y channel of the target image. If the target image is in the YUV domain, the existing YUV Y channel is used directly. If it is in the RGB domain, the conversion coefficients corresponding to bt601 and bt709 are used to convert the RGB image to YUV and then the Y channel is used.

[0147] The above scheme calculates the contrast, saturation, and exposure of pixels and calculates fusion weights based on these indicators. This allows the preservation of image details and texture information during the fusion process. The reasonable allocation of fusion weights ensures that the contribution of each pixel matches its visual importance, thereby preserving more details in the enhanced image. On the other hand, by constructing a third and fourth image pyramid, images can be processed at different scales. This multi-scale processing method can capture the features of images at different resolutions, thus preserving more details and texture information during the restoration process.

[0148] Optionally, the third image pyramid mentioned above includes the Gaussian pyramid.

[0149] The Gaussian pyramid described above is an image representation method that generates a series of images with different resolutions by continuously downsampling an image. One method for generating a Gaussian pyramid involves first applying Gaussian filtering to the image, then downsampling it at a certain step size, repeating this process to obtain a pyramid image sequence with the image resolution gradually decreasing from top to bottom. Another method involves calculating normalized fusion weights based on contrast, saturation, and exposure, as described in the above scheme, and then obtaining the pyramid through normalized weight decomposition.

[0150] The third image pyramid in the above scheme can be a Gaussian pyramid. The Gaussian pyramid can perform layer-by-layer downsampling and Gaussian filtering on the normalized fusion weights, thereby generating a multi-scale weight distribution. This is beneficial to reduce information mismatch in the multi-scale fusion process and to alleviate the contrast imbalance defect that occurs when enhancing the contrast of the image. On the other hand, the construction and operation complexity of the Gaussian pyramid is relatively low, which makes the above image enhancement method applicable to more application scenarios and helps to improve the adaptability of the above image enhancement method.

[0151] Optionally, the second image pyramid of the exposed image includes a Laplacian pyramid; the fourth image pyramid of the exposed image includes a Laplacian pyramid.

[0152] The Laplacian Pyramid is a multi-resolution image representation method based on Gaussian pyramids.

[0153] Both the second and fourth image pyramids in the above scheme can be Laplacian pyramids. Laplacian pyramids can effectively separate image features at different scales and can also restore the original image details by layer-by-layer superposition of residuals during the image restoration process, thereby improving the image enhancement effect.

[0154] Optionally, the first image pyramid of the enhanced image described above includes the Laplacian pyramid.

[0155] The first image pyramid in the above scheme can be a Laplacian pyramid. The Laplacian pyramid captures high-frequency detail information of the image layer by layer through residual calculation, which can effectively separate image features at different scales. In the process of image restoration, it can also restore the original image details by layer-by-layer superposition of residuals, avoiding information loss caused by multi-scale fusion, thereby improving the image enhancement effect.

[0156] To further illustrate the working principle of the above image enhancement method, the following describes a specific implementation method for contrast enhancement or dynamic range compression of a single-frame image in a certain scene. Let the input single-frame image be I″. The main steps of the above image enhancement method include:

[0157] Step 1: Obtain a series of images with different brightness by linearly scaling the brightness of the input image I;

[0158] I i "=Min(1,I×gain i ); gain i ∈{0.1,0.5,1,2,4,6,8,12……}

[0159] Step 2: Calculate the input image I at different brightness levels. i The Y channel pixel value of " i ;

[0160] Step 3: Calculate the input image I at different brightness levels. i The normalized fusion weights of "".

[0161] (1) Calculate local contrast pixel by pixel:

[0162]

[0163] (2) Calculate saturation pixel by pixel;

[0164] (3) Calculate exposure pixel by pixel:

[0165] Exposure = e -(I″-0.6)

[0166] (4) Calculate the fusion weights pixel by pixel:

[0167] weight=Contrast.*Colorful.*Exposure

[0168] (5) Calculate multiple input images I with different brightness levels. i Normalized weights of "":

[0169]

[0170] Step 4: Construct a Laplacian pyramid based on the Y channel of each input image;

[0171] Step 5: Construct a Gaussian pyramid for the normalized weights;

[0172] Step Six: Merge different levels within the pyramid to obtain the merged Laplacian pyramid, which is the first image pyramid L{R} of the enhanced image.

[0173]

[0174] Step 7: Use the Laplacian pyramid of the enhanced image to perform image restoration and obtain the enhanced image of the input single frame image I″;

[0175] Based on empirical values, a preset cropping threshold range for the pixel values ​​of the reconstructed image pixels during the pyramid restoration process is given. The upper limit of the preset cropping threshold range is MaxValue = 1.1, and the lower limit is MinValue = -0.1.

[0176] In the iterative process of image restoration using the first image pyramid enhanced by the image, the pixel values ​​of the reconstructed image are cropped and normalized. The iterative process includes:

[0177] A. Upsampling the zeroth layer image in the first image pyramid L{R} requires enlarging the zeroth layer image of the first image pyramid to the same image size as the first layer image of the first image pyramid L{R}, denoted as:

[0178] Lowpass = 2 × UPscale(LR{0})

[0179] Where Lowpass is the magnified low-frequency layer; LR{0} represents the zeroth layer of the first image pyramid; 2×Upscale(LR{0}) means that the image LR{0} is upsampled by a factor of two.

[0180] B. Stacking residual layer and amplified low-frequency layer:

[0181] New = Lowpass + LR{1}

[0182] Wherein, LR{1} is the first layer of the first image pyramid of the enhanced image, which can also be called the residual layer.

[0183] C. Calculate the maximum and minimum pixel values ​​for each pixel in New:

[0184] M1 = Max(New)

[0185] M2 = Min(New)

[0186] D. Crop and correct the value range of the fused image New in the current layer:

[0187] M1′=Min(MaxValue,M1)

[0188] M2′=Max(MinValue,M2)

[0189] N1 = Max(M2′, Nrw)

[0190] N2 = Min(M1′, N1)

[0191] E. Normalize the fused image N2 after cropping:

[0192]

[0193] Where N is the pixel value after normalization; M1 and M2 are the minimum and maximum pixel values ​​of each pixel in the fused image after cropping, respectively; N2 is used to represent the pixel values ​​after lower and upper bound cropping.

[0194] F. Upsample the normalized reconstructed image N to the image size corresponding to the next residual layer, and repeat B to E above until the final image is restored, that is, the last reconstructed image reaches the preset image resolution.

[0195] A comparative illustration of the image enhancement effect of the image enhancement method provided in this application embodiment in certain application scenarios and the image enhancement effect of related technologies is shown below. Figure 8 As shown, where, Figure 8 (a) and Figure 8 (b) are schematic diagrams of the image enhancement effect of the image enhancement scheme in the relevant technology in the first application scenario and the image enhancement effect of the image enhancement method provided in the embodiment of this application, respectively. Figure 8 (c) and Figure 8 (d) are schematic diagrams of the image enhancement effect of the image enhancement scheme in the related technology in the second application scenario and the image enhancement effect of the image enhancement method provided in the embodiment of this application, respectively. Figure 8(e) and Figure 8 (f) are schematic diagrams of the image enhancement effect of the image enhancement scheme in the related technology in the third application scenario and the image enhancement effect of the image enhancement method provided in the embodiment of this application, respectively. Figure 8 (g) and Figure 8 (h) are schematic diagrams of image enhancement effects of image enhancement schemes in related technologies in the fourth application scenario and schematic diagrams of image enhancement effects of the image enhancement method provided in the embodiments of this application, respectively; Figure 8 (i) and Figure 8 (j) are schematic diagrams illustrating the image enhancement effects of image enhancement schemes in related technologies in the fifth application scenario and the image enhancement effects of the image enhancement method provided in the embodiments of this application, respectively. Figure 8 As can be seen, the image enhancement method provided in this application embodiment can effectively solve the pixel value out-of-bounds problem and has a better suppression effect on Halo artifacts.

[0196] Based on the same inventive concept, this application also provides an image enhancement device, which includes:

[0197] The image restoration module is used to perform N-1 image restorations based on the first image pyramid to obtain an enhanced image; wherein, the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image after enhancement of the image to be enhanced; and N is the number of layers in the first image pyramid;

[0198] The image restoration module performs the nth image restoration, 1≤n≤N-1, including:

[0199] The target image is upsampled; when n=1, the target image is the zeroth layer image of the first image pyramid, and when n>1, the target image is the (n-1)th layer reconstructed image.

[0200] The upsampled target image is fused with the nth layer image to obtain the nth layer reconstructed image;

[0201] Based on a preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are cropped and normalized to obtain the nth layer reconstructed image.

[0202] Optionally, the image enhancement device described above further includes:

[0203] The first image pyramid acquisition module is used to acquire a multi-exposure image sequence of the image to be enhanced; wherein, the multi-exposure image sequence includes multiple exposure images of the same scene with different brightness; calculates the second image pyramid of at least two exposure images in the multi-exposure image sequence to acquire at least two second image pyramids; and fuses the second image pyramids of at least two exposure images to acquire the first image pyramid of the enhanced image.

[0204] Optionally, the above first image pyramid acquisition module acquires exposure images in the multi-exposure image sequence, including:

[0205] Acquire multiple exposure images with different exposure times.

[0206] Optionally, the above first image pyramid acquisition module acquires exposure images in the multi-exposure image sequence, including:

[0207] Perform linear brightness scaling on the image to be enhanced, and acquire multiple exposure images for the same scene with different brightness levels.

[0208] Optionally, the above first image pyramid acquisition module calculates the second image pyramid of the exposure image, including: obtaining the contrast, saturation, and exposure of each pixel point in the exposure image; calculating the fusion weight of each pixel point based on the contrast, saturation, and exposure; performing normalization processing on the fusion weight to obtain a normalized fusion weight; decomposing the normalized fusion weight into a third image pyramid; obtaining the fourth image pyramid of the exposure image based on the third image pyramid; where the zeroth layer image of the fourth image pyramid is the zeroth layer image of the third image pyramid; the mth (1 < m ≤ N) layer image of the fourth image pyramid is obtained based on the image difference between the mth layer image of the third image pyramid and the initial upsampled image; the initial upsampled image is the upsampled image of the (m - 1)th layer image of the third image pyramid; fuse the fourth image pyramid and the third image pyramid of the exposure image to obtain the second image pyramid of the target image.

[0209] Optionally, the above third image pyramid includes a Gaussian pyramid.

[0210] Optionally, the above second image pyramid of the exposure image includes a Laplacian pyramid; the fourth image pyramid of the exposure image includes a Laplacian pyramid.

[0211] Optionally, the above first image pyramid of the enhanced image includes a Laplacian pyramid.

[0212] Optionally, the above image restoration module performs clipping processing on the pixel values of each pixel point in the nth layer reconstructed image based on a preset clipping threshold range, including:

[0213] Perform hard clipping processing on the pixel values of each pixel point in the nth layer reconstructed image based on the preset clipping threshold range.

[0214] Optionally, the above image restoration module performs normalization processing on the pixel values of each pixel point in the nth layer reconstructed image, including:

[0215] Perform linear normalization processing on the nth layer reconstructed image after the clipping processing.

[0216] Figure 9 This is a schematic diagram of an electronic device provided in an embodiment of this application. (Refer to...) Figure 9 The electronic device 300 includes a processor 310, a memory 320, and a communication interface 330. These components are interconnected and communicate with each other via a communication bus 340 and / or other forms of connection mechanism (not shown).

[0217] The memory 320 includes one or more (only one is shown in the figure), which may be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The processor 310 and other possible components may access the memory 320 to read and / or write data therein.

[0218] Processor 310 includes one or more (only one is shown in the figure), which can be an integrated circuit chip with signal processing capabilities. The processor 310 can be a data processing core of a GPU (Graphics Processing Unit), CPU (Central Processing Unit), AI (Artificial Intelligence), NPU (Neural Network Processing Unit), ISP (Image Signal Processor), DPU (Display Processing Unit), VPU (Video Processing Unit), or DSP (Digital Signal Processor), or a processor chip applied to scenarios such as large-scale data computation. The above are merely examples and should not be construed as limiting this application.

[0219] Communication interface 330 includes one or more (only one is shown in the figure) and can be used to communicate directly or indirectly with other devices to exchange data. For example, communication interface 330 can be an Ethernet interface; it can be a mobile communication network interface, such as an interface for 3G, 4G, or 5G networks; or it can be other types of interfaces with data transmission and reception functions.

[0220] One or more computer program instructions may be stored in the memory 320, and the processor 310 may read and run these computer program instructions to implement the image enhancement method provided in the embodiments of this application and other desired functions.

[0221] Understandable. Figure 9 The structure shown is for illustrative purposes only; the electronic device 300 may also include components that are more advanced than those shown. Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown. Figure 9 The components shown can be implemented using hardware, software, or a combination thereof. For example, electronic device 300 can be a single server (or other device with computing power), a combination of multiple servers, a cluster of a large number of servers, etc., and can be either a physical device or a virtual device.

[0222] This application also provides a computer-readable storage medium storing computer program instructions. These instructions are read and executed by a computer's processor to perform the image enhancement method and other desired functions provided in this application. For example, the computer-readable storage medium can be implemented as follows: Figure 9 The memory 320 in the electronic device 300.

[0223] Based on the same inventive concept, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described image enhancement method.

[0224] In the embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. The apparatus embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Furthermore, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Additionally, the displayed or discussed mutual couplings, direct couplings, or communication connections may be through some communication interfaces; indirect couplings or communication connections between devices or units may be electrical, mechanical, or other forms.

[0225] Furthermore, the units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0226] The above are merely embodiments of this application and are not intended to limit the scope of protection of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. An image enhancement method, characterized in that, The method includes: Based on the first image pyramid, performing N-1 times of image restoration to obtain an enhanced image; wherein, the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image obtained by enhancing the image to be enhanced; N is the number of layers of the first image pyramid; The nth image restoration, where 1≤n≤N-1, includes: Performing upsampling on the target image; wherein, when n = 1, the target image is the 0th layer image of the first image pyramid, and the 0th layer image is the image with the lowest image resolution in the first image pyramid; when n>1, the target image is the (n-1)th layer reconstructed image; Fusing the upsampled target image with the nth layer image of the first image pyramid to obtain the nth layer reconstructed image; Based on a preset clipping threshold range, performing clipping and normalization processing on the pixel values of each pixel point in the nth layer reconstructed image to obtain the nth layer reconstructed image.

2. The image enhancement method according to claim 1, characterized in that, Before performing N-1 times of image restoration based on the first image pyramid, the method further includes: Obtaining a multi-exposure image sequence of the image to be enhanced; wherein, the multi-exposure image sequence includes multiple exposure images for the same scene with different brightnesses; Calculating the second image pyramids of at least two exposure images in the multi-exposure image sequence respectively to obtain at least two second image pyramids; Fusing the second image pyramids of at least two of the exposure images to obtain the first image pyramid of the enhanced image.

3. The image enhancement method according to claim 2, characterized in that, Obtaining the exposure images in the multi-exposure image sequence includes: Obtaining multiple exposure images with different exposure times.

4. The image enhancement method according to claim 2, characterized in that, Obtaining the exposure images in the multi-exposure image sequence includes: Performing linear brightness scaling on the image to be enhanced to obtain multiple exposure images for the same scene with different brightnesses.

5. The image enhancement method according to claim 2, characterized in that, Calculating the second image pyramid of the exposure image includes: Obtaining the contrast, saturation, and exposure of each pixel point in the exposure image; Based on the contrast, the saturation, and the exposure, calculating the fusion weight of each pixel point; Performing normalization processing on the fusion weight to obtain a normalized fusion weight; Decomposing the normalized fusion weight into a third image pyramid; Based on the third image pyramid, obtaining the fourth image pyramid of the exposure image; wherein, the 0th layer image of the fourth image pyramid is the 0th layer image of the third image pyramid; the mth layer image of the fourth image pyramid is obtained based on the image difference between the mth layer image of the third image pyramid and the initial upsampled image, where 1<m≤N; the initial upsampled image is the upsampled image of the (m-1)th layer image of the third image pyramid; Fusing the fourth image pyramid and the third image pyramid of the exposure image to obtain the second image pyramid of the exposure image.

6. The image enhancement method according to claim 5, characterized in that, The third image pyramid includes a Gaussian pyramid.

7. The image enhancement method according to claim 5, characterized in that, The second image pyramid includes a Laplacian pyramid; the fourth image pyramid includes a Laplacian pyramid.

8. The image enhancement method according to any one of claims 1 to 7, characterized in that, The first image pyramid includes a Laplacian pyramid.

9. The image enhancement method according to any one of claims 1 to 7, characterized in that, Based on the preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are cropped, including: Based on the preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are subjected to hard cropping.

10. The image enhancement method according to any one of claims 1 to 7, characterized in that, The pixel values ​​of each pixel in the nth layer reconstructed image are normalized, including: The reconstructed image of the nth layer after cropping is subjected to linear normalization.

11. An image enhancement device, characterized in that, The device includes: An image restoration module is used to perform N-1 image restorations based on a first image pyramid to obtain an enhanced image; wherein, the first image pyramid is the image pyramid of the enhanced image; the enhanced image is the image after enhancement of the image to be enhanced; and N is the number of layers in the first image pyramid. The image restoration module performs the nth image restoration, 1≤n≤N-1, including: The target image is upsampled; wherein, when n=1, the target image is the zeroth layer image of the first image pyramid, and when n>1, the target image is the (n-1)th layer reconstructed image; The upsampled target image is fused with the nth layer image to obtain the nth layer reconstructed image; Based on a preset cropping threshold range, the pixel values ​​of each pixel in the nth layer reconstructed image are cropped and normalized to obtain the nth layer reconstructed image.

12. An electronic device, characterized in that, include: A processor, a memory, and a communication bus, wherein the processor and the memory communicate with each other via the communication bus; The memory stores program instructions that can be executed by the processor, and the processor can execute the method as described in any one of claims 1 to 10 by calling the program instructions.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, which, when executed by a computer, cause the computer to perform the method as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 10.

Images