An image enhancement method, device, storage medium and electronic equipment

Abstract

Embodiments of the present application disclose an image enhancement method and device, a storage medium and an electronic device. The method comprises: extracting a detail layer component from an original image to obtain a detail image; performing noise reduction processing on the original image and the detail image to obtain a noise-reduced image and a noise-reduced detail image, and performing fusion processing on the noise-reduced image and the noise-reduced detail image to obtain an enhanced image; wherein the noise reduction processing and the fusion processing use a convolutional neural network model. The technical solution can obtain a high-quality enhanced image.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular to an image enhancement method, apparatus, storage medium, and electronic device. Background Technology

[0002] To improve the quality of images or videos with low signal-to-noise ratios, the first step is noise reduction. However, existing noise reduction methods, whether based on traditional techniques or advanced deep learning, struggle to completely avoid the loss of detail. In other words, during noise reduction, the original details in the image or video are weakened or even lost. The root cause of this phenomenon is that current noise reduction technologies are unable to completely distinguish between noise and real details in an image.

[0003] To enhance image details, existing methods typically involve sharpening the image after noise reduction, which can restore details lost during the noise reduction process to some extent. Sharpening usually involves first extracting the detail layer of the image, then mapping and processing this detail layer according to certain rules before superimposing it back onto the original image.

[0004] However, the existing cascaded processing method of first reducing noise and then sharpening does not result in good image enhancement. Summary of the Invention

[0005] This application provides an image enhancement method, apparatus, storage medium, and electronic device. The method first extracts the detail layer components of the original image, and then performs noise reduction and fusion processing on the original image and the detail layer components to obtain a high-quality enhanced image.

[0006] In a first aspect, embodiments of this application provide an image enhancement method, the method comprising:

[0007] The detail layer components are extracted from the original image to obtain the detail image;

[0008] The original image and the detail image are denoised to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are then fused to obtain an enhanced image. The denoising and fusion processes are performed using a convolutional neural network model.

[0009] Secondly, embodiments of this application provide an image enhancement apparatus, the apparatus comprising:

[0010] The detail image acquisition module is used to extract detail layer components from the original image to obtain a detail image;

[0011] The image enhancement module is used to perform noise reduction processing on the original image and the detail image to obtain a noise-reduced image and a noise-reduced detail image, and to fuse the noise-reduced image and the noise-reduced detail image to obtain an enhanced image; wherein, the noise reduction processing and the fusion processing adopt a convolutional neural network model.

[0012] Thirdly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image enhancement method as described in embodiments of this application.

[0013] Fourthly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the image enhancement method as described in embodiments of this application.

[0014] The technical solution provided in this application involves extracting detail layer components from the original image to obtain a detail image; denoising the original image and the detail image to obtain a denoised image and a denoised detail image; and then fusing the denoised image and the denoised detail image to obtain an enhanced image. This technical solution, by first extracting the detail layer components of the original image and then performing denoising and fusing processing on the original image and the detail layer components, can obtain a high-quality enhanced image. Attached Figure Description

[0015] Figure 1 This is a flowchart of the image enhancement method provided in Embodiment 1 of this application;

[0016] Figure 2 This is a schematic diagram of the image enhancement processing procedure provided in Embodiment 1 of this application;

[0017] Figure 3 This is a schematic diagram of the image enhancement process provided in Embodiment 2 of this application;

[0018] Figure 4 This is a schematic diagram of the loss function setting for model training provided in Embodiment 2 of this application;

[0019] Figure 5 This is a schematic diagram of the image enhancement device provided in Embodiment 3 of this application;

[0020] Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this application. Detailed Implementation

[0021] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the application and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present application, not the entire structure.

[0022] Before discussing the exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe the steps as sequential processes, many of these steps can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the steps can be rearranged. The process can be terminated when its operation is complete, but may also have additional steps not included in the figures. The process can correspond to a method, function, procedure, subroutine, subroutine, etc.

[0023] Example 1

[0024] Figure 1 This is a flowchart of the image enhancement method provided in Embodiment 1 of this application. This embodiment can be applied to improving the quality of low signal-to-noise ratio images. The method can be executed by the image enhancement device provided in this embodiment. The device can be implemented by software and / or hardware and can be integrated into devices such as smart terminals used for image enhancement processing.

[0025] like Figure 1 As shown, the image enhancement method includes:

[0026] S110. Extract detail layer components from the original image to obtain a detail image;

[0027] In this scheme, the detail layer components mainly include high-frequency information from the original image, such as edges, noise, and texture. The detail image and the original image have the same dimensions, including the same spatial resolution and the same number of channels.

[0028] In this embodiment, the detail layer components can be directly extracted by convolving the original image with a traditional high-frequency spatial filtering template to obtain a detail image; alternatively, the low-frequency layer components can be obtained by convolving the original image with a traditional low-frequency spatial filtering template, and then the detail layer components can be calculated by subtracting the low-frequency layer components from the original image to obtain a detail image; or the original image can be input into a pre-trained CNN (Convolutional Neural Network) to extract its detail layer components to obtain a detail image, etc.

[0029] For example, for the original image I, a low-pass filtered spatial template F is used to perform a convolution operation on it to obtain the low-frequency layer component L of image I. Then, the low-frequency layer L is subtracted from image I to obtain the detail layer component D:

[0030]

[0031]

[0032] D = IL.

[0033] In this embodiment, extracting detail layer components from the original image avoids the detail loss problem caused by the noise reduction effect of the previous step in the traditional serial processing mode of first denoising and then sharpening. This minimizes the loss of detail layer information.

[0034] S120. The original image and the detail image are denoised to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are then fused to obtain an enhanced image. The denoising and fusion processes are performed using a convolutional neural network model.

[0035] In this scheme, noise reduction and fusion processing can be performed on the original image and detail image based on deep learning technology to obtain an enhanced image. Preferably, the original image and detail image can be input into a pre-trained convolutional neural network model, and the original image and detail image can be denoised and fused based on the convolutional neural network model to obtain an enhanced image.

[0036] In this technical solution, optionally, the original image and the detail image are subjected to noise reduction processing to obtain a denoised image and a denoised detail image, and the denoised image and the denoised detail image are fused to obtain an enhanced image, including:

[0037] The original image and the detail image are used as input, and the original image and the detail image are denoised based on a predetermined denoising model to obtain a denoised image and a denoised detail image.

[0038] The denoised image and the denoised detail image are used as input, and the denoised image and the denoised detail image are fused based on a predetermined fusion model to obtain an enhanced image.

[0039] The denoising model is used to reduce noise in images, while the fusion model is used to sharpen them. The denoising and fusion models are two independent convolutional neural network (CNN) models; their training is simultaneous, i.e., end-to-end training. The CNN models can be pre-trained based on a dataset to obtain a pre-trained CNN model for processing both the original and detailed images.

[0040] In this embodiment, the original image and the detail image are input into a pre-trained denoising model for processing to obtain a denoised image and a denoised detail image. The denoising of the original image and the extraction of high-quality detail layer components are achieved simultaneously through a single denoising model. This avoids the detail loss problem caused by the denoising effect of the previous step in the traditional serial processing mode of denoising first and then sharpening.

[0041] In this approach, after obtaining the denoised image and the denoised detail image, they are input into a pre-trained fusion model for processing to obtain the final enhanced image. Since the denoised image and the detail layer are fused, the resulting enhanced image has both denoising and sharpening (detail enhancement) effects.

[0042] By directly extracting the denoised image and denoised detail image from the original image and its detail image using a denoising model, the loss of detail layer information can be minimized. Then, a fusion model is used to fuse the denoised image and the denoised detail image to obtain the enhanced image. This improves the image enhancement effect and solves the problem of incomplete sharpening detail extraction, which leads to poor enhancement results, commonly found in existing methods.

[0043] In this technical solution, the original image and the detail image are used as input, including:

[0044] The original image and the detail image are stitched together to construct the image to be processed, and the image to be processed is used as input.

[0045] In this approach, the original image and detail image can be stacked along the channel dimension to construct the image to be processed.

[0046] By stitching together the original image and the detail image, it becomes easier to input the noise reduction model for processing.

[0047] Optionally, in this technical solution, the original image and the detail image are stitched together to construct the image to be processed, including:

[0048] The original image and the detail image are stacked in the channel dimension to construct the image to be processed for the noise reduction model;

[0049] Accordingly, the denoised image and the denoised detail image are used as input, including:

[0050] The denoised image and the denoised detail image are stacked in the channel dimension to construct the denoised image to be processed for the fusion model, and the denoised image to be processed is used as input.

[0051] In this embodiment, the original image and the detail image can be stacked along the channel dimension to construct the image to be processed for the denoising model. For example, if the height, width, and number of channels of image A and image B are H, W, and C, respectively, then the height, width, and number of channels of the stitched image are H, W, and 2*C, respectively. The first 1 to C layers along the channel dimension are image A, and the C+1 to 2*C layers are image B; the order of images A and B can be interchanged.

[0052] In this scheme, the original image and the detail image are stacked in the channel dimension to construct the image to be processed for the denoising model. The image to be processed is denoised based on the denoising model to obtain the denoised image and the denoised detail image. The denoised image and the denoised detail image are stacked in the channel dimension to construct the image to be denoised based on the fusion model. The image to be denoised is then fused based on the fusion model to obtain the enhanced image.

[0053] Stacking images along the channel dimension facilitates processing by both the input denoising model and the fusion model.

[0054] For example, Figure 2 This is a schematic diagram of the image enhancement processing procedure provided in Embodiment 1 of this application, as shown below. Figure 2 As shown, the first step is to extract the detail layer component D0 from the original input image I0. The second step is to concatenate the original image I0 and the original detail layer component D0, and then input this concatenation into a pre-trained CNN denoising model M0 to obtain the denoised image I1 and the denoised detail layer D1. The third step is to concatenate the denoised image I1 and the denoised detail layer D1, and then input this concatenation into a pre-trained CNN fusion model M1 to obtain the final enhanced image I2.

[0055] The technical solution provided in this application extracts detail layer components from the original image to obtain a detail image; it then performs denoising processing on the original image and the detail image to obtain a denoised image and a denoised detail image, and finally fuses the denoised image and the denoised detail image to obtain an enhanced image. This technical solution directly obtains the denoised image and the denoised detail image from the original image through a denoising model, thus minimizing the loss of detail layer information. A fusion model then fuses the denoised image and the denoised detail image to obtain the enhanced image. This improves the image enhancement effect and solves the problem of incomplete sharpening detail extraction, which is common in existing solutions and leads to poor enhancement results.

[0056] Example 2

[0057] Figure 3This is a schematic diagram of the image enhancement process provided in Embodiment 2 of this application. Embodiment 2 is a further optimization based on Embodiment 1. Specifically, the optimization includes: the training process of the convolutional neural network model includes: determining a training dataset; wherein the training dataset consists of pairs of noisy images and noise-free images; setting a loss function for the convolutional neural network model, and training the convolutional neural network model based on the training dataset to obtain a trained convolutional neural network model. For details not described in this embodiment, please refer to Embodiment 1. Figure 3 As shown, the method includes the following steps:

[0058] S310. Determine the training dataset; wherein the training dataset consists of pairs of noisy images and noise-free images;

[0059] In this approach, the denoising dataset typically consists of multiple pairs of noisy and noise-free images. The paired noisy and noise-free images maintain the same scene content; the only difference is the presence or absence of noise. The denoising dataset can be either a real noise dataset or a synthetic noise dataset; this paper does not impose any restrictions. Specifically, a real noise dataset refers to noisy images acquired in actual low-light conditions, while a synthetic noise dataset refers to noisy images obtained by superimposing artificially created noise onto noise-free images.

[0060] S320. Set a loss function for the convolutional neural network model and train the convolutional neural network model based on the training dataset to obtain a trained convolutional neural network model.

[0061] In this embodiment, to complete effective supervised training of the denoising model M0 and the fusion model M1, corresponding image labels and loss functions need to be set for the predicted outputs I1, D1, and I2 of the two models, respectively. By combining the training dataset (including input images and image labels), the loss function, the CNN model training optimizer, and other configurations, effective supervised training of the denoising model M0 and the fusion model M1 can be completed. The CNN model training optimizer and other configurations can use conventional settings. For example, the CNN model training optimizer can choose stochastic gradient descent, ADAM algorithm, etc. Some other parameters during model training can be tuned according to the specific training results. For example, the learning rate and the number of training samples per batch can be tuned based on the training performance.

[0062] In this approach, the training of the denoising model and the fusion model can be supervised by setting a loss function. For example, the loss function can be MAE (Mean Absolute Error), MSE (Mean Squared Error), or SSIM (Structural Similarity).

[0063] In this technical solution, optionally, a loss function is set for the convolutional neural network model, including:

[0064] Determine the output image of the convolutional neural network model, and the image label corresponding to the output image;

[0065] A loss function is constructed based on the output image, image label, and preset weights.

[0066] In this scheme, the output image of the model and the image label of the output image are determined, and the output image and image label are combined according to the preset calculation rules to construct the loss function.

[0067] By setting a loss function, the training effect of convolutional neural network models can be improved, resulting in high-quality enhanced images.

[0068] In this technical solution, optionally, the output image includes a predicted denoised image, a predicted denoised detail image, and a predicted enhanced image;

[0069] The image label corresponding to the predicted denoised image is the noise-free image;

[0070] The image label corresponding to the predicted denoised detail image is the target detail image; the target detail image is used to characterize the detail layer components in the noise-free image;

[0071] The image label corresponding to the predicted enhanced image is the sharpened image; the sharpened image is used to characterize the sharpening result of the noise-free image;

[0072] Accordingly, based on the output image, image labels, and preset weights, a loss function is constructed, including:

[0073] According to the preset calculation rules, a first loss term is constructed based on the predicted denoised image and the noise-free image, a second loss term is constructed based on the predicted denoised detail image and the target detail image, and a third loss term is constructed based on the predicted enhanced image and the sharpened image;

[0074] The first loss term, the second loss term, and the third loss term are combined using the weights to construct a loss function.

[0075] In this scheme, the output image of the denoising model includes a predicted denoised image and a predicted denoised detail image; the output image of the fusion model includes a predicted enhanced image. The noise-free image paired with the original image I0 of the denoising model M0 in the denoising training set is set as the image label for the predicted denoised image. The target detail image is obtained from the detail layer component of the noise-free image and used as the image label for the predicted denoised detail image. The noise-free image is then sharpened to obtain a sharpened image, which is used as the image label for the predicted enhanced image.

[0076] For example, Figure 4 This is a schematic diagram of the loss function setting for model training provided in Embodiment 2 of this application, as shown below. Figure 4 As shown, loss functions are set based on the model's output image and the corresponding image labels.

[0077] The loss function is calculated as follows, and includes three loss terms:

[0078] Loss=λ1Loss1+λ2Loss2+λ3Loss3;

[0079] Loss1 = lossfunc(I1, TI1);

[0080] Loss2 = lossfunc(D1, TD1);

[0081] Loss3 = lossfunc(I2, TI2).

[0082] Where I1 and D1 are the predicted denoised image and predicted denoised detail image output by the CNN denoising model M0, respectively, and I2 is the predicted enhanced image output by the CNN fusion model M1. TI1 is the image label for I1, TD1 is the image label for D1, and TI2 is the image label for I2. lossfunc(x, y) represents the calculation of the loss term between the two input images x and y. The coefficients λ1, λ2, and λ3 of the three loss terms need to be tuned according to the training effect. By default, they can be set to λ1 = 1, λ2 = 0.5, and λ3 = 0.5, respectively.

[0083] In this embodiment, lossfunc(x, y) can be calculated using conventional rules. For example, it can be calculated using conventional methods such as MAE (Mean Absolute Error), MSE (Mean Squared Error), or SSIM (Structural Similarity).

[0084] By setting a loss function and image labels, the training performance of convolutional neural network models can be improved, resulting in high-quality enhanced images. This solves the problem common in existing solutions where incomplete extraction of sharpening details leads to poor enhancement results.

[0085] The technical solution provided in this application involves determining a training dataset, setting a loss function for a convolutional neural network model, and training the convolutional neural network model based on the training dataset to obtain a trained convolutional neural network model. This technical solution, by setting a loss function and image labels, trains the convolutional neural network model to obtain high-quality enhanced images.

[0086] Example 3

[0087] Figure 5 This is a schematic diagram of the image enhancement device provided in Embodiment 3 of this application, as shown below. Figure 5 As shown, the image enhancement device includes:

[0088] The detail image acquisition module 510 is used to extract detail layer components from the original image to obtain a detail image;

[0089] The image enhancement module 520 is used to perform noise reduction processing on the original image and the detail image to obtain a noise-reduced image and a noise-reduced detail image, and to perform fusion processing on the noise-reduced image and the noise-reduced detail image to obtain an enhanced image; wherein, the noise reduction processing and fusion processing adopt a convolutional neural network model.

[0090] In this technical solution, optionally, the image enhancement module 520 includes:

[0091] The image acquisition unit is used to take the original image and the detail image as input, and perform noise reduction processing on the original image and the detail image based on a predetermined noise reduction model to obtain a noise-reduced image and a noise-reduced detail image;

[0092] The enhanced image obtaining unit is used to take the denoised image and the denoised detail image as input, and perform fusion processing on the denoised image and the denoised detail image based on a predetermined fusion model to obtain the enhanced image.

[0093] In this technical solution, optionally, the image enhancement unit includes:

[0094] The image to be processed construction subunit is used to stitch together the original image and the detail image to construct the image to be processed, and to use the image to be processed as input.

[0095] In this technical solution, optionally, the image to be processed construction subunit is specifically used for:

[0096] The original image and the detail image are stacked in the channel dimension to construct the image to be processed for the noise reduction model;

[0097] Accordingly, the image enhancement unit includes:

[0098] The denoised image construction subunit is used to stack the denoised image and the denoised detail image in the channel dimension to construct the denoised image of the fusion model, and to use the denoised image as input.

[0099] In this technical solution, optionally, the image enhancement module 520 further includes:

[0100] A training dataset determination unit is used to determine the training dataset; wherein the training dataset consists of pairs of noisy images and noise-free images;

[0101] A convolutional neural network model obtaining unit is used to set a loss function for the convolutional neural network model and train the convolutional neural network model based on the training dataset to obtain a trained convolutional neural network model.

[0102] In this technical solution, optionally, the convolutional neural network model obtains units, including:

[0103] An image label determination subunit is used to determine the output image of the convolutional neural network model and the image label corresponding to the output image;

[0104] The loss function construction subunit is used to construct a loss function based on the output image, image label, and preset weights.

[0105] In this technical solution, optionally, the output image includes a predicted denoised image, a predicted denoised detail image, and a predicted enhanced image;

[0106] The image label corresponding to the predicted denoised image is the noise-free image;

[0107] The image label corresponding to the predicted denoised detail image is the target detail image; the target detail image is used to characterize the detail layer components in the noise-free image;

[0108] The image label corresponding to the predicted enhanced image is the sharpened image; the sharpened image is used to characterize the sharpening result of the noise-free image;

[0109] Correspondingly, the loss function constructs sub-units, specifically used for:

[0110] According to the preset calculation rules, a first loss term is constructed based on the predicted denoised image and the noise-free image, a second loss term is constructed based on the predicted denoised detail image and the target detail image, and a third loss term is constructed based on the predicted enhanced image and the sharpened image;

[0111] The first loss term, the second loss term, and the third loss term are combined using the weights to construct a loss function.

[0112] The above-mentioned products can perform the methods provided in the embodiments of this application, and have the corresponding functional modules and beneficial effects of performing the methods.

[0113] Example 4

[0114] This application also provides a storage medium containing computer-executable instructions, which, when executed by a computer processor, are used to perform an image enhancement method, the method comprising:

[0115] The detail layer components are extracted from the original image to obtain the detail image;

[0116] The original image and the detail image are denoised to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are then fused to obtain an enhanced image. The denoising and fusion processes are performed using a convolutional neural network model.

[0117] Storage medium – any type of memory device or storage device. The term “storage medium” is intended to include: mounting media, such as CD-ROM, floppy disk, or magnetic tape devices; computer system memory or random access memory, such as DRAM, DDR RAM, SRAM, EDO RAM, Rambus RAM, etc.; non-volatile memory, such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. Storage medium may also include other types of memory or combinations thereof. Furthermore, storage medium may reside in a computer system in which a program is executed, or it may reside in a different second computer system connected to the computer system via a network (such as the Internet). The second computer system can provide program instructions to the computer for execution. The term “storage medium” can include two or more storage media that may reside in different locations (e.g., in different computer systems connected via a network). Storage medium may store program instructions (e.g., specifically implemented as a computer program) that can be executed by one or more processors.

[0118] Of course, the computer-executable instructions provided in the embodiments of this application are not limited to the image enhancement operations described above, but can also perform related operations in the image enhancement methods provided in any embodiment of this application.

[0119] Example 5

[0120] This application provides an electronic device that can integrate the image enhancement device provided in this application. Figure 6 This is a schematic diagram of the structure of an electronic device provided in Embodiment 5 of this application. Figure 6 As shown, this embodiment provides an electronic device 600, which includes: one or more processors 620; and a storage device 610 for storing one or more programs. When the one or more programs are executed by the one or more processors 620, the one or more processors 620 implement the image enhancement method provided in this embodiment, the method including:

[0121] The detail layer components are extracted from the original image to obtain the detail image;

[0122] The original image and the detail image are denoised to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are then fused to obtain an enhanced image. The denoising and fusion processes are performed using a convolutional neural network model.

[0123] Of course, those skilled in the art will understand that the processor 620 also implements the technical solutions of the image enhancement methods provided in any embodiment of this application.

[0124] Figure 6 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0125] like Figure 6 As shown, the electronic device 600 includes a processor 620, a storage device 610, an input device 630, and an output device 640; the number of processors 620 in the electronic device can be one or more. Figure 6 Taking a processor 620 as an example; the processor 620, storage device 610, input device 630, and output device 640 in the electronic device can be connected via a bus or other means. Figure 6 Taking the connection between China and Israel via bus 650 as an example.

[0126] Storage device 610, as a computer-readable storage medium, can be used to store software programs, computer-executable programs, and module units, such as the program instructions corresponding to the image enhancement method in the embodiments of this application.

[0127] Storage device 610 may primarily include a program storage area and a data storage area. The program storage area may store the operating system and at least one application program required for a function; the data storage area may store data created based on terminal usage. Furthermore, storage device 610 may include high-speed random access memory and non-volatile memory, such as at least one disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, storage device 610 may further include memory remotely located relative to processor 620, and these remote memories can be connected via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0128] Input device 630 can be used to receive input digital, character, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. Output device 640 may include electronic devices such as a display screen and a speaker.

[0129] The electronic device provided in this application embodiment can achieve the purpose of improving image enhancement effect.

[0130] The image enhancement apparatus, storage medium, and electronic device provided in the above embodiments can execute the image enhancement method provided in any embodiment of this application, and have the corresponding functional modules and beneficial effects for executing the method. Technical details not described in detail in the above embodiments can be found in the image enhancement method provided in any embodiment of this application.

[0131] Note that the above are merely preferred embodiments and the technical principles employed in this application. Those skilled in the art will understand that this application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions can be made without departing from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include many other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the appended claims.

Claims

1. An image enhancement method, characterized in that, include: The detail layer components are extracted from the original image to obtain the detail image; The original image and the detail image are denoised to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are then fused to obtain an enhanced image. The denoising and fusion processes are performed using a convolutional neural network model. The loss function construction process of the convolutional neural network model includes: According to preset calculation rules, a first loss term is constructed based on the predicted denoised image and the noise-free image; a second loss term is constructed based on the predicted denoised detail image and the target detail image; and a third loss term is constructed based on the predicted enhanced image and the sharpened image. The predicted denoised image, the predicted denoised detail image, and the predicted enhanced image are the output images of the convolutional neural network model. The image label corresponding to the predicted denoised image is the noise-free image; the image label corresponding to the predicted denoised detail image is the target detail image; the target detail image is used to characterize the detail layer components in the noise-free image; and the image label corresponding to the predicted enhanced image is the sharpened image; the sharpened image is used to characterize the sharpening result of the noise-free image. The first, second, and third loss terms are combined using weights to construct a loss function; The process involves denoising the original image and the detail image to obtain a denoised image and a denoised detail image, and then fusing the denoised image and the denoised detail image to obtain an enhanced image, including: The original image and the detail image are used as input, and the original image and the detail image are denoised based on a predetermined denoising model to obtain a denoised image and a denoised detail image. The denoised image and the denoised detail image are used as input, and the denoised image and the denoised detail image are fused based on a predetermined fusion model to obtain an enhanced image.

2. The method according to claim 1, characterized in that, The original image and the detail image are used as input, including: The original image and the detail image are stitched together to construct the image to be processed, and the image to be processed is used as input.

3. The method according to claim 2, characterized in that, The original image and the detail image are stitched together to construct the image to be processed, including: The original image and the detail image are stacked in the channel dimension to construct the image to be processed for the noise reduction model; Accordingly, the denoised image and the denoised detail image are used as input, including: The denoised image and the denoised detail image are stacked in the channel dimension to construct the denoised image to be processed for the fusion model, and the denoised image to be processed is used as input.

4. The method according to claim 1, characterized in that, The training process of the convolutional neural network model includes: Determine the training dataset; wherein the training dataset consists of pairs of noisy images and noise-free images; A loss function is set for the convolutional neural network model, and the convolutional neural network model is trained based on the training dataset to obtain a trained convolutional neural network model.

5. The method according to claim 4, characterized in that, Setting a loss function for the convolutional neural network model includes: Determine the output image of the convolutional neural network model, and the image label corresponding to the output image; A loss function is constructed based on the output image, image label, and preset weights.

6. An image enhancement device, characterized in that, include: The detail image acquisition module is used to extract detail layer components from the original image to obtain a detail image; The image enhancement module is used to perform noise reduction processing on the original image and the detail image to obtain a noise-reduced image and a noise-reduced detail image, and to fuse the noise-reduced image and the noise-reduced detail image to obtain an enhanced image; wherein, the noise reduction processing and the fusion processing adopt a convolutional neural network model; The enhanced image acquisition module includes a convolutional neural network model acquisition unit; the convolutional neural network model acquisition unit includes a loss function construction subunit; the loss function construction subunit is specifically used for: According to preset calculation rules, a first loss term is constructed based on the predicted denoised image and the noise-free image; a second loss term is constructed based on the predicted denoised detail image and the target detail image; and a third loss term is constructed based on the predicted enhanced image and the sharpened image. The predicted denoised image, the predicted denoised detail image, and the predicted enhanced image are the output images of the convolutional neural network model. The image label corresponding to the predicted denoised image is the noise-free image; the image label corresponding to the predicted denoised detail image is the target detail image; the target detail image is used to characterize the detail layer components in the noise-free image; and the image label corresponding to the predicted enhanced image is the sharpened image; the sharpened image is used to characterize the sharpening result of the noise-free image. The first, second, and third loss terms are combined using weights to construct a loss function; The image enhancement module includes: The image acquisition unit is used to take the original image and the detail image as input, and perform noise reduction processing on the original image and the detail image based on a predetermined noise reduction model to obtain a noise-reduced image and a noise-reduced detail image; The enhanced image obtaining unit is used to take the denoised image and the denoised detail image as input, and perform fusion processing on the denoised image and the denoised detail image based on a predetermined fusion model to obtain the enhanced image.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the image enhancement method as described in any one of claims 1-5.

8. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the image enhancement method as described in any one of claims 1-5.

Images