Image enhancement methods, systems, devices and storage media

Abstract

This invention provides an image enhancement method, system, device, and storage medium, belonging to the field of image processing. The method includes: acquiring an image to be processed; obtaining a trained first image enhancement model, the first image enhancement model including a first network branch, a second network branch, and an output module; inputting the image to be processed into the first image enhancement model, performing local enhancement processing on the image to be processed through the first network branch to obtain local enhancement information, performing global enhancement processing on the image to be processed through the second network branch to obtain global enhancement information, and performing enhancement processing on the image to be processed based on the local enhancement information and the global enhancement information through the output module; and obtaining the output image of the first image enhancement model as the enhanced image. By employing this invention, the enhancement effect on low-light images is improved, which helps to reduce hardware resource consumption.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to an image enhancement method, system, device, and storage medium. Background Technology

[0002] In low-light conditions, image quality is severely affected. On the one hand, the captured image is unclear and noisy, resulting in a low signal-to-noise ratio. On the other hand, due to the large dynamic range of the scene in low light, the brightness enhancement achieved by traditional ISP (Image Signal Processing) is limited, resulting in an overall dark image. There are three main methods to improve image quality: (1) improve hardware; (2) extend exposure time; and (3) use image enhancement algorithms. The first two methods place higher demands on hardware and require more hardware resources. However, existing image enhancement algorithms often directly perform global enhancement processing on the image, failing to consider local pixel information in detail, which easily leads to poor local enhancement effects. Therefore, existing methods still cannot achieve good low-light image enhancement effects. Summary of the Invention

[0003] To address the problems in the prior art, the present invention aims to provide an image enhancement method, system, device, and storage medium that improves the enhancement effect on low-light images by combining local enhancement and global enhancement, and helps reduce hardware resource consumption by providing an end-to-end first image enhancement model.

[0004] This invention provides an image enhancement method, comprising the following steps:

[0005] Acquire low-light images to be processed;

[0006] Obtain a trained first image enhancement model, which includes a first network branch, a second network branch, and an output module;

[0007] The low-light image to be processed is input into the first image enhancement model. The first network branch performs local enhancement processing on the low-light image to be processed to obtain local enhancement information. The second network branch performs global enhancement processing on the low-light image to be processed to obtain global enhancement information. The output module enhances the low-light image to be processed based on the local enhancement information and the global enhancement information.

[0008] Obtain the output image of the first image enhancement model as the enhanced image.

[0009] In the method of this application, after acquiring the low-light image to be processed and the trained first image training model, the low-light image to be processed is input into the first image enhancement model, and the enhanced image output by the first image enhancement model is obtained. Since the first image enhancement model divides the image enhancement processing into local image processing and global image processing, and implements them by two network branches respectively, it enhances the image at both the local pixel level and the global image level, which is beneficial to improving the image enhancement effect. Furthermore, the model no longer depends on the camera's image signal processing algorithm, and the model can be extended to any chip platform while unifying image quality and style. This model is an end-to-end machine learning network, which is more conducive to board-side portability and helps reduce hardware resource consumption.

[0010] In some embodiments, the local enhancement process includes at least one of demosaicing, noise reduction, sharpening, and tone mapping; and / or,

[0011] The global enhancement process includes color correction and / or gamma correction.

[0012] In some embodiments, the first network branch includes at least two sub-branches, wherein the first sub-branch is used to perform local enhancement processing on the low-light image to be processed, and the i-th sub-branch is used to perform k-level enhancement processing on the low-light image to be processed. i After downsampling, local enhancement processing is performed, i∈[2,n], where n is the total number of sub-branches, n≥2,k i It is an integer greater than 1.

[0013] In some embodiments, the second network branch is used to perform global enhancement processing on the low-light image to be processed based on a self-attention mechanism.

[0014] In some embodiments, the low-light image to be processed is a RAW image, and the enhanced image is an RGB image.

[0015] In some embodiments, before obtaining the trained first image enhancement model, the method further includes training the first image enhancement model using the following steps:

[0016] Collect multiple low-light first sample images;

[0017] Obtain the augmented baseline images corresponding to multiple first sample images;

[0018] The first sample image is input into the first image enhancement model. The first sample image is locally enhanced through the first network branch to obtain local enhancement information. The first sample image is globally enhanced through the second network branch to obtain global enhancement information. The output module enhances the first sample image based on the local enhancement information and the global enhancement information to obtain the output image.

[0019] A loss function is constructed based on the output image of the first image enhancement model and the reference image, and the network parameters of the first image enhancement model are iteratively optimized based on the loss function.

[0020] In some embodiments, obtaining the enhanced reference image corresponding to the multiple first sample images includes the following steps:

[0021] Obtain the trained second image augmentation model;

[0022] Each of the first sample images is input into the second image enhancement model to obtain the first processed image corresponding to each of the first sample images;

[0023] Obtain a second processed image after processing each of the first sample images using a camera image signal processing algorithm;

[0024] The first processed image and the second processed image are fused together to obtain the image-enhanced reference image corresponding to each of the first sample images.

[0025] In some embodiments, before obtaining the trained second image enhancement model, the method further includes training the second image enhancement model using the following steps:

[0026] Multiple second sample images and a corresponding reference image are acquired, wherein the exposure time of the reference image is longer than that of the corresponding second sample image;

[0027] The reference image is processed using a camera image signal processing algorithm to obtain a reference label image;

[0028] The second sample image is input into the second image enhancement model to obtain the enhanced sample image;

[0029] A loss function is constructed based on the enhanced sample image and the reference label image, and the network parameters of the second image enhancement model are iteratively optimized based on the loss function.

[0030] This invention also provides an image enhancement system applied to the image enhancement method, the system comprising:

[0031] The image acquisition module is used to acquire low-light images to be processed.

[0032] The model acquisition module is used to acquire a trained first image enhancement model, which includes a first network branch, a second network branch, and an output module.

[0033] The image processing module is configured to input the low-light image to be processed into the first image enhancement model, perform local enhancement processing on the low-light image to be processed through the first network branch to obtain local enhancement information, perform global enhancement processing on the low-light image to be processed through the second network branch to obtain global enhancement information, and perform enhancement processing on the low-light image to be processed based on the local enhancement information and the global enhancement information through the output module; and obtain the output image of the first image enhancement model as the enhanced image.

[0034] This application acquires the low-light image to be processed and the trained first image model using the aforementioned image acquisition module and model acquisition module, respectively. Then, the image processing module inputs the low-light image to be processed into the first image enhancement model and obtains the enhanced image output by the first image enhancement model. Since the first image enhancement model divides image enhancement processing into local image processing and global image processing, each implemented by a separate network branch, and enhances the image at both the local pixel level and the global image level, it improves the image enhancement effect. Furthermore, the model no longer relies on the camera's image signal processing algorithm, allowing it to be extended to any chip platform while maintaining consistent image quality and style. This end-to-end machine learning network is more conducive to board-side portability and reduces hardware resource consumption.

[0035] This invention also provides an image enhancement device, comprising:

[0036] processor;

[0037] A memory in which executable instructions of the processor are stored;

[0038] The processor is configured to perform the steps of the image enhancement method by executing the executable instructions.

[0039] By employing the image enhancement device provided by the present invention, the processor executes the image enhancement method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned image enhancement method.

[0040] This invention also provides a computer-readable storage medium for storing a program that, when executed by a processor, implements the steps of the image enhancement method.

[0041] By employing the computer-readable storage medium provided by the present invention, wherein the stored program, when executed, implements the steps of the image enhancement method described above, the beneficial effects of the image enhancement method described above can be obtained. Attached Figure Description

[0042] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings.

[0043] Figure 1 This is a flowchart of an image enhancement method according to an embodiment of the present invention;

[0044] Figure 2 This is a schematic diagram of the structure of a first image enhancement model according to an embodiment of the present invention;

[0045] Figure 3 This is a flowchart of the training process of a first image enhancement model according to an embodiment of the present invention;

[0046] Figure 4 This is a schematic diagram of an image enhancement system according to an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the structure of an image enhancement device according to an embodiment of the present invention;

[0048] Figure 6 This is a schematic diagram of the structure of a computer storage medium according to an embodiment of the present invention. Detailed Implementation

[0049] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the embodiments set forth herein; rather, they are provided so that the invention will be thorough and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The same reference numerals in the drawings denote the same or similar structures, and therefore repeated descriptions of them will be omitted.

[0050] like Figure 1 As shown, in one embodiment, the present invention provides an image enhancement method, including the following steps S210 to S240. The image enhancement method is used to enhance an acquired original low-light RAW image to obtain an enhanced RGB image with brightening, noise reduction, and color correction. Steps S210 to S240 are described in detail below.

[0051] S210: Acquire low-light images to be processed.

[0052] The low-light image to be processed here is the original RAW format image, such as a low-light raw image acquired from a surveillance system that has not yet been processed by an image signal processing algorithm. The low-light image referred to in this application refers to a dark, indistinct image where the brightness of the subject is so low that the video signal level output by the camera is too low, resulting in images that are difficult to distinguish in terms of detail, such as surveillance images at night without lights. When distinguishing low-light images, the brightness value of the image can be judged. If the proportion of pixels with brightness values ​​below a first threshold is greater than a second threshold, the image is considered a low-light image. The values ​​of the first and second thresholds can be set as needed.

[0053] S220: Obtain the trained first image enhancement model, which includes a first network branch, a second network branch, and an output module.

[0054] S230: Input the low-light image to be processed into the first image enhancement model, perform local enhancement processing on the low-light image to be processed through the first network branch to obtain local enhancement information, perform global enhancement processing on the low-light image to be processed through the second network branch to obtain global enhancement information, and perform enhancement processing on the low-light image to be processed based on the local enhancement information and global enhancement information through the output module.

[0055] In this embodiment, the first image enhancement model can be a model built based on a deep learning network. Local enhancement processing includes at least one of demosaic, noise reduction, sharpening, and tone mapping, and the corresponding local enhancement information is at least one of the demosaic parameters, noise reduction coefficients, sharpening coefficients, and tone mapping parameters corresponding to the image size. Global enhancement processing includes color correction and / or gamma correction. Color correction includes white balance processing and color conversion processing, and the corresponding global enhancement information is at least one of the white balance coefficients, color matrix, and gamma values.

[0056] S240: Obtain the output image of the first image enhancement model as the enhanced image.

[0057] Through the aforementioned enhancement processes of depixelation, noise reduction, toning, tone mapping, color correction, and / or gamma correction, the enhanced image here is an RGB image that has undergone enhancements such as brightening, noise reduction, and color correction.

[0058] This application, after acquiring the low-light image to be processed and the trained first image training model through steps S210 and S220 respectively, inputs the low-light image to be processed into the first image enhancement model in step S230, and obtains the enhanced image output by the first image enhancement model in step S240. Since the first image enhancement model divides image enhancement processing into local image processing and global image processing, each implemented by a separate network branch, and enhances the image at both the local pixel level and the global image level, it improves the image enhancement effect. Furthermore, the model no longer relies on the camera's image signal processing algorithm, allowing it to be extended to any chip platform while maintaining consistent image quality and style. This end-to-end machine learning network is more conducive to board-side portability and reduces hardware resource consumption. This image enhancement method can be applied to electronic police and checkpoint scenarios in surveillance systems to enhance acquired low-light images through brightening, noise reduction, and color correction. However, this invention is not limited to this; the image enhancement method can also be applied to other scenarios to process images of various low light conditions.

[0059] like Figure 2 The diagram shows the network structure of a first image enhancement model in a specific embodiment. In this embodiment, the first image enhancement model includes a first network branch W110, a second network branch W120, and an output module W200. The first network branch W110 is responsible for predicting pixel-level information of the image, such as texture, detail, and tone. The second network branch W120 is responsible for predicting color parameters and gamma values ​​with global image information. The output module W200 uses a multi-layer convolutional network to aggregate the aforementioned local and global enhancement information to process the low-light image to obtain the final enhanced image. Unlike traditional image signal processing algorithms, the white balance coefficient, color matrix, and gamma value in this embodiment can be adaptively adjusted according to each image to complete the enhancement task.

[0060] In this embodiment, the first network branch includes at least two sub-branches, which perform local enhancement processing on the low-light image to be processed at different scales. Specifically, the first sub-branch performs local enhancement processing on the low-light image to be processed, and the i-th sub-branch performs k-scale enhancement processing on the low-light image to be processed. i After downsampling, local enhancement processing is performed, i∈[2,n], where n is the total number of sub-branches, n≥2,k i k is an integer greater than 1. i The specific values ​​can be set according to requirements. The local enhancement information obtained by performing local enhancement processing on the i-th sub-branch is then processed by k... iAfter upsampling, the local enhancement information is fused with that of the first sub-branch. Since the i-th sub-branch performs local enhancement processing on the downsampled low-light image, the processing workload is reduced, which is beneficial for the lightweighting of the first image enhancement model. For example... Figure 2 As shown, in this embodiment, the first network branch includes three parallel sub-branches W111, W112, and W113. The first sub-branch W111 performs local enhancement processing on the original input low-light image to be processed. The second sub-branch W112 performs local enhancement processing on the low-light image to be processed after downsampling by a factor of 2, and then upsamples the local enhancement information by a factor of 2 before outputting it. The third sub-branch W113 performs local enhancement processing on the low-light image to be processed after downsampling by a factor of 4, and then upsamples the local enhancement information by a factor of 4 before outputting it. The computational complexity of the three sub-branches W111, W112, and W113 is kept as equal as possible to utilize parallelization. Each sub-branch can be implemented using a residual group, but this application is not limited to this.

[0061] In one embodiment, the second network branch is used to perform global enhancement processing on the low-light image to be processed based on a self-attention mechanism, that is, to capture global enhancement information through a self-attention mechanism. The self-attention mechanism reduces dependence on external information and is better at capturing the internal correlations of data or features. Furthermore, in this embodiment, the second network branch may also include three sub-branches, used for white balance processing, color correction, and gamma correction, respectively, to obtain white balance coefficients, color matrices, and gamma matrices. Each sub-branch can use a self-attention mechanism to predict the corresponding parameters. By employing a self-attention mechanism, the dependence on external information during the global enhancement information capture process can be reduced, allowing for greater focus on the global enhancement of the image itself, which is beneficial for improving the overall image enhancement effect.

[0062] like Figure 3 As shown, in one embodiment, before obtaining the trained first image enhancement model, the method further includes training the first image enhancement model using the following steps:

[0063] S110: Collect multiple low-light first sample images.

[0064] The first sample image here can be multiple RAW low-light images with different exposure lengths.

[0065] S120: Obtain the base image after image enhancement corresponding to multiple first sample images.

[0066] The reference image is an RGB image that has been brightened and denoised. Specifically, the first sample image can be processed by another trained second image enhancement model to obtain the enhanced reference image. The second image enhancement model will be described in detail below. Alternatively, the first sample image in RAW format can be converted to RGB format by the camera device hardware, and then the first sample image can be enhanced by a software image enhancement algorithm to obtain the reference image.

[0067] S130: Input the first sample image into the first image enhancement model, perform local enhancement processing on the first sample image through the first network branch to obtain local enhancement information, perform global enhancement processing on the first sample image through the second network branch to obtain global enhancement information, and perform enhancement processing on the first sample image based on the local enhancement information and global enhancement information through the output module to obtain the output image;

[0068] S140: Construct a loss function based on the output image and the reference image of the first image enhancement model, and iteratively optimize the network parameters of the first network branch and the second network branch in the first image enhancement model based on the loss function.

[0069] Specifically, the output image is used as the predicted value, and the reference image is used as the label value. The loss between the predicted value and the label value is calculated based on the constructed loss function. Training is complete when the loss value is less than a preset loss threshold. In this embodiment, the loss function can be a weighted sum of L1 loss, VGG loss, and SSIM loss functions. The L1 loss function compares differences pixel-by-pixel and then takes the absolute value. The VGG loss function is a perceptual loss, which compares the features extracted by convolution of the reference image with the features extracted by convolution of the output image after processing the first sample image using the first image enhancement model, making the high-level information similar, i.e., perceptual. The SSIM (structural similarity) loss function considers brightness, contrast, and structural indicators, providing more detail than the L1 loss function. By combining multiple loss functions, various aspects of the model can be optimized during training. In other embodiments, other types of loss functions can be used, such as a combination of only L1 and VGG loss functions, or a softmax loss function.

[0070] Therefore, in this embodiment, the first image enhancement model can be directly learned end-to-end using the first sample image and the reference image, making the training process simpler. Furthermore, the trained first image enhancement model itself is an end-to-end model; that is, after inputting the low-light image to be processed, it directly obtains the enhanced image that meets the requirements of global and local enhancement, without needing multiple processing steps, thus improving the algorithm's processing efficiency and reducing hardware resources. Further, in scenarios where the enhanced RGB image is obtained from a RAW format low-light image, the trained first image enhancement model can directly enhance the RAW format low-light image end-to-end to obtain an RGB image with better display effects; that is, it directly obtains the RGB enhanced image from the RAW low-light image, without first obtaining the RGB low-light image from the RAW low-light image and then obtaining the RGB enhanced image. This improves the processing efficiency of the image enhancement algorithm, effectively enhances the brightness, color effect, and clarity of dark areas, and maintains the sensory realism and naturalness.

[0071] In one specific embodiment, step S120: obtaining the image-enhanced reference image corresponding to multiple first sample images includes the following steps:

[0072] Obtain the trained second image augmentation model;

[0073] Each of the first sample images is input into the second image enhancement model to obtain the first processed image corresponding to each of the first sample images;

[0074] Obtain a second processed image after processing each of the first sample images using a camera image signal processing algorithm;

[0075] Using multi-exposure fusion technology, the first processed image and the second processed image are fused to obtain an enhanced reference image corresponding to each of the first sample images.

[0076] Specifically, the second image enhancement model described above can also be constructed based on a deep learning network architecture. In this embodiment, the second image enhancement model can be constructed using a Unet structure, but the present invention is not limited to this. In other embodiments, the second image enhancement model can be constructed using other types of networks, such as residual networks. The first sample image is a low-light RAW image. The first sample image is input into the second image enhancement model, which brightens and denoises the first sample image to obtain a first processed image. The first processed image is a first RGB image that is brightened and denoised based on the low-light RAW image. The first sample image is processed using the ISP processing algorithm to obtain a second RGB image that has undergone color correction, brightening, and denoising. The first RGB image and the second RGB image are fused using multi-exposure fusion technology to obtain the final RGB image as the reference image. This reference image is a fused RGB image with better brightness, better detail, higher dynamic range, and better color. In this embodiment, the fusion of the first RGB image and the second RGB image to obtain the final RGB image is used as the reference image. This is beneficial to improving the optimization effect of the model parameters during the training process of the first image enhancement model, so that the final first image enhancement model has better image optimization capabilities and can combine the advantages of both the camera image signal processing algorithm and the second image enhancement model.

[0077] In one embodiment, before obtaining the trained second image enhancement model, the method further includes training the second image enhancement model using the following steps:

[0078] Multiple second sample images and a corresponding reference image are acquired. The reference image has a longer exposure time than the corresponding second sample image, i.e., the second sample image is a short exposure image and the reference image is a long exposure image.

[0079] The reference image is processed using a camera image signal processing algorithm to obtain a reference label image, that is, the reference image is processed using the ISP algorithm to obtain a reference label image in RGB format;

[0080] The second sample image is input into the second image enhancement model to obtain the enhanced sample image;

[0081] A loss function is constructed based on the enhanced sample image and the reference label image. The network parameters of the second image enhancement model are iteratively optimized based on this loss function. Training continues until the loss function is less than a preset loss threshold, at which point the training of the second image enhancement model is considered complete. In this embodiment, using a reference image with a longer exposure time to train the second image enhancement model is beneficial for obtaining a first processed image with better image enhancement effects when acquiring the aforementioned baseline image. Furthermore, the reference image can be obtained simply by increasing the exposure time, making the method simple and practical.

[0082] like Figure 4 As shown, this embodiment of the invention also provides an image enhancement system applied to the image enhancement method described above, the system comprising:

[0083] The image acquisition module M100 is used to acquire low-light images to be processed.

[0084] The model acquisition module M200 is used to acquire a trained first image enhancement model, which includes a first network branch, a second network branch, and an output module.

[0085] The image processing module M300 is used to input the low-light image to be processed into the first image enhancement model, perform local enhancement processing on the low-light image to be processed through the first network branch to obtain local enhancement information, perform global enhancement processing on the low-light image to be processed through the second network branch to obtain global enhancement information, and perform enhancement processing on the low-light image to be processed based on the local enhancement information and global enhancement information through the output module; and obtain the output image of the first image enhancement model as the enhanced image.

[0086] This application acquires the low-light image to be processed and the trained first image training model using the image acquisition module M100 and the model acquisition module M200, respectively. Then, the image processing module M300 inputs the low-light image to be processed into the first image enhancement model and obtains the enhanced image output by the first image enhancement model. Since the first image enhancement model divides image enhancement processing into local image processing and global image processing, each implemented by a separate network branch, and enhances the image at both the local pixel level and the global image level, it improves the image enhancement effect. Furthermore, the model no longer relies on the camera's image signal processing algorithm, allowing it to be extended to any chip platform while maintaining consistent image quality and style. This end-to-end machine learning network is more conducive to board-side portability and reduces hardware resource consumption.

[0087] In this embodiment, the functions of each module of the image enhancement system can be implemented using the specific implementation methods of each step in the above-described image enhancement method. For example, the function of the image acquisition module M100 can be implemented using the specific implementation method of step S210, the function of the model acquisition module M200 can be implemented using the specific implementation method of step S220, and the function of the image processing module M300 can be implemented using the specific implementation methods of steps S230 and S240, which will not be elaborated here.

[0088] In this embodiment, the image enhancement system may further include a model training module for training the first image enhancement model. Specifically, the model training module may use the steps S110 to S140 described above to train the first image enhancement model.

[0089] This invention also provides an image enhancement device, including a processor; a memory storing executable instructions of the processor; wherein the processor is configured to perform the steps of the image enhancement method by executing the executable instructions.

[0090] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0091] The following reference Figure 5 To describe an electronic device 600 according to this embodiment of the present invention. Figure 5 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0092] like Figure 5 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0093] The storage unit stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the above-described section on the electronic prescription circulation processing method according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 1 The steps are shown in the figure.

[0094] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache storage unit 6202, and may further include a read-only memory unit (ROM) 6203.

[0095] The storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0096] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0097] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0098] By employing the image enhancement device provided by the present invention, the processor executes the image enhancement method when executing the executable instructions, thereby obtaining the beneficial effects of the above-mentioned image enhancement method.

[0099] This invention also provides a computer-readable storage medium for storing a program that, when executed by a processor, implements the steps of the image enhancement method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code that, when run on a terminal device, causes the terminal device to perform the steps described in the above-described electronic prescription transfer processing method section of this specification according to various exemplary embodiments of the invention.

[0100] refer to Figure 6 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0101] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0102] The computer-readable storage medium may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The readable storage medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0103] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or cluster. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or they can be connected to external computing devices (e.g., via the Internet using an Internet service provider).

[0104] By employing the computer-readable storage medium provided by the present invention, wherein the stored program, when executed, implements the steps of the image enhancement method described above, the beneficial effects of the image enhancement method described above can be obtained.

[0105] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of the present invention. It should not be construed that the specific implementation of the present invention is limited to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of the present invention, and all such modifications and substitutions should be considered within the scope of protection of the present invention.

Claims

1. An image enhancement method, characterized in that, Includes the following steps: Acquire low-light images to be processed; Obtain a trained first image enhancement model, which includes a first network branch, a second network branch, and an output module; the first network branch includes at least two parallel sub-branches. The low-light image to be processed is input into the first image enhancement model. Local enhancement processing at different scales is performed on the low-light image to be processed through each sub-branch of the first network branch. Local enhancement information is obtained based on the output of the at least two parallel sub-branches. Global enhancement processing is performed on the low-light image to be processed through the second network branch. The global enhancement processing includes color correction and gamma correction to obtain global enhancement information. The output module uses a multi-layer convolutional network to enhance the low-light image to be processed based on the local enhancement information and the global enhancement information. Obtain the output image of the first image enhancement model as the enhanced image; The first image enhancement model is trained using a low-light first sample image and a corresponding enhanced baseline image. The baseline image is obtained through the following steps: A trained second image enhancement model is obtained. The second image enhancement model is trained based on multiple second sample images and corresponding reference label images. The reference label images are obtained by processing the reference images corresponding to the second sample images using a camera image signal processing algorithm. The exposure time of the reference images is longer than that of the corresponding second sample images. Each of the first sample images is input into the second image enhancement model to obtain the first processed image corresponding to each of the first sample images; Obtain a second processed image after processing each of the first sample images using a camera image signal processing algorithm; The first processed image and the second processed image are fused together to obtain the image-enhanced reference image corresponding to each of the first sample images.

2. The image enhancement method according to claim 1, characterized in that, The local enhancement process includes at least one of depigmentation, noise reduction, sharpening, and tone mapping.

3. The image enhancement method according to claim 1, characterized in that, The first sub-branch of the first network branch is used to perform local enhancement processing on the low-light image to be processed, and the i-th sub-branch is used to perform k-level enhancement processing on the low-light image to be processed. i After downsampling, local enhancement processing is performed, i∈[2,n], where n is the total number of sub-branches, n≥2,k i It is an integer greater than 1.

4. The image enhancement method according to claim 1, characterized in that, The second network branch is used to perform global enhancement processing on the low-light image to be processed based on a self-attention mechanism.

5. The image enhancement method according to claim 1, characterized in that, The low-light image to be processed is a RAW image, and the enhanced image is an RGB image.

6. The image enhancement method according to claim 1, characterized in that, Before obtaining the trained first image enhancement model, the method further includes training the first image enhancement model using the following steps: Collect multiple low-light first sample images; Obtain the augmented baseline images corresponding to multiple first sample images; The first sample image is input into the first image enhancement model. The first network branch performs local enhancement processing on the first sample image to obtain local enhancement information. The second network branch performs global enhancement processing on the first sample image to obtain global enhancement information. The output module enhances the first sample image based on the local enhancement information and the global enhancement information to obtain the output image. A loss function is constructed based on the output image of the first image enhancement model and the reference image, and the network parameters of the first image enhancement model are iteratively optimized based on the loss function.

7. The image enhancement method according to claim 6, characterized in that, Before obtaining the trained second image enhancement model, the second image enhancement model is trained using the following steps: Collect multiple second sample images and the corresponding reference images of the second sample images; The reference image is processed using a camera image signal processing algorithm to obtain the reference label image; The second sample image is input into the second image enhancement model to obtain the enhanced sample image; A loss function is constructed based on the enhanced sample image and the reference label image, and the network parameters of the second image enhancement model are iteratively optimized based on the loss function.

8. An image enhancement system, characterized in that, The system, applied to the image enhancement method according to any one of claims 1 to 7, comprises: The image acquisition module is used to acquire low-light images to be processed. The model acquisition module is used to acquire a trained first image enhancement model, which includes a first network branch, a second network branch, and an output module. The image processing module is configured to input the low-light image to be processed into the first image enhancement model, perform local enhancement processing on the low-light image to be processed through the first network branch to obtain local enhancement information, perform global enhancement processing on the low-light image to be processed through the second network branch to obtain global enhancement information, and perform enhancement processing on the low-light image to be processed based on the local enhancement information and the global enhancement information through the output module; and obtain the output image of the first image enhancement model as the enhanced image.

9. An image enhancement device, characterized in that, include: processor; A memory in which executable instructions of the processor are stored; The processor is configured to perform the steps of the image enhancement method according to any one of claims 1 to 7 by executing the executable instructions.

10. A computer-readable storage medium for storing a program, characterized in that, When the program is executed by the processor, it implements the steps of the image enhancement method according to any one of claims 1 to 7.

Images