An ultrahigh-resolution image exposure error correction method and system

Abstract

The application belongs to the field of image processing, and provides a kind of super high resolution image exposure error correction method and system, including separating high and low frequency information in current image needing exposure correction, decomposed into multiple high frequency information layers and single low frequency information layer;Low frequency information layer is down-sampled to low resolution, and a kind of bilateral grid network similar to affine transformation is calculated and obtained, and the low frequency information layer after correction is obtained through the bilateral grid similar to it;Multiple high frequency information layers are corrected using shallow network, to obtain the high frequency information layer after correction;The high frequency information layer after correction and the low frequency information layer after correction are reconstructed into the picture after exposure correction.The application solves the real-time and effectiveness problems of super high resolution image exposure correction task by applying a kind of lossless reversible image content decoupling method and new multi-layer residual extraction correction transformation prediction module.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method and system for correcting exposure errors in ultra-high resolution images. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] Image exposure errors refer to underexposed or overexposed areas in a photographic image due to poor lighting conditions or inappropriate shutter speed, aperture, and ISO settings. Traditional methods for correcting image exposure errors, as well as most deep learning-based methods, require enormous computational resources and several seconds of processing time for an image with tens of millions of pixels, making them difficult to deploy on today's mobile devices.

[0004] Existing methods for accelerating image exposure correction are all based on a similar framework: downsampling the image, predicting a transformation for exposure correction, and then applying this transformation to the original-size image. These methods avoid the high computational cost of correcting exposure errors at the original image size or predicting the correction transformation, effectively accelerating the image exposure correction process. However, these methods still have significant limitations.

[0005] First, the spatial adaptability of exposure correction transforms is limited; applying them to the original image after high-magnification upsampling produces severe artifacts. This limits the speedup effect of existing algorithms; for images with tens of millions of pixels, to balance efficiency and performance, a large amount of computation is still needed at higher resolutions to predict a suitable exposure correction transform. Second, the transform needs to be simple and capable of high-speed parallel implementation, allowing for high-speed processing of original-size images. However, some existing methods use exposure correction transforms with high time complexity, making them unsuitable for processing ultra-high-resolution images.

[0006] For the reasons mentioned above, although existing methods for accelerating image exposure correction have some effect, their processing speed and performance on ultra-high resolution images are limited. Summary of the Invention

[0007] To address the aforementioned issues, this invention proposes a method and system for correcting exposure errors in ultra-high resolution images. By applying a lossless and reversible image content decoupling method and a novel multi-layer residual extraction, correction, transformation, and prediction module, this invention designs a novel lightweight image exposure correction network to achieve efficient correction of 4K images at speeds exceeding 100fps, thus solving the real-time and effectiveness problems of ultra-high resolution image exposure correction tasks.

[0008] According to some embodiments, the first aspect of the present invention provides a method for correcting exposure errors in ultra-high resolution images, employing the following technical solution:

[0009] A method for correcting exposure errors in ultra-high resolution images includes:

[0010] The high and low frequency information in the image that needs exposure correction is separated and decomposed into multiple high frequency information layers and a single low frequency information layer.

[0011] The low-frequency information layer is downsampled to a low resolution, and a quasi-bilateral mesh network storing affine transformations is calculated and obtained. The corrected low-frequency information layer is obtained through the quasi-bilateral mesh.

[0012] Multiple high-frequency information layers are corrected using shallow networks to obtain the corrected high-frequency information layers;

[0013] The corrected high-frequency information layer and the corrected low-frequency information layer are reconstructed into an exposure-corrected image.

[0014] Furthermore, the step of separating high- and low-frequency information in the image currently requiring exposure correction, decomposing it into multiple high-frequency information layers and a single low-frequency information layer, includes:

[0015] The input image is decoupled using a Laplacian pyramid decomposition network, separating it into several high-frequency information layers with high to low resolution, which mainly contain contour and texture information, and a single low-frequency information layer containing illumination, color, and content.

[0016] The method of decoupling the input image using a Laplacian pyramid-like decomposition network includes:

[0017] The image is blurred and downsampled multiple times to obtain its Gaussian pyramid sequence;

[0018] The final result of the Gaussian pyramid sequence is blurred and upsampled multiple times, and then subtracted from each layer of the Gaussian pyramid to obtain the final Laplace pyramid.

[0019] Further, the step of downsampling the low-frequency information layer to a low resolution, calculating and obtaining a quasi-bilateral mesh network that stores affine transformations, and obtaining the corrected low-frequency information layer through the quasi-bilateral mesh includes:

[0020] The low-frequency information layer is input into the self-modulation feature extraction module in the double-sided mesh to obtain the guidance map features;

[0021] The low-frequency information layer is downsampled, and the downsampling result is input into the hierarchical feature decomposition module in the double-sided grid to obtain the predicted double-sided grid coefficients;

[0022] The coefficients of the bilateral grid are obtained by slicing the coefficients of the bilateral grid with the features of the guide map.

[0023] Based on the bilateral grid coefficients after slicing, a quasi-bilateral grid network is obtained. Using the quasi-bilateral grid network, the affine transformation is applied pixel by pixel to the low-frequency information layer to obtain the corrected low-frequency information layer.

[0024] Furthermore, the hierarchical feature decomposition module has three layers, each consisting of a context-aware feature extraction module, a self-modulation feature extraction module, a 1*1 convolutional layer, and a ReLU activation function layer;

[0025] After the first-layer context-aware feature extraction module performs decomposition processing, it obtains context-aware features and residual features. The context features are extracted by passing through a 1*1 convolutional layer and a ReLU activation function layer. The residual features are extracted by passing through a self-modulation feature extraction module and then to the next layer's context-aware feature extraction module, where they undergo the same processing as the first layer, until the third-layer self-modulation feature extraction module outputs the final residual feature.

[0026] The extracted features from the context-aware features of each layer and the final residual features are added together, and the predicted bilateral grid coefficients are obtained using a 1*1 convolution kernel and reshaped dimension.

[0027] Furthermore, the self-modulation feature extraction module takes the input and passes it through a 1*1 convolution kernel to obtain initial features. Then, the initial features are successively passed through a 1*1 convolution, a global average pooling layer, and a 1*1 convolution to obtain average feature information. The average feature information is multiplied by the initial features and then added to the initial features. Finally, the output features are obtained by passing the ReLU activation function layer.

[0028] The context-aware feature extraction module passes the input features through a global average pooling layer and a global standard deviation pooling layer, respectively, and then adds the results before passing them through a Sigmoid activation layer; the output of the Sigmoid activation layer is multiplied by the initial input features to obtain the context-aware features;

[0029] The residual features are obtained by subtracting the initial input features from the context-aware features.

[0030] Furthermore, the step of correcting multiple high-frequency information layers using shallow networks to obtain the corrected high-frequency information layers includes:

[0031] The lowest resolution high-frequency information layer is spliced ​​together with the upsampled low-frequency information layer and the corrected low-frequency information layer.

[0032] The concatenated result is taken as input and passed through a correction prediction module consisting of two convolutional layers and a Leaky ReLU layer in the middle. The output is a correction layer m used to correct high-frequency information. n-1This correction layer first interacts with the high-frequency information layer h. n-1 Multiplying them together yields the corrected high-frequency information layer h. n ′ -1 ;

[0033] Use bilinear interpolation to m n-1 Upsampling to h n-2 The resolution size is then used to obtain the correction layer m of each high-frequency information layer after passing through the same correction prediction module. n-2 , will high-frequency information layer h n-2 With the corresponding corrective layer m n-2 Pixel-by-pixel multiplication yields the corrected high-frequency information layer h. n ′ -2 ;

[0034] Repeat this operation on the newly obtained correction layers to finally obtain the corrected high-frequency layers h1', h'2, h3'...h' n-3 .

[0035] Further, the step of reconstructing the corrected high-frequency information layer and the corrected low-frequency information layer into an exposure-corrected image includes:

[0036] A network similar to the Laplacian pyramid is used to reconstruct the image after exposure correction. Starting from the corrected low-frequency information layer, the network is convolved and upsampled layer by layer and added to the previous layer.

[0037] The upsampling structure of the Laplacian pyramid reconstruction network shares parameters with the upsampling structure of the Laplacian pyramid image decomposition network.

[0038] According to some embodiments, a second aspect of the present invention provides an ultra-high resolution image exposure error correction system, employing the following technical solution:

[0039] An ultra-high resolution image exposure error correction system includes:

[0040] The image decoupling module is configured to separate high- and low-frequency information in the image whose exposure needs to be corrected, decomposing it into multiple high-frequency information layers and a single low-frequency information layer.

[0041] The low-frequency information processing module is configured to downsample the low-frequency information layer to a low resolution, calculate and obtain a quasi-bilateral mesh network that stores affine transformations, and obtain the corrected low-frequency information layer through the quasi-bilateral mesh.

[0042] The high-frequency information processing module is configured to use shallow networks to correct multiple high-frequency information layers to obtain corrected high-frequency information layers.

[0043] The image reconstruction module is configured to reconstruct the corrected high-frequency information layer and the corrected low-frequency information layer into an exposure-corrected image.

[0044] According to some embodiments, a third aspect of the present invention provides a computer-readable storage medium.

[0045] A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the ultra-high resolution image exposure error correction method as described in the first aspect above.

[0046] According to some embodiments, a fourth aspect of the present invention provides a computer device.

[0047] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the steps of the ultra-high resolution image exposure error correction method as described in the first aspect above.

[0048] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0049] This invention designs a novel lightweight image exposure correction network by applying a lossless and reversible image content decoupling method and a novel hierarchical feature decomposition module, so as to achieve efficient correction of 4K images at speeds exceeding 100fps, thus solving the real-time and effectiveness problems of ultra-high resolution image exposure correction tasks. Attached Figure Description

[0050] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0051] Figure 1 This is a flowchart of an ultra-high resolution image exposure error correction method according to an embodiment of the present invention;

[0052] Figure 2 This is a technical roadmap of an ultra-high resolution image exposure error correction method according to an embodiment of the present invention;

[0053] Figure 3 This is a schematic diagram of a double-sided mesh network structure as described in an embodiment of the present invention;

[0054] Figure 4 This is a schematic diagram of the hierarchical feature decomposition module structure according to an embodiment of the present invention;

[0055] Figure 5 This is a schematic diagram of the context-aware feature decomposition module structure according to an embodiment of the present invention;

[0056] Figure 6 This is a schematic diagram of the self-modulation feature extraction module structure according to an embodiment of the present invention. Detailed Implementation

[0057] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0058] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0059] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0060] Where there is no conflict, the embodiments and features in the embodiments of the present invention can be combined with each other.

[0061] Example 1

[0062] This embodiment provides a method for correcting exposure errors in ultra-high resolution images. This embodiment uses the application of this method to a server as an example for illustration. It is understood that this method can also be applied to terminals, and can also be applied to systems including terminals, servers, and other components, and implemented through interaction between the terminal and the server. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network servers, cloud communication, middleware services, domain name services, CDN security services, and big data and artificial intelligence platforms. The terminal can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these. The terminal and server can be directly or indirectly connected via wired or wireless communication, which is not limited herein. In this embodiment, the method includes the following steps:

[0063] The high and low frequency information in the image that needs exposure correction is separated and decomposed into multiple high frequency information layers and a single low frequency information layer.

[0064] The low-frequency information layer is downsampled to a low resolution, and a quasi-bilateral mesh network storing affine transformations is calculated and obtained. The corrected low-frequency information layer is obtained through the quasi-bilateral mesh.

[0065] Multiple high-frequency information layers are corrected using shallow networks to obtain the corrected high-frequency information layers;

[0066] The corrected high-frequency information layer and the corrected low-frequency information layer are reconstructed into an exposure-corrected image.

[0067] To address the real-time performance and effectiveness issues of ultra-high resolution image exposure correction tasks, this embodiment designs a novel lightweight image exposure correction network to achieve efficient correction of 4K images at speeds exceeding 100fps.

[0068] Specifically, such as Figure 1 As shown, this embodiment proposes a technique for image exposure correction by decoupling image depth and predicting exposure correction transform at extremely low resolution. The technical process includes:

[0069] The image is decomposed into several high-frequency information layers and a single low-frequency information layer arranged from high to low resolution using a Laplacian pyramid-like image decomposition network.

[0070] For the high-frequency information layer, a simple shallow network and the prediction layer of the next network are used for correction. For the low-frequency information layer, it is first downsampled and then input into the hierarchical feature decomposition module to obtain a bilateral grid storing the affine transformation. Finally, based on the output of the low-frequency information layer input to the self-modulation feature extraction module, the obtained affine transformation is applied to the corresponding pixels of the low-frequency information layer to obtain the corrected low-frequency information layer.

[0071] Based on the parameters of the Laplacian pyramid-like image decomposition network, a Laplacian reconstruction process is performed on the corrected high-frequency and low-frequency information layers to obtain the final corrected image.

[0072] like Figure 2 As shown, this embodiment proposes an image exposure correction technology that can process ultra-high resolution images in real time at high speed. It consists of four modules: an image decoupling module, a low-frequency information processing module, a high-frequency information processing module, and an image reconstruction module.

[0073] Its method for correcting exposure errors in ultra-high resolution images is as follows:

[0074] First, the image decoupling module uses a Laplacian pyramid-like decomposition network to decouple the input image, separating it into several high-frequency information layers (denoted as h1', h'2, h3'...h') that mainly contain contour and texture information. n-1 ) and a low-frequency information layer containing information such as lighting, color, and content (denoted as l) n This allows the network to process information in a targeted manner based on its characteristics.

[0075] Then, for the low-frequency information layer, the main part of the low-frequency information processing module is a bilateral mesh network that predicts and stores affine transformations. This network processes the input low-frequency information layer l n After correction, l is obtained n This module mainly consists of three parts: the first part inputs the low-frequency information layer into the self-modulation feature extraction module (SFE) to obtain the guiding map features; the second part downsamples the low-frequency information layer and then inputs the downsampled result into the hierarchical feature decomposition module (HFD), which mainly consists of a context-aware feature extraction module, a self-modulation feature extraction module, and a 1*1 convolution. First, the downsampled result is predicted by the hierarchical feature decomposition module to obtain the coefficients of the bilateral grid; the third part performs a slicing operation on the coefficients of the bilateral grid and the guiding map features to obtain the sliced ​​bilateral grid coefficients B, and then applies an affine transformation pixel-by-pixel to the input low-frequency information layer l. n Get l n The network structure of the low-frequency information processing module is as follows: Figure 3 As shown.

[0076] Specifically, based on the pixel intensity search of the double-sided grid in the low-frequency information layer, the affine transformation corresponding to the pixel is obtained;

[0077] The resulting affine transformation is applied to the corresponding pixels of the low-frequency information layer.

[0078] The corrected low-frequency information layer is obtained.

[0079] Secondly, for the high-frequency information layer, the main structure of the high-frequency information processing module is a shallow network that predicts an exposure correction layer and continuously upsamples it to correct each high-frequency information layer.

[0080] Finally, the image reconstruction module performs Laplacian pyramid reconstruction on the corrected high-frequency and low-frequency information layers based on the convolution parameters in the Laplacian pyramid decomposition network to obtain the final corrected image.

[0081] Specifically, the main part of the image decoupling module is a Laplacian pyramid-like decomposition network. It is based on the traditional Gaussian pyramid and Laplacian pyramid methods: first, the image is blurred and downsampled multiple times to obtain its Gaussian pyramid sequence; then, the final result of the sequence is blurred and upsampled multiple times, and subtracted from each layer of the Gaussian pyramid to obtain the final Laplacian pyramid.

[0082] To enhance the ability of traditional Laplacian pyramids to decouple high- and low-frequency information in images and improve model speed, this scheme replaces the traditional Gaussian blurring and downsampling process and the Gaussian blurring and upsampling process with learnable convolutional kernels to achieve fast, adaptive, and learnable Laplacian pyramid-like decomposition. The Laplacian pyramid-like decomposition network uses learnable convolutions (3x3 convolutions with a stride of 2) to perform multiple downsampling operations instead of the traditional Gaussian blurring and downsampling process, resulting in a Gaussian pyramid sequence. This Gaussian pyramid sequence is then layer by layer from low to high frequency through 3x3 convolutions with a stride of 1, upsampled to the size of the previous layer, and subtracted from the previous layer to obtain the Laplacian pyramid sequence. It should be noted that in the Laplacian pyramid sequence, the bottom layer represents the decomposed low-frequency information layer, and the higher layers represent the decomposed high-frequency information layers.

[0083] like Figure 4 As shown, the main part of the low-frequency information processing module is a hierarchical feature extraction module that predicts and stores the bilateral grid of affine transformation. This module has three layers, each consisting of two parts. After each layer is processed by a context-aware feature decomposition module, the feature information is separated into context-aware features and residual features. For the context-aware feature part, it is directly output to the end of the feature extraction block through a 1*1 convolutional layer and a ReLU activation function layer. For the residual feature part, it is input to the self-modulation feature extraction module, and the result is input to the next layer for the same processing, repeated twice. Finally, at the end of the feature extraction block, the four feature parts are added together (three of which are the outputs of the context-aware feature parts of each layer, and the other feature is the output of the last self-modulation feature extraction module), and then a 1*1 convolutional kernel and reshaping the dimension are used to obtain the predicted bilateral grid coefficients. The obtained affine transformation is applied to the corresponding pixels to obtain the corrected low-frequency information layer.

[0084] Among them, such as Figure 6 As shown, the self-modulation feature extraction module includes:

[0085] The input is passed through a 1*1 convolution kernel to obtain the output, which is denoted as feature 1.

[0086] Feature 1 is input into a 1*1 convolutional kernel, passed through a global average pooling layer, and then passed through another 1*1 convolutional kernel, and is denoted as Feature 2.

[0087] Multiply feature 1 and feature 2 together, add the result back to feature 1, and then pass the result through a ReLU activation layer to obtain the output.

[0088] like Figure 5 As shown, the context-aware feature extraction module includes:

[0089] The input features are passed through a global average pooling layer and a global standard deviation pooling layer respectively, and the two results are added together and then passed through a Sigmoid activation layer.

[0090] The context-aware features are obtained by multiplying the output of the Sigmoid activation layer with the initial input features.

[0091] The residual features are obtained by subtracting the initial input features from the context-aware features.

[0092] like Figure 4 As shown, the hierarchical feature decomposition module includes:

[0093] The input is successively passed through a 1*1 convolutional kernel and a context-aware feature extraction module to obtain context-aware feature 1 and residual feature 1;

[0094] The context-aware features are input into the 1*1 convolutional kernel connected to the ReLU activation layer, and the residual features are input into the self-modulation feature extraction module to obtain features 3 and features 4 respectively.

[0095] Repeat the above two steps for feature 4 to obtain ReLU activation layer output feature 5 and self-modulation feature extraction module output feature 6.

[0096] Repeat the operation of feature 4 on feature 6 to obtain ReLU activation layer output feature 7 and self-modulation feature extraction module output feature 8;

[0097] Features 3, 5, 7, and 8 are added together, and then passed through a 1*1 convolutional kernel and a reshaping layer to obtain the output.

[0098] The high-frequency information processing module mainly consists of a shallow network. This shallow network comprises multiple convolutional layers and an activation function layer, namely the correction prediction module mentioned below, which outputs the correction layer. It first concatenates the penultimate layer of the Laplacian pyramid, the upsampled low-frequency information layer, with the corrected low-frequency information layer, thus forming the lowest-resolution high-frequency information layer h. n-1 Upsampled low-frequency information layer l n and correction of low-frequency information layer l n The concatenation is performed by concatenating the two layers, and the concatenation result is then used as input. This input is passed through a correction prediction module consisting of two 1*1 convolutional layers and a Leaky ReLU layer in the middle, which outputs a correction layer m for correcting high-frequency information. n-1 This correction layer first interacts with the high-frequency information layer h. n-1 Multiply by the dot product to obtain the corrected h. n ′ -1 Then bilinear interpolation was used to transform m n-1 The resolution of each high-frequency information layer is upsampled, and then passed through the correction and prediction module to obtain the correction layers m1, m2, m3...m of each high-frequency information layer. n-2 The high-frequency information layers h1, h2, h3...hn-3 With the corresponding corrective layers m1, m2, m3....m n-2 By multiplying each pixel, we obtain the corrected h1', h'2, ..., h' n-2 .

[0099] The main body of the image reconstruction module is a Laplacian pyramid-like reconstruction network. Its upsampling structure shares parameters with the upsampling structure of the Laplacian pyramid-like decomposition network in the image decoupling module to ensure the rationality of the reconstruction. Starting from the corrected low-frequency information layer, the module processes each layer through a 3x3 convolutional kernel with a stride of 1, upsampling to the size of the previous information layer, and then adding it to the previous layer to finally reconstruct the image after exposure correction.

[0100] This embodiment proposes a high-speed, real-time technique for correcting exposure errors in ultra-high resolution images based on data-driven deep learning technology. The proposed network structure can process 4K images at over 100fps, and the total number of parameters is 8.098K, making it easy to port or deploy to mobile devices to improve the photography quality on mobile devices.

[0101] This embodiment achieves exposure correction processing of 4K images at over 100fps in an Intel i9-10900X CPU and Titan RTX GPU environment. In fair comparisons, this speed outperforms existing traditional and deep learning methods for image exposure correction, and achieves state-of-the-art performance on publicly available benchmark datasets.

[0102] Example 2

[0103] This embodiment provides an ultra-high resolution image exposure error correction system, including:

[0104] The image decoupling module is configured to separate high- and low-frequency information in the image whose exposure needs to be corrected, decomposing it into multiple high-frequency information layers and a single low-frequency information layer.

[0105] The low-frequency information processing module is configured to downsample the low-frequency information layer to a low resolution, calculate and obtain a quasi-bilateral mesh network that stores affine transformations, and obtain the corrected low-frequency information layer through the quasi-bilateral mesh.

[0106] The high-frequency information processing module is configured to use shallow networks to correct multiple high-frequency information layers to obtain corrected high-frequency information layers.

[0107] The image reconstruction module is configured to reconstruct the corrected high-frequency information layer and the corrected low-frequency information layer into an exposure-corrected image.

[0108] like Figure 2As shown, this embodiment proposes an image exposure correction system capable of high-speed real-time processing of ultra-high-resolution images, which consists of four modules: an image decoupling module, a high-frequency information processing module, a low-frequency information processing module, and an image reconstruction module. The technical process is as follows:

[0109] The image decoupling module is used to separate high- and low-frequency information in the image that needs exposure correction, decomposing the input image into several high-frequency information layers (denoted as h1, h2, h3...h). n-1 ) and a single low-frequency information layer (denoted as l) n ).

[0110] The low-frequency information processing module is used to correct the low-frequency information layer of the image. It processes the low-frequency information layer... n The sample is downsampled to a lower resolution, and a bilateral mesh storing the affine transformation is calculated. The low-frequency information layer is then corrected pixel-by-pixel based on the affine transformation in the bilateral mesh.

[0111] The high-frequency information processing module is used to correct the high-frequency information layer of the image. This module first processes the upsampled low-frequency information layer. n Corrected low-frequency information layer n ′ and the lowest resolution high-frequency information layer h n-1 The illumination correction layer m is obtained by concatenating the layers and passing them through a correction prediction module consisting of two simple convolutional layers and a LeakyReLU layer in the middle. n-1 And the high-frequency information layer h n-1 Correction. Then, multiple convolutions and upsampling m. n-1 Until it is connected to each high-frequency information layer h1, h2, h3...h n-2 With the same resolution, the data are then processed through the same correction prediction module and multiplied by the corresponding h. i The resulting high-frequency information layers are corrected.

[0112] The image reconstruction module reconstructs the corrected high-frequency and low-frequency information layers into an exposure-corrected image.

[0113] The examples and application scenarios implemented by the above modules and corresponding steps are the same, but are not limited to the content disclosed in Embodiment 1 above. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer-executable instructions.

[0114] The descriptions of each embodiment in the above embodiments have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0115] The proposed system can be implemented in other ways. For example, the system embodiments described above are merely illustrative, and the division of modules described above is only a logical functional division. In actual implementation, there may be other division methods. For example, multiple modules may be combined or integrated into another system, or some features may be ignored or not executed.

[0116] Example 3

[0117] This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the steps in the ultra-high resolution image exposure error correction method described in Embodiment 1 above.

[0118] Example 4

[0119] This embodiment provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, it implements the steps in the ultra-high resolution image exposure error correction method described in Embodiment 1 above.

[0120] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of hardware embodiments, software embodiments, or embodiments combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage and optical storage) containing computer-usable program code.

[0121] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0122] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0123] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0124] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0125] While the specific embodiments of the present invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the present invention. Those skilled in the art should understand that various modifications or variations that can be made by those skilled in the art without creative effort based on the technical solutions of the present invention are still within the scope of protection of the present invention.

Claims

1. A method for correcting exposure errors in ultra-high resolution images, characterized in that, include: The high and low frequency information in the image that needs exposure correction is separated and decomposed into multiple high frequency information layers and a single low frequency information layer. The low-frequency information layer is downsampled to a low resolution, and a near-bilateral mesh network storing affine transformations is calculated and obtained. The corrected low-frequency information layer is then obtained through the near-bilateral mesh network; including: The low-frequency information layer is input into the self-modulation feature extraction module in the double-sided mesh to obtain the guidance map features; The low-frequency information layer is downsampled, and the downsampling result is input into the hierarchical feature decomposition module in the double-sided grid to obtain the predicted double-sided grid coefficients; The coefficients of the bilateral grid are obtained by slicing the coefficients of the bilateral grid with the features of the guide map. Based on the bilateral grid coefficients after slicing, a quasi-bilateral grid network is obtained. Using the quasi-bilateral grid network, the affine transformation is applied pixel by pixel to the low-frequency information layer to obtain the corrected low-frequency information layer. Multiple high-frequency information layers are corrected using shallow networks to obtain the corrected high-frequency information layers; The corrected high-frequency information layer and the corrected low-frequency information layer are reconstructed into an exposure-corrected image.

2. The method for correcting exposure errors in ultra-high resolution images as described in claim 1, characterized in that, The step of separating high- and low-frequency information in the image that needs exposure correction and decomposing it into multiple high-frequency information layers and a single low-frequency information layer includes: The input image is decoupled using a Laplacian pyramid decomposition network, separating it into several high-frequency information layers with high to low resolution, which mainly contain contour and texture information, and a single low-frequency information layer containing illumination, color, and content. The method of decoupling the input image using a Laplacian pyramid-like decomposition network includes: The image is blurred and downsampled multiple times to obtain its Gaussian pyramid sequence; The final result of the Gaussian pyramid sequence is blurred and upsampled multiple times, and then subtracted from each layer of the Gaussian pyramid to obtain the final Laplace pyramid.

3. The method for correcting exposure errors in ultra-high resolution images as described in claim 1, characterized in that, The hierarchical feature decomposition module has three layers, each consisting of a context-aware feature extraction module, a self-modulation feature extraction module, a 1*1 convolutional layer, and a ReLU activation function layer; After the first-layer context-aware feature extraction module performs decomposition processing, it obtains context-aware features and residual features. The context-aware features are extracted by passing through a 1*1 convolutional layer and a ReLU activation function layer. The residual features are extracted by passing through a self-modulation feature extraction module and then to the next layer's context-aware feature extraction module, where they undergo the same processing as the first layer, until the third-layer self-modulation feature extraction module outputs the final residual features. The extracted features from the context-aware features of each layer and the final residual features are added together, and the predicted bilateral grid coefficients are obtained using a 1*1 convolution kernel and reshaped dimension.

4. The method for correcting exposure errors in ultra-high resolution images as described in claim 3, characterized in that, The self-modulation feature extraction module takes the input and passes it through a 1*1 convolution kernel to obtain initial features. Then, the initial features are successively passed through a 1*1 convolution, a global average pooling layer, and another 1*1 convolution to obtain average feature information. The average feature information is multiplied by the initial features and then added back to the initial features. Finally, the output features are obtained by passing the ReLU activation function layer. The context-aware feature extraction module passes the input features through a global average pooling layer and a global standard deviation pooling layer, respectively, and then adds the results before passing them through a Sigmoid activation layer; the output of the Sigmoid activation layer is multiplied by the initial input features to obtain the context-aware features; The residual features are obtained by subtracting the initial input features from the context-aware features.

5. The method for correcting exposure errors in ultra-high resolution images as described in claim 1, characterized in that, The method of correcting multiple high-frequency information layers using shallow networks to obtain corrected high-frequency information layers includes: The lowest resolution high-frequency information layer is spliced ​​together with the upsampled low-frequency information layer and the corrected low-frequency information layer. The concatenated result is taken as input and passed through a correction prediction module consisting of two convolutional layers and a LeakyReLU layer in the middle. The output is a correction layer used to correct high-frequency information. This correction layer first interacts with the high-frequency information layer. Multiply to obtain the corrected high-frequency information layer. ; Using bilinear interpolation Upsampling to The resolution size is then used to obtain the correction layer of each high-frequency information layer after passing through the same correction prediction module. , will high-frequency information layer With the corresponding corrective layer Pixel-by-pixel multiplication yields the corrected high-frequency information layer. ; Repeat this process for the newly obtained correction layers to finally obtain each corrected high-frequency layer. .

6. The method for correcting exposure errors in ultra-high resolution images as described in claim 1, characterized in that, The process of reconstructing the corrected high-frequency information layer and the corrected low-frequency information layer into an exposure-corrected image includes: A network similar to the Laplacian pyramid is used to reconstruct the image after exposure correction. Starting from the corrected low-frequency information layer, the network is convolved and upsampled layer by layer and added to the previous layer. The upsampling structure of the Laplacian pyramid reconstruction network shares parameters with the upsampling structure of the Laplacian pyramid image decomposition network.

7. A system for correcting exposure errors in ultra-high resolution images, characterized in that, include: The image decoupling module is configured to separate high- and low-frequency information in the image whose exposure needs to be corrected, decomposing it into multiple high-frequency information layers and a single low-frequency information layer. The low-frequency information processing module is configured to downsample the low-frequency information layer to a low resolution, calculate and obtain a near-bilateral mesh network that stores affine transformations, and obtain the corrected low-frequency information layer through the near-bilateral mesh; including: The low-frequency information layer is input into the self-modulation feature extraction module in the double-sided mesh to obtain the guidance map features; The low-frequency information layer is downsampled, and the downsampling result is input into the hierarchical feature decomposition module in the double-sided grid to obtain the predicted double-sided grid coefficients; The coefficients of the bilateral grid are obtained by slicing the coefficients of the bilateral grid with the features of the guide map. Based on the bilateral grid coefficients after slicing, a quasi-bilateral grid network is obtained. Using the quasi-bilateral grid network, the affine transformation is applied pixel by pixel to the low-frequency information layer to obtain the corrected low-frequency information layer. The high-frequency information processing module is configured to use shallow networks to correct multiple high-frequency information layers to obtain corrected high-frequency information layers. The image reconstruction module is configured to reconstruct the corrected high-frequency information layer and the corrected low-frequency information layer into an exposure-corrected image.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps in the ultra-high resolution image exposure error correction method as described in any one of claims 1-6.

9. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the ultra-high resolution image exposure error correction method as described in any one of claims 1-6.

Images