A laplacian three-layer cyclic high-definition image enhancement method

Abstract

The present application relates to a kind of laplace three-layer cyclic high-definition image enhancement method, first (sensation expression is problematic) proposed a multi-objective layered learning deep neural network model LPTCGAN, is trained in an end-to-end manner, to generate color harmony, clear structure, detailed high-definition portrait image.First, the input image is decomposed using Laplacian pyramid, separate three size halved high-frequency residual and a low-pass component containing color information.For high-frequency residual and low-pass component, use different networks for conversion, use different loss functions and cyclic way to learn and fuse multi-objective information, and connect high-frequency component for compensation.GAN structure is used for further enhancing the generation ability of generator.Compared with the traditional method, the experimental results show that the method can obtain higher quality high-definition portrait image.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a Laplacian three-layer loop high-definition image enhancement method. Background Technology

[0002] The digital image formation process is affected by various factors. For example, images captured under less than ideal lighting conditions may exhibit low overall quality, localized blurring, and loss of detail, making it difficult to obtain aesthetically pleasing images that meet human perception of quality. Traditional image enhancement processes involve manual adjustments and editing by photographers, which is not only inefficient but also cumbersome. Therefore, designing an image enhancement method that improves aesthetics while reducing workload is crucial.

[0003] To improve image quality, researchers have proposed numerous image enhancement methods. These methods are mainly divided into two categories: traditional methods and deep learning-based methods. Histogram equalization, a traditional method, aims to improve the overall contrast of an image through a specific mapping function. Deep learning-based methods use end-to-end networks, inputting low-quality images into the network and learning the overall mapping relationship to obtain an enhanced image. Other studies incorporate prior knowledge into the enhancement pipeline to guide the network's learning. Image enhancement essentially involves learning nonlinear mapping relationships between images. Currently, most image enhancement models focus on local features and cannot coordinate the overall structure, color, and details of the image during the enhancement process. Therefore, the objective evaluation of network-enhanced results deviates from human aesthetic perception. Summary of the Invention

[0004] In view of the above-mentioned problems in the existing technology, the technical problem to be solved by the present invention is: how to generate high-definition human portrait images that meet the quality of human perception.

[0005] To solve the above technical problems, the present invention adopts the following technical solution: 1. A Laplacian three-layer loop high-definition image enhancement method, characterized in that it includes the following steps:

[0006] S1: Construct and train a deep neural network model LPTCGAN for multi-objective hierarchical learning, which includes a Laplacian decomposition module, a generator module, and a discriminator module.

[0007] S1-1: The h×w image I0 is obtained by three decompositions using the Laplacian decomposition module. I3, H0, H1, and H2 represent the high-frequency components obtained by the Laplace decomposition module on I0 for the first, second, and third iterations, respectively. I0 represents the high-frequency residual sequence, and I3 represents the low-frequency image of I0.

[0008] S1-2: The generator module consists of a high-frequency conversion module and a low-frequency conversion module. The high-frequency conversion module is responsible for migration. The low-frequency conversion module is responsible for transferring I3 and the results of each generation, and finally obtains the converted high-frequency component sequence H′=[H′0,H′1,H′2] and the final output sequence I′=[I′0,I′1,I′2], where I′0, I′1, and I′2 represent the generated graphs of I3 after the third, second, and first cycles, respectively, and H′0, H′1, and H′2 represent the transferred high-frequency components of H0, H1, and H2 after processing by the high-frequency conversion module, respectively.

[0009] The high-frequency conversion module consists of multiple convolutional layers and a channel spatial attention mechanism module, while the low-frequency conversion module includes multiple convolutional layers, residuals, and upsampling.

[0010] S1-3: Input the target image Y and I′0 of I0 into the discriminator. The discriminator determines that I′0 obtained by the generator is a fake image and the target image Y is a real image.

[0011] S1-4: Obtain multiple images to form a training set, and use the images in the training set to train LPTCGAN. When the loss function... When the value of is continuously reduced until it no longer changes, a well-trained LPTCGAN is obtained.

[0012] S2: Prediction. Input an image X to be processed into the trained LPTCGAN, and the output is the corresponding enhanced image X′.

[0013] As an improvement, the step of obtaining H′=[H′0,H′1,H′2] using the high-frequency conversion module in S1-2 is as follows:

[0014] First of all The H2 in the matrix is ​​convolved, and then convolved with Instance Norm and Leaky ReLU, and then convolved again to a feature map of size h / 4*w / 4*64, denoted as H. 2FM Then H 2FM Two 1×1×c features are obtained by global average pooling and global max pooling, respectively. Then, the features of these two 1×1×c features are input into a fully connected layer with shared parameters before being concatenated. Finally, the features are normalized using the Sigmoid activation function and then compared with H. 2FM Channel feature map H is obtained based on element-wise multiplication. 2c Then H 2c Two h×w×1 features are obtained based on channel-wise average pooling and max pooling, and then concatenated according to channels. Next, a convolutional layer with a sigmoid activation function is used to reduce the dimensionality to 1 channel, and then this feature is concatenated with H. 2cH is obtained by performing element-wise multiplication. 2cs H 2cs Then with H 2FM The temporary output H is obtained by performing element-wise multiplication. 2middle Then H 2middle After a certain convolution and then the tanh activation function, we obtain H′2 of size h / 4*w / 4*3. H0 and H1 are processed in the same way as H2 to obtain H′0 and H′1 respectively.

[0015] As an improvement, the step in S1-2 using the low-frequency conversion module to obtain the final output sequence I′=[I′0,I′1,I′2] is as follows:

[0016] enter As the first loop of the low-frequency conversion module begins, after two convolutional layers, its channels are expanded to 64. Then, it undergoes Leaky ReLU normalization, followed by four stacked residual blocks to deepen the network, each with a depth of 64. Next, the feature maps from the first four residual blocks are upsampled, doubling the h and w values ​​while keeping the number of channels unchanged. After three more stacked residual blocks, it undergoes two convolutions to reduce the number of channels to 3. Finally, I0 is connected to it, followed by a Tanh activation, and then... get

[0017] Then I′2 is re-input into the low-frequency conversion module for a second loop. After a Tanh activation, H′1 is obtained. Using the same method, I′1 is re-inputted into the low-frequency conversion module for a third loop to obtain...

[0018] As an improvement, S1-4 The definition is:

[0019] The generator uses a loss function And discriminator using loss function They are:

[0020]

[0021]

[0022] Among them, E Y P represents the expected value for input Y. data (Y) represents the distribution space of the target image Y, P data(I0) represents the distribution space of I0, D(Y) represents the value output by the discriminator after the input target image Y is processed, D(G(I0)) represents the output value obtained by generating I′0 for I0 and then inputting I′0 into the discriminator D, and G(I0) represents the value of generating I′0 for I0.

[0023] The loss functions for the third, second, and first iterations are as follows:

[0024] L1, the loss for the first iteration, is defined as follows:

[0025]

[0026] in This represents the L1 loss function for the output of the first loop layer, where Y is the target sample and ρ represents the index of a single pixel.

[0027] The L2 loss function, used as the loss in the second iteration, is defined as follows:

[0028]

[0029] in This represents the L2 loss function for the output of the second loop.

[0030] The combination of L2 and MS-SSIM is used as the loss for the third cycle, defined as follows:

[0031]

[0032]

[0033]

[0034]

[0035] in This represents the perceptual motivation loss function for the third-level loop, where M represents the decomposition level, and C1 and C2 are two constants. Let L2 and MS-SSIM represent the loss functions, respectively.

[0036]

[0037] Where α, β, γ, λ, k, v are coefficients controlling the proportion of each loss, μ Y This represents the average value of Y, and similarly... This represents the average value of I′0. σ represents the covariance of Y and I′0. Y 2This represents the variance of Y. This represents the variance of I′0. Let l represent the covariance of I′0 and Y. M (ρ), cs j (ρ) represents the result of the intermediate equation system and has no special meaning.

[0038] Compared with the prior art, the present invention has at least the following advantages:

[0039] This invention proposes a multi-objective hierarchical learning deep neural network model, LPTCGAN. It employs a Laplacian pyramid to decompose images, designs different networks for transforming high-frequency residuals and low-pass components, uses different loss functions to control the learning of specific objectives, and utilizes a cyclic approach for adaptive fusion of multi-objective information. Furthermore, a high-frequency information compensation structure is designed to effectively mitigate the problem of high-frequency loss during ultra-high-definition image convolution. Finally, experiments were conducted on different datasets, and the results show that LPTCGAN's enhancement results outperform the baseline model in both subjective and objective evaluations, while also exhibiting good generalization ability. Attached Figure Description

[0040] Figure 1 This demonstrates the enhanced results of the model of this invention on different datasets.

[0041] Figure 2 This is a simplified structural diagram of LPTCGAN.

[0042] Figure 3 This is a simplified diagram of the generator's structure.

[0043] Figure 4 This is a simplified structural diagram of the discriminator.

[0044] Figure 5 To compare the visual results of different networks, they were tested on the RCA dataset, where bounding boxes represent the detailed regions that need to be focused on.

[0045] Figure 6 The translation results of state-of-the-art image augmentation networks were visually tested on the MIT-Adobe fiveK dataset, with bounding boxes representing the detailed regions that need attention.

[0046] Figure 7 The images show some of the visual results of the model enhancement in this invention. The left side shows the results from the MIT-Adobe fiveK dataset, and the right side shows the results from the DCA dataset. Detailed Implementation

[0047] The present invention will now be described in further detail.

[0048] This paper proposes a Laplacian three-layer recurrent high-resolution image enhancement method and presents the LPTCGAN deep neural network model for multi-objective hierarchical learning. This method utilizes the Laplacian pyramid to decompose the image, transforming and concatenating the high-frequency residuals and low-pass images using two networks. Different loss functions are employed to control the learning of specific objectives, and adaptive fusion of learned features is performed in a recurrent manner. Experimental results show that LPTCGAN can generate higher-quality images compared to existing image enhancement networks. Experiments on two datasets demonstrate that LPTCGAN exhibits good generalization ability.

[0049] See Figure 1-7 A Laplacian three-layer loop high-resolution image enhancement method includes the following steps:

[0050] S1: Construct and train a deep neural network model LPTCGAN for multi-objective hierarchical learning, which includes a Laplacian decomposition module, a generator module, and a discriminator module.

[0051] To address the issue of high-frequency image loss during image downsampling, the image is linearly decomposed into high-frequency and low-frequency components to achieve accurate image reconstruction. Specifically, given an h×w image I0, its low-frequency prediction is first calculated and then convolved with a fixed Gaussian kernel to obtain... Specifically, low-frequency predictions are obtained by convolving image I0 with a fixed Gaussian kernel. To reconstruct the image, the high-frequency residual is calculated using H0 = I0 - UP(I1), where This is obtained by interpolating I1 and convolving it with a Gaussian kernel of four times its normal value. By repeating the above operation on I1, a set of bandpass filters can be obtained.

[0052] Given an image I0 is decomposed into a set of components, where the high-frequency components are used as... This indicates that the low-frequency image I from the last decomposition was also saved. n+1 'n' represents the specific number of layers in the decomposition, and the number of pixels in the nth layer image is... Decomposing an image into a Laplacian pyramid allows for restoration of the original image through a series of mirroring operations, significantly saving storage space while minimizing the loss of detail that is imperceptible to the naked eye. (Qualcomm component) Only high-frequency details are preserved, with most pixel values ​​set to 0, whereas the low-pass image I... n+1 The process of removing "high-frequency" elements layer by layer reflects the properties of the original image in a content-independent manner.

[0053] S1-1: The h×w image I0 is obtained by three decompositions using the Laplacian decomposition module. I3, H0, H1, and H2 represent the high-frequency components obtained by the Laplace decomposition module on I0 for the first, second, and third iterations, respectively. I0 represents the high-frequency residual sequence, and I3 represents the low-frequency image of I0; thus, we obtain... Figure 2 The loop begins with I3 as the first input to the low-frequency conversion network, while simultaneously... The high-frequency information is input to the high-frequency migration module to achieve high-frequency information compensation.

[0054] Image I0 is downsampled three times while retaining high-frequency residuals. A cyclical structure is used to transform low-frequency components, and the high-frequency residuals transformed by the attention-guided network are continuously superimposed. This approach takes into account that the main part of image enhancement is the transformation of low frequencies, and the three-cycle method can also promote network learning and the integration of more information.

[0055] S1-2: The generator module consists of a high-frequency conversion module and a low-frequency conversion module. The high-frequency conversion module is responsible for migration. The low-frequency conversion module is responsible for transferring I3 and the results of each generation, and finally obtains the converted high-frequency component sequence H′=[H′0,H′1,H′2] and the final output sequence I′=[I′0,I′1,I′2], where I′0, I′1, and I′2 represent the generated graphs of I3 after the third, second, and first cycles, respectively, and H′0, H′1, and H′2 represent the transferred high-frequency components of H0, H1, and H2 after processing by the high-frequency conversion module, respectively.

[0056] The high-frequency conversion module consists of multiple convolutional layers and a channel spatial attention mechanism module, while the low-frequency conversion module includes multiple convolutional layers, residuals, and upsampling, and it is a cyclic backbone.

[0057] When the low-frequency conversion module receives its first input, the high-frequency conversion module simultaneously converts the corresponding high-frequency component H2 to obtain H′2. When the low-frequency conversion module outputs, H′2 is concatenated to obtain the final output of the first loop. The same operation is performed on the remaining two high-frequency components of different sizes, using concatenation to compensate for high-frequency information. This effectively eliminates high-frequency loss caused by excessive convolution and promotes adaptive high-frequency transfer to generate high-definition portrait images that meet human perception quality. It is important to note that the high-frequency conversion network does not change the image size; therefore, a convolution method with a kernel size of 3×3 and a stride of 1 is chosen.

[0058] S1-3: Input the target image Y and I′0 of I0 into the discriminator. The discriminator determines that I′0 obtained by the generator is a fake image and the target image Y is a real image. The purpose of inputting them into the discriminator is to make the discriminator determine that the generated I′0 is a fake image and Y is a real image. The purpose is only to enhance the generator's generation ability, that is, to make the generated I′0 more closely match Y.

[0059] The detailed network structure of the discriminator is as follows: Figure 4 As shown, it attempts to determine the authenticity of the generated image, thus establishing a game-theoretic relationship with the generator to enhance the generator's generation capabilities. The target image Y and the generated image I′0 are input into the discriminator, and after a convolution with a step size of 2 and a kernel size of 3×3, its channels are widened to 16. Then, the same convolution method is used to continuously widen its channels to 128 while compressing the size. Finally, convolution is used to output the final features.

[0060] S1-4: Obtain multiple images to form a training set, and use the images in the training set to train LPTCGAN. When the loss function... When the value of is continuously reduced until it no longer changes, a well-trained LPTCGAN is obtained.

[0061] S2: Prediction. Input an image X to be processed into the trained LPTCGAN, and the output is the corresponding enhanced image X′.

[0062] Specifically, the steps in S1-2 to obtain H′=[H′0,H′1,H′2] using the high-frequency conversion module are as follows:

[0063] like Figure 3 As shown, the high-frequency transformation module consists of a small number of convolutional layers, because too many convolutions can easily lead to unexpected losses in high-frequency components. In addition, a channel spatial attention mechanism module is introduced, which prompts the network to focus on features of interest and weaken less important features. Although the intensity of most high-frequency components is close to zero, they contain many high-frequency details such as hair strands and freckles, necessitating selective transformation of high-frequency components. The attention mechanism is essentially the dynamic adjustment of feature map weights. The channel attention mechanism focuses on capturing which features are more important.

[0064] Taking the input H2 in the first loop as an example, firstly... The H2 in the matrix is ​​convolved, and then convolved with Instance Norm and Leaky ReLU, and then convolved again to a feature map of size h / 4*w / 4*64, denoted as H. 2FM (ps: This refers to the operations on the generator graph. h and w represent the length and width of the input image I0, and the subscript fm means feature map.) Then H 2FMTwo 1×1×c features (where c is 64) are obtained by global average pooling and global max pooling, respectively. Then, the features from these two 1×1×c features are input into a fully connected layer with shared parameters, followed by concatenation. Finally, the features are normalized using the Sigmoid activation function and then compared with H. 2FM Channel feature map H is obtained based on element-wise multiplication. 2c Then H 2c Two h×w×1 features are obtained based on channel-wise average pooling and max pooling, and then concatenated according to channels. Next, a convolutional layer with a sigmoid activation function is used to reduce the dimensionality to 1 channel, and then this feature is concatenated with H. 2c H is obtained by performing element-wise multiplication. 2cs (i.e., the feature map obtained through channel and spatial attention mechanisms), H 2cs Then with H 2FM The temporary output H is obtained by performing element-wise multiplication. 2middle Then H 2middle After a certain convolution and then the tanh activation function, we obtain H′2 of size h / 4*w / 4*3. We then use the same method as with H2 to obtain H′0 and H′1.

[0065] Specifically, the steps in S1-2 to obtain the final output sequence I′=[I′0,I′1,I′2] using the low-frequency conversion module are as follows:

[0066] LP's multi-level decomposition successfully separates high-frequency residuals and low-frequency attributes, enabling reversible reconstruction with almost no loss. A more refined reconstruction is achieved by cyclically transforming low-frequency components and continuously superimposing high-frequency components. The advantages of this approach are twofold: first, low-frequency attributes reflect the overall image content and color information in a coarse-grained manner, saving network computation without sacrificing learning efficiency; second, the cyclical approach encourages the network to learn more feature information and perform adaptive fusion. Finally, a simple and effective connection method is employed to compensate for high-frequency information.

[0067] The low-frequency transfer module is the backbone of the loop, but it does not only translate the input in the loop, but also transfer the output.

[0068] enter As the first loop of the low-frequency conversion module begins, after two convolutional layers, its channels are expanded to 64. Then, it undergoes Leaky ReLU normalization, followed by four stacked residual blocks to deepen the network, each with a depth of 64. Next, the feature maps from the first four residual blocks are upsampled, doubling the h and w values ​​while keeping the number of channels unchanged. After three more stacked residual blocks, it undergoes two convolutions to reduce the number of channels to 3. Finally, I0 (the original input image I0, resized to match its output and then added with 30% weights) is connected to it, followed by a Tanh activation, and then... get

[0069] Then I′2 is re-input into the low-frequency conversion module for a second loop. After a Tanh activation, H′1 is obtained. Using the same method, I′1 is re-inputted into the low-frequency conversion module for a third loop to obtain... The final output sequence of the generator, I′=[I′0,I′1,I′2], is obtained through the above method.

[0070] High-frequency information is compensated using a connection-based approach. This effectively eliminates high-frequency loss caused by excessive convolution, while also promoting adaptive high-frequency transfer to generate high-resolution portrait images that meet human perception quality. It's important to note that the high-frequency transformation network does not change the image size; therefore, a convolution method with a kernel size of 3×3 and a stride of 1 is chosen. Specific details are as follows... Figure 3 As shown.

[0071] Specifically, S1-4 The definition is:

[0072] The generator uses a loss function And discriminator using loss function They are:

[0073]

[0074]

[0075] Among them, E Y P represents the expected value for input Y. data (Y) represents the distribution space of the target image Y, P data(I0) represents the distribution space of I0, D(Y) represents the value output by the discriminator after the input target image Y is processed, D(G(I0)) represents the output value obtained by generating I′0 for I0 and then inputting I′0 into the discriminator D, and G(I0) represents the value of generating I′0 for I0.

[0076] The loss functions for the third, second, and first iterations are as follows:

[0077] L1, the loss for the first iteration, is defined as follows:

[0078]

[0079] in This represents the L1 loss function for the output of the first loop layer, where Y is the target sample, ρ represents the index of a single pixel, and P represents the entire pixel space.

[0080] Although the L2 loss function tends to smooth out high-frequency learning, it is more efficient at learning image structure content. Therefore, the L2 loss function is chosen as the loss for the second loop, and is defined as follows:

[0081]

[0082] in This represents the L2 loss function for the output of the second loop.

[0083] For the high-level output being the final image, a perception-driven loss function is used to control the overall structure and details of the image. The combination of L2 and MS-SSIM is used as the loss in the third loop, defined as follows:

[0084]

[0085]

[0086]

[0087]

[0088] in This represents the perceptual motivation loss function for the third loop, where M represents the decomposition level [M=5], μ represents the mean, σ represents the variance, and C1 and C2 are two constants. Let μ represent the L2 and MS-SSIM loss functions, respectively. Y This represents the average value of Y, and similarly... This represents the average value of I′0. σ represents the covariance of Y and I′0. Y2 This represents the variance of Y. This represents the variance of I′0. Let l represent the covariance of I′0 and Y. M (ρ), cs j (ρ) represents the result of the intermediate system of equations and has no special meaning.

[0089]

[0090] Where α, β, γ, λ, k, and v are coefficients that control the proportion of each loss.

[0091] experiment

[0092] A. Dataset

[0093] Experiments were conducted on our own dataset RCA and the benchmark dataset MIT-Adobe fiveK dataset.

[0094] RCA: Portrait Retouching (RCA) contains 2000 portrait images taken with a digital camera. Four professional retouchers professionally retouched the images to meet human aesthetic standards. The retouched images were then cropped to a size of 912×1368 pixels, approximately 1k. A network was trained using 1800 randomly selected images, with the remaining 200 serving as a validation set.

[0095] MIT-Adobe fiveK: It contains 5000 original RGB images and their corresponding sRGB images, manually rendered by five professional photographers. The results of expert C photographers' retouching were selected as the target images, with resolutions ranging from 2k to 4k. From these, 801 high-resolution portrait images were chosen, with 721 used as the training set and the remainder as the validation set.

[0096] B. Parameter Settings

[0097] The input image is randomly cropped to 256×256 for training. Adam is used for optimization, with β1 = 0.9, β2 = 0.99, and a learning rate of 10. -4 A decay strategy was adopted as the number of iterations increased, with a total of 150,000 iterations. For equation (9), α = 0.1, β = 0.135, γ = 0.225, k = 0.45, and v = 0.09 were set. Finally, training was implemented on a single NVIDIA Titan VGPU.

[0098] C. Baseline Model Comparison

[0099] The following baseline models were selected for comparative experiments: LPTN, CycleGAN, DPED, RetinexNet, and DPE. Qualitative and quantitative evaluations were performed on the DCA dataset and the MIT-Adobe fiveK dataset, respectively.

[0100] D. Evaluation Indicators

[0101] PSNR: Peak Signal-to-Noise Ratio (PSNR) is an objective image evaluation metric. However, it is based on the error between corresponding pixels, which is an error-sensitive image quality evaluation.

[0102] SSIM: Structure Similarity Index Measure (SSIM) can measure the degree of distortion in an image, as well as the similarity between two images. It integrates three features: brightness, contrast, and structure, and evaluates from the perspective of human perception, making it more consistent with human visual characteristics.

[0103] Comparison of experimental results on the E.DCA dataset

[0104] Table 1 shows the quantitative comparison results of DCA, comparing the method of this invention with state-of-the-art methods.

[0105]

[0106] 1) Qualitative evaluation:

[0107] Qualitative results such as Figure 5 As shown in the image, zooming in on the details reveals that the model of this invention has advantages in color learning, detail learning, and image structure learning. In the second row, images generated by CycleGAN, RetinexNet, and DPED show deficiencies in color learning and detail texture. While images generated by DPE and LPTN perform well in detail texture learning, they struggle to maintain color accuracy. In contrast, the model of this invention performs better in both color learning and texture, and its overall image structure is closer to the target image. In the fourth row, images generated by CycleGAN and DPED fail to maintain detail texture, and images generated by DPE and LPTN models are not accurate enough in color learning. In comparison, the models using RetinexNet and DPTN achieve better results in detail learning, but for the overall logical structure of the image, the model of this invention is clearly closer to the target image.

[0108] The model of this invention achieves significant advantages in learning detailed textures thanks to its high-frequency compensation structure. This invention continuously adds high-frequency components during recurrent training to compensate for the loss of high frequencies during convolution. Furthermore, the model also performs well in color learning and image structure content, thanks to the targeted loss function used in the recurrent structure. In summary, the model of this invention can generate high-quality portrait images that align with human perceptual motivations and is more suitable for enhancing high-resolution portrait images.

[0109] 2) Quantitative evaluation:

[0110] The augmentation results on the RCA dataset were quantitatively evaluated, and the results are shown in Table 1. As can be seen from the table, the model of this invention outperforms other baseline models. The SSIM index of the model of this invention is 0.9016, and the PSNR index is 25.73.

[0111] Comparison of experimental results on the F.MIT-Adobe fiveK dataset

[0112] 1) Qualitative evaluation:

[0113] The method of this invention was compared on the public dataset MIT-Adobe fiveK, see [link / reference]. Figure 6 As shown in the image, compared to a 4K ultra-high-definition portrait image, the DPED result shows insufficient learning, resulting in a noticeably darker image. While the LPTN translation network's generated result presents a generally good effect, it is slightly brighter. These two comparison images demonstrate that the result of this invention is closer to the target image, exhibiting better overall visual quality. Zooming in on the details reveals that the model of this invention learns color and detail more accurately.

[0114] The reason why the model of this invention achieves good visual perception quality is that it allows the same model to learn multiple targets and adaptively fuse them in an appropriate manner. Furthermore, the high-frequency information compensation method also plays a significant role, minimizing the loss of high-frequency detail information even for 4K ultra-high-definition images.

[0115] 2) Quantitative evaluation:

[0116] The augmentation results were quantitatively evaluated on the MIT-Adobe fiveK dataset, and the results are shown in Table 1. As can be seen from the table, the SSIM and PSNR indices of the model of this invention are slightly higher than those of LPTN, with an SSIM of 0.8743 and a PSNR of 21.91.

[0117] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A Laplacian three-layer loop high-definition image enhancement method, characterized in that: Includes the following steps: S1: Construct and train a deep neural network model LPTCGAN for multi-objective hierarchical learning, wherein LPTCGAN includes a Laplacian decomposition module, a generator module, and a discriminator module; S1-1: For image After three decompositions by the Laplace decomposition module, the following was obtained: and , These represent the Laplace decomposition module for... The high-frequency components obtained in the first, second, and third trials. express High-frequency residual sequences, express Low-frequency images; S1-2: The generator module consists of a high-frequency conversion module and a low-frequency conversion module. The high-frequency conversion module is responsible for migration. The low-frequency conversion module is responsible for migration. The results of each generation ultimately yield the transformed high-frequency component sequence. and the final output sequence ,in They represent The generated graph obtained after the third iteration, the second iteration, and the first iteration. They represent The corresponding high-frequency component after processing by the high-frequency conversion module; The high-frequency conversion module consists of multiple convolutional layers and a channel spatial attention mechanism module, while the low-frequency conversion module includes multiple convolutional layers, residuals, and upsampling. Obtained using a high-frequency conversion module The steps are as follows: First of all In Perform convolution, then convolution with Instance Norm and Leaky ReLU, then convolution again to a size of The feature map is denoted as Then Two results were obtained by global average pooling and global max pooling, respectively. Secondly, the two 1×1×c features are input into a fully connected layer with shared parameters and then connected. After normalization using the Sigmoid activation function, they are then compared with... Channel feature maps are obtained based on element-wise multiplication. Then Two h×w×1 features are obtained based on channel-wise average pooling and max pooling, and then concatenated according to channels. Next, a convolutional layer with a sigmoid activation function is used to reduce the dimensionality to 1 channel, and then this feature is combined with... Obtain by performing element-wise multiplication , Again with Temporary output is obtained based on element-wise multiplication. Then After a certain convolution followed by a tanh activation function, we obtain... Size ; right Adopted and The same method is used to process the corresponding results. ; The final output sequence is obtained using the low-frequency conversion module. The steps are as follows: enter As the first loop of the low-frequency conversion module begins, after two convolutional layers, its channels are expanded to 64. Then, it undergoes Leaky ReLU normalization, followed by four stacked residual blocks to deepen the network, each with a depth of 64. Next, the feature maps from the first four residual blocks are upsampled, doubling h and w while keeping the number of channels unchanged. After three more stacked residual blocks, it undergoes two convolutions to reduce the number of channels to 3. Finally, the... Connect to it, activate it via Tanh, and then add it. get ; After that The data is re-input into the low-frequency conversion module for a second cycle, after which it undergoes a Tanh activation. get Using the same method The data is re-entered into the low-frequency conversion module for a third iteration. ; S1-3: Will Target map and The input is fed into the discriminator, which then judges the generator's output. It's a fake image, the target image. It's a genuine image; S1-4: Obtain multiple images to form a training set, and use the images in the training set to train LPTCGAN. When the loss function... When the value of is continuously reduced until it no longer changes, a well-trained LPTCGAN is obtained; S2: Prediction. Input an image X to be processed into the trained LPTCGAN, and the output is the corresponding enhanced image X'.

2. The Laplacian three-layer loop high-definition image enhancement method as described in claim 1, characterized in that: The S1-4 The definition is: The generator uses a loss function And discriminator using loss function They are: (1) (2) in, This represents the expected value for input Y. Represents the distribution space of the target image Y. express The distribution space, This represents the value output by the discriminator after processing the input target image Y. It means for The result generated by the generator Then The output value obtained by inputting into discriminator D, It means for The result generated by the generator ; The loss functions for the third, second, and first iterations are as follows: L1, the loss for the first iteration, is defined as follows: (3) in This represents the L1 loss function for the output of the first loop. It is the target sample. An index representing a single pixel; The L2 loss function, used as the loss in the second iteration, is defined as follows: (4) in This represents the L2 loss function for the output of the second loop; The combination of L2 and MS-SSIM is used as the loss for the third cycle, defined as follows: (5) (6) (7) (8) in This represents the perceptual motivation loss function for the third loop. Indicates the levels of decomposition. For two constants, , Let L2 and MS-SSIM loss functions be represented respectively; (9) in It is a coefficient that controls the proportion of each type of loss. This represents the average value of Y, and similarly... express The average value, Representing Y and covariance, This represents the variance of Y. express variance express covariance of Y This represents the result of the intermediate system of equations and has no special meaning.

Images