Method and system for improving image processing quality by using perceptual loss technology

Abstract

The invention discloses a method and system for improving image processing quality by using a perception loss technology, and the method comprises the steps: S1, obtaining image data under different illuminance conditions, and carrying out the preprocessing of the collected image data; s2, constructing an image enhancement model based on a deep learning algorithm, selecting a network architecture, and then dividing the preprocessed image data into a training set and a test set; s3, inputting the training set and the test set into an image enhancement model to realize training and testing of the model; and S4, finally inputting to-be-processed image data into the image enhancement model to realize enhancement of image quality. Through the perception loss technology, the reconstructed image can be closer to a high-quality reference image, the problems of noise, blurring, color distortion and the like are effectively reduced, and the definition and detail performance of the image are improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to a method and system for improving image processing quality by using perceptual loss technology. Background Art

[0002] With the rapid development of deep learning technology, its application in the field of image processing is becoming more and more extensive. As a key method in deep learning, perceptual loss technology provides effective guidance for image reconstruction by comparing the feature representation differences between the reconstructed image and the reference image in the pre-trained deep neural network, making the reconstructed image closer to the high-quality reference image. However, in the field of full-color night vision images, the application of perceptual loss technology is still relatively limited, and the existing processing methods have many shortcomings.

[0003] Adaptability issues of deep learning models: Full-color night vision images have significant differences in light intensity, color distribution, and detail features compared to images under normal daylight. Most existing deep learning models are not specifically designed for these characteristics, which results in the inability to fully capture and utilize their unique feature information when processing full-color night vision images, affecting the image enhancement effect. For example, in low-light environments, images may have serious noise interference, dim and color cast, and blurred details. General models are difficult to effectively deal with these complex situations.

[0004] Lack of loss function combination and optimization: When using perceptual loss technology, most existing methods fail to fully consider the comprehensive application and optimization of different types of loss functions. A single loss function can often only measure the difference in image quality from a certain aspect, and cannot comprehensively evaluate the image's performance in multiple aspects such as visual effects, color accuracy, and structural integrity. For example, relying only on pixel-level mean square error loss may result in an image that is numerically close to the reference image, but still has obvious distortion in visual perception, such as blurred edges, texture loss, and other problems.

[0005] Insufficient use of training datasets: The diversity and complexity of full-color night vision images require the model to use a large amount of rich training data during training to learn a wide range of image features and patterns. However, existing methods generally lack effective use of large-scale training datasets, resulting in limited generalization capabilities of the model. When faced with full-color night vision images of different scenes, lighting conditions, and objects, the model may not be able to accurately perform image enhancement, showing poor adaptability and stability.

[0006] In summary, there is an urgent need for an innovative method and system to overcome the difficulties of existing technologies in full-color night vision image processing in order to improve image quality and meet the needs of practical applications. Summary of the invention

[0007] The purpose of the present invention is to provide a method and system for improving image processing quality by using perceptual loss technology. Through perceptual loss technology, the reconstructed image can be closer to the high-quality reference image, effectively reducing problems such as noise, blur and color distortion, and improving the clarity and detail expression of the image.

[0008] To achieve the above object, the present invention provides the following solutions:

[0009] A method for improving image processing quality using perceptual loss technology comprises the following steps:

[0010] S1, acquiring image data under different illumination conditions, and preprocessing the acquired image data;

[0011] S2. Build an image enhancement model based on the deep learning algorithm, select the network architecture, and then divide the preprocessed image data into a training set and a test set;

[0012] S3, input the training set and the test set into the image enhancement model to implement model training and testing;

[0013] S4. Finally, the image data to be processed is input into the image enhancement model to enhance the image quality.

[0014] Preferably, the image data under different illumination conditions acquired in S1 includes image data under low illumination conditions and image data under normal illumination conditions.

[0015] Preferably, the image enhancement model constructed based on the deep learning algorithm in S2 specifically adopts the VGG16 model, and the VGG16 model is compressed and optimized through pruning and quantization techniques.

[0016] Preferably, the selection of the network architecture in S2 includes:

[0017] ResNet is used as the feature extraction network and image reconstruction network. The image reconstruction network adopts a structure in which multiple deconvolution layers and convolution layers are alternately connected to gradually increase the size of the image and restore a high-resolution image. Jump connections are introduced to connect features at different levels in the feature extraction network with features at corresponding levels in the image reconstruction network.

[0018] Preferably, the training of the image enhancement model in S3 includes:

[0019] The image under low-light conditions is input into the image enhancement model, features are extracted through the feature extraction network, and then the image is reconstructed by the image reconstruction network; the perceptual loss between the reconstructed image and the reference image under normal light conditions is calculated; the results of the loss function are combined using a weighted summation method; and the performance of the model is optimized by continuously adjusting the weight coefficients.

[0020] Preferably, the perceptual loss calculation method is:

[0021] First, the reconstructed image and the reference image are respectively input into the pre-trained feature extraction network to obtain the corresponding feature maps; for the feature maps of each layer, the mean square error loss is calculated:

[0022] For two feature maps F 1 and F 2 , the mean square error loss calculation formula is:

[0023]

[0024] Among them, N is the number of elements in the feature map, and i represents a layer in the feature extraction network. The mean square error losses of different layers are weighted and summed to obtain the perceptual loss. The weight coefficient is determined based on experience or through experiments to highlight the importance of features at different levels.

[0025] A system for improving image processing quality by using perceptual loss technology, and a method for improving image processing quality by using perceptual loss technology applied to any of the above items, comprising:

[0026] A data acquisition module is used to acquire image data under different illumination conditions and pre-process the acquired image data;

[0027] The model building module is used to build an image enhancement model based on a deep learning algorithm, select a network architecture, and then divide the preprocessed image data into a training set and a test set;

[0028] Model training and testing module, used to input training set and test set into image enhancement model to realize model training and testing;

[0029] The image enhancement module is used to finally input the image data to be processed into the image enhancement model to enhance the image quality.

[0030] A non-transitory computer-readable storage medium stores a computer executable program thereon, and when the computer executable program is executed by a processor, it implements any of the above-mentioned methods for improving image processing quality by using perceptual loss technology.

[0031] According to the specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0032] (1) The present invention can significantly improve the quality of full-color night vision images through carefully designed perceptual loss technology and deep learning model architecture. The reconstructed image is closer to the high-quality reference image in terms of clarity, color saturation and detail expression, effectively reducing problems such as noise, blur and color distortion. The enhanced image visual effect is more realistic, providing more reliable image data support for security monitoring, night vision enhancement and other fields.

[0033] (2) The method has wide applicability and is suitable for full-color night vision image processing tasks under various low-light conditions. A large number of tests on images of different scenes, lighting conditions, and objects show that the method can stably and effectively improve image quality and meet the needs of a variety of practical applications.

[0034] (3) Based on the deep learning algorithm, the model can automatically learn image features without manually designing complex image processing algorithms, which greatly improves processing efficiency and accuracy. At the same time, by optimizing the model and improving the training strategy, the consumption of computing resources is reduced and the practicality and scalability of the model are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0036] Figure 1 A flow chart of a method for improving image processing quality using perceptual loss technology provided by the present invention. DETAILED DESCRIPTION

[0037] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0038] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0039] like Figure 1 As shown, the present invention provides a method for improving image processing quality by using perceptual loss technology, comprising the following steps:

[0040] S1, acquiring image data under different illumination conditions, and preprocessing the acquired image data;

[0041] S2. Build an image enhancement model based on the deep learning algorithm, select the network architecture, and then divide the preprocessed image data into a training set and a test set;

[0042] S3, input the training set and the test set into the image enhancement model to implement model training and testing;

[0043] S4. Finally, the image data to be processed is input into the image enhancement model to enhance the image quality.

[0044] Specifically, S4 also includes:

[0045] The enhanced image is subjected to appropriate post-processing, such as denoising, sharpening, etc., to further enhance the visual effect of the image. Existing image post-processing algorithms, such as median filtering, Laplace sharpening, etc., may be used. In the embodiment, median filtering is used to remove noise in the image, so that the peak signal-to-noise ratio of the image is increased by 2 decibels; Laplace sharpening is used to enhance the edges and details of the image, so that the subjective evaluation score of the image is increased by 3 points.

[0046] Effect evaluation: 10 professional image processing personnel were invited to conduct subjective evaluation of the processed images. The evaluation criteria included image clarity, color accuracy, and detail richness. The evaluation results were scored on a scale of 1-100, with an average score of 80 points, indicating that the visual effect of the processed images was significantly improved. Objective indicator calculation, the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators of the image were calculated. The PSNR value of the processed image increased by 2.5 decibels compared to the original image, and the SSIM value increased by 0.15. The objective data proved the effectiveness of the method of the present invention in improving image quality.

[0047] Furthermore, the image data under different illumination conditions acquired in S1 includes image data under low illumination conditions and image data under normal illumination conditions.

[0048] Specifically, we used professional night vision image acquisition equipment to collect 20,000 low-light full-color night vision images and their corresponding high-quality daytime images as reference images in different scenes such as urban streets, rural roads and forests. During the acquisition process, we ensured the clarity, color accuracy and representativeness of the lighting conditions of the images, covering different degrees of low-light environments to enrich the diversity of the data set. We performed Gaussian noise removal on the collected images, using a 3x3 Gaussian filter kernel, and dynamically adjusted the filter parameters according to the image noise level to effectively reduce noise interference. The image pixel values ​​were normalized to the [0,1] interval to give the image data better numerical stability and consistency in subsequent processing.

[0049] Furthermore, the image enhancement model built based on the deep learning algorithm in S2 specifically adopts the VGG16 model, and the VGG16 model is compressed and optimized through pruning and quantization techniques.

[0050] Specifically, considering the computational complexity and performance of the model, VGG16 was selected as the basis for perceptual loss calculation. In the experiment, the effects of VGG16 and VGG19 in perceptual loss calculation were compared, and it was found that VGG16 has advantages in computational efficiency and accuracy.

[0051] The pre-trained model can be properly compressed and optimized to reduce the consumption of computing resources. For example, the size of the VGG16 model was compressed by 40% through techniques such as pruning and quantization, while maintaining a high accuracy.

[0052] Furthermore, the network architecture choices in S2 include:

[0053] ResNet-50 is used as the feature extraction and image reconstruction network, which contains 49 convolutional layers and residual connections. The convolution kernel size is adjusted to 3x3, and the number of some layers is optimized to improve the model's ability to extract and reconstruct full-color night vision image features. In the image reconstruction network, 3 deconvolution layers and 4 convolution layers are set to be connected alternately. The features of different levels in the ResNet feature extraction network are spliced ​​and fused with the corresponding levels of the image reconstruction network through jump connections to improve the image reconstruction quality. The ReLU activation function is selected, and the slope parameter of the ReLU is dynamically adjusted according to the model convergence during the training process. The initial slope is set to 0.1. When the model loss function no longer decreases after 50 consecutive iterations, the slope is adjusted to 0.2 to speed up the model convergence.

[0054] During the model training process, the activation function can be properly adjusted and optimized to improve the performance of the model. For example, during the training process, the slope parameter of the ReLU activation function is adjusted according to the convergence of the model to further improve the performance of the model. The learning rate is initialized to 0.0008, and an exponential decay strategy is adopted. After every 500 iterations, the learning rate decays to the original 0.8; the batch size is set to 32.

[0055] Furthermore, the training of the image enhancement model in S3 includes:

[0056] The low-light full-color night vision image is input into the model. After feature extraction and reconstruction, the perceptual loss (perceptual loss weight is set to 0.7, mean square error loss weight is set to 0.3) and structural similarity loss (weight is set to 0.1) between the reconstructed image and the reference image are calculated, and the final loss function is obtained by weighted summation. The model parameters are updated using the back propagation algorithm, and the model converges after 15,000 iterations of training. During the training process, data enhancement techniques such as random rotation (angle range is -10° to 10°) and flipping (horizontally and vertically) are used to increase the training data by 4 times, improving the generalization ability of the model.

[0057] Furthermore, the perceptual loss is calculated as:

[0058] First, the reconstructed image and the reference image are respectively input into the pre-trained feature extraction network to obtain the corresponding feature maps; for the feature maps of each layer, the mean square error loss is calculated:

[0059] For two feature maps F 1 and F 2 , the mean square error loss calculation formula is:

[0060]

[0061] Among them, N is the number of elements in the feature map, and i represents a layer in the feature extraction network. The mean square error losses of different layers are weighted and summed to obtain the perceptual loss. The weight coefficient is determined based on experience or through experiments to highlight the importance of features at different levels.

[0062] A system for improving image processing quality by using perceptual loss technology, and a method for improving image processing quality by using perceptual loss technology applied to any of the above items, comprising:

[0063] A data acquisition module is used to acquire image data under different illumination conditions and pre-process the acquired image data;

[0064] The model building module is used to build an image enhancement model based on a deep learning algorithm, select a network architecture, and then divide the preprocessed image data into a training set and a test set;

[0065] Model training and testing module, used to input training set and test set into image enhancement model to realize model training and testing;

[0066] The image enhancement module is used to finally input the image data to be processed into the image enhancement model to enhance the image quality.

[0067] A non-transitory computer-readable storage medium stores a computer executable program thereon, and when the computer executable program is executed by a processor, it implements any of the above-mentioned methods for improving image processing quality by using perceptual loss technology.

[0068] In a specific embodiment, 4,000 low-light full-color night vision images under different scenes and lighting conditions are collected from multiple night vision devices, and corresponding high-quality reference images are obtained. These images contain different types of objects and complex lighting environments, further increasing the diversity of training data. The data is preprocessed and optimized. In the denoising process, an adaptive median filtering algorithm is used to automatically adjust the filter window size according to the local noise characteristics of the image, so as to better remove noise while retaining image details. The image is preprocessed for color correction, and the color channel is adjusted according to the color temperature information of the image to reduce color cast and make the image color closer to the real scene.

[0069] Adjust the structure and parameters of the deep learning model, and try different combinations of network architectures and loss functions to improve the image processing effect. For example, try using ResNet-101 as the feature extraction network and increase the network depth to further improve the feature extraction capability. At the same time, adjust the parameters of the deconvolution layer and convolution layer in the image reconstruction network, such as increasing the step size of the deconvolution layer to 2, increase the image magnification and optimize the image detail restoration effect. Change the feature fusion method, use weighted fusion instead of splicing fusion, set weights according to the importance of features at different levels, and determine the optimal weight distribution after multiple experiments to further improve the image reconstruction quality.

[0070] The activation function was changed to Leaky ReLU (with a slope of 0.01), and the slope value was gradually increased during the training process, increasing by 0.01 every 1000 iterations until the slope reached 0.1, and the changes in model performance were observed. Finally, the best application method of the LeakyReLU activation function in this embodiment was determined.

[0071] Model training optimization: adjust the training parameters, initialize the learning rate to 0.001, adopt the step decay strategy, after every 1000 iterations, the learning rate decays to the original 0.9; the batch size is set to 16. During the training process, a training process similar to Example 1 is adopted to calculate the perceptual loss (the perceptual loss weight is set to 0.6, the mean square error loss weight is set to 0.3, and the structural similarity loss weight is set to 0.1), and the model parameters are updated by back propagation. After 20,000 iterations of training, the model converged. During the training process, a data enhancement method of random cropping (cropping ratio of 0.8-1.0) was added to enrich the diversity of training data and improve the adaptability of the model to different image contents and structures.

[0072] Image enhancement and subjective evaluation: The newly collected low-light full-color night vision images were enhanced using the improved model and parameter settings. Eight professionals were then invited to conduct a visual evaluation, with the evaluation criteria including overall image quality, visual comfort, and object recognizability. The evaluation results showed an average score of 75 points, a significant improvement over the original images, indicating that the processed images were more in line with the expectations of professionals in terms of visual effects.

[0073] Objective index calculation: The PSNR and SSIM indexes of the processed image were calculated. The PSNR value increased by 2 decibels and the SSIM value increased by 0.12. At the same time, the information entropy of the image was calculated to evaluate the image information richness. The information entropy of the processed image increased by 0.3 bits / pixel compared with the original image, further proving the effectiveness of the method of the present invention in improving image quality and retaining image information. According to the evaluation results, the model parameters were further fine-tuned, such as adjusting the feature fusion weights and the activation function slope parameters, so that the image processing effect was continuously optimized.

[0074] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, or of course by hardware. Based on this understanding, the above technical solution, in essence or in other words, the part that contributes to the prior art, can be embodied in the form of a software product, which can be stored in the embedded memory of the SoC, in the computer-readable storage medium, and includes several instructions for a computer device to execute the methods described in each embodiment or some parts of the embodiment.

[0075] The principles and implementation methods of the present invention are described in this article using specific examples. The description of the above embodiments is only used to help understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, according to the idea of ​​the present invention, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims

1. A method for improving image processing quality using perceptual loss technology, characterized in that: The following steps are involved: S1, acquiring image data under different illumination conditions, and preprocessing the acquired image data; S2. Build an image enhancement model based on the deep learning algorithm, select the network architecture, and then divide the preprocessed image data into a training set and a test set; S3, input the training set and the test set into the image enhancement model to implement model training and testing; S4. Finally, the image data to be processed is input into the image enhancement model to enhance the image quality.

2. The method for improving image processing quality using perceptual loss technology according to claim 1, characterized in that: The image data under different illumination conditions acquired in S1 includes image data under low illumination conditions and image data under normal illumination conditions.

3. The method for improving image processing quality using perceptual loss technology according to claim 1, characterized in that: The image enhancement model constructed based on the deep learning algorithm in S2 specifically adopts the VGG16 model, and the VGG16 model is compressed and optimized through pruning and quantization techniques.

4. The method for improving image processing quality using perceptual loss technology according to claim 1, characterized in that: The network architecture choices in S2 include: ResNet is used as a feature extraction network and an image reconstruction network. The image reconstruction network adopts a structure in which multiple deconvolution layers and convolution layers are alternately connected to gradually increase the size of the image and restore a high-resolution image. Jump connections are introduced to connect features at different levels in the feature extraction network with features at corresponding levels in the image reconstruction network.

5. The method for improving image processing quality using perceptual loss technology according to claim 1, characterized in that: Training of the image enhancement model in S3 includes: An image under low light conditions is input into the image enhancement model, features are extracted through a feature extraction network, and then the image is reconstructed by an image reconstruction network; the perceptual loss between the reconstructed image and the reference image under normal light conditions is calculated; the results of the loss function are combined by weighted summation; and the performance of the model is optimized by continuously adjusting the weight coefficients.

6. The method for improving image processing quality using perceptual loss technology according to claim 5, characterized in that: The perceptual loss is calculated as: First, the reconstructed image and the reference image are respectively input into the pre-trained feature extraction network to obtain the corresponding feature maps; for the feature maps of each layer, the mean square error loss is calculated: For the two feature maps F1 and F2, the mean square error loss calculation formula is: Among them, N is the number of elements in the feature map, and i represents a layer in the feature extraction network. The mean square error losses of different layers are weighted and summed to obtain the perceptual loss. The weight coefficient is determined based on experience or through experiments to highlight the importance of features at different levels.

7. A system for improving image processing quality using perceptual loss technology, applied to execute the method for improving image processing quality using perceptual loss technology as claimed in any one of claims 1 to 6, characterized in that: include: A data acquisition module is used to acquire image data under different illumination conditions and pre-process the acquired image data; The model building module is used to build an image enhancement model based on a deep learning algorithm, select a network architecture, and then divide the preprocessed image data into a training set and a test set; Model training and testing module, used to input training set and test set into image enhancement model to realize model training and testing; The image enhancement module is used to finally input the image data to be processed into the image enhancement model to enhance the image quality.

8. A non-transitory computer-readable storage medium having a computer-executable program stored thereon, characterized in that: When the computer executable program is executed by the processor, it can implement the method for improving image processing quality by using perceptual loss technology as described in any one of claims 1 to 6.

Images