A lightweight image just-perceptible distortion prediction method based on reversible networks

Abstract

The present invention provides a lightweight image just-perceptible distortion prediction method based on a reversible network, comprising: step S1, constructing a training set and a test set, and preprocessing the training set to obtain multiple Y-channel images; step S2, building an image just-perceptible distortion threshold prediction network to sequentially perform feature enhancement, discrete wavelet transform, high- and low-frequency image extraction, high-frequency modulation, high- and low-frequency image inversion, discrete wavelet transform, and feature enhancement on the Y-channel image to obtain a critical perceptual lossless image; step S3, training the image just-perceptible distortion threshold prediction network using the training set according to step S2 to obtain an image just-perceptible distortion threshold prediction model; step S4, testing the test set and outputting an absolute difference map between the critical perceptual lossless image and the Y-channel image as a just-perceptible distortion threshold map. The beneficial effect is that the present invention can significantly reduce the cost of just-perceptible distortion threshold annotation and achieve high-quality pixel-level just-perceptible distortion threshold prediction.

Description

Technical Field

[0001] The present invention relates to the technical field of image distortion prediction, and in particular to a lightweight image just-perceptible distortion prediction method based on a reversible network. Background Art

[0002] In the fields of psychology and perceptual science, Just Noticeable Difference (JND) is an important concept used to describe the smallest stimulus change that the human visual system can perceive. This concept was first proposed by German psychologist Gustav Fechner in his psychophysics research and has become one of the foundations of understanding human perceptual abilities.

[0003] Studying just-perceptible distortion can help us better understand how humans perceive changes in visual information, and thus optimize applications such as image compression, image enhancement, and information hiding. In recent years, deep learning technology has made breakthrough progress, and models represented by convolutional neural networks (CNNs) have demonstrated outstanding performance in the field of computer vision.

[0004] The prediction of just-perceptible distortion thresholds currently faces two major challenges: manual annotation of pixel-level just-perceptible distortion thresholds is highly subjective and costly, and there is a lack of adapted end-to-end training datasets. To address this technical bottleneck, some recent studies have adopted a compromise strategy, modeling the just-perceptible distortion threshold prediction as a binary classification task. However, these image-level just-perceptible distortion threshold predictors cannot provide pixel-level just-perceptible distortion thresholds, resulting in reduced practicality of the prediction results. Pixel-level just-perceptible distortion thresholds are essentially necessary in various perceptual image processing applications. Therefore, there is an urgent need to study a method that can reduce annotation costs and achieve lightweight pixel-level just-perceptible distortion prediction. Summary of the Invention

[0005] The technical problem to be solved by the present invention is how to significantly reduce the cost of labeling the just-noticeable distortion threshold and realize lightweight pixel-level just-noticeable distortion threshold prediction. In order to overcome the defects of the above-mentioned existing technologies (or related technologies), the present invention provides a lightweight image just-noticeable distortion prediction method based on a reversible network.

[0006] The present invention provides a method for predicting just perceptible distortion of a lightweight image based on a reversible network, comprising the following steps:

[0007] Step S1, obtaining multiple original images and inputting them into a high dynamic range visual difference predictor to construct a training set and a test set, and preprocessing the training set to obtain multiple Y channel images;

[0008] Step S2: Use a deep learning framework to build an image just perceptible distortion threshold prediction network, and perform feature enhancement, discrete wavelet transform, and high and low frequency image extraction on the Y channel image through the image just perceptible distortion threshold prediction network to obtain a low frequency feature map. and high-frequency feature maps , and the high frequency feature map Perform high-frequency modulation to obtain high-frequency characteristic graph , and the low-frequency feature map And the high frequency feature map The high- and low-frequency image inversion, discrete wavelet transform, and feature enhancement are performed in sequence to obtain a critical perception lossless image;

[0009] Step S3, training the image just perceptible distortion threshold prediction network using the training set according to step S2 to obtain an image just perceptible distortion threshold prediction model;

[0010] Step S4: Testing the test set using the image just-perceptible distortion threshold prediction model to obtain a critical perceptual lossless image corresponding to the test set, and obtaining an absolute difference map based on the pairwise corresponding critical perceptual lossless image and Y channel image of the test set as a predicted just-perceptible distortion threshold map.

[0011] Compared with the existing technology, the lightweight image just perceptible distortion prediction method based on reversible network has the following advantages:

[0012] In the present invention, the just noticeable distortion threshold is redefined as the absolute difference between the original Y channel image and its critically perceptually lossless image, which provides a new idea for the prediction of the pixel-level just noticeable distortion threshold. The high dynamic range visual difference predictor is used to predict the pseudo-label partitioning to generate the training set and test set, which effectively solves the high labeling cost problem faced when large-scale original images are used as real labels. By building an image just noticeable distortion threshold prediction network to focus on the modulation of high-frequency information, based on the reversible characteristics of the image just noticeable distortion threshold prediction network, through a forward processing process including feature enhancement, discrete wavelet transform, high- and low-frequency image extraction and a reverse processing process including high- and low-frequency image inversion, discrete wavelet transform, and feature enhancement, the corrected high-frequency information is fused with the intact low-frequency information to obtain the critically perceptually lossless image, thereby achieving high-quality image prediction.

[0013] In one possible implementation, in step S1, a Kalhonen-Loev transform is performed on each original image to generate degraded versions of images to varying degrees, and the high dynamic range visual difference predictor is used to predict the perceptual distortion probability between each original image and the corresponding degraded version of the image. The degraded versions of the images with a perceptual distortion probability of 0.75 are selected to form the training set and the test set.

[0014] In a possible implementation, in step S1, the process of preprocessing the training set includes:

[0015] Each of the degraded version images in the training set is uniformly cropped with a fixed size of 256×256 pixels to obtain a cropped image, and then each of the cropped images is converted into a YUV color space and the Y channel brightness component thereof is extracted to obtain the Y channel image.

[0016] In a possible implementation, the image perceptible distortion threshold prediction network constructed in step S2 includes a feature enhancement module, a discrete wavelet transform module, a reversible module, and a high-frequency modulation module connected in sequence, and the feature enhancement module receives the Y channel image to perform feature enhancement to obtain a feature map. , the feature map is transformed into Decompose into low-frequency feature maps and high-frequency feature maps , the low-frequency feature map is reversibly And the high frequency feature map Extract high and low frequency images to obtain the low frequency feature map And the high frequency feature map , the high frequency characteristic map is modulated by the high frequency modulation module Modulation is performed to obtain the high frequency characteristic graph , the low-frequency feature map is reversibly And the high frequency feature map Perform high- and low-frequency image inversion to obtain low-frequency feature maps and high-frequency feature maps , the low-frequency feature map is transformed by the discrete wavelet transform module And the high frequency feature map Perform inverse discrete wavelet transform to obtain the reconstructed feature map , the feature map is enhanced by the feature enhancement module Feature enhancement is performed to obtain the critical perception lossless image.

[0017] In a possible implementation, the feature enhancement module includes a first dense block, a first convolutional layer, and a first splicing layer connected in sequence, wherein the first dense block includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer connected in sequence, and the feature map is obtained by receiving the Y channel image through the first convolutional layer and performing a convolution operation. And the feature map Add the Y channel image to obtain the feature map , receiving the feature map through the second convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the third convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the fourth convolutional layer Perform convolution operation to obtain feature map , the feature map is transformed into and the feature map Perform channel addition and set the output channel to 1 through the first convolutional layer to obtain the feature map , the feature map is transformed into Perform residual connection with the Y channel image to obtain the feature map .

[0018] In a possible implementation, the feature enhancement module in step S2 includes a second dense block, a second convolutional layer, and a second convolutional layer connected in sequence, the second dense block includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer connected in sequence, and the feature map is received through the first convolutional layer. Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the second convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the third convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the fourth convolutional layer Perform convolution operation to obtain feature map , the feature map is transformed into and the feature map Perform channel addition and set the output channel to 1 through the second convolutional layer to obtain the feature map , the feature map is transformed into and the feature map Perform residual connection to obtain the critical perceptual lossless image.

[0019] In a possible implementation, the reversible module includes four reversible blocks connected in sequence, and the low-frequency feature map is reversibly converted by the four reversible blocks. And the high frequency feature map Extract high and low frequency images in sequence to obtain the low frequency feature map And the high frequency feature map ;or

[0020] The low-frequency feature map is processed by four reversible blocks. And the high frequency feature map Invert the high and low frequency images in sequence to obtain the low frequency feature map And the high frequency feature map .

[0021] In a possible implementation, the high frequency modulation module includes a first convolution layer, a second convolution layer, and a third convolution layer connected in sequence, and the high frequency feature map is modulated by the first convolution layer. Perform the first modulation to obtain the high-frequency characteristic map , through the second convolutional layer to the high-frequency feature map Perform the second modulation to obtain the high-frequency feature map , through the third convolutional layer to the high-frequency feature map Perform the third modulation to obtain the high frequency characteristic graph .

[0022] In one possible implementation, the convolution kernel size of the first convolution layer of the high-frequency modulation module is 1, the padding size is 0, the number of input channels is 3, and the number of output channels is 128; the convolution kernel size of the second convolution layer is 3, the padding size is 1, the number of input channels is 128, and the number of output channels is 128; the convolution kernel size of the third convolution layer is 1, the padding size is 0, the number of input channels is 128, and the number of output channels is 3. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 is a flow chart of the steps of the present invention;

[0024] Figure 2 Schematic diagram of the framework of the image just perceptible distortion threshold prediction network of the present invention;

[0025] Figure 3 Schematic diagram of the framework of the dense block in the feature enhancement module of the present invention;

[0026] Figure 4 Schematic diagram of the prediction results of the existing just-perceptible distortion threshold prediction methods Yang05, Wu13, Wu17, and Jakh18;

[0027] Figure 5 Schematic diagram of the prediction effects of the method of the present invention and the existing just-perceptible distortion threshold prediction methods Chen19, Wang21, and Jiang22. DETAILED DESCRIPTION

[0028] First, those skilled in the art should understand that these embodiments are merely for explaining the technical principles of the embodiments of the present invention and are not intended to limit the scope of protection of the embodiments of the present invention. Those skilled in the art may make adjustments as needed to adapt to specific application scenarios.

[0029] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0030] See also Figure 1 The embodiment of the present invention discloses a lightweight image perceptible distortion prediction method based on a reversible network, comprising the following steps:

[0031] Step S1, obtaining multiple original images and inputting them into a high dynamic range visual difference predictor to construct a training set and a test set, and preprocessing the training set to obtain multiple Y channel images;

[0032] Step S2: Use the deep learning framework to build an image just perceptible distortion threshold prediction network, and perform feature enhancement, discrete wavelet transform, and high and low frequency image extraction on the Y channel image through the image just perceptible distortion threshold prediction network to obtain a low frequency feature map. and high-frequency feature maps , and the high frequency feature map Perform high-frequency modulation to obtain high-frequency characteristic graph , and the low-frequency feature map and high-frequency feature maps The high- and low-frequency image inversion, discrete wavelet transform, and feature enhancement are performed in sequence to obtain a critical perception lossless image;

[0033] Step S3, training the image just perceptible distortion threshold prediction network using the training set according to step S2 to obtain an image just perceptible distortion threshold prediction model;

[0034] Step S4: Testing the test set using the image just-perceptible distortion threshold prediction model to obtain a critical perceptual lossless image corresponding to the test set, and obtaining an absolute difference map based on the pairwise corresponding critical perceptual lossless image and Y channel image of the test set as a predicted just-perceptible distortion threshold map.

[0035] In an embodiment of the present application, a pixel-level just-noticeable distortion threshold dataset is constructed in step S1. In order to solve the problem that manual labeling of pixel-level just-noticeable distortion thresholds is highly subjective and costly, a high dynamic range visual difference predictor that can predict the probability of perceptual distortion based on an image pair containing an original image and a distorted image is introduced to provide pseudo labels as a training set and a validation set, and a portion of the original images is manually labeled as a test set.

[0036] In the embodiment of the present application, in order to address the two core defects of the traditional manual labeling method: the inconsistency of labeling caused by subjective perception differences and the high time cost required for pixel-level labeling, a high dynamic range visual difference predictor is innovatively introduced. The high dynamic range visual difference predictor simulates the multi-channel perception mechanism of the human visual system, combines the adaptive contrast sensitivity function with the brightness perception nonlinear compensation algorithm, and can predict the pixel-level visual difference probability (range 0-1) of the input original image-distorted image pair, and generate pseudo JND labeling data with perceptual consistency. In the specific implementation process, each original image is subjected to the Kalhonen-Loev transform (KLT transform) to generate 256 degraded versions of images with different degrees of consistency, and uses the high dynamic range visual system to generate 256 degraded versions of images with different degrees of consistency. The perceptual difference predictor predicts the perceptual distortion probability between the original image and its degraded version in turn, and selects the image with a perceptual distortion probability of 0.75 (i.e., a 75% probability of being predicted as distorted) as the critical perceptual lossless image corresponding to the original image. Subsequently, the high dynamic range visual difference predictor is used to batch process 8,965 groups of critical perceptual lossless images to construct a large-scale pseudo-annotated training set (70%) and validation set (10%), while retaining 20% of manually annotated real data as a test set to ensure that the image just perceptible distortion threshold prediction model has reliable real-world generalization capabilities while maintaining the advantages of automation. This hybrid annotation strategy not only significantly improves data preparation efficiency, but also enhances the robustness of the image just perceptible distortion threshold prediction model.

[0037] In the embodiment of the present application, since the input of the deep neural network usually has a fixed form, it is necessary to preprocess the critical perception lossless images in the test set. Specifically, the critical perception lossless images are uniformly cropped into , to ensure that the image input to the deep neural network has consistent dimensions. Subsequently, the critical-perceptual lossless image in RGB format is converted to YUV color space, and the brightness component (Y channel) is extracted as the input feature of the neural network. This processing method not only reduces the computational complexity, but also retains the main structural information of the critical-perceptual lossless image, which is conducive to subsequent feature learning and model training.

[0038] In the embodiment of the present application, in step S2, a deep learning framework is used to build an image perceptible distortion threshold prediction network based on a reversible network, such as Figure 2 As shown in FIG, the image just perceptible distortion threshold prediction network includes a feature enhancement module, a discrete wavelet transform module, a reversible module and a high-frequency modulation module, and has two processes: forward processing and inverse processing.

[0039] In the embodiment of the present application, the feature enhancement modules are distributed at the entrance and exit of the image perceptible distortion threshold prediction network. The input image in the forward processing process is The feature enhancement module consists of a first dense block and a first convolutional layer. There are 4 convolutional layers in the first dense block. Each layer adds 16 channels, and the number of output channels of the previous layer is added to the number of input channels of the current layer. Here, the feature addition operation is a conventional operation in the convolutional neural network. Suppose the Y channel image is , the feature map output by the first convolutional layer in the first dense block is , the feature maps of the second convolutional layer input and output are and , the feature maps of the third convolutional layer input and output are and , the feature maps of the fourth convolutional layer input and output are and After the first dense block is processed, the number of channels is adjusted to 1 through the first convolution layer (consistent with the number of output channels of the residual connection). The output channel number of this convolution operation is 1, and the feature map is obtained. Finally, in the first concatenation layer, the Y channel image The identity mapping is used to add features and the Y channel image And the feature map output after feature enhancement Add to obtain feature map ; Since the number of input and output channels matches, the residual connection does not change the size. Specifically, Figure 3 As shown, the Y channel image The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , in this embodiment .

[0040] In the embodiment of the present application, the complete reversibility of the discrete wavelet transform module is its important characteristic, which ensures the lossless transmission of information of the feature map during the forward transformation and inverse transformation process. This characteristic makes it of great value in many image processing tasks. In the embodiment of the present invention, the discrete wavelet transform module can convert the feature map into Decompose into low-frequency feature maps and high-frequency feature maps , where the low-frequency feature map The size is , high-frequency feature map The size is .

[0041] In the embodiment of the present application, the input of the reversible module is the low-frequency feature map output by the discrete wavelet transform module. and high-frequency feature maps , the output is a low-frequency feature map and high-frequency feature maps ,The reversible module contains 4 reversible blocks, each reversible block contains multiple convolution blocks, and each convolution block consists of two convolution layers composed of convolution and activation function ReLU, such as Figure 2 As shown, taking the first reversible block of the forward processing as an example, the forward processing process is as follows:

[0042] High-frequency feature map of the input After the convolution block Get feature map , after the convolution block Get feature map , then the feature map and low-frequency feature maps Multiply to obtain feature map , then, the feature map With feature map Add to obtain the low-frequency feature map output by the first reversible block ; To prevent the gradient from disappearing or exploding during training, the low-frequency feature map Perform Sigmoid activation operation to obtain high-frequency feature maps ; High-frequency feature map of the input After the convolution block Get feature map , after the convolution block Get feature map , then the feature map and high-frequency feature maps Multiply to obtain feature map , then, the feature map With feature map Add to obtain the high-frequency feature map output by the first reversible block ;

[0043] Then, the low-frequency feature map and high-frequency feature maps Input into the following three reversible blocks with the same structure but different parameters, that is, after a total of 4 reversible block operations, the low-frequency feature map is output and high-frequency feature maps , where the feature map , feature map , feature map , low-frequency feature map , high-frequency feature map , low-frequency feature map The size is , feature map , feature map , feature map , high-frequency feature map , high-frequency feature map The size is .

[0044] In the embodiment of the present application, the high-frequency modulation module is composed of three cascaded convolutional layers, which is responsible for the high-frequency feature map output by the reversible module. Modulation is performed to potentially learn the connection between the perceptible distortion threshold and high-frequency information. Specifically, the high-frequency feature map Input into the first convolutional layer to obtain high-frequency feature maps , input into the second convolutional layer to obtain high-frequency feature maps , and finally input into the third convolutional layer to obtain the modulated high-frequency feature map , where the high-frequency feature map The size is , high-frequency feature map The size is , high-frequency feature map The size is .

[0045] In an embodiment of the present application, the high-frequency modulation module is composed of three cascaded convolution layers with convolution kernels of 1, 3, and 1. Specifically, the convolution kernel size of the first convolution layer is 1, the padding size is 0, the number of input channels is 3, and the number of output channels is 128; the convolution kernel size of the second convolution layer is 3, the padding size is 1, the number of input channels is 128, and the number of output channels is 128; the convolution kernel size of the third convolution layer is 1, the padding size is 0, the number of input channels is 128, and the number of output channels is 3.

[0046] In the embodiment of the present application, after completing all the forward propagation processes, the modulated high-frequency feature map and low-frequency feature maps As the input of the reverse processing of the reversible module, taking the first reversible block of the reverse processing as an example, the reverse processing process is as follows:

[0047] Low-frequency feature map of the input After the convolution block Get feature map , after the convolution block Get feature map , followed by the low-frequency feature map Subtract feature map Get feature map , and then divided by the feature map Obtain high-frequency feature maps ; To prevent the gradient from disappearing or exploding during training, the low-frequency feature map Perform Sigmoid activation operation to obtain high-frequency feature maps ; High-frequency feature map of the input After the convolution block Get feature map , after the convolution block Get feature map , followed by high-frequency feature maps Subtract feature map Get feature map , and then divided by the feature map Obtain low-frequency feature maps ;

[0048] Similarly, the low-frequency feature map and high-frequency feature maps As the input of the next reversible block, it is input into the following three reversible blocks with the same structure but different parameters, that is, after a total of 4 reversible block inversion operations, the low-frequency feature map is output. and high-frequency feature maps , where the feature map , feature map , feature map , low-frequency feature map The size is , feature map , feature map , feature map , high-frequency feature map , high-frequency feature map The size is ; Then the low frequency feature map and high-frequency feature maps The reconstructed feature map is obtained by inverse transformation through the discrete wavelet transform module Finally, the predicted critical perception lossless image is obtained through the feature enhancement module at the export , where the feature map and criticality-aware lossless images The size is .

[0049] In the embodiment of the present application, the input image of the feature enhancement module during the inverse processing is Feature map of size , the feature enhancement module includes a second dense block and a second convolutional layer. There are 4 convolutional layers in the second dense block. Each layer adds 16 channels, and the number of output channels of the previous layer is added to the number of input channels of the current layer. Suppose the feature map output by the first convolutional layer in the second dense block is , the feature maps of the second convolutional layer input and output are and , the feature maps of the third convolutional layer input and output are and , the feature maps of the fourth convolutional layer input and output are and After the second dense block is processed, the number of channels is adjusted to 1 through the second convolution layer (consistent with the number of output channels of the residual connection). The output channel number of this convolution operation is 1, and the feature map is obtained. Finally, in the second concatenation layer, The identity mapping is used to add features and add the feature map And the feature map output after feature enhancement Add to obtain critical perceptual lossless image ; Since the number of input and output channels matches, the residual connection does not change the size, where the feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , feature map The size is , in this embodiment .

[0050] In the embodiment of the present application, in step S3, the training set is used to train the image perceptual distortion threshold prediction network based on the reversible network. After each round of training, the output critical perceptual lossless image is obtained. And the input Y channel image Calculate the loss of the network, recorded as , , MSE represents the mean squared error loss, the initial learning rate of the SGD optimizer is 0.01, and the batch size is 16 for network parameter adjustment.

[0051] In an embodiment of the present application, in step S3, the image just noticeable distortion threshold prediction network is trained for a total of 200 rounds according to the process of step S2. At the 100th round, the learning rate of the SGD optimizer is decayed to 0.001, and finally the image just noticeable distortion threshold prediction network model based on the reversible network is trained.

[0052] In the embodiment of the present application, the first dense block and the second dense block in the feature enhancement module both have 4 layers of convolution blocks, each layer adds 16 channels, and the number of output channels of the previous layer is added to the number of input channels of the current layer, and finally a convolution block restores the number of channels to 1. Specifically, the convolution kernel size of the first layer convolution block is 3, the padding size is 1, the number of input channels is 1, and the number of output channels is 16. The number of output channels of the first layer is added to the number of input channels of the current layer to obtain 17 input channels of the second layer; the convolution kernel size of the second layer convolution block is 3, the padding size is 1, the number of input channels is 17, the number of output channels is 16, and the number of output channels of the second layer is 16. Adding it to the number of input channels of the current layer, the number of input channels of the third layer is 33; the convolution kernel size of the third layer convolution block is 3, the padding size is 1, the number of input channels is 33, and the number of output channels is 16. The number of output channels of the third layer is added to the number of input channels of the current layer, and the number of input channels of the fourth layer is 49; the convolution kernel size of the fourth layer convolution block is 3, the padding size is 1, the number of input channels is 49, and the number of output channels is 16. The convolution kernel size of the last layer convolution block is 1, the padding size is 0, the number of input channels is 16, and the number of output channels is 1. Finally, the output features of the last layer convolution block are added to the input features to obtain the final output.

[0053] In an embodiment of the present application, the discrete wavelet transform module can decompose the input feature map into low-frequency and high-frequency signal components. For 2D images, the discrete wavelet transform is mainly decomposed into the following four sub-bands: a low-frequency part, a horizontal high-frequency part, a vertical high-frequency part, and a diagonal high-frequency part. In an embodiment of the present invention, the Haar wavelet basis is used to perform wavelet transformation on the feature map output by the feature enhancement module to obtain the low-frequency part, the horizontal high-frequency part, the vertical high-frequency part, and the diagonal high-frequency part of the feature map, thereby generating a low-frequency feature map and a high-frequency feature map.

[0054] In the embodiment of the present application, the reversible module uses 4 reversible blocks, each of which contains 4 convolution blocks. Since the four reversible blocks have the same structure but different weights, only the specific structure of one reversible block is explained in detail. The convolution blocks mentioned above are , convolutional blocks , convolutional blocks , convolutional blocks Contains two convolutional layers, each of which is connected to a ReLU activation function. Specifically, the convolution block The convolution kernel size of the first convolution layer is 1, the padding size is 0, the number of input channels is 3, and the number of output channels is 64; the convolution block The convolution kernel size of the second convolution layer is 1, the padding size is 0, the number of input channels is 64, and the number of output channels is 1; the convolution block The convolution kernel size of the first convolution layer is 1, the padding size is 0, the number of input channels is 3, and the number of output channels is 64; the convolution block The convolution kernel size of the second convolution layer is 1, the padding size is 0, the number of input channels is 64, and the number of output channels is 1; the convolution block The convolution kernel size of the first convolution layer is 1, the padding size is 0, the number of input channels is 1, and the number of output channels is 64; the convolution block The convolution kernel size of the second convolution layer is 1, the padding size is 0, the number of input channels is 64, and the number of output channels is 3; the convolution block The convolution kernel size of the second convolution layer is 1, the padding size is 0, the number of input channels is 1, and the number of output channels is 64; the convolution block The convolution kernel size of the second convolutional layer is 1, the padding size is 0, the number of input channels is 64, and the number of output channels is 3.

[0055] In order to further verify the feasibility and effectiveness of the method of the present invention, the following experiments were conducted on the method of the present invention:

[0056] The experiment directly selected the pixel-level just-perceptible distortion threshold dataset constructed by the method of the present invention. To further verify the generalization ability of the image just-perceptible distortion threshold prediction model, this experiment further tested it on the existing visually lossless threshold VLT dataset (Mikhailiuk et al., 2021);

[0057] First, the performance of the image just-perceptible distortion threshold prediction model proposed by the method of the present invention in predicting the just-perceptible distortion threshold is evaluated. Table 1 below summarizes the experimental results of the image just-perceptible distortion threshold prediction model on the just-perceptible distortion threshold dataset test set proposed by the method of the present invention:

[0058] Table 1 Root mean square error results of the image just perceptible distortion threshold prediction model in the method of the present invention

[0059] ;

[0060] The images are then classified by scene type. Several key observations are worth noting. Notably, our method achieves the lowest root mean square error across all scene categories and the entire test set, demonstrating state-of-the-art performance. This demonstrates the effectiveness of constructing pseudo-labeled critical perceptually lossless images for training and emphasizes the importance of focusing on high-frequency information in predicting the just-perceptible distortion threshold.

[0061] To further evaluate the generalization ability of the image perceptible distortion threshold prediction model, its performance was tested on the VLT dataset. The experimental results are shown in Table 2 below:

[0062] Table 2 Root mean square error results of the method of the present invention on the VLT dataset

[0063] ;

[0064] As shown in Table 2 above, in this experiment, the image just-perceptible distortion threshold prediction model proposed by the present invention was trained solely on the pseudo-labels proposed by the present invention and then tested directly on the VLT dataset without any fine-tuning. Although the image just-perceptible distortion threshold prediction model was trained entirely on pseudo-label data, it surpassed all competing methods in 5 of the 20 test images and achieved the second-best overall average performance. These results demonstrate that the image just-perceptible distortion threshold prediction model proposed by the present invention has strong generalization capabilities and can effectively adapt to unknown data from different image datasets.

[0065] In addition to directly comparing the prediction performance, the effectiveness of the image just-perceptible distortion threshold prediction model can also be indirectly verified by evaluating its noise hiding ability. For a given image, the image just-perceptible distortion threshold prediction model is first used to calculate its corresponding visibility just-perceptible distortion threshold. Then, based on the calculated just-perceptible distortion threshold as a regulation factor for noise embedding, Gaussian white noise of the same intensity (peak signal-to-noise ratio equal to 26dB) is injected into the Y channel image, and a visual quality comparison analysis is performed. Figure 4 and Figure 5A schematic diagram of the prediction effects of the method of the present invention and existing just-noticeable distortion threshold prediction methods (Yang05, Wu13, Wu17, Jakh18, Chen19, Wang21, Jiang22) is shown, as well as the corresponding noise-injected images guided by each method. The results show that the image just-noticeable distortion threshold prediction model proposed by the method of the present invention outperforms other comparison methods and exhibits more superior performance.

[0066] In the description of the present invention, the reference terms "one embodiment", "some embodiments", "in the present embodiment", "specific examples", or "some examples" mean that the specific features, mechanisms, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, mechanisms, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0067] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the present invention should be included in the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be based on the scope of protection of the claims.

Claims

1. A lightweight image perceptible distortion prediction method based on a reversible network, characterized by: The following steps are involved: Step S1, obtaining multiple original images and inputting them into a high dynamic range visual difference predictor to construct a training set and a test set, and preprocessing the training set to obtain multiple Y channel images; Step S2: Use a deep learning framework to build an image just perceptible distortion threshold prediction network, and perform feature enhancement, discrete wavelet transform, and high and low frequency image extraction on the Y channel image through the image just perceptible distortion threshold prediction network to obtain a low frequency feature map. and high-frequency feature maps , and the high frequency feature map Perform high-frequency modulation to obtain high-frequency characteristic graph , and the low-frequency feature map And the high frequency feature map The high- and low-frequency image inversion, discrete wavelet transform, and feature enhancement are performed in sequence to obtain a critical perception lossless image; Step S3, training the image just perceptible distortion threshold prediction network using the training set according to step S2 to obtain an image just perceptible distortion threshold prediction model; Step S4: Testing the test set using the image just-perceptible distortion threshold prediction model to obtain a critical perceptual lossless image corresponding to the test set, and obtaining an absolute difference map based on the pairwise corresponding critical perceptual lossless image and Y channel image of the test set as a predicted just-perceptible distortion threshold map.

2. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 1, characterized in that: In step S1, a Kalhonen-Loev transform is performed on each original image to generate degraded versions of images to varying degrees, and the high dynamic range visual difference predictor is used to predict the perceptual distortion probability between each original image and the corresponding degraded version of the image. The degraded versions of the images with a perceptual distortion probability of 0.75 are selected to form the training set and the test set.

3. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 2, characterized in that: In step S1, the process of preprocessing the training set includes: Each of the degraded version images in the training set is uniformly cropped with a fixed size of 256×256 pixels to obtain a cropped image, and then each of the cropped images is converted into a YUV color space and the Y channel brightness component thereof is extracted to obtain the Y channel image.

4. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 1, wherein: The image perceptible distortion threshold prediction network constructed in step S2 includes a feature enhancement module, a discrete wavelet transform module, a reversible module and a high-frequency modulation module connected in sequence, and the feature enhancement module receives the Y channel image to perform feature enhancement to obtain a feature map. , the feature map is transformed into Decompose into low-frequency feature maps and high-frequency feature maps , the low-frequency feature map is reversibly And the high frequency feature map Extract high and low frequency images to obtain the low frequency feature map And the high frequency feature map , the high frequency characteristic map is modulated by the high frequency modulation module Modulation is performed to obtain the high frequency characteristic graph , the low-frequency feature map is reversibly And the high frequency feature map Perform high- and low-frequency image inversion to obtain low-frequency feature maps and high-frequency feature maps , the low-frequency feature map is transformed by the discrete wavelet transform module And the high frequency feature map Perform inverse discrete wavelet transform to obtain the reconstructed feature map , the feature map is enhanced by the feature enhancement module Feature enhancement is performed to obtain the critical perception lossless image.

5. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 4, characterized in that: The feature enhancement module includes a first dense block, a first convolutional layer and a first splicing layer connected in sequence, wherein the first dense block includes a first convolutional layer, a second convolutional layer, a third convolutional layer and a fourth convolutional layer connected in sequence, and the feature map is obtained by receiving the Y channel image through the first convolutional layer and performing a convolution operation. And the feature map Add the Y channel image to obtain the feature map , receiving the feature map through the second convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the third convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the fourth convolutional layer Perform convolution operation to obtain feature map , the feature map is transformed into and the feature map Perform channel addition and set the output channel to 1 through the first convolutional layer to obtain the feature map , the feature map is transformed into Perform residual connection with the Y channel image to obtain the feature map .

6. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 4, characterized in that: The feature enhancement module in step S2 includes a second dense block, a second convolutional layer, and a second splicing layer connected in sequence, the second dense block includes a first convolutional layer, a second convolutional layer, a third convolutional layer, and a fourth convolutional layer connected in sequence, and the feature map is received through the first convolutional layer. Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the second convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the third convolutional layer Perform convolution operation to obtain feature map And the feature map and the feature map Add channels to get feature maps , receiving the feature map through the fourth convolutional layer Perform convolution operation to obtain feature map , the feature map is transformed into and the feature map Perform channel addition and set the output channel to 1 through the second convolutional layer to obtain the feature map , the feature map is transformed into and the feature map Perform residual connection to obtain the critical perceptual lossless image.

7. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 4, characterized in that: The reversible module includes four reversible blocks connected in sequence, and the low-frequency feature map is processed by the four reversible blocks. And the high frequency feature map Extract high and low frequency images in sequence to obtain the low frequency feature map And the high frequency feature map ;or The low-frequency feature map is processed by four reversible blocks. And the high frequency feature map Invert the high and low frequency images in sequence to obtain the low frequency feature map And the high frequency feature map .

8. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 4, characterized in that: The high frequency modulation module includes a first convolution layer, a second convolution layer and a third convolution layer connected in sequence, and the high frequency feature map is modulated by the first convolution layer. Perform the first modulation to obtain the high-frequency characteristic map , through the second convolutional layer to the high-frequency feature map Perform the second modulation to obtain the high-frequency feature map , through the third convolutional layer to the high-frequency feature map Perform the third modulation to obtain the high frequency characteristic graph .

9. The method for predicting just perceptible distortion of lightweight images based on a reversible network according to claim 8, characterized in that: The convolution kernel size of the first convolution layer of the high-frequency modulation module is 1, the padding size is 0, the number of input channels is 3, and the number of output channels is 128; the convolution kernel size of the second convolution layer of the high-frequency modulation module is 3, the padding size is 1, the number of input channels is 128, and the number of output channels is 128; the convolution kernel size of the third convolution layer of the high-frequency modulation module is 1, the padding size is 0, the number of input channels is 128, and the number of output channels is 3.

Images