Image bit depth enhancement method, device and product based on reversible neural network

Abstract

The application provides a reversible neural network-based image bit depth enhancement method, device and product. The reversible neural network is used to bidirectionally model an image bit depth degradation process and an enhancement process, wherein the reversible neural network comprises a bit depth segmentation block and an affine coupling block. In a forward process, a high bit depth image is input into the reversible neural network to generate a low bit depth image and a latent variable subject to a Gaussian distribution. In a reverse process, the low bit depth image and the latent variable sampled from the Gaussian distribution are input into the reversible neural network to generate a high bit depth image. A loss function is used to constrain the difference between the low bit depth image and a real low bit depth image, the distribution matching property of the latent variable and the reconstruction accuracy of the high bit depth image. The loss function comprises a low bit depth degradation loss, a distribution matching loss and a high bit depth reconstruction loss. The application can realize high-quality reconstruction of a low bit depth image into a high bit depth image.

Description

Technical Field

[0001] This invention relates to the field of image bit depth enhancement technology, and in particular to an image bit depth enhancement technology solution based on a reversible neural network. Background Technology

[0002] Bit depth is a fundamental concept in digital image processing, representing the number of bits used to encode color information for each pixel. Higher bit depths can render more subtle color variations and finer textures, thus improving visual fidelity. With the rise of high dynamic range (HDR) displays supporting 10-bit or higher encoding, the limitations associated with bit depth have become increasingly prominent. Despite continuous hardware advancements, a significant portion of digital content still uses traditional 8-bit formats. This is related to earlier image acquisition standards and storage costs. Directly displaying these low bit depth images on HDR displays often results in artifacts such as false contours, color shifts, and blurring. Image bit depth enhancement aims to reconstruct high bit depth images from low bit depth corresponding images and is a key technology for adapting low bit depth images to HDR displays for high dynamic range content. However, due to quantization ambiguity, multiple high bit depth images may correspond to a single low bit depth image input, thus posing a challenge to this enhancement task.

[0003] Image bit-depth enhancement based on traditional methods mainly focuses on two directions: classic pixel-based methods and traditional spatial context-aware methods. The former emphasizes performing simple numerical operations or statistical analyses on low-bit-depth pixel values ​​to further expand the bit depth. While computationally fast, it neglects the spatial features of the image, leading to a lack of local consistency and resulting in visual artifacts and false contours. The latter incorporates background and structural information from the image, using techniques such as filtering and interpolation to alleviate the problem of false contours to some extent. However, it remains ineffective in handling edge and texture details, and this method relies heavily on prior knowledge, resulting in high computational complexity and unsuitability for processing large-scale image data.

[0004] Zero-padding (ZP) extends bit depth by directly padding the least significant bit of low-bit-depth pixels with zeros. However, its drawback is that it does not consider the spatial characteristics of the image, leading to a lack of local consistency in the generation of high-bit-depth images and resulting in visual artifacts.

[0005] The Adaptive Bit-Depth Expansion (ABDE) method first uses an image generated with zero padding as a base and dynamically calculates the non-negative offset of each pixel to expand the low-bit-depth pixel values ​​to a high-bit-depth. Then, for residual pseudo-contours, it first segments smooth regions in the low-bit-depth image based on regional pixel differences and variance thresholds, and then applies multiple low-pass filters to the marked regions to eliminate artifacts. However, its drawback is that this method relies heavily on prior knowledge, resulting in very high computational complexity and making it difficult to handle large-scale datasets.

[0006] The aforementioned traditional methods enhance image bit depth by targeting pixel values ​​or prior knowledge. However, these methods can generate false contours and artifacts, and are insufficient to produce high bit depth images with good visual effects.

[0007] In recent years, with the development of deep learning methods, deep learning-based spatial bit depth enhancement methods have been proposed. These methods enhance bit depth through convolutional neural networks, which can capture spatial correlations and important visual details in low-bit-depth images, thereby correctly inferring the values ​​of pixels and their neighboring pixels. Some encoder-decoder architectures based on U-Net are used to capture multi-scale contextual information, while there are also some variants, including dual-branch networks that handle smooth and textured regions separately, and iterative frameworks that progressively reconstruct the bit plane from coarse to fine. These methods alleviate problems such as false contours and color shifts to some extent, but their performance improvement still faces significant bottlenecks, mainly because they implicitly handle information loss during the degradation process from high-bit-depth images to low-bit-depth images.

[0008] The residual-guided multiscale fusion network (RMFNet) constructs a multiscale network based on transformation operations. The top-level subnetwork is responsible for extracting global information to distinguish between false contours and real edges, while other subnetworks focus on extracting finer features to supplement details at different scales. However, its drawback lies in the fact that generating a high-bit-depth image from a low-bit-depth image is a one-to-many process, making it difficult to guarantee that the generated image closely resembles the real image. This method ignores the information lost during the transition from high-bit-depth to low-bit-depth images, resulting in poor controllability and greater difficulty in training.

[0009] Patent document CN107481278A discloses an image bit depth enhancement method and apparatus based on a hybrid framework. By fusing traditional debanding algorithms and deep network-based learning algorithms, it can remove unnatural effects in flat regions of an image while restoring the numerical information of missing bits. However, this method has a somewhat vague definition of flat and non-flat regions. Because this invention processes flat and non-flat regions of an image differently, misidentification of flat and non-flat regions will affect the image bit depth enhancement effect.

[0010] Patent document CN116263940A provides a method for image bit depth extension based on convolutional neural networks to fuse noise. However, when introducing noise, the noise model is too simple and lacks adaptation to real-world scenarios. At the same time, the network structure of this invention is relatively complex. During training, it does not consider the processing of information lost from high bit depth images to low bit depth images, which makes network training more complex and thus increases the complexity of the network.

[0011] Therefore, while existing deep learning-based bit depth enhancement methods can alleviate artifacts, they implicitly handle information loss and cannot model the degradation process from high bit depth to low bit depth, leading to uncontrollable generation results. Furthermore, the same low bit depth image may correspond to multiple high bit depth solutions, and existing methods lack explicit modeling for this uncertainty. Summary of the Invention

[0012] Currently, large-scale images are still primarily stored using 8-bit color depth, while new display devices widely support high bit depth imaging. This results in a significant mismatch between data representation capabilities and visual presentation effects. To address the uncertainties inherent in existing image bit depth enhancement technologies, this invention provides an image bit depth enhancement method based on a reversible neural network.

[0013] This invention provides an image bit depth enhancement method based on a reversible neural network, comprising the following steps:

[0014] A reversible neural network is used to model the image bit-depth degradation and enhancement processes bidirectionally, wherein the reversible neural network includes a bit-depth segmentation block and an affine coupling block;

[0015] In the forward process, the high-depth image is input into the reversible neural network to generate a low-depth image and latent variables that follow a Gaussian distribution;

[0016] In the reverse process, the low-bit depth image and latent variables sampled from a Gaussian distribution are input into the reversible neural network to generate a high-bit depth image;

[0017] The loss function constrains the difference between the low-depth image and the real low-depth image, the distribution matching of the latent variables, and the reconstruction accuracy of the high-depth image. The loss function includes low-depth degradation loss, distribution matching loss, and high-depth reconstruction loss.

[0018] Moreover, the bit-depth segmentation block divides the input image into multiple bit-depth components according to channels, and achieves segmentation through iterative remainder and integer division operations.

[0019] Furthermore, the affine coupling block includes densely connected layers, where the input of each layer is composed of the outputs of all previous layers, and affine transformation is achieved through convolution operations and the LeakyReLU activation function.

[0020] Moreover, the low-level degradation loss is calculated using binary cross-entropy loss.

[0021] Moreover, the distribution matching loss constrains the latent variable z to follow an isotropic Gaussian distribution by cross-entropy constraint.

[0022] Moreover, the high-depth reconstruction loss is calculated using L1 loss.

[0023] Furthermore, the reversible neural network employs the Adam optimizer during training, with an initial learning rate of 0.0002, which is adjusted through a multi-step decay strategy, and the number of affine coupling block layers is set to 16.

[0024] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the image bit depth enhancement method based on a reversible neural network as described above.

[0025] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, characterized in that: when the computer program is executed by a processor, it implements the image bit-depth enhancement method based on a reversible neural network as described above.

[0026] On the other hand, the present invention also provides a computer program product, including a computer program, characterized in that: when the computer program is executed by a processor, it implements the image bit depth enhancement method based on a reversible neural network as described above.

[0027] This invention designs a framework for image bit-depth enhancement based on a reversible neural network. By modeling lost information, it improves the controllability of image bit-depth enhancement. Based on low-bit-depth images and trained latent variables that follow a Gaussian distribution, it generates visually appealing high-bit-depth images while avoiding issues such as false contours and color shifts, significantly improving the visual friendliness and accuracy of image bit-depth enhancement. This invention primarily targets the processing of natural images and has been tested on common natural image datasets. It can be applied to high dynamic range displays, meeting the current hardware requirements for displaying high-bit-depth images.

[0028] This invention addresses the significant performance bottlenecks faced by current methods, which, while mitigating issues like false contours and color shifts to some extent, still struggle with performance improvement. This is primarily due to the implicit information loss inherent in the degradation process from high-bit-depth to low-bit-depth images, coupled with the generation bottleneck caused by uncertainty. The invention proposes an image bit-depth enhancement method based on a reversible neural network. This method models the information lost during the degradation process from high-bit-depth to low-bit-depth images and leverages the bidirectional nature of the reversible neural network. By constraining both the forward and inverse networks, it can effectively generate visually appealing high-bit-depth images from low-bit-depth images. It has the following two key points:

[0029] 1) Reversible Modeling of Degradation and Enhancement: This invention proposes a reversible bit depth enhancement network framework to address the shortcomings of existing technologies. This framework explicitly models the mutually reversible bit depth degradation and enhancement tasks by using the dual projection of a reversible network. The proposed image bit depth enhancement framework based on a reversible neural network can effectively capture the inherent uncertainties in image bit depth enhancement tasks and improve its controllability by modeling latent variables. Therefore, compared with current methods, this invention demonstrates better controllability and utilization of lost information.

[0030] 2) Novel Architecture and Training Objectives: This invention proposes a reversible bit depth segmentation module based on a reversible neural network. This module can segment image information across channels while maintaining strict reversibility. Training is guided by three constraints: a degradation loss constraint for low-bit depth images to simulate realistic low-bit depth images; a distribution matching loss to normalize the latent variable z, ensuring that the corresponding information can be sampled through a specific distribution; and an enhancement loss constraint for high-bit depth images to ensure accurate high-bit depth image reconstruction. These objectives collectively ensure bidirectional consistency and improve the fidelity of both low-bit depth and high-bit depth image outputs. Such training objectives are also relatively simple and can be used for training on large-scale data. Attached Figure Description

[0031] Figure 1 This is a flowchart of the training and testing process according to an embodiment of the present invention.

[0032] Figure 2 This is a framework diagram of the image bit depth enhancement process according to an embodiment of the present invention.

[0033] Figure 3 This is a diagram of the bit-depth segmentation block structure according to an embodiment of the present invention.

[0034] Figure 4 This is a structural diagram of the affine coupling block according to an embodiment of the present invention.

[0035] Figure 5 This is a qualitative result (DIV2K dataset) of the scene details and sky outline in an embodiment of the present invention.

[0036] Figure 6 These are qualitative results (Kodak dataset) of two typical sky outline scenes from embodiments of the present invention. Detailed Implementation

[0037] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and embodiments.

[0038] This invention is primarily based on reversible neural networks. Considering the reversibility of the degradation and enhancement processes in image bit-depth enhancement techniques, as well as the ill-posedness of information loss in these techniques, a method for image bit-depth enhancement based on reversible neural networks is proposed. This method fully considers modeling the lost information during image degradation, thereby better utilizing the lost information to assist in the generation of high bit-depth images. Through this invention, higher bit-depth images with better performance and superior visual effects can be obtained.

[0039] This invention proposes a novel reversible bit depth enhancement network for high-quality reconstruction of low-bit-depth images to high-bit-depth images. By using reversible bit depth segmentation blocks, the bit-plane information loss is decoupled into learnable latent variables while maintaining strict reversibility. Simultaneously, a bidirectional learning paradigm is guaranteed, combining degradation loss, latent distribution matching loss, and enhancement loss to ensure consistent optimization in both directions.

[0040] This invention proposes an image bit-depth enhancement method based on a reversible neural network. The loss of low-bit-depth information in the image bit-depth enhancement process is treated as a latent variable z, which follows a specific distribution during forward training. In backward inference, the latent variable can be resampled from the distribution and reconstructed together with the low-bit-depth image to form a high-bit-depth image. By utilizing the bidirectional reversibility of the reversible network, the two mutually reversible tasks of bit-depth degradation and enhancement are explicitly modeled. Furthermore, the proposed framework for image bit-depth enhancement based on a reversible neural network effectively captures the inherent uncertainties in the image bit-depth enhancement task and improves its controllability through latent modeling.

[0041] like Figure 1 As shown, the process of this embodiment of the invention includes two parts: a network model training part and an image bit depth enhancement part.

[0042] The network model training part consists of: configuring training parameters; reading and preprocessing training set images; segmenting the input image into channel components using bit-depth segmentation blocks; transforming the input bit-depth components into low-bit-depth images and latent variables using affine coupling blocks; combining the generated low-bit-depth images with latent variables obtained from random Gaussian sampling, and inputting them into the inverse transform affine coupling blocks and bit-depth segmentation blocks to obtain the high-bit-depth images generated by the model; and determining the relationship between the number of model iterations and a threshold. If the number of model iterations is lower than the threshold, the model continues training; if the number of iterations reaches the threshold, training ends.

[0043] Figure 2 This is a framework diagram of the key technology of this invention, focusing on how the invention obtains a low-bit-depth image and corresponding latent variables from an input high-bit-depth image through a network. This is the pre-training process of this invention. Since each part in the framework diagram is reversible, the latent variables obtained by sampling from a Gaussian distribution and the generated low-bit-depth image are input back into the network to obtain the network-generated high-bit-depth image. In the framework diagram, N represents that the high-bit-depth image is an N-bit image, and n represents that the generated low-bit-depth image is an n-bit image. This represents the bitmap obtained by each bit-depth segmentation block of an image. In this example, N images are obtained by passing through N bit-depth segmentation blocks. i=1,2,…,N and These represent the high-bit-depth component and the low-bit-depth component, respectively. The low-bit-depth component contains the image. For i=1,2,…,Nn, the higher-order deep components contain images. Let i = N-n+1, ​​..., N. Then, after M affine transformation blocks, a low-bit-depth image and the corresponding latent variables are obtained.

[0044] The specific implementation steps of the network model training part in this embodiment are as follows:

[0045] Step 1.1: Configure training parameters, including configuring the number of bits for bit-depth segmentation blocks, the number of bits for processing images in affine coupling blocks, the selection of sub-networks, etc., and setting the number of affine coupling blocks, the number of iterations, the loss calculation function, etc., to complete the setting of training parameters;

[0046] The preferred configuration scheme adopted in the embodiment is as follows: the bit depth of the segmentation block is configured, the bit depth of the processed image of the affine coupling block and the selection of sub-networks are configured, and the number of blocks, iterations, and loss calculation function of the affine coupling block are set. For the configuration of the optimizer and learning rate, the generator learning rate is set to 0.0002 to control the parameter update step size; the first momentum decay rate of the Adam optimizer is set to 0.9; and the second momentum decay rate of the Adam optimizer is set to 0.999. For the parameter settings of training iterations and learning rate scheduling, the total number of training iterations is set to 500,000; the number of learning rate warm-up steps is set to -1 (indicating that warm-up is not enabled); the learning rate decay strategy adopts multi-step decay (i.e., the learning rate decays at certain decay step nodes) (MultiStepLR); the learning rate decay step nodes are set to [100,000, 200,000, 300,000, 400,000]; the learning rate decay coefficient is set to 0.5 (i.e., each decay is half of the original learning rate). Simultaneously, the affine coupling blocks are set to M=16 layers. In this embodiment, 16 layers are preferred to ensure that the memory used during training does not exceed the GPU memory. If a GPU with more memory is available later, more affine coupling blocks can be considered.

[0047] Step 1.2: Preprocessing of training set images: This includes normalizing the training set images, data augmentation (including random horizontal flipping, vertical flipping, and rotation operations), and cropping operations (ensuring that the input images are of the same size).

[0048] In this embodiment, the image is first normalized and then converted into a tensor data type, which helps with subsequent network training. Afterwards, the image undergoes data preprocessing. Specifically, the preprocessing steps include, but are not limited to, random image cropping (ensuring the image meets network input requirements, for example, cropping all images to 144×144 size) and data augmentation (including random horizontal flipping, vertical flipping, and rotation operations).

[0049] Step 1.3: Train the entire image bit depth enhancement network IBEN:

[0050] This invention proposes a bidirectional modeling approach for image bit-depth degradation and enhancement processes using a reversible neural network. The reversible neural network includes a bit-depth segmentation block and an affine coupling block. In the forward process, a high-bit-depth image is input into the reversible neural network to generate a low-bit-depth image and latent variables following a Gaussian distribution. In the reverse process, the low-bit-depth image and latent variables sampled from the Gaussian distribution are input into the reversible neural network to generate a high-bit-depth image. This invention refers to the entire network as the Image Bit-Depth Enhancement Network (IBEN).

[0051] The loss function constrains the difference between the low-depth image and the real low-depth image, the distribution matching of the latent variables, and the reconstruction accuracy of the high-depth image. The loss function includes low-depth degradation loss, distribution matching loss, and high-depth reconstruction loss.

[0052] The working process of the image bit depth enhancement network IBEN is as follows:

[0053] 1) Input the image into a bit depth segmentation block. Segment the image obtained in step 1.2 by channel to obtain the bit depth components of each channel:

[0054] In the embodiment, the specific bit-depth segmentation block structure diagram is as follows: Figure 3 As shown, where, This represents the bitmap of the remaining bits after each bit-depth segmentation block iteration. Each time, the remaining image bits are used as input to the bit-depth segmentation block, and after a modulo 2 operation, the result is obtained. The result This serves as the output of each bit-depth segmentation block iteration. The input and output are then subtracted, and the result is divided by 2 (div2) to obtain the input for the next model iteration. The entire bit-depth segmentation block iterates N times, which is the number of bits in the high-bit-depth image. The bit-depth segmentation block divides the image by channels; for example, an 8-bit image will be divided into 24 channels, each with a value of either 0 or 1. This facilitates subsequent constraints on each generated channel to obtain better results. After the input image passes through the bit-depth segmentation block, the bit-depth components of each channel are obtained.

[0055] 2) Input the bit depth components of each channel obtained in 1) into the affine coupling block. After passing through multiple affine transformation layers, the low bit depth image and latent variable z generated by the model are obtained, and the generated low bit depth image and latent variable z are constrained.

[0056] The channel bit depth components obtained in step 1) are input into the affine coupling block, the structure of which is as follows: Figure 4 As shown, where, and These represent the representations of the high-depth and low-depth components at the k-th affine coupling block, respectively. The affine transformation process can be expressed as:

[0057]

[0058]

[0059] in, , , For learnable functions, For element-wise multiplication, exp() is the exponential function.

[0060] After multiple affine transformation layers, the low-bit depth image and latent variable z generated by the model are obtained, and constraints are applied to the generated low-bit depth image and latent variable z.

[0061] In the embodiments, each affine coupling block is implemented using a densely connected block, wherein the densely connected block defines the input tensor. Output tensor The structure is implemented as shown in the following formula:

[0062]

[0063]

[0064]

[0065]

[0066] As can be seen from the algorithm, For each layer's output, each layer first undergoes a Conv2d convolution operation, and then uses the LeakyReLU activation function to obtain the input for the next layer. Each input is a concatenation of the original input and the outputs of all preceding convolutional layers. Furthermore, the LeakyReLU function is applied after each convolution to balance gradient propagation and non-linearity. The advantages of this approach are that dense connections explicitly preserve shallow features, avoid gradient vanishing, improve the fusion efficiency of features at different scales, and directly transfer low-level texture information to deeper layers through concatenation, making it suitable for tasks requiring detailed restoration.

[0067] As can be seen from the structure of the densely connected blocks, the input of each layer is composed of the outputs of all previous layers. This ensures that as the model progresses, the input of each layer becomes more complex and contains more information, allowing the model to learn more information. Furthermore, each layer includes, but is not limited to, convolutional operations and activation function operations, while maintaining a consistent output size. This ensures the feasibility of subsequent concatenation operations and guarantees the model's learning effectiveness.

[0068] The optimization and learning rate parameters are set according to the above scheme in order to ensure that the learning rate of 0.0002 is used to quickly decrease to the optimal region in the early stage of training, and the learning rate is halved at nodes such as [100000, 200000, ...] in the middle and later stages to fine-tune the model with finer step sizes. Not enabling warm-up means going full speed ahead from the beginning, which is suitable for relatively robust small batch or medium-sized models.

[0069] As a preferred embodiment, the network structure and parameters of the densely connected layer are shown in Table 1 below. In the last column of the table, "Output Size" represents the size of the data output by this layer, expressed as a triplet array. The first two numbers represent the feature map size, and the third number represents the number of feature maps. The default value for gc is 32.

[0070] Table 1. Network structure and parameters of the dense connection layer

[0071]

[0072] The specific network settings are as follows:

[0073] The densely connected layer consists of six layers: the first layer is the input layer, the second to fifth layers are hidden layers, and the sixth layer is the output layer. The structure of each layer is as follows (where gc is 32 by default):

[0074] First layer: Input layer, inputting the bit depth components of each channel after bit depth segmentation in step 1.3.

[0075] The second layer is hidden layer 1, which includes a convolutional layer with a kernel count of gc, a kernel size of 3×3, a stride of 1, and edge padding of 1, and a LeakyReLU activation function layer.

[0076] The third layer: Hidden layer 2, which includes a convolutional layer with a kernel count of gc, a kernel size of 3×3, a stride of 1 and an edge padding of 1, and a LeakyReLU activation function layer.

[0077] The fourth layer: Hidden layer 3, which includes a convolutional layer with a kernel count of gc, a kernel size of 3×3, a stride of 1 and edge padding of 1, and a LeakyReLU activation function layer.

[0078] The fifth layer: Hidden layer 4, which includes a convolutional layer with a kernel count of gc, a kernel size of 3×3, a stride of 1 and an edge padding of 1, and a LeakyReLU activation function layer.

[0079] The sixth layer is the output layer, which consists of a convolutional layer with an out number of kernels, a kernel size of 3×3, a stride of 1, and edge padding of 1. It outputs the bit components of each channel generated by the model.

[0080] This invention further proposes to impose constraints on the generated low-bit-depth image and the latent variable z: 1) Low-bit-depth image degradation loss constraint: by comparing the differences in each channel between the low-bit-depth image generated by the model and the original low-bit-depth image, and using binary cross-entropy loss, the low-bit-depth image generated during the degradation process is constrained to be visually good; 2) Distribution matching error constraint: by constraining the cross-entropy loss between the distribution of high-bit components of the high-bit-depth image and the specified distribution, the latent variable z is constrained to follow the corresponding distribution (isotropic Gaussian distribution), thereby ensuring that the sampling of the latent variable is independent of specific situations.

[0081] After passing through multiple affine transformation blocks, constraints need to be applied to the low-bit depth image and latent variable z generated by the network to constrain the model's output and ensure a better result. For the error constraint of the low-bit depth image generated by the network, the loss function in this embodiment is set as follows:

[0082] Let the original true low-bit depth image be... The low-bit depth image generated by the network is The degradation constraint loss error of low-bit depth images Defined as:

[0083]

[0084] Where C represents the number of channels and c represents the channel index of the image. The constraint on low bit depth images is mainly achieved by using binary cross-entropy loss. After segmenting each image by channel, the value generated by the model is constrained to be closer to 0 or 1, that is, closer to the true value.

[0085] Simultaneously, the latent variable z must be constrained to ensure it more closely approximates a Gaussian distribution. This ensures that the results obtained during subsequent random Gaussian sampling are closer to the true situation, while also guaranteeing that the sampling is independent of specific circumstances. Distribution matching error. Defined as:

[0086]

[0087] in, and These are high-depth image I H and the distribution of the latent variable z, It is a partial output of the latent variable. It is the cross-entropy function. The distribution-matching loss is used to constrain the latent variable z to follow an isotropic Gaussian distribution, thus ensuring that the sampling of the latent variable is independent of specific situations. It can perform random sampling during the inverse reconstruction process. This reduces the uncertainty of the reconstruction process.

[0088] 3) The low-bit depth image generated by the model and the latent variable z obtained by Gaussian sampling are concatenated and then input into the multi-block affine coupling block and bit depth segmentation block of the inverse transform to obtain the high-bit depth image generated by the model, and the generated high-bit depth image is constrained.

[0089] The loss function in this embodiment is set as follows:

[0090] The input, consisting of a low-bit-depth image generated by the model and a randomly sampled Gaussian image (z-thickness), undergoes multiple inverse transforms through affine coupling blocks and bit-depth segmentation blocks to generate a high-bit-depth image. Therefore, constraints must be applied to the high-bit-depth image to ensure a high-quality high-bit-depth image is generated. The reconstruction constraint loss error of the high-bit-depth image is discussed. Defined as:

[0091]

[0092] in, Indicates by (i.e., the low-depth image generated by the model) and the high-depth image reconstructed by a reversible transformation of randomly sampled z. This is a metric that measures the difference between the reconstructed image and the original image; the preferred method used in this example is L1 loss. z is derived from a predefined distribution. Random sampling is used. By limiting the reconstruction loss, it can be ensured that the model can accurately reconstruct high-depth images from low-depth images and randomly sampled latent variables.

[0093] After considering the three loss functions mentioned above, the total loss function defined in this invention is set as follows:

[0094]

[0095] in, , , These are the degradation loss error weights for low-depth images, the reconstruction loss error weights for high-depth images, and the distribution matching error weights.

[0096] This invention proposes to extract training images from the training dataset in batches, with the goal of minimizing the loss of the output layer, and update the parameters of each layer by backpropagating the loss, so that the network parameters are updated, until all training images in the training dataset are used up, which means that one training cycle is completed, and then proceed to step 1.6 for iterative judgment.

[0097] When the preset iteration stopping conditions are met, the structure is saved, completing the training of the Image Bit Depth Enhancement Network (IBEN). This invention further proposes conducting corresponding tests. This invention sets up a model saving process every 5000 iterations and a testing process every 2000 iterations. The testing only utilizes the affine coupling block and bit-depth segmentation block of the inverse transform in the IBEN network, primarily relying on the calculation of PSNR and SSIM metrics to objectively assess the quality of model generation. This approach also facilitates timely adjustments to the model's generation performance, allowing for parameter tweaks to ensure better results.

[0098] The image bit depth enhancement part includes: configuring test parameters; inputting a test image (i.e., a low bit depth image) and performing preprocessing; and generating a high bit depth image by passing through the affine coupling block and bit depth segmentation block of the inverse transform in the trained image bit depth enhancement network IBEN.

[0099] The image bit depth enhancement part in this embodiment is specifically implemented by the following sub-steps:

[0100] Step 2.1: Output the low bit depth image for testing and perform preprocessing. The low bit depth image needs to be randomly cropped, data augmented, and other operations to ensure that the input image can meet the 144×144 size required by the image bit depth augmentation network. In other words, it must correspond to the input size of the network model training part.

[0101] Step 2.2 introduces z obtained by random Gaussian sampling and concatenates it with the low bit depth image. The concatenated result is used as the input of the network. The advantage of doing this is that z obtained by random Gaussian sampling is non-special and has strong well-stability, ensuring that the sampling is independent of specific situations.

[0102] Step 2.3: Use the Image Bit Depth Enhancement Network IBEN (which only uses inverse transform affine coupling blocks and bit depth segmentation blocks) to generate a high bit depth image, including multiple inverse transform affine coupling blocks and bit depth segmentation blocks. Save the generated image result. This way, you get a high bit depth image result generated by the network after giving a low bit depth image.

[0103] In practice, all of the above processes can be automated using computer software technology.

[0104] For a better understanding of the technical effects of this invention, please refer to... Figure 5 and Figure 6 The method provided by this invention was tested on the DIV2K and Kodak datasets. This invention provides visualization of the generated images, specific regions, and residual maps, ensuring that the generated images are of the best quality from a subjective visual perspective.

[0105] Judging from the generated images, the images generated by this invention do not have obvious false contours, and the residual image is the closest to the real image. This further ensures that the method proposed in this invention can generate an image with good visual effects.

[0106] Meanwhile, PSNR and SSIM indices were calculated to further confirm that the present invention is optimal.

[0107] Table 24-8 shows the comparative test results of the image bit depth enhancement algorithm on the Kodak dataset.

[0108]

[0109] Table 34-8 shows the comparative test results of the 8-bit image bit depth enhancement algorithm on the DIV2K dataset.

[0110]

[0111] As can be seen from Tables 2 and 3, the present invention outperforms the current method in both PSNR and SSIM metrics in the 4-8 bit image bit depth enhancement task.

[0112] Therefore, this invention can achieve the best results in both qualitative and quantitative aspects.

[0113] In specific implementation, the method proposed in the technical solution of this invention can be automatically executed by those skilled in the art using computer software technology. System devices for implementing the method, such as computer-readable storage media storing the corresponding computer program of the technical solution of this invention and computer equipment including the computer program running the corresponding computer program, should also be within the protection scope of this invention.

[0114] The image bit depth enhancement device based on a reversible neural network provided by the present invention is described below. The image bit depth enhancement device based on a reversible neural network described below can be referred to in correspondence with the image bit depth enhancement method based on a reversible neural network described above.

[0115] On the other hand, the present invention also provides an electronic device, which may include: a processor, a communications interface, a memory, and a communication bus, wherein the processor, the communications interface, and the memory communicate with each other through the communication bus. The processor can call logical instructions in the memory to execute the above-described image bit-depth enhancement method based on a reversible neural network.

[0116] Furthermore, the logical instructions in the aforementioned memory can be implemented as software functional units and sold or used as independent products, and can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0117] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer is able to execute the image bit depth enhancement method based on the reversible neural network provided by the above methods.

[0118] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image bit-depth enhancement method based on the reversible neural network provided by the above methods.

[0119] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0120] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image bit depth enhancement method based on a reversible neural network, characterized in that, Includes the following processes: A reversible neural network is used to model the image bit-depth degradation and enhancement processes bidirectionally, wherein the reversible neural network includes a bit-depth segmentation block and an affine coupling block; In the forward process, the high-depth image is input into the reversible neural network to generate a low-depth image and latent variables that follow a Gaussian distribution; In the reverse process, the low-bit depth image and latent variables sampled from a Gaussian distribution are input into the reversible neural network to generate a high-bit depth image; The loss function constrains the difference between the low-depth image and the real low-depth image, the distribution matching of the latent variables, and the reconstruction accuracy of the high-depth image. The loss function includes low-depth degradation loss, distribution matching loss, and high-depth reconstruction loss.

2. The method according to claim 1, characterized in that: The bit-depth segmentation block divides the input image into multiple bit-depth components according to channels, and achieves segmentation through iterative remainder and integer division operations.

3. The method according to claim 1, characterized in that: The affine coupling block includes densely connected layers, where the input of each layer is composed of the outputs of all previous layers, and the affine transformation is achieved through convolution operations and the LeakyReLU activation function.

4. The method according to claim 1, characterized in that: The low-level degradation loss is calculated using binary cross-entropy loss.

5. The method according to claim 1, characterized in that: The distribution matching loss constrains the latent variable z to follow an isotropic Gaussian distribution by cross-entropy constraint.

6. The method according to claim 1, characterized in that: The high-depth reconstruction loss is calculated using L1 loss.

7. The method according to claim 1, characterized in that: The reversible neural network is trained using the Adam optimizer with an initial learning rate of 0.0002, which is adjusted through a multi-step decay strategy. The number of affine coupling block layers is set to 16.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the image bit depth enhancement method based on a reversible neural network as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the image bit depth enhancement method based on any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the image bit depth enhancement method based on any one of claims 1 to 7.

Images