Image denoising method and system based on degenerate complex quaternion convolutional neural network

Abstract

The application relates to an image denoising method and system based on a degenerate complex quaternion convolutional neural network, which comprises the following steps: obtaining a target image; inputting the target image into a trained image denoising model to obtain a denoised image corresponding to the target image; wherein the trained image denoising model adopts a degenerate complex quaternion convolutional neural network model, the degenerate complex quaternion convolutional neural network model is obtained by replacing the convolutional layers of a real-value neural network model except the first and last convolutional layers with degenerate complex quaternion convolutional layers, and the operation method of the degenerate complex quaternion convolutional layers is that each axis of a quaternion convolution kernel and each axis of a feature map are respectively and one-to-one subjected to convolution operation. The degenerate complex quaternion convolutional layers are used to replace ordinary real-value convolutional layers, in the convolution process, the multi-dimensional features of the picture are fused, and the correlation between multiple channels is considered. Therefore, the picture has more abundant features extracted and learned in the denoising, and can be better distinguished from noise points.

Description

Technical Field

[0001] This invention relates to the field of image denoising, and in particular to an image denoising method and system based on a degenerate complex quaternion convolutional neural network. Background Technology

[0002] With the rapid development of information science and technology, people require clear digital images as input in many applications such as target detection, object recognition, and image retrieval. However, digital images can be contaminated by noise during the acquisition and storage process for various reasons, so image denoising is a very important topic.

[0003] In recent years, deep network-based image denoising methods have become mainstream due to their promising denoising results. However, existing deep learning-based image denoising methods using real-valued neural networks do not consider the correlation between image pixel dimensions; they simply treat the RGB dimensions as three unrelated image features, which easily leads to the loss of feature information. Summary of the Invention

[0004] The purpose of this invention is to provide an image denoising method and system based on degenerate complex quaternion convolutional neural networks. This method replaces ordinary real-valued convolutional layers with degenerate complex quaternion convolutional layers, fusing multi-dimensional features of the image during the convolution process and considering the correlation between multiple channels. This allows for the extraction and learning of richer features during the denoising process, resulting in better differentiation from noise points.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] An image denoising method based on a degenerate complex quaternion convolutional neural network includes:

[0007] Acquire the target image;

[0008] The target image is input into the trained image denoising model to obtain the denoised image corresponding to the target image; the trained image denoising model is a model trained with a sample image as input and a clear image of the sample image before adding noise as a label.

[0009] The trained image denoising model employs a degenerate complex quaternion convolutional neural network model. This model is obtained by replacing all convolutional layers of a real-valued neural network model except for the first and last convolutional layers with degenerate complex quaternion convolutional layers. The first convolutional layer of the real-valued neural network contains four convolutional kernels. The degenerate complex quaternion convolutional layer is operated by performing convolution operations between each axis of the quaternion convolutional kernel and each axis of the feature map.

[0010] Optionally, the real-valued neural network model adopts the Unet model.

[0011] Optionally, the degenerate complex quaternion convolutional neural network model uses the Leakly ReLU function to activate neurons in each layer.

[0012] Optionally, the degenerate complex quaternion convolutional neural network model adds a batch normalization layer to the hidden layers.

[0013] Optionally, before inputting the target image into the trained image denoising model, the method further includes training the image denoising model, the training process of which is as follows:

[0014] Acquire a sample image, which is an image obtained by adding noise to a clear image;

[0015] Using the sample image as input and the clear image of the sample image before noise addition as a label, the image denoising model is trained to obtain the trained image denoising model.

[0016] Optionally, before training the image denoising model, the method further includes initializing the weights of the degenerate complex quaternion convolution kernel; the initialization method adopts the He initialization method.

[0017] The present invention also provides an image denoising system based on a degenerate complex quaternion convolutional neural network, comprising:

[0018] The target image acquisition module is used to acquire the target image;

[0019] The denoising module is used to input the target image into a trained image denoising model to obtain a denoised image corresponding to the target image; the trained image denoising model is a model trained with a sample image as input and a clear image of the sample image before adding noise as a label.

[0020] The trained image denoising model employs a degenerate complex quaternion convolutional neural network model. This model is obtained by replacing all convolutional layers of a real-valued neural network model except for the first and last convolutional layers with degenerate complex quaternion convolutional layers. The first convolutional layer of the real-valued neural network contains four convolutional kernels. The degenerate complex quaternion convolutional layer is operated by performing convolution operations between each axis of the quaternion convolutional kernel and each axis of the feature map.

[0021] Optionally, the system further includes:

[0022] Image denoising model training module: used to train the image denoising model before inputting the target image into the trained image denoising model.

[0023] Optionally, the image denoising model training module specifically includes:

[0024] The sample image acquisition submodule is used to acquire sample images, which are images obtained by adding noise to a clear image.

[0025] The model training submodule is used to train the image denoising model by taking the sample image as input and using the clear image of the sample image before denoising as the label, so as to obtain the trained image denoising model.

[0026] Optionally, the system further includes a weight initialization module for initializing the weights of the degenerate complex quaternion convolution kernel before training the image denoising model; the initialization method adopts the He initialization method.

[0027] According to specific embodiments provided by the present invention, the present invention discloses the following technical effects:

[0028] This invention provides an image denoising method and system based on a degenerate complex quaternion convolutional neural network. First, a target image is acquired. Then, the target image is input into a trained image denoising model to obtain a denoised image corresponding to the target image. The trained image denoising model is a model trained using sample images as input and a clear image of the sample image before noise addition as a label. The trained image denoising model employs a degenerate complex quaternion convolutional neural network model, which is obtained by replacing the convolutional layers of a real-valued neural network model (except for the first and last layers) with degenerate complex quaternion convolutional layers. The first convolutional layer of the real-valued neural network contains four convolutional kernels. The operation method of the degenerate complex quaternion convolutional layer involves performing convolution operations between each axis of the quaternion convolutional kernel and each axis of the feature map one by one. Because this invention uses degenerate complex quaternion convolutional layers instead of ordinary real-valued convolutional layers, the multi-dimensional features of the image are integrated during the convolution process, and the correlation between multiple channels is considered. This allows for the extraction and learning of richer features in the image during denoising, enabling better differentiation from noise points. Attached Figure Description

[0029] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0030] Figure 1 A structural diagram showing the network structure and parameter settings details of the prior art solution provided by the present invention;

[0031] Figure 2 The two activation function images provided by this invention;

[0032] Figure 3 The flowchart of the image denoising method based on degenerate complex quaternion convolutional neural network provided by the present invention is shown below.

[0033] Figure 4 The main neural network structure diagram provided by this invention;

[0034] Figure 5 The flowchart for model training provided by this invention;

[0035] Figure 6 This is a schematic diagram of the quaternion convolution process provided by the present invention;

[0036] Figure 7 A schematic diagram of the degenerate complex quaternion convolution process provided by the present invention;

[0037] Figure 8 This is a schematic diagram illustrating the visualization test results of the model denoising performance provided by the present invention;

[0038] Figure 9 This is a comparison image of the denoising effect provided by the present invention with that of DnCNN. Detailed Implementation

[0039] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0040] The purpose of this invention is to provide an image denoising method and system based on a degenerate complex quaternion convolutional neural network. Specifically, it designs a novel deep network structure, the Degenerate Biquaternion Convolutional Neural Network (RQCNN), to address the shortcomings of traditional deep learning-based image denoising methods that rely on Convolutional Neural Networks (CNNs). This structure, by considering the correlations between different dimensions of image pixels, achieves better image processing results compared to real-valued convolutional neural networks. RQCNN demonstrates its unique advantages, particularly when processing images containing Gaussian noise with a large standard deviation.

[0041] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0042] Before describing the solution of this invention, some of the terms used in this invention will be explained:

[0043] Quaternions: Quaternions are simple hypercomplex numbers, generally represented as q = a + bi + cj + dk, where a, b, c, and d are real numbers, and i, j, and k are the imaginary units. 2 =k 2 =j 2 =-1.

[0044] Pure quaternions: Quaternions with a real part of 0 are generally represented as q = 0 + bi + cj + dk, where b, c, and d are real numbers, and i, j, and k are the imaginary units. 2 =k 2 =j 2 =-1.

[0045] Degenerate complex quaternions: Degenerate complex quaternions are also a type of hypercomplex number, generally represented as b = a + bi + cj + dk, where a, b, c, d are real numbers, i, j, k are the imaginary unit, and i 2 =k 2 =-j 2 =-1.

[0046] Batch normalization: Since the changes in the distribution of nodes within a deep network during training are considered internal covariate transfers, eliminating these transfers can provide faster training. Batch normalization is used to reduce internal covariate transfers, which can greatly accelerate the training of deep neural networks.

[0047] Currently, image denoising can be divided into three main categories: filter-based methods, model-based methods, and learning-based methods.

[0048] Learning-based methods, which focus on learning the latent mapping from noisy images to clean images, can be divided into traditional learning-based methods and deep network-based learning methods. In recent years, deep network-based methods have become the mainstream approach because they have achieved more promising denoising results than filtering-based, model-based, and traditional learning-based methods.

[0049] Common learning-based networks include Denoising Convolutional Neural Network (DnCnn) and Convolutional Blind Denoising Neural Network (CBDnet). While these methods have achieved impressive denoising results, the inherent structural limitations of real-valued neural networks mean they do not consider the correlations between image pixel dimensions, and the latent relationships between input features remain poorly investigated. Therefore, they do not perform well when dealing with strong noise with large mean differences.

[0050] This invention proposes a degenerate complex quaternion-based convolutional neural network, RQCNN, which effectively considers the correlation between different dimensions of the input image, and the proposed degenerate complex quaternion multiplication is commutative. When applied to image denoising, RQCNN outperforms ordinary real-valued CNNs with the same structure when processing Gaussian noise with the same mean-difference distribution, and exhibits better resilience when dealing with high-intensity noise.

[0051] Currently, the most successful deep learning-based image denoising methods are those mentioned in the paper "Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising." The main technical approach is as follows:

[0052] Training and testing data: For the network training data, 400 clear images were initially used. These 400 images were then randomly cropped and subjected to data augmentation processing such as adding Gaussian noise with different standard deviations, ultimately producing 128×1600 40×40 training images. For the testing data, another 80 clear images were used, and the same noise-adding method was applied. Of these, 12 images were used for model evaluation during training, and the other 68 images were used for performance testing of the final trained model.

[0053] Parameter settings and network training: The VGGNet network model was modified, removing the pooling layers from the original network and using the ReLU function to activate neurons in each layer. Finally, a residual learning formula was used to train a residual mapping, and residual learning and batch normalization were integrated to improve the network's denoising performance and efficiency. Details of the specific network structure and parameter settings are as follows... Figure 1 .

[0054] Figure 1 In the first layer, Conv+ReLU consists of 64 convolutional kernels of size 3×3×c, which are used to generate 64 feature maps.

[0055] Conv+BN+ReLU: 2 to (d-1) layers, using 64 convolutional kernels of size 3×3×64, and adding batch normalization between convolutions and activation functions.

[0056] Conv: The last layer, the d-th layer, has c filters of size 3×3×64 used to reconstruct the input.

[0057] Parameter settings: To capture sufficient spatial information for denoising, the neural network is configured with 18 layers. During training, weights are first initialized, and then the mean squared error between the desired residual image and the image estimated from the noisy input is used as the loss function.

[0058]

[0059] (Note: In the above formula, N represents the number of training samples, R represents the final output residual mapping, θ represents the trainable parameters, y represents the noisy training sample image after adding noise, and x represents the clean sample image after adding noise.) The algorithm was optimized using a stochastic gradient descent algorithm with a weight decay rate of 0.0001 and an impulse of 0.9. The mini-batch size was set to 128, and the training run was set to 50 epochs. During training, the learning rate decayed exponentially from 1e-1 to 1e-4.

[0060] 3) Model Evaluation and Comparative Analysis: After training and optimization with a fixed number of rounds (50 rounds in this scheme), the final model's performance is evaluated using test data, with peak signal-to-noise ratio (based on mean squared error, MSE) as the metric.

[0061]

[0062]

[0063] (Note: In the above formula, MSE is the mean square error of two grayscale images I and K, each with a size of m×n, and MAX is the mean square error of the image.) I The denoising performance of the test data on the final trained model is evaluated using the maximum gray level of the image points (where the maximum gray level is the maximum value). The results are compared and analyzed with traditional denoising methods such as BM3D, WNNM, EPLL, MLP, CSF, and TNRD. Finally, the performance of the existing method is summarized.

[0064] Experimental setup and environment for this method: The experiment used the MatConvNet package to train the neural network model. The experiment was conducted in the Matlab (R2015b) environment, running on a PC with an Intel(R)core(TM) i7-5820K CPU 3.30GHz and an Nvidia TitanX GPU.

[0065] While the aforementioned existing methods utilize convolutional neural network algorithms to leverage external image information and address the shortcomings of traditional denoising algorithms in denoising images with limited usable internal information, particularly those acquired under conditions of high Gaussian noise standard deviation and low lighting, they also achieve denoising of images with Gaussian noise of varying standard deviations by training a single neural network model. Experimental data demonstrate that these methods significantly improve denoising performance compared to traditional algorithms, overcoming the bottleneck of traditional denoising methods. The experimental data comparison, using average peak signal-to-noise ratio (PNSR) as a metric, is as follows (unit: dB):

[0066] Methods BM3D WNNM EPLL MLP TNRD DnCNN θ = 15 31.07 31.17 31.21 --- 31.42 31.73 θ = 25 28.57 28.83 28.68 28.96 28.92 29.23 θ = 50 25.62 25.87 25.67 26.03 25.97 26.23

[0067] However, this method has the following shortcomings:

[0068] 1. The real-valued neural network in this method does not consider the correlation between image pixel dimensions; it simply treats the RGB dimensions as three unrelated image features. Although each convolutional kernel summarizes the convolution results of different channels into a single output during convolution, this still ignores the interrelationships between the RGB color channels, leading to the loss of feature information.

[0069] 2. The denoising network model used in this method is a combination of Conv+BN+ReLU. While this linear structure is simple and intuitive, and can extract most of the image features, it lacks the ability to extract more detailed and deeper features. Consequently, in actual denoising processes, this network model does not thoroughly denoise noisy images, has poor generalization ability, and cannot effectively process truly noisy images.

[0070] 3. The convolutional neural network algorithm in this method uses the popular ReLU activation function in its hidden layers, and its mathematical expression is:

[0071] f(x) = max(0, x)

[0072] ReLU outputs 0 when the input signal is less than 0, and outputs equal to the input when the input signal is greater than 0. This characteristic makes ReLU a sparse activation function, solving the gradient vanishing problem during neural network training. Compared to sigmoid / tanh, it avoids a lot of complex calculations, speeding up computation and convergence. However, ReLU has an inherent drawback: for values ​​less than 0, the gradient of this neuron will always be 0. Consequently, in practice, if the learning rate is large, a very large gradient flowing through a ReLU neuron and updating its parameters will render the neuron inactive on any data. This can also happen with smaller learning rates.

[0073] To address problem 3 above, this invention employs the LeakyReLU activation function, which is more suitable for network training. Its expression is:

[0074] f(x)=max(0,x)+leak*min(0,x)

[0075] Leak is a very small number, which preserves some values ​​of the complex axis, ensuring that information from the negative axis is not completely lost, effectively solving the "death" problem of the ReLU function during training. Please refer to the graphs of the two activation functions. Figure 2 .

[0076] In summary, this invention addresses the three shortcomings mentioned above. The most significant issue addressed is the first technical problem. For problem 1, a degenerate complex quaternion convolutional layer is used instead of a real-valued convolutional layer. For problem 2, a more efficient model structure, UNet, is employed to extract richer features from the image. For problem 3, the Leakly ReLU function is used instead of the ReLU function. Experiments show that the proposed solution significantly improves denoising efficiency for Gaussian images while achieving better denoising performance. Furthermore, it exhibits greater resilience when dealing with Gaussian noise with a large variance distribution.

[0077] The specific solution of the present invention will be described in detail below:

[0078] This invention provides an image denoising method based on a degenerate complex quaternion convolutional neural network. Please refer to [link / reference]. Figure 3 ,include:

[0079] S1. Obtain the target image.

[0080] S2. Input the target image into the trained image denoising model to obtain the denoised image corresponding to the target image; the trained image denoising model is a model trained with a sample image as input and a clear image of the sample image before adding noise as a label.

[0081] The trained image denoising model employs a degenerate complex quaternion convolutional neural network (DNN) model. This DNN model is obtained by replacing all convolutional layers (except the first and last layers) of a real-valued neural network (RVN) with degenerate complex quaternion convolutional layers. The first convolutional layer of the RVN contains four convolutional kernels. The degenerate complex quaternion convolutional layer operates by performing convolution operations between each axis of the quaternion kernel and each axis of the feature map. The operational method of the degenerate complex quaternion convolutional layer will be described in detail later.

[0082] As an optional implementation, before inputting the target image into the trained image denoising model, the method further includes training the image denoising model, the training process of which is as follows:

[0083] Acquire a sample image, which is an image obtained by adding noise to a clear image;

[0084] Using the sample image as input and the clear image of the sample image before noise addition as a label, the image denoising model is trained to obtain the trained image denoising model.

[0085] During model training, 1) it is necessary to first obtain the dataset and preprocess the data. The preprocessed dataset images are the sample images. Specifically, the BSD-100 and Kodak-24 datasets can be used for model training. From the BSD-100 dataset, 80 color images can be randomly selected as the training set, 10 as the validation set, and 10 as the test set. Similarly, from the Kodak-24 dataset, 16 color images can be randomly selected as the training set, 4 as the validation set, and 4 as the test set. Then, the pixel values ​​of the color images in both datasets are normalized to [0,1], and each 512*512 image is divided into 64 sub-images of size 64*64 to improve the computational efficiency of the system. During network training, additive Gaussian noise with a mean of zero and a variance of [10,20,30,40] is added to the training set images to obtain the sample images. During the testing process, the same method was used to add noise to the clear images. The images in the validation set were used for model evaluation during training, while the images in the test set were used for performance testing of the final trained model.

[0086] 2) After obtaining the dataset and preprocessing it, a neural network model still needs to be built.

[0087] As an optional implementation, the real-valued neural network model adopts the Unet model. This technical feature can address problem 2 in the prior art (i.e., the existing Conv+BN+ReLU combined structure model lacks the ability to extract more detailed and deeper features of images).

[0088] The main body of the degenerate complex quadruple convolutional neural network is modified using the Unet model. This structure contains 5 upsampling processes and 5 downsampling processes. There is a skip connection between the upsampling and downsampling processes of the same size. This can fully extract the feature information of the image.

[0089] Specifically, the input three-channel image is first convolved with four real-valued convolutional kernels to transform it into four-channel image features. This four-channel information is then input into the main body of the degenerate complex quadruple convolutional neural network—Unet. After a series of operations, the output is finally convolved with three real-valued convolutional kernels to restore the image information to three channels. This facilitates the connection between the degenerate complex quadruple convolutional layer and other layers, making quadruple convolution more convenient and flexible.

[0090] Please refer to the main neural network structure diagram. Figure 4 Throughout the convolution process, the first layer has four real-valued convolutional kernels of size c×3×3, and the last layer has c real-valued convolutional kernels of size 64×3×3. In layers 2-26 (Unet structure), each degenerate complex quaternion convolutional layer has n quaternion convolutional kernels of size 3×3×1 (n=8,16,32,64,128), where c=1 if the input image is grayscale and c=3 if it is color image.

[0091] As an optional implementation, before training the image denoising model, the method further includes initializing the weights of the degenerate complex quaternion convolution kernel; the initialization method adopts the He initialization method, which convolves the initialized weights with the input tensor to obtain the output of the convolution kernel. During training, the weights are updated according to the gradient descent method, and the final weight parameter model is obtained after multiple iterations.

[0092] As an optional implementation, the degenerate complex quaternion convolutional neural network model adds a batch normalization layer (BN layer) to the hidden layer.

[0093] Adding a batch normalization (BN) layer to the hidden layers can effectively improve network performance. This ensures that the data distribution is automatically normalized after each hidden layer extracts features, thus saving the network time to learn and adapt to new data distributions and improving efficiency.

[0094] As an optional implementation, the degenerate complex quaternion convolutional neural network model uses the LeaklyReLU function to activate neurons in each layer.

[0095] The Leakly ReLU function is used in the hidden layer. It has all the advantages of the ReLU function, while also solving the shortcomings of the ReLU function, such as some neurons not being activated or "die". Experiments have shown that combining BN and LeaklyReLU function has indeed greatly improved the denoising efficiency and performance.

[0096] Finally, residual learning is used to learn image features, and stochastic gradient descent with a batch size of 32 is used for backpropagation optimization of trainable parameters. The total number of training epochs is set to 100. The experiment uses the Adam optimizer, with an initial learning rate of 0.0005 and an exponential decay method with a learning decay rate of 0.9. The loss function is the mean squared error (MSE) as follows:

[0097]

[0098] 3) After building the neural network model, the model training process begins. The steps 1) sample image acquisition and 2) model building mentioned above are preparatory work before model training. The following section details the specific training process; please refer to [link to relevant documentation] for more information. Figure 5 .

[0099] The neural network uses a noisy image (made by adding Gaussian noise to the original clean image) as input to the network and the original clean image as the target value. The goal is to continuously update the weights through multiple rounds of training, reduce the gap between the output result and the target value, and achieve a non-linear mapping relationship from noisy image to clean image.

[0100] a. Input each original, noise-free, clean image of size 512*512 from the BSD-100 or Kodak-24 training set.

[0101] b. Divide each input 512*512 image into 64 sub-images of size 64*64. Use these sub-images as a new training set to improve image utilization and computational efficiency. During network training, add additive Gaussian noise with a mean of zero and variances of [10, 20, 30, 40] to the sub-images in the training set to complete the image preprocessing.

[0102] c. Input the preprocessed training dataset into the network.

[0103] d. The weights of the degenerate complex quaternion convolution kernel are initialized using the He Kaiming initialization method.

[0104] e. Input the noisy image into the network model, and successively perform calculation processes such as real-valued convolution, degenerate complex quaternion convolution, downsampling, and upsampling to carry out the forward propagation of the neural network and the feature extraction of the image.

[0105] f. The minimum mean square error (MSE) is used as the loss function to calculate the error between the training result after the forward propagation of the neural network and the original clean image. The error is then propagated back to each weight node to update the weights.

[0106] g. Determine whether the specified number of training iterations has been reached based on the target loss function value. If yes, continue; otherwise, repeat steps e through f.

[0107] h. Training is complete. Save the final trained learning model and use the final model to test and evaluate the test dataset.

[0108] The initialization process for the weights of the degenerate complex quaternion convolution kernel is as follows:

[0109] A degenerate complex quaternion b can be written in the form of a combination of two complex numbers:

[0110] b = b m+n e1+b m-n e2

[0111] Among them, b m+n =(b r +b j )+(b i +b k i and b m-n =(b r -b j )+(b i -b k )i. and They are two idempotent special elements that satisfy the following rules:

[0112]

[0113] Suppose two complex numbers are z1 = a + bi and z2 = c + di, then we have:

[0114]

[0115] The four-dimensional weight matrix of a degenerate complex quaternion can then be calculated by first initializing the real and imaginary parts (a, b, c, d) of two complex numbers, and then using the formula to obtain the four-dimensional components (b... r b i b j b k ).

[0116] Finally, this weight initialization calculation method is incorporated into the custom initialization method of the degenerate complex quaternion convolution kernel to initialize the parameters of the degenerate complex quaternion convolution kernel. The initialization strategy adopted in this invention is "he" initialization, which initializes the weight parameters of the degenerate complex quaternion convolution layer to a suitable initial value that satisfies the quaternion structure of the degenerate complex quaternion. This ensures that the quaternion structure of the internal parameters of the convolution kernel is not deformed, and also greatly reduces the training time of the network.

[0117] The degenerate complex quaternion convolution process will be described in detail below. To enable those skilled in the art to better understand the degenerate complex quaternion convolution process in this invention, it is necessary to explain it in conjunction with the quaternion convolution process. For the quaternion convolution process, please refer to [link to relevant documentation]. Figure 6 For the degenerate complex quaternion convolution process, please refer to [link / reference]. Figure 7 .

[0118] A quaternion q = a + bi + cj + dk is a hypercomplex number extended from the complex number field. It was first discovered by the Irish mathematician William Nouwen-Hamilton in 1843. Its multiplication does not satisfy the commutative law, which is reflected in the following aspects during calculation:

[0119] ij = k, ji = -k;

[0120] jk = i, kj = -i;

[0121] ki = j, ik = -j.

[0122] In the forward propagation computation of neural networks, the non-commutative nature of multiplication makes quaternion convolution operations unnormalizable from ordinary real-valued convolution operations. Therefore, a new type of hypercomplex number, the reduced biquaternion, is introduced. Like quaternions, the reduced biquaternion b = a + bi + cj + dk is also an algebraic expression in four-dimensional space, and its computation satisfies the following:

[0123]

[0124] Clearly, degenerate complex quaternion multiplication is commutative. Given two degenerate complex quaternions b and c, their addition and multiplication rules can be expressed as follows:

[0125] b+c=(b r +c r )+(b i +c i )i+(b j +c j )j+(b k +c k )k

[0126] b·c=(b r c r -b i c i +b j c j -b k c k )+(b r c i +b i c r +b j c k +b k c j )i+(b r c j -b i c k +b j c r -b k c i )j+(b r c k -b i c j +b j c i +b k c r )k

[0127] The matrix representation of degenerate complex quaternions is determined by their multiplication rules:

[0128]

[0129] Therefore, the degenerate complex quaternion convolution process can be represented in matrix form as follows:

[0130]

[0131] Where w is the degenerate complex quaternion convolution weight matrix, and x is the convolution input.

[0132] After model training, the denoising performance of the finally trained denoising model is tested using a test dataset. The test results are visualized as follows. Figure 8 (Two sets of images are selected here for visual comparison, where σ = 20).

[0133] After testing, the final denoising effect of this invention was measured using two metrics: mean peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), and compared with the denoising effects of traditional methods and DnCNN (PSNR unit: dB):

[0134]

[0135]

[0136] The comparison data in the table above shows that the method of the present invention is indeed superior to the denoising method based on DnCNND in terms of denoising performance. At the same time, under the same conditions as DnCNN, when training the same training dataset, the method of the present invention saves about 1.5 hours of training convergence time compared with DnCNN, which is very impressive in terms of training efficiency. Figure 9 A comparison of the denoising results with DnCNN is shown (the image here contains noisy data with σ=20).

[0137] This invention uses degenerate complex quaternion convolutional layers instead of ordinary real-valued convolutional layers. During the convolution process, multi-dimensional features of the image are integrated, and the correlation between multiple channels is considered. This allows for the extraction and learning of richer features in the image denoising experiments, enabling better differentiation from noise points.

[0138] Compared to DnCnn, this invention employs a Unet structure with a richer hierarchical structure. This structure provides multi-level feature representations for the network's input images, achieving multi-scale receptive fields. Furthermore, the skip-layer connections between layers effectively reduce the vanishing gradient problem during training, ensuring maximum information flow between layers and encouraging feature reuse.

[0139] By using the Leakly ReLU function instead of the ReLU function, the advantages of the ReLU function are retained while overcoming its shortcomings. This fully utilizes the available information in the image, making image feature learning more efficient and comprehensive. Experimental comparison data shows that this method further improves denoising performance.

[0140] The advantages of this invention are mainly twofold:

[0141] Structurally, a degenerate complex quaternion convolutional neural network structure, different from the real-valued convolutional neural network, is proposed. It uses a more holistic and scientific degenerate complex quaternion convolution calculation method for image feature extraction and network learning.

[0142] In terms of performance, the final denoising effect of this invention, measured by mean peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), shows through experimental data that, at σ=10, σ=20, σ=30, and σ=40, the method of this invention improves the PSNR by 0.29dB, 0.83dB, 0.67dB, and 0.48dB respectively compared to DnCNN, and improves the SSIM by 0.002, 0.017, 0.018, and 0.015 respectively. The degenerate complex quadrupole convolutional neural network maintains a high level of denoising performance even when facing higher intensity noise, demonstrating greater resilience compared to real-valued convolutional neural networks.

[0143] As an alternative to this approach, pure quaternions can be introduced into image denoising to establish a pure quaternion convolutional neural network model. This pure quaternion network model, like the degenerate complex quaternion network model, can consider the correlation between image dimensions and perform calculations on them as a whole.

[0144] For the structure proposed in the alternative scheme above, a weight initialization method for pure quaternion convolution kernels is proposed using the computational rules of pure quaternions, and a pure quaternion convolution layer is designed:

[0145] A pure quaternion can generally be represented as q = 0 + bi + cj + dk, where b, c, and d are real numbers, and i, j, and k are the imaginary units. 2 =k 2 =j 2 =-1.

[0146] Pure quaternion weight initialization

[0147] Proper initialization of weights is crucial for the convergence of deep networks. In ordinary real-valued neural networks, convolution can be viewed as scaling the pixels of an image in space, while pure quaternion (or quaternion) convolution simultaneously scales and rotates the pixels in space. Suppose we have a three-dimensional vector q = [q1 q2 q3]. T Around the grayscale line w = [w1 w2 w3] T Rotating by an angle θ yields the vector p = [p1 p2 p3] T ,in This process can be described as:

[0148]

[0149] in, and It consists of the pure quaternions corresponding to vectors q and p. According to the Fourier transform formula, the weights of the pure quaternion convolution kernel can be set as:

[0150]

[0151] in, It is a unit vector on the grayscale axis, and θ is the rotation angle. This is the modulus of the pure quaternion convolution kernel weights. Based on this formula, the rotation angles are first randomly initialized (θ∈(0,180)). Then, the modulus s of the weights is calculated using the He Kaiming initialization method. Multiplying this by the corresponding μ value yields the initial weight values ​​f1, f2, and f3 for the three dimensions, as well as the weight matrix W.

[0152]

[0153] in,

[0154] Pure Quaternion Convolutional Layer

[0155] Similar to degenerate complex quaternion convolutional layers, the convolution operation of a quaternion convolutional layer involves calculating a weight matrix W = f1i + f2j + f3k and an input vector h = ix + jy + kz, with the following calculation rules:

[0156]

[0157] Using the pure quaternion weight initialization method described above, a three-dimensional pure quaternion weight matrix W can be obtained. Performing a pure quaternion convolution multiplication with the input h yields the convolution result. This pure quaternion multiplication structure allows each dimension of the convolution kernel to interact with each dimension of the image input, making the convolution operation holistic.

[0158] The noisy image is trained through a pure quaternion convolutional neural network to obtain a reconstructed image. Finally, the mean squared error between the clear target image and the reconstructed image is used as the target loss function. The target loss function is iteratively trained and optimized using a stochastic gradient descent optimization algorithm for a fixed number of rounds (the number of rounds is determined based on the experimental data).

[0159] Due to the similarity of its quaternion structure, the denoising effect of this scheme can be comparable to that of the technical solution proposed in this invention. However, the denoising performance is reduced when dealing with Gaussian noise with a large variance distribution. Even so, this type of quaternion convolutional neural network can still slightly outperform the real-valued convolutional neural network denoising method of DnCNN.

[0160] The present invention also provides an image denoising system based on a degenerate complex quaternion convolutional neural network, comprising:

[0161] Target image acquisition module M1 is used to acquire target images;

[0162] The denoising module M2 is used to input the target image into the trained image denoising model to obtain the denoised image corresponding to the target image; the trained image denoising model is a model trained with a sample image as input and a clear image of the sample image before adding noise as a label.

[0163] The trained image denoising model employs a degenerate complex quaternion convolutional neural network model. This model is obtained by replacing the convolutional layers of a real-valued neural network model (excluding the first and last layers) with degenerate complex quaternion convolutional layers. The first convolutional layer of the real-valued neural network contains four convolutional kernels. The degenerate complex quaternion convolutional layer is operated by performing convolution operations between each axis of the quaternion convolutional kernel and each axis of the feature map.

[0164] Optionally, the system further includes:

[0165] Image denoising model training module: used to train the image denoising model before inputting the target image into the trained image denoising model.

[0166] Optionally, the image denoising model training module specifically includes:

[0167] The sample image acquisition submodule is used to acquire sample images, which are images obtained by adding noise to a clear image.

[0168] The model training submodule is used to train the image denoising model by taking the sample image as input and using the clear image of the sample image before denoising as the label, so as to obtain the trained image denoising model.

[0169] Optionally, the system further includes a weight initialization module for initializing the weights of the degenerate complex quaternion convolution kernel before training the image denoising model; the initialization method adopts the He initialization method.

[0170] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0171] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for image denoising based on degenerate complex quaternion convolutional neural network, characterized in that, The method comprises the following steps: obtaining a target image; inputting the target image into a trained image denoising model to obtain a denoised image corresponding to the target image; the trained image denoising model is a model trained by taking a sample image as input and taking a clear image before the sample image is added with noise as a label; wherein the trained image denoising model adopts a degenerate complex quaternion convolutional neural network model, the degenerate complex quaternion convolutional neural network model is obtained by replacing the convolutional layers of a real-valued neural network model except the first convolutional layer and the tail convolutional layer with degenerate complex quaternion convolutional layers, the first convolutional layer of the real-valued neural network comprises four convolutional kernels, and the operation method of the degenerate complex quaternion convolutional layer is a method of performing convolution operation one by one between each axis of a quaternion convolutional kernel and each axis of a feature map.

2. The method of claim 1, wherein, The real-valued neural network model adopts a Unet model.

3. The method according to claim 1 or 2, characterized in that, The degenerate complex quaternion convolutional neural network model adopts a Leakly ReLU function to activate each layer of neurons.

4. The method of claim 3, wherein, The degenerate complex quaternion convolutional neural network model adds a batch normalization layer in a hidden layer.

5. The method of claim 1, wherein, Before the target image is inputted into the trained image denoising model, the method further comprises training the image denoising model, and the training process is as follows: obtaining a sample image, the sample image being an image obtained by adding noise to a clear image; training an image denoising model by taking the sample image as input and taking a clear image before the sample image is added with noise as a label to obtain a trained image denoising model.

6. The method of claim 5, wherein, Before the image denoising model is trained, the method further comprises initializing the weight of the degenerate complex quaternion convolutional kernel; and the initialization method adopts a He initialization method.

7. An image denoising system based on degenerate complex quaternion convolutional neural network, characterized in that, The system comprises: a target image acquisition module configured to acquire a target image; a denoising module configured to input the target image into a trained image denoising model to obtain a denoised image corresponding to the target image; the trained image denoising model is a model trained by taking a sample image as input and taking a clear image before the sample image is added with noise as a label; wherein the trained image denoising model adopts a degenerate complex quaternion convolutional neural network model, the degenerate complex quaternion convolutional neural network model is obtained by replacing the convolutional layers of a real-valued neural network model except the first convolutional layer and the tail convolutional layer with degenerate complex quaternion convolutional layers, the first convolutional layer of the real-valued neural network comprises four convolutional kernels, and the operation method of the degenerate complex quaternion convolutional layer is a method of performing convolution operation one by one between each axis of a quaternion convolutional kernel and each axis of a feature map.

8. The system of claim 7, wherein, The system further comprises: an image denoising model training module configured to train the image denoising model before the target image is inputted into the trained image denoising model.

9. The system of claim 8, wherein, The image denoising model training module specifically comprises: a sample image acquisition submodule configured to acquire a sample image, the sample image being an image obtained by adding noise to a clear image; a model training submodule configured to train an image denoising model by taking the sample image as input and taking a clear image before the sample image is added with noise as a label to obtain a trained image denoising model.

10. The system of claim 8, wherein, The system further comprises a weight initialization module configured to initialize weights of a degraded complex quaternion convolution kernel before the image denoising model is trained. The initialization method adopts a He initialization method.

Images