Mobile device image denoising method based on double-branch residual sparse network

Abstract

The application discloses a kind of mobile device image denoising methods based on double-branch residual sparse network, belong to image processing technical field, which includes: residual sparse module, by mixed dilated convolution and residual connection capture local feature in image, while reducing the number of parameters and computational complexity of model;Attention guiding residual sparse module introduces channel attention and pixel attention mechanism on the basis of residual sparse block to adjust the weight of feature map, pay attention to important area in image, improve denoising effect and image quality;Feature fusion module, by adding double-branch output after residual module processing, combined with attention mechanism and activation function, realize high-frequency detail preservation and noise suppression.The application carries out image denoising in unmanned aerial vehicle and other resource-limited scenarios, solves the problems of large parameter quantity, low computational efficiency, insufficient detail preservation and other problems of existing denoising model.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and to an image denoising method based on a dual-branch residual sparse network with a series-parallel structure, applicable to resource-constrained mobile devices such as drones and mobile robots. Background Technology

[0002] Image denoising is a crucial task in image processing, aiming to reduce or eliminate noise in images while preserving as much detail and feature as possible. With the rapid development of mobile communication technology and digital imaging sensors, mobile phones and cameras have become the most commonly used shooting devices in daily life, leading to an ever-increasing demand for high-quality images. Therefore, developing a denoising technique that can effectively process noisy images and achieve good visual reproduction quality at low computational cost is particularly important.

[0003] Against this backdrop, image denoising techniques have made significant progress. For example, block-matching and 3D filtering algorithms produce sharper images while preserving details by grouping similar image blocks, applying 3D filtering to reduce noise, and averaging the results. The BM3D-Net convolutional neural network, inspired by the BM3D algorithm, models the image by unfolding the BM3D computation flow and introducing "extraction" and "aggregation" layers. In addition, there are various other denoising algorithms, such as weighted nuclear norm minimization, multi-channel WNNM, block-based sparse cooperative low-rank algorithms based on sparse and cooperative low-rank matrix factorization, and 3D magnetic resonance imaging denoising models.

[0004] While traditional denoising models perform well under specific conditions, they are often limited by the large number of parameters. Deep neural network-based denoising models improve performance by learning complex features and patterns, typically requiring fewer hyperparameters and offering faster inference speeds, thus achieving better results in a variety of denoising tasks. For example, deep convolutional neural networks (CNNs) improve denoising performance by learning the noise distribution in an image and the residuals during the denoising process. Fast and flexible denoising networks (FCLs) achieve efficient image denoising with lower computational cost and smaller model size.

[0005] However, as network depth increases, the number of parameters and operation time also increase, making the model difficult to deploy in practical applications. To address this issue, researchers have enhanced the representational power of denoising networks by broadening the network structure, such as Deep Convolutional Neural Networks (DCNNs) and Batch Renormalization Deep Networks (BRNNs). Furthermore, attention mechanisms are playing an increasingly important role in image denoising, enabling models to better understand image content, eliminate noise, and thus improve image quality and clarity.

[0006] While existing deep learning networks perform admirably in image denoising, some limitations remain. Larger network models, although achieving good denoising results, place higher demands on hardware, have longer runtimes, and consume more memory. Conversely, smaller network models, while meeting hardware requirements in terms of speed and space, suffer from relatively insufficient denoising performance. Therefore, researching a network model that satisfies performance requirements while maintaining low complexity is a challenging direction. Summary of the Invention

[0007] The purpose of this invention is to propose a mobile device image denoising method based on a dual-branch residual sparse network to solve the problems in the existing technology, such as the difficulty in balancing denoising performance and model complexity, the difficulty in scaling up on small mobile devices, and the inability to be well applied to various image processing scenarios.

[0008] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0009] A mobile device image denoising method based on a dual-branch residual sparse network includes the following steps:

[0010] S1. Data Preparation and Preprocessing:

[0011] Images in various scenarios are captured using mobile devices. The images are then labeled and enhanced to construct a mobile device image dataset. The images in the dataset are then preprocessed, including normalization, grayscale conversion, resolution adjustment, and data segmentation.

[0012] S2. Construct a dual-branch residual sparse denoising network based on a series-parallel structure:

[0013] The dual-branch residual sparse denoising network includes an upper branch network, a lower branch network, and a feature fusion module. The upper branch network consists of five residual sparse modules (RSBs) connected in series through upsampling and downsampling, used to progressively extract image features and capture multi-scale information. The lower branch network consists of five attention-guided residual sparse modules (ARSBs) connected in series. The feature fusion module is used to integrate the feature information from the upper and lower branch networks, perform final residual learning, and output the denoised image.

[0014] S3. The loss function L of the bi-branch residual sparse denoising network is designed as follows:

[0015]

[0016] in, and These represent the image after denoising by the model and the corresponding clean image, respectively; N represents the total number of images.

[0017] The mobile device images obtained in S4 and S1 are divided into training and test sets after data segmentation. The training set images are input into the constructed bi-branch residual sparse denoising network. The gradient of the loss function is calculated using the backpropagation method until the loss function tends to stabilize, and the trained bi-branch residual sparse denoising network model is obtained.

[0018] S5. Input the noisy images in the test set into the trained dual-branch residual sparse denoising network to obtain the denoised images.

[0019] Preferably, each of the residual sparse modules (RSBs) includes standard convolution and dilated convolution, and the input features are added to the output features through residual connections to avoid the gradient vanishing problem and improve training stability;

[0020] The Attention-Guided Residual Sparse Module (ARSB) adds channel attention and pixel attention mechanisms to the residual sparse module, enabling the network to focus on key areas of the image and improve the denoising effect.

[0021] The Residual Sparse Module (RSB) and Attention-Guided Residual Sparse Module (ARSB) utilize hybrid dilated convolutions to increase the receptive field without increasing computational cost; dilated convolutions expand the coverage by introducing gaps into the convolution kernel without increasing parameters, thereby improving the model's receptive field and feature extraction capabilities without increasing computational cost, thus achieving model lightweighting.

[0022] Preferably, the method for constructing the upper branch network in S2 specifically includes:

[0023] 1) A standard convolutional module is composed of a standard convolutional layer with a kernel size of 3×3 and a stride of 1, batch normalization, and ReLU activation function;

[0024] 2) The dilated convolutional module is composed of a 3×3 kernel, a stride of 1, and dilation rates of 2 and 3, along with batch normalization and ReLU activation function.

[0025] 3) Connect the standard convolutional module, the dilated convolutional module with a dilation rate of 2, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3 in sequence, and make residual connections with the input to form a residual sparse module.

[0026] 4) Connect the five residual sparse modules in sequence. The first three residual sparse modules are connected by downsampling, and the last three residual sparse modules are connected by upsampling to form an upper branch network.

[0027] The function representation of the upper branch network is as follows:

[0028]

[0029] in, This represents the output of each RSB module, i∈1,2,3,4,5; O shallow Represents the input of the upper branch network; RSB represents the residual sparse module; Down and Up represent downsampling and upsampling, respectively; This represents the output of the upper branch network of the deep feature extraction module.

[0030] Preferably, the residual sparse module in the upper branch network expands the receptive field of the upper branch network by combining standard convolution and dilated convolution operations to capture multi-scale feature information, as detailed below:

[0031] First, image features are extracted using standard convolutional layers. Then, dilated convolutions (with dilation rates of 2 and 3) are used to increase the receptive field, thereby obtaining a wider range of contextual information while ensuring computational efficiency.

[0032] Residual connections are applied between the input and output of modules to alleviate the vanishing gradient problem in deep networks, promote information flow, and accelerate network training and convergence. Their functional representation is as follows:

[0033] O RSB =RSB(f rsb )+f rsb =DBR3(CBR(DBR2(CBR(f) rsb ))))+f rsb

[0034] Among them, f rsb1 represents the input of the RSB module; CBR represents the standard convolution module; DBR2 represents the dilated convolution module with a dilation rate of 2; DBR3 represents the dilated convolution module with a dilation rate of 3; the standard convolution and dilated convolution both have 64 channels and a 3×3 kernel.

[0035] Preferably, the method for constructing the lower branch network in S2 specifically includes:

[0036] 1) Connect the standard convolutional module and the dilated convolutional module with a dilation rate of 2 and perform residual connections to form an internal residual sparse module;

[0037] 2) The internal residual sparse module, channel attention mechanism, pixel attention mechanism, standard convolution module and dilated convolution module with dilation rate of 3 are connected in sequence and residually connected to the input to form an attention-guided residual sparse module.

[0038] 3) Connect the five attention-guided residual sparse modules sequentially to form the lower branch network;

[0039] The function representation of the lower branch network is as follows:

[0040]

[0041] in, This represents the output of each ARSB module, i∈1,2,3,4,5; O shallow Represents the input to the upper branch network; ARSB represents the attention-guided residual sparse module; This represents the output of the lower branch network of the deep feature extraction module.

[0042] Preferably, the attention-guided residual sparse module in the lower branch network specifically includes the following:

[0043] First, standard convolutional modules and dilated convolutional modules with a dilation rate of 2 are sequentially connected and internal residual sparse modules are formed through residual connections. Local features are extracted using standard convolution, while dilated convolution captures broader contextual information by increasing the receptive field.

[0044] Next, the internal residual sparse module is sequentially connected with the channel attention mechanism, the pixel attention mechanism, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3 to form the attention-guided residual sparse module. In the attention-guided residual sparse module, the channel and pixel attention mechanisms adaptively assign weights to different features, thereby enhancing the network's attention to important features.

[0045] Finally, the module's output is added to the initial input via residual connections to ensure efficient information flow and enhance gradient propagation during training. Attention-guided residual sparse modules, combining multi-scale feature extraction, attention mechanisms, and residual connections, improve the performance and stability of the lower branch network in complex tasks. Their functional representation is as follows:

[0046] O ARSB =ARSB(f arsb )+f arsb =DBR3(CBR(CAB(PAB(DBR2(CBR(f arsb )))))+f arsb )+f arsb

[0047] Among them, f arsb This represents the input of the ARSB module; CAB represents the channel attention module; and PAB represents the pixel attention module.

[0048] Preferably, the feature fusion module in S2 specifically includes the following:

[0049] Feature fusion input: The outputs of the upper branch network and the lower branch network are added together and used as the input to the feature fusion module for feature fusion.

[0050] Module composition: The feature fusion module consists of a standard convolution module, an attention-guided residual sparse module, two standard convolutions, and a sigmoid activation function;

[0051] Feature fusion process:

[0052] 1) The input of the feature fusion module is passed sequentially through the standard convolution module, the attention-guided residual sparse module, and the first standard convolution;

[0053] 2) Concatenate the above output with the normalized noisy image, and then pass it through the Sigmoid activation function and the second standard convolution in sequence;

[0054] 3) Multiply the output of the first standard convolution with the output of the second standard convolution, and finally calculate the residual with the original noisy image to obtain the final denoised image;

[0055] The feature fusion process can be represented as follows:

[0056]

[0057] Among them, f rb Represents the input of the RB module; cat represents the cascade operation; C represents standard convolution; I n This represents the input noisy image; Sig represents the Sigmoid function.

[0058] Preferably, the step S4, which involves calculating the gradient of the loss function using backpropagation to train the model, is implemented as follows:

[0059] 1) Set the dual-branch residual sparse denoising network to training mode;

[0060] 2) Input the noisy images in the training set into the dual-branch residual sparse denoising network, perform forward propagation, calculate and output the fused denoised image;

[0061] 3) Calculate the loss between the model output and the original clean image based on the loss function;

[0062] 4) Calculate the gradient of the loss function using the backpropagation method;

[0063] 5) Use the Adam optimizer to update the model parameters based on the gradient change of the loss function. The learner rate of the optimizer is set to 0.001, and the hyperparameters β1 = 0.9 and β2 = 0.99.

[0064] 6) Repeat the above process until the loss function reaches a stable state, and obtain the trained bi-branch residual sparse denoising network model.

[0065] Based on the above method, a mobile device image denoising system applying this method is further proposed, comprising:

[0066] The Residual Sparse Module (RSB) contains standard convolutional modules and dilated convolutional modules to capture local and global features of an image. It also utilizes batch normalization and ReLU activation functions to enhance the nonlinear expressive power of the network and alleviates the gradient vanishing problem in deep network training through residual connections, thereby accelerating the training process and improving the model's learning efficiency of image features.

[0067] Attention-Guided Residual Sparse Module (ARSB) incorporates standard convolutions and dilated convolutions to capture image features. By combining dilated convolutions and residual connections, it achieves a balance between network depth and breadth, effectively avoiding the limitations of standard convolutions and the inefficiency of dilated convolutions. At the same time, it introduces channel attention and pixel attention mechanisms to enhance the network's attention to important features, thereby improving the denoising effect.

[0068] The Feature Fusion Module (RB) includes convolutional layers, an attention-guided module, and skip connections. The convolutional layers extract image features, the attention-guided module identifies and enhances important features in the image while suppressing noise, and the skip connections ensure that the input is directly passed to the output, enabling the network to learn the residual between the input and output, thereby avoiding the gradient vanishing problem in deep network training.

[0069] Compared with the prior art, the present invention has the following advantages:

[0070] (1) Balance between lightweight and high performance: This invention maintains excellent noise reduction performance while significantly reducing the number of network parameters through a unique dual-branch serial-parallel structure and optimized network design.

[0071] (2) Enhanced detail preservation and noise reduction capabilities: This invention introduces residual sparse blocks and attention-guided residual sparse blocks, which, combined with hybrid dilated convolution and attention mechanisms, can extract the structural and texture information of the image more comprehensively, while better preserving details during the noise reduction process. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings involved in the embodiments are now briefly described. Obviously, the drawings in the following description are merely illustrative of some embodiments of the present invention. For those skilled in the art, other forms of drawings can be constructed based on these drawings without creative effort.

[0073] Figure 1 This is an overall flowchart of the mobile device image denoising method based on a dual-branch residual sparse network proposed in this invention.

[0074] Figure 2 This is a schematic diagram of the dual-branch residual sparse denoising network structure constructed in Embodiment 1 of the present invention;

[0075] Figure 3 For the present invention Figure 2 Residual sparse module structure diagram;

[0076] Figure 4 For the present invention Figure 2 The structure diagram of the attention-guided residual sparse module in the diagram;

[0077] Figure 5 For the present invention Figure 2 Block diagram of the feature fusion module in the middle;

[0078] Figure 6 This is a comparison chart showing the denoising effects of the present invention and several existing denoising methods in high-noise scenarios. Detailed Implementation

[0079] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0080] This invention proposes a mobile device image denoising method based on a dual-branch residual sparse network to balance denoising performance and model complexity. The invention employs a serial-parallel structure, including a residual sparse module (RSB) and an attention-guided residual sparse module (ARSB), connected by upsampling and downsampling operations to effectively capture multi-scale information of the image. The RSB and ARSB modules utilize hybrid dilated convolutions to increase the receptive field without increasing computational cost. Dilated convolutions expand the coverage by introducing gaps into the convolution kernel without increasing parameters, thereby improving the model's receptive field and feature extraction capabilities without increasing computational cost, thus achieving a lightweight model. This invention aims to remove various types of noise while preserving image details and edge information, making it suitable for deployment and application on mobile devices.

[0081] The embodiments and effects of the present invention will be further described in detail below with reference to the accompanying drawings and specific examples.

[0082] Example 1:

[0083] Reference Figure 1 This invention proposes a mobile device image denoising method based on a dual-branch residual sparse network, the implementation steps of which include the following:

[0084] Step 1: Acquisition and preprocessing of the dataset.

[0085] To improve the performance of the model in image denoising, the Waterloo Exploration Database (Waterloo Exploration Dataset) was used for training, containing 4744 natural images. To improve computational resource utilization, the training images were randomly cropped into 128×128 image patches according to the receptive field size of the model. For real-world noise reduction evaluation, the SIDD (Smartphone Image Denoising Dataset) medium-sized dataset was selected as the training set. This dataset contains 320 pairs of high-resolution noisy images and their corresponding clean images. The high-resolution images were cropped into 128×128 image patches, and the training samples were enhanced through rotation and flipping operations. During the model evaluation phase, seven public datasets were used as the test set, including Set12 and BSD68 for grayscale images, CBSD68, Kodak24, and McMaster for color images, and the SIDD validation set and DND sRGB dataset for real images.

[0086] Step 2: Construct an image denoising network based on a dual-branch residual sparse network with a series-parallel structure.

[0087] like Figure 2As shown, the image denoising network based on a serial-parallel structure bi-branch residual sparse network includes an upper branch network, a lower branch network, and a feature fusion module (RB).

[0088] 2.1) Constructing the upper branch network:

[0089] like Figure 2 As shown, the upper branch network consists of five residual sparse modules connected in series. The first three residual sparse modules are connected through downsampling, and the last three residual sparse modules are connected through upsampling.

[0090] The formula for the upper branch network is as follows:

[0091]

[0092] in, This is the output of each RSB module, O shallow This represents the input to the upper branch network, RSB represents the residual sparse module, and Down and Up represent downsampling and upsampling, respectively. This represents the output of the upper branch network of the deep feature extraction module.

[0093] like Figure 3 As shown, the residual sparse module consists of two standard convolutional modules and two dilated convolutional modules. The standard convolutional modules consist of a 3×3 kernel layer with a stride of 1, batch normalization, and a ReLU activation function. The dilated convolutional modules consist of dilated convolutional layers with kernel sizes of 3×3, strides of 1, and dilation rates of 2 and 3 respectively, batch normalization, and a ReLU activation function. The residual sparse module is formed by sequentially connecting the standard convolutional modules, the dilated convolutional module with a dilation rate of 2, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3, and then performing a residual connection with the initial input of the module. Its expression is as follows:

[0094] O RSB =RSB(f rsb )+f rsb =DBR3(CBR(DBR2(CBR(f) rsb ))))+f rsb

[0095] Among them, f rsb 1 represents the input of the RSB module; CBR represents the standard convolution module; DBR2 represents the dilated convolution module with a dilation rate of 2; DBR3 represents the dilated convolution module with a dilation rate of 3; the standard convolution and dilated convolution both have 64 channels and a 3×3 kernel.

[0096] 2.2) Construct the lower branch network:

[0097] like Figure 2As shown, it consists of five attention-guided residual sparse modules connected in series.

[0098] The formula for the lower branch network is as follows:

[0099]

[0100] in, Indicates the output of each ARSB module, O shallow This represents the input to the upper branch network, and ARSB represents the attention-guided residual sparse module. This represents the output of the lower branch network of the deep feature extraction module.

[0101] like Figure 4 As shown, the attention-guided residual sparse module consists of two standard convolutional modules, two dilated convolutional modules, a channel attention mechanism, and a pixel attention mechanism. The standard convolutional modules consist of a 3×3 kernel layer with a stride of 1, batch normalization, and a ReLU activation function. The dilated convolutional modules consist of dilated convolutional layers with kernel sizes of 3×3, strides of 1, and dilation rates of 2 and 3 respectively, batch normalization, and a ReLU activation function. First, the standard convolutional modules and the dilated convolutional modules with a dilation rate of 2 are connected and residually connected to form the inner residual sparse module. Then, the inner residual sparse module, the channel attention mechanism, the pixel attention mechanism, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3 are sequentially connected and residually connected to the initial input of the module to form the attention-guided residual sparse module. Its specific expression is as follows:

[0102] O ARSB =ARSB(f arsb )+f arsb =DBR3(CBR(CAB(PAB(DBR2(CBR(f arsb )))))+f arsb )+f arsb

[0103] Among them, f arsb This represents the input of the ARSB module, CAB represents the channel attention module, and PAB represents the pixel attention module.

[0104] 2.3) Constructing a feature fusion module:

[0105] like Figure 5As shown, the feature fusion module consists of a standard convolution module, an attention-guided residual sparse module, two standard convolutions, and a sigmoid activation function. The input of the feature fusion module is passed through the standard convolution module, the attention-guided residual sparse module, and the first standard convolution in sequence. The output of the first standard convolution is concatenated with the normalized noise image and then passed through the sigmoid activation function and the second standard convolution in sequence. The outputs of the first and second standard convolutions are multiplied together, and finally, the residual is calculated with the original noise image to obtain the denoised image.

[0106] Its specific expression is as follows:

[0107]

[0108] Among them, f rb Represents the input of the RB module; cat represents the cascade operation; C represents standard convolution; I n This represents the input noisy image; Sig represents the Sigmoid function.

[0109] Step 3, construct the loss function L for the bi-branch residual sparse denoising network based on a series-parallel structure: Use the MSE (Mean Squared Error) loss function as the loss function for the bi-branch residual sparse denoising network:

[0110]

[0111] in, and These represent the image after denoising by the model and the corresponding clean image, respectively, where N is the total number of images;

[0112] Step 4: Train the dual-branch residual sparse denoising network based on a series-parallel structure:

[0113] 4.1) Set the dual-branch residual sparse denoising network to training mode;

[0114] 4.2) The noisy images in the training set are forward-propagated through a two-branch residual sparse denoising network to calculate and output the fused denoised image;

[0115] 4.3) Calculate the loss between the model output and the original image based on the loss function;

[0116] 4.4) Calculate the gradient of the loss function using the backpropagation method;

[0117] 4.5) The Adam optimizer is used to update the model parameters based on the gradient change of the loss function. The learning rate of the optimizer is set to 0.001, and the hyperparameters β1 = 0.9 and β2 = 0.99.

[0118] 4.6) Repeat steps 4.2) to 45) until the loss function stabilizes, and you will get the trained bi-branch residual sparse denoising network model.

[0119] Step 5: Use the trained denoising network to denoise the image:

[0120] 5.1) Set the feature interaction complementarity learning denoising network to test mode;

[0121] 5.2) Input the noisy images in the test set into the trained dual-branch residual sparse denoising network to obtain the denoised images.

[0122] The effects of the present invention will be further explained below with reference to experiments.

[0123] 1. Experimental conditions:

[0124] The experiments of this invention were conducted on a high-performance computing platform, with hardware configuration including an AMD EPYC 8255C / 2.50GHz 12-core CPU, 128GB of memory, and an Nvidia GeForce GTX 2080Ti GPU. The software environment was based on the Ubuntu 20.04 operating system, developed and implemented using the PyTorch 1.11.0 framework and Python 3.8, and accelerated deep learning tasks using CUDA 11.3 and cuDNN 8.04. This combination of hardware and software ensured that the model could efficiently handle large-scale datasets and significantly improved training speed and computational efficiency.

[0125] 2. Experiment Content:

[0126] The training dataset for this experiment is divided into two parts: a synthetic noise image training dataset and a real noise image training dataset.

[0127] For synthetic noise removal, the invented model was trained using the Waterloo Exploration Database (WED), which contains 4744 natural images. To fully utilize computational resources, the receptive field size of the model was calculated, and images in the training dataset were randomly cropped to 128×128 pixels for image denoising. For real-world noise reduction evaluation, the SIDD medium-sized dataset was selected as the training set for the network model. This dataset contains 320 pairs of noisy HR images and their corresponding clean images. These HR images were arbitrarily cropped into 128×128 image patches, and rotation and flipping operations were used to enhance the training samples.

[0128] During the model evaluation phase, to test the model's performance in removing additive white Gaussian noise (AWGN), five public datasets were used as test sets: Set12 and BSD68 for grayscale images, and CBSD68, Kodak24, and McMaster for color images. To test the model's performance in denoising real images, the SIDD validation set and the DND sRGB dataset were selected as test datasets.

[0129] This invention uses PSNR (Peak Signal-to-Noise Ratio) and SSIM (Structural Similarity Index) as evaluation metrics for image denoising performance.

[0130] PSNR (Peak Signal-to-Noise Ratio) is a commonly used evaluation metric in image denoising, used to quantify the quality of image restoration. PSNR measures the difference between a noise-reduced image and the original image; a higher value indicates higher restoration quality. The PSNR formula is defined as:

[0131]

[0132] Among them, MAX I It is the maximum possible pixel value in the image (MAX for an 8-bit image). I =255), MSE is the Mean Squared Error, defined as the average of the sum of squares of the pixel differences between the recovered image and the original image, and the formula is:

[0133]

[0134] Where I(i,j) and K(i,j) are the pixel values ​​of the original image and the restored image, respectively, and m and n are the dimensions of the image.

[0135] SSIM (Structural Similarity Index) is a metric used to measure the similarity between two images, focusing specifically on the structural information of the images rather than just differences in pixel values. It is designed based on the principles of human visual perception, aiming to evaluate image quality by simulating the perceptual mechanisms of the human eye. SSIM values ​​range from 0 to 1; the closer a value is to 1, the more similar the two images are; the closer a value is to 0, the greater the difference between the two images.

[0136] Gaussian white noise with σ = 15, 25, and 50 was added to the set12 grayscale image test dataset. Image denoising was performed using existing mainstream methods and the method of this invention, respectively. The PSNR and SSIM of the denoised images obtained by each method and the standard denoised image are shown in Table 1:

[0137] Table 1. Comparison of the set12 dataset metrics between this invention and five existing mainstream methods.

[0138]

[0139] Gaussian white noise with σ = 15, 25, and 50 was added to the CBSD68 color image test dataset. Image denoising was performed using existing mainstream methods and the method of this invention, respectively. The PSNR of the denoised images obtained by each method and the standard denoised image are shown in Table 2.

[0140] Table 2 Comparison of CBSD68 dataset metrics between the present invention and five existing mainstream methods

[0141]

[0142] On the SIDD and DND real image test datasets, image denoising was performed using existing mainstream methods and the method of this invention, respectively. The PSNR and SSIM of the denoised images obtained by each method and the standard denoised image are shown in Table 3:

[0143] Table 3 Comparison of SIDD and DND dataset metrics between this invention and five existing mainstream methods

[0144] Noise reduction methods SIDD DND BM3D 33.40 / 0.879 37.38 / 0.929 TNRD 35.33 / 0.933 37.94 / 0.940 FFDNet 36.24 / 0.943 38.63 / 0.946 DnCNN 23.66 / 0.583 37.90 / 0.943 VDN 39.28 / 0.956 39.38 / 0.952 This invention 39.51 / 0.957 39.66 / 0.957

[0145] The higher the Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity (SSIM) in the table, the closer the denoised image is to the standard image in the dataset, and the better the denoising effect.

[0146] As shown in Tables 1, 2 and 3, the method proposed in this invention is superior to other mainstream methods in both low-noise and high-noise scenarios.

[0147] The comparison of the denoising effects of this invention and several existing denoising methods in a high-noise scene with σ=15 is shown in the figure below. Figure 6 As shown, (a) is the original clean image, (b) is the noisy image, (c) is the denoising effect of the BM3D method, (d) is the denoising effect of the DnCNN method, (e) is the denoising effect of the ADNet method, and (f) is the denoising effect of the method of the present invention.

[0148] from Figure 6 As can be seen, the present invention can retain more image texture features while removing noise, thus obtaining a higher quality denoised image.

[0149] The complexity of the proposed model was comprehensively evaluated. The lightweight nature and performance advantages of the model were demonstrated through comparisons of the number of parameters, floating-point operations (flops), average runtime, and memory usage. To ensure fairness, all denoising models were implemented on the same device. As shown in Table 4, tests were conducted on images of different sizes (128×128, 256×256, and 512×512) at a noise level of 15. The average runtime was calculated based on 100 runs of the denoising model for each image. Experimental results show that compared to the DamNet model with a similar number of parameters, the proposed model reduces the number of floating-point operations by 51.3% and the runtime by 62.2%. This indicates that the proposed model has a significant advantage in computational efficiency and can complete the denoising task faster. Furthermore, the memory usage of the proposed model is reduced by 33.8% compared to DamNet, meaning that the proposed model is more efficient in terms of resource consumption and suitable for deployment on memory-constrained devices.

[0150] Despite optimizations in model complexity, the proposed model still exhibits superior peak signal-to-noise ratio (PSNR) performance in experiments. This demonstrates that the proposed model is not only lightweight but also performs exceptionally well in terms of image denoising quality and detail preservation, achieving efficient image denoising with low computational cost and making it suitable for various practical applications.

[0151] Table 4. Flops (G), runtime (ms), memory usage (M), and PSNR results of this invention compared to existing mainstream methods.

[0152]

[0153]

[0154] The specific embodiments described herein are merely illustrative of the technical solutions of the present invention and do not constitute a limitation thereof. Although the present invention has been described in detail through specific embodiments, those skilled in the art should understand that various modifications, variations, and equivalent substitutions can be made to the present invention without departing from the basic idea and scope of the technical solutions. All such modifications, variations, and substitutions should be considered within the scope of protection of the claims of the present invention and therefore should be covered by the patent claims of the present invention.

Claims

1. A mobile device image denoising method based on a dual-branch residual sparse network, characterized in that, It includes the following steps: S1. Data Preparation and Preprocessing: Images in various scenarios are captured using mobile devices. The images are then labeled and enhanced to construct a mobile device image dataset. The images in the dataset are then preprocessed, including normalization, grayscale conversion, resolution adjustment, and data segmentation. S2. Construct a dual-branch residual sparse denoising network based on a series-parallel structure: The dual-branch residual sparse denoising network includes an upper branch network, a lower branch network, and a feature fusion module. The upper branch network is composed of five residual sparse modules connected in series through upsampling and downsampling, used to progressively extract image features and capture multi-scale information. The lower branch network is composed of five attention-guided residual sparse modules connected in series. The feature fusion module is used to integrate the feature information of the upper branch network and the lower branch network, perform final residual learning, and output the denoised image. Each of the residual sparse modules includes standard convolution and dilated convolution, which add the input features and output features through residual connections to avoid the gradient vanishing problem and improve training stability; The attention-guided residual sparse module adds channel attention and pixel attention mechanisms on the basis of the residual sparse module, so that the network focuses on the key areas of the image and improves the denoising effect. The residual sparse module and the attention-guided residual sparse module utilize hybrid dilated convolution to increase the receptive field without increasing computational cost; dilated convolution expands the coverage by introducing gaps in the convolution kernel without increasing parameters, thereby improving the model's receptive field and feature extraction capabilities without increasing computational cost, thus achieving model lightweighting. The feature fusion module specifically includes the following components: Feature fusion input: The outputs of the upper branch network and the lower branch network are added together and used as the input to the feature fusion module for feature fusion. Module composition: The feature fusion module consists of a standard convolution module, an attention-guided residual sparse module, two standard convolutions, and a sigmoid activation function; Feature fusion process: 1) The input of the feature fusion module is passed sequentially through the standard convolution module, the attention-guided residual sparse module, and the first standard convolution; 2) Concatenate the above output with the normalized noisy image, and then pass it through the Sigmoid activation function and the second standard convolution in sequence; 3) Multiply the output of the first standard convolution with the output of the second standard convolution, and finally calculate the residual with the original noisy image to obtain the final denoised image; The feature fusion process can be represented functionally as follows: in, f rb This represents the input to the RB module; RB represents the feature fusion module. cat Indicates a cascading operation; C Represents standard convolution; I n This represents the input noisy image; Sig Represents the Sigmoid function; This represents the output of the upper branch network of the deep feature extraction module; This represents the output of the lower branch network of the deep feature extraction module; CBR This represents a standard convolutional module; ARSB This indicates an attention-guided residual sparse module; S3. Design the loss function for a bi-branch residual sparse denoising network. L for: in, and These represent the image after denoising by the model and the corresponding clean image, respectively. N Indicates the total number of images; The mobile device images obtained in S4 and S1 are divided into training and test sets after data segmentation. The training set images are input into the constructed bi-branch residual sparse denoising network. The gradient of the loss function is calculated using the backpropagation method until the loss function tends to stabilize, and the trained bi-branch residual sparse denoising network model is obtained. S5. Input the noisy images in the test set into the trained dual-branch residual sparse denoising network to obtain the denoised images.

2. The mobile device image denoising method based on a dual-branch residual sparse network according to claim 1, characterized in that, The method for constructing the upper branch network described in S2 specifically includes: 1) A standard convolutional module is composed of a standard convolutional layer with a kernel size of 3×3 and a stride of 1, batch normalization, and ReLU activation function; 2) The dilated convolutional module is composed of a 3×3 kernel, a stride of 1, and dilation rates of 2 and 3, along with batch normalization and ReLU activation function. 3) Connect the standard convolutional module, the dilated convolutional module with a dilation rate of 2, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3 in sequence, and make residual connections with the input to form a residual sparse module; 4) Connect the five residual sparse modules in sequence. The first three residual sparse modules are connected by downsampling, and the last three residual sparse modules are connected by upsampling to form an upper branch network. The function representation of the upper branch network is as follows: in, Indicate each RSB The module's output, i ∈1,2,3,4,5; O shallow This represents the input to the upper branch network; RSB Represents a residual sparse module; Down and Up These represent downsampling and upsampling, respectively. This represents the output of the upper branch network of the deep feature extraction module.

3. The mobile device image denoising method based on a dual-branch residual sparse network according to claim 2, characterized in that, The residual sparse module in the upper branch network expands the receptive field of the upper branch network by combining standard convolution and dilated convolution operations to capture multi-scale feature information, as detailed below: First, image features are extracted using standard convolutional layers. Then, dilated convolutions are used to increase the receptive field, thereby obtaining a wider range of contextual information while ensuring computational efficiency. Residual connections are applied between the input and output of modules to alleviate the vanishing gradient problem in deep networks, promote information flow, and accelerate network training and convergence. Their functional representation is as follows: in, f rsb express RSB Module input; CBR This represents a standard convolutional module; DBR 2 indicates a dilated convolutional module with a dilation rate of 2; DBR 3 indicates a dilated convolution module with a dilation rate of 3; both standard convolution and dilated convolution have 64 channels and a 3×3 kernel.

4. The lower branch network of the mobile device image denoising method based on a dual-branch residual sparse network according to claim 1, characterized in that, The method for constructing the lower branch network described in S2 specifically includes: 1) Connect the standard convolutional module and the dilated convolutional module with a dilation rate of 2 and perform residual connections to form an internal residual sparse module; 2) The internal residual sparse module, channel attention mechanism, pixel attention mechanism, standard convolution module and dilated convolution module with dilation rate of 3 are connected in sequence and residually connected to the input to form an attention-guided residual sparse module. 3) Connect the five attention-guided residual sparse modules sequentially to form the lower branch network; The function representation of the lower branch network is as follows: in, Indicate each ARSB The module's output, i ∈1,2,3,4,5; O shallow This represents the input to the upper branch network; ARSB This indicates an attention-guided residual sparse module; This represents the output of the lower branch network of the deep feature extraction module.

5. The mobile device image denoising method based on a dual-branch residual sparse network according to claim 4, characterized in that, The attention-guided residual sparse module in the lower branch network specifically includes the following: First, standard convolutional modules and dilated convolutional modules with a dilation rate of 2 are sequentially connected and internal residual sparse modules are formed through residual connections. Local features are extracted using standard convolution, while dilated convolution captures broader contextual information by increasing the receptive field. Next, the internal residual sparse module is sequentially connected with the channel attention mechanism, the pixel attention mechanism, the standard convolutional module, and the dilated convolutional module with a dilation rate of 3 to form the attention-guided residual sparse module. In the attention-guided residual sparse module, the channel and pixel attention mechanisms adaptively assign weights to different features, thereby enhancing the network's attention to important features. Finally, the module's output is added to the initial input via residual connections to ensure efficient information flow and enhance gradient propagation during training. Attention-guided residual sparse modules, combining multi-scale feature extraction, attention mechanisms, and residual connections, improve the performance and stability of the lower branch network in complex tasks. Their functional representation is as follows: in, f arsb express ARSB Module input; CAB This indicates the channel attention module; PAB This represents the pixel attention module.

6. The mobile device image denoising method based on a dual-branch residual sparse network according to claim 1, characterized in that, The steps described in S4 for calculating the gradient of the loss function using backpropagation and training the model are as follows: 1) Set the dual-branch residual sparse denoising network to training mode; 2) Input the noisy images in the training set into the dual-branch residual sparse denoising network, perform forward propagation, calculate and output the fused denoised image; 3) Calculate the loss between the model output and the original clean image based on the loss function; 4) Calculate the gradient of the loss function using the backpropagation method; 5) Use the Adam optimizer to update the model parameters based on the gradient change of the loss function. The learner rate of the optimizer is set to 0.001, and the hyperparameters are... β 1 = 0.9 β 2 = 0.99; 6) Repeat the above process until the loss function stabilizes, and you will get the trained bi-branch residual sparse denoising network model.

Images