Image noise reduction method and device

Abstract

The invention provides an image noise reduction method and device. The method comprises the following steps: preprocessing an original Bayer image; carrying out spatial domain feature coding processing on the preprocessed image; performing frequency domain feature coding processing on the preprocessed image; and carrying out spatial domain and frequency domain feature fusion and reconstruction on the image subjected to spatial domain feature coding processing and frequency domain feature coding processing to obtain a denoised image. The method can give consideration to the balance of noise reduction and detail protection.

Description

Technical Field

[0001] This invention belongs to the field of image and video processing technology, and specifically relates to an image noise reduction method and apparatus. Background Technology

[0002] In scenarios such as mobile terminals, security monitoring, and vehicle cameras, raw Bayer mode images (RAW images) captured under low-light conditions are often subject to various noise interferences, including photon noise, readout noise, and quantization noise, resulting in insufficient image brightness, blurred details, and color distortion. Traditional image denoising methods typically suppress noise through spatial domain filtering or block matching methods, such as mean filtering, bilateral filtering, and BM3D. These methods are simple to implement but tend to lose edge and texture details. In recent years, deep learning methods have been introduced into RAW image denoising tasks. These end-to-end convolutional neural networks can automatically learn complex noise distributions, thereby significantly improving denoising performance.

[0003] Insufficient balance between noise reduction and detail preservation: While traditional filtering methods or some deep learning methods can reduce noise, they lack the joint utilization of multi-scale features, making it difficult to simultaneously preserve high-frequency details and low-frequency structural information. High computational resource consumption, hindering real-time applications: Most deep learning methods have large model parameters and complex computations, making them particularly difficult to meet real-time requirements on mobile terminals or embedded devices. Limitations of single-domain feature utilization: Existing methods mostly utilize only spatial or frequency domain features, failing to effectively fuse spatial and frequency domain information, resulting in limited accuracy in image detail restoration and limited ability to model complex noise.

[0004] How to provide an effective, reliable, and easy-to-build image denoising method that balances image denoising with image detail preservation is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of this, the present invention provides an image noise reduction method and apparatus to overcome the shortcomings of the prior art.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] This invention provides an image noise reduction method, comprising:

[0008] Preprocess the original Bayer image;

[0009] Spatial domain feature encoding is performed on the preprocessed image;

[0010] The preprocessed image is then subjected to frequency domain feature encoding.

[0011] The image after spatial and frequency domain feature fusion and reconstruction is obtained by processing the image with spatial and frequency domain features.

[0012] Furthermore, the preprocessing of the original Bayer image specifically includes:

[0013] The Bayer image is split into four channels: one red channel (R), two green channels (Gr and Gb), and one blue channel (B), ensuring that the pixel information of each channel is independent and complete.

[0014] Linear normalization is performed on the images of the four channels respectively, mapping the pixel values ​​to a preset range to ensure that the input data format is consistent for subsequent encoding processing.

[0015] Furthermore, the linear normalization process performed on the images of the four channels is carried out using the following formula:

[0016] x_norm=(x-blacklevel) / (bit_depth-blacklevel)

[0017] Where x is the original pixel value, blacklevel is the image sensor dark current reference value, bit_depth is the maximum bit depth of the image, and x_norm is the normalized value.

[0018] Furthermore, the spatial feature encoding process for the preprocessed image adopts a multi-stage progressive encoding architecture; the multi-stage progressive encoding architecture includes, in sequence: spatial initialization, multiple concatenated spatial encodings, and intermediate feature transformation;

[0019] The spatial initialization process uses a 3×3 depthwise separable convolution to extract features from a 4-channel Bayer image and increases the number of channels to 16 to generate a low-dimensional feature map.

[0020] Each of the multiple cascaded spatial codes contains a residual coding unit with a step size of 2 and several residual coding units with a step size of 1, which are used for deep feature extraction and spatial scale compression.

[0021] The intermediate feature transformation process uses a 3×3 depthwise separable convolution to compress the input feature map to 64 channels, which provides support for subsequent frequency domain feature fusion.

[0022] Furthermore, the residual coding unit used in the spatial domain coding process is specifically processed in parallel through the main path and the shortcut path; the shortcut path directly performs an identity mapping when the input and output channels are consistent and the stride is 1, otherwise it adjusts the number of channels and spatial size through a 3×3 convolution to match the output of the main path; the main path consists of two layers of 3×3 depthwise separable convolutions, the first layer is activated with a corrected linear unit, and the second layer only performs feature mapping; the output of the main path and the output of the shortcut path are added element-wise to form a residual connection, which is then activated by a corrected linear unit to generate the unit output.

[0023] Furthermore, the frequency domain feature encoding process for the preprocessed image specifically includes: frequency domain transformation, frequency domain enhancement, and multi-scale encoding; the frequency domain transformation process inputs the obtained preprocessed image into a preset wavelet transform unit to perform sub-band decomposition in low frequency and three types of high frequency directions for each color channel; the frequency domain enhancement process performs local feature fusion and detail enhancement on the multi-sub-band information within each color channel; the multi-scale encoding process adopts a multi-stage progressive encoding architecture, which includes: frequency domain initialization, multiple cascaded frequency domain encodings, and intermediate feature transformation; the frequency domain initialization process uses 3×3 depthwise separable convolution combined with modified linear unit activation; each of the multiple cascaded frequency domain encodings contains a residual encoding unit with a stride of 2 and several residual encoding units with a stride of 1 for joint noise reduction; the number of cascaded frequency domain encodings is one less than the number of cascaded spatial domain encodings.

[0024] Furthermore, the frequency domain enhancement processing in the frequency domain feature encoding process specifically includes: compressing the spatial dimension into a scalar representation per channel through two-dimensional adaptive average pooling, then performing two layers of 1×1 convolution to achieve linear mapping between channels and inter-layer modified linear unit activation, then generating attention weights for each channel through an activation function, and finally multiplying the weights back into the input feature map channel by channel to achieve adaptive recalibration and enhancement of channel features; then performing local feature fusion and detail enhancement on the multi-subband information within each color channel through 3×3 group convolution, and further enhancing the expressive power of frequency domain features with modified linear unit activation; wherein the calculation formula of the activation function is: Sigmoid(x)=1 / (1+e x ).

[0025] Furthermore, the step of fusing and reconstructing the images processed by spatial and frequency domain feature encoding to obtain the denoised image specifically includes:

[0026] For images processed by frequency domain feature encoding, bilinear interpolation is used for upsampling to align their resolution with the spatial domain features.

[0027] The aligned frequency domain features and spatial domain features are then concatenated along the channel dimension to form a fused feature map.

[0028] The fused feature map and the corresponding spatial coding features are reconstructed using multiple decoding units.

[0029] Furthermore, both the spatial and frequency domain feature fusion and reconstruction employ skip connection processing.

[0030] The image transformed by intermediate features in the spatial domain is fused with the image transformed by intermediate features in the frequency domain in the first stage; the image fused in the first stage and the image in the second stage of spatial coding are then decoded and reconstructed by the first decoding unit.

[0031] The image decoded and reconstructed by the first decoding unit is fused with the image from the frequency domain coding stage 2 in the second stage; the image fused in the second stage is then decoded and reconstructed with the image from the spatial domain coding stage 1 through the second decoding unit.

[0032] The image decoded and reconstructed by the second decoding unit is fused with the image from frequency domain coding stage 1 in the third stage; the image fused in the third stage is then decoded and reconstructed with the image initialized in the spatial domain through the third decoding unit.

[0033] The image decoded and reconstructed by the third decoding unit is processed by two layers of convolutional mapping and then added to the pixel-by-pixel residual of the preprocessed image to generate a four-channel Bayer format denoised image with the same size as the input image.

[0034] This invention also provides an image noise reduction device, comprising: a preprocessing module, a spatial domain coding processing module, a frequency domain coding processing module, and a fusion reconstruction module;

[0035] The preprocessing module is used to preprocess the original Bayer image; the spatial domain coding module is used to perform spatial domain feature coding on the preprocessed image; the frequency domain coding module is used to perform frequency domain feature coding on the preprocessed image; the fusion and reconstruction module is used to perform spatial and frequency domain feature fusion and reconstruction on the images output by the spatial domain coding module and the frequency domain coding module to obtain the denoised image.

[0036] This invention employs a dual-branch design of spatial and frequency domain feature coding to jointly model local image structure and frequency domain noise information, thereby achieving lightweight denoising of the original image while preserving detail and texture integrity. This method balances lightweight and high-fidelity denoising, and is not limited by implementation parameters or network structure, exhibiting strong adaptability and operability. Through lightweight spatial coding, fixed or lightweight frequency domain transformation, multi-stage joint feature fusion, and cross-layer connection reconstruction, high-quality original image denoising is achieved with low computational cost and a small model size. This method combines efficiency and deployability, making it suitable for real-time or near-real-time denoising tasks in mobile terminals, embedded platforms, and image signal processing systems (ISPs), significantly improving texture fidelity and detail integrity in low-light and noisy scenes. Attached Figure Description

[0037] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0038] Figure 1 This is a flowchart of an image noise reduction method provided in an embodiment of the present invention;

[0039] Figure 2 yes Figure 1 Data flow diagram;

[0040] Figure 3 This is a flowchart of the residual coding unit processing data;

[0041] Figure 4 This is a flowchart of frequency domain enhancement processing;

[0042] Figure 5 This is a flowchart for spatial-frequency domain feature fusion. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] like Figure 1 As shown, an embodiment of the present invention provides an image noise reduction method, including:

[0045] S1: Preprocess the original Bayer image.

[0046] The raw Bayer pattern image acquired by the image sensor is obtained, with pixels arranged in a 2×2 periodic pattern (R-Gr-Gb-B). The Bayer image is then split into four channels: red (R), green (Gr, Gb), and blue (B) to ensure independent and complete pixel information for each channel. Subsequently, the four-channel image undergoes linear normalization, mapping pixel values ​​to a preset range acceptable to the network, such as [0,1] or [-1,1]. This preprocessing ensures a consistent input data format, providing a consistent foundation for subsequent spatial and frequency domain feature extraction and adapting to the training and inference requirements of deep learning networks.

[0047] The following formula can be used to perform normalization:

[0048] x_norm=(x-blacklevel) / (bit_depth-blacklevel)

[0049] Where x is the original pixel value, blacklevel is the image sensor's dark current reference value (e.g., 512, which can be adjusted according to the sensor model), and bit_depth is the maximum bit depth of the image (e.g., 255 for 8 bits, 1023 for 10 bits, and 4095 for 12 bits). The range of x_norm obtained by the normalization formula above is [0,1].

[0050] The preset normalization interval can be [0,1], [0,255], or [-1,1]. [0,1] is preferred to suit the training and inference needs of deep learning networks. If the network has specific requirements for the input range, it can be flexibly adjusted to other intervals. To map to the interval [0,a] (a>0), simply multiply x_norm by a. To map to the interval [-1,1], simply multiply (x_norm-0.5)*2.

[0051] S2: Perform spatial feature encoding on the preprocessed image.

[0052] Spatial structure information is extracted at multiple scales using lightweight convolutional networks. Spatial domain feature encoding is mainly used to extract spatial texture and structural information from images. To achieve a lightweight design, various convolution methods can be used, such as small-size depthwise separable convolution, group convolution, or small kernel convolution. The number of channels can be adjusted according to actual needs to reduce computational load.

[0053] The spatial feature encoding process employs a multi-stage progressive encoding architecture, including spatial initialization, multiple cascaded spatial encoding stages (spatial encoding stages 1, 2, and 3), and intermediate feature transformation. Each spatial encoding stage contains a residual encoding unit with a stride of 2 and several residual encoding units with a stride of 1, adapting to different computing resources and application requirements. To enhance information flow and feature reuse, the encoding units can selectively introduce residual connections or cross-layer connections. These connections only add or fine-tune the channels of the feature maps, having minimal impact on computational overhead. The final output is a multi-scale spatial feature map, whose dimensions can be adjusted through optional channel compression or feature mapping modules to adapt to subsequent frequency domain feature fusion. The spatial feature encoding module can select the convolution type, number of channels, number of stages, and residual connection method according to the actual application, achieving flexible and lightweight spatial feature encoding. The image processed in step S1 is input into the spatial feature encoding module, which extracts the local spatial structure and texture information of the image through a multi-scale convolutional structure, and is designed to be lightweight for subsequent frequency domain feature fusion.

[0054] Spatial initialization uses 3×3 depthwise separable convolution to extract features from a 4-channel Bayer image and increases the number of channels to 16 to generate a low-dimensional feature map.

[0055] Depthwise separable convolution decomposes ordinary convolution into a sequence of a K×K grouped convolution with both input and output channels C_in and a 1×1 convolution with input channel C_in and output channel C_out. The computational cost is H×W×(C_in×K×K+C_in×C_out). When C_in and C_out are relatively large, the computational cost of depthwise separable convolution is only about 1 / (K×K) of that of ordinary convolution. The computational overhead is significantly reduced.

[0056] The spatial coding stage n (n = 1, 2, 3) includes one residual coding unit with a stride of 2 and several residual coding units with a stride of 1, used for deep feature extraction and spatial scale compression. Each residual coding unit is processed in parallel through the main path and shortcut paths. For example... Figure 3 As shown, the shortcut path directly performs an identity mapping when the input and output channels are consistent and the stride is 1; otherwise, it adjusts the number of channels and spatial size using a 3×3 (depth-separable) convolution to match the main path output. The main path consists of two 3×3 depth-separable convolution layers: the first layer is activated with ReLU (Rectified Linear Unit), and the second layer only performs feature mapping. The main path output and the shortcut path output are element-wise added to form a residual connection, which is then activated by ReLU to generate the unit output.

[0057] The intermediate feature transformation uses a 3×3 depthwise separable convolution to compress the input high-channel feature map from 256 channels to 64 channels, providing support for subsequent frequency domain feature fusion.

[0058] like Figure 2 As shown, the outputs of spatial initialization, spatial encoding stages 1 and 2 are not only serialized, but also output to decoding stages 3, 2, and 1 respectively. The intermediate feature transformations output to a feature fusion module.

[0059] S3: Perform frequency domain feature encoding on the preprocessed image.

[0060] The input image undergoes frequency domain transformation to extract multi-scale frequency domain features, which are further enhanced by a lightweight convolutional coding module. The frequency domain feature coding module performs sub-band decomposition on the input image to extract multi-scale, multi-frequency features to capture noise distribution and detail information. This module also employs a lightweight design: convolutional operations primarily use small-size, depthwise separable convolutions with a limited number of channels to reduce computational cost; decomposition methods can employ wavelet transform, Fourier transform, learnable transform, or other frequency decomposition methods, allowing each color channel to independently extract sub-bands to form structured frequency domain features. The extracted frequency domain features can be further processed through lightweight attention weighting, feature refinement, or convolutional coding to achieve multi-scale feature representation. The output feature dimension can be matched with intermediate features in the spatial domain, supporting spatial-frequency feature fusion while balancing feature expressiveness and computational efficiency.

[0061] The frequency domain feature coding module follows the process of "frequency domain transformation → frequency domain enhancement → multi-scale coding" to capture noise features from the frequency dimension and provide support for joint noise reduction.

[0062] The specific process of frequency domain transformation is as follows: First, the preprocessed image obtained in step S1 is input into a fixed Haar wavelet transform unit. The Haar wavelet transform unit consists of four predefined Haar wavelet bases, and each color channel is decomposed into subbands in the low-frequency (LL) and three types of high-frequency (LH, HL, HH) directions. The decomposition process is implemented using grouped convolution with a convolution stride of 2, thereby reducing the spatial size of the output feature map to half that of the original image. Finally, the four subbands of each color channel are concatenated along the channel dimension in the order of "channel × subband" to form a structured frequency domain representation.

[0063] The four predefined Haar wavelet bases are:

[0064] ll = [[s,s];[s,s]]

[0065] lh = [[-s, -s]; [s, s]]

[0066] hl = [[-s,s];[-s,s]]

[0067] hh = [[s,-s];[-s,s]]

[0068] in

[0069] The specific process of frequency domain enhancement is as follows: Figure 4As shown: The input feature map is first compressed into a scalar representation per channel through global average pooling in the spatial dimension. Then, it undergoes two layers of inter-channel linear mapping (implemented by 1×1 convolution) and ReLU activation. Next, attention weights for each channel are generated by Sigmoid. Finally, the weights are multiplied back into the input feature map channel by channel to achieve adaptive recalibration and enhancement of channel features. Subsequently, 3×3 grouped convolutions (the number of groups is the same as the color channels, i.e., 4 groups) are used to perform local feature fusion and detail enhancement on the multi-subband information within each color channel, and ReLU activation is used to further enhance the expressive power of frequency domain features.

[0070] The output of the two-dimensional adaptive average pooling is the average of the four channels; MUL represents channel-wise multiplication. The formula for calculating Sigmoid is: Sigmoid(x) = 1 / (1+e^x) x ).

[0071] The enhanced features are fed into a multi-scale encoding network, employing an architecture of "initial feature transformation - multi-stage encoding - intermediate feature transformation". The initial transformation uses 3×3 depthwise separable convolutions combined with ReLU activation; the multi-stage encoding includes at least two encoding stages, and finally, the deep features are compressed to a dimension matching the intermediate features in the spatial domain through 3×3 depthwise separable convolutions. This step outputs multi-scale, multi-frequency domain features for joint noise reduction.

[0072] The structure of the initial feature transformation, encoding stage and intermediate feature transformation is the same as in step S2, only the number of channels is different.

[0073] S4: Perform spatial-frequency feature fusion and reconstruction on the images processed by spatial-domain feature coding and frequency-domain feature coding to obtain the denoised image.

[0074] First, spatial and frequency domain features are fused. The specific steps of feature fusion are as follows: Figure 5 As shown, the frequency domain features are first upsampled layer by layer to align their resolution with the spatial domain features. Upsampling methods can employ computationally inexpensive techniques such as bilinear interpolation or lightweight transposed convolution. Bilinear interpolation is preferred to smoothly restore spatial resolution and reduce computational cost. Subsequently, the aligned frequency and spatial domain features are concatenated along the channel dimension to form a fused feature map, balancing spatial texture and frequency details. The fusion method is not limited to channel concatenation; it can also include weighted summation or attention-weighted summation. The fused feature map is then reconstructed using multiple decoding units along with the corresponding spatial domain encoded features.

[0075] Each feature fusion and reconstruction uses a skip connection: the image transformed from intermediate features in the spatial domain is fused with the image transformed from intermediate features in the frequency domain in the first stage; the image fused in the first stage is then decoded and reconstructed with the image from the second stage of spatial coding through the first decoding unit; the image decoded and reconstructed by the first decoding unit is then fused with the image from the second stage of frequency coding through the second stage; the image fused in the second stage is then decoded and reconstructed with the image from the first stage of spatial coding through the second decoding unit; the image decoded and reconstructed by the second decoding unit is then fused with the image from the first stage of frequency coding through the third stage; the image fused in the third stage is then decoded and reconstructed with the image initialized in the spatial domain through the third decoding unit; the image decoded and reconstructed by the third decoding unit is then processed by two layers of convolutional mapping and added to the pixel-by-pixel residual of the preprocessed image to generate a four-channel Bayer format denoised image with the same size as the input image. Integrating the spatial coding features and fused features at the corresponding scale enhances low-level texture and edge information.

[0076] The reconstruction consists of multiple decoding units, each containing lightweight convolutions (such as small-size depthwise separable convolutions or group convolutions), computationally inexpensive upsampling operations, and residual connections. These are used to extract fused features and progressively restore the image's spatial resolution, while controlling the number of parameters and computational overhead to achieve a lightweight design. The final output is added to the pixel-by-pixel residual of the input image after preprocessing in step S1 to enhance low-frequency information preservation, ensuring high fidelity in texture, edges, and details, generating a four-channel Bayer format denoised image with the same size as the input image. This design balances multi-scale feature integration, joint spatial-frequency information modeling, and lightweight design to achieve efficient and high-fidelity original image denoising. The fused features then enter multiple decoding stages. Each stage contains a residual decoding unit consisting of two 3×3 depthwise separable convolutions, used to enhance local representations while keeping the feature map size unchanged. Then, a 2×2 deconvolution upsampling is used to double the spatial resolution. At the same time, the spatial features from the corresponding scale of the encoding stage are channel-projected through a 3×3 depthwise separable convolution and added element-wise with the upsampled features to achieve the fusion of shallow textures and decoded semantic features at the same scale.

[0077] The image decoded and reconstructed by the third decoding unit passes through the output stage unit, which uses two 3×3 convolution layers to generate a four-channel (R, Gr, Gb, B) image with the same size as the input image. After mapping processing through two convolution layers, the image is directly added pixel-by-pixel to the preprocessed image via residual concatenation to maximize the preservation of low-frequency information. The output is the denoised Bayer image, which retains image details and color information.

[0078] This step aligns and fuses spatial and frequency domain features across multiple scales to form a joint feature representation. A lightweight decoding network is then used to progressively restore the image's spatial resolution, achieving high-fidelity noise reduction and detail preservation. The fusion process combines skip connections and residual convolution units, balancing low-level texture enhancement with deep feature integration, ultimately generating a four-channel Bayer image. In the final output stage, a 3×3 depthwise separable convolution is used to generate a four-channel (R, Gr, Gb, B) image with the same size as the input image. This image is then directly added pixel-by-pixel to the preprocessed image via residual connections to maximize the preservation of low-frequency information. The output is the denoised Bayer image, retaining both image details and color information.

[0079] This invention provides an image denoising device, comprising: a preprocessing module, a spatial domain coding processing module, a frequency domain coding processing module, and a fusion reconstruction module. The preprocessing module preprocesses the original Bayer image. The spatial domain coding processing module performs spatial feature coding on the preprocessed image. The frequency domain coding processing module performs frequency domain feature coding on the preprocessed image. The fusion reconstruction module fuses and reconstructs the output images from the spatial domain coding processing module and the frequency domain coding processing module to obtain a denoised image. The image denoising device and image denoising method provided in this invention belong to the same concept; their specific implementation process is detailed in the full specification and will not be repeated here.

[0080] This invention also provides a computer storage medium, which is a computer-readable storage medium storing a computer program. This computer program can be executed by a coordination control processor to complete the steps of the aforementioned image noise reduction method. The computer-readable storage medium can be a magnetic random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM), etc.

[0081] In the several embodiments provided by this invention, it should be understood that the disclosed methods and smart devices can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0082] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An image denoising method, characterized in that, include: Preprocess the original Bayer image; Spatial domain feature encoding is performed on the preprocessed image; The preprocessed image is then subjected to frequency domain feature encoding. The image after spatial and frequency domain feature fusion and reconstruction is obtained by processing the image with spatial and frequency domain features.

2. The method according to claim 1, characterized in that, The preprocessing of the original Bayer image specifically includes: The Bayer image is split into four channels: one red channel (R), two green channels (Gr and Gb), and one blue channel (B), ensuring that the pixel information of each channel is independent and complete. Linear normalization is performed on the images of the four channels respectively, mapping the pixel values ​​to a preset range to ensure that the input data format is consistent for subsequent encoding processing.

3. The method according to claim 2, characterized in that, The linear normalization process for the four channels of the image is performed using the following formula: x_norm=(x-blacklevel) / (bit_depth-blacklevel) Where x is the original pixel value, blacklevel is the image sensor dark current reference value, bit_depth is the maximum bit depth of the image, and x_norm is the normalized value.

4. The method according to claim 1, characterized in that, The spatial feature encoding process for the preprocessed image adopts a multi-stage progressive encoding architecture; the multi-stage progressive encoding architecture includes, in sequence: spatial initialization, multiple concatenated spatial encodings, and intermediate feature transformation; The spatial initialization process uses a 3×3 depthwise separable convolution to extract features from a 4-channel Bayer image and increases the number of channels to 16 to generate a low-dimensional feature map. Each of the multiple cascaded spatial codes contains a residual coding unit with a step size of 2 and several residual coding units with a step size of 1, which are used for deep feature extraction and spatial scale compression. The intermediate feature transformation process uses a 3×3 depthwise separable convolution to compress the input feature map to 64 channels, which provides support for subsequent frequency domain feature fusion.

5. The method according to claim 4, characterized in that, The residual coding unit used in the spatial domain coding process is specifically processed in parallel through the main path and the shortcut path. The shortcut path directly performs an identity mapping when the input and output channels are consistent and the stride is 1; otherwise, it adjusts the number of channels and spatial size through a 3×3 convolution to match the output of the main path. The main path consists of two layers of 3×3 depthwise separable convolutions, with the first layer activated by a corrected linear unit and the second layer performing only feature mapping. The output of the main path and the output of the shortcut path are added element-wise to form a residual connection, which is then activated by a corrected linear unit to generate the unit output.

6. The method according to claim 1, characterized in that, The frequency domain feature encoding process for the preprocessed image specifically includes: frequency domain transformation, frequency domain enhancement, and multi-scale encoding. The frequency domain transformation process inputs the obtained preprocessed image into a preset wavelet transform unit to perform sub-band decomposition in low-frequency and three types of high-frequency directions for each color channel. The frequency domain enhancement process performs local feature fusion and detail enhancement on the multi-sub-band information within each color channel. The multi-scale encoding process adopts a multi-stage progressive encoding architecture, which includes: frequency domain initialization, multiple cascaded frequency domain encodings, and intermediate feature transformation. The frequency domain initialization process uses 3×3 depthwise separable convolution combined with modified linear unit activation. Each of the multiple cascaded frequency domain encodings contains a residual encoding unit with a stride of 2 and several residual encoding units with a stride of 1 for joint noise reduction. The number of cascaded frequency domain encodings is one less than the number of cascaded spatial domain encodings.

7. The method according to claim 6, characterized in that, The frequency domain enhancement process in the frequency domain feature encoding process specifically includes: compressing the spatial dimension to a scalar representation per channel through two-dimensional adaptive average pooling, then performing two layers of 1×1 convolution to achieve linear mapping between channels and modified linear unit activation between layers, then generating attention weights for each channel through an activation function, and finally multiplying the weights back into the input feature map channel by channel to achieve adaptive recalibration and enhancement of channel features; then performing local feature fusion and detail enhancement on the multi-subband information within each color channel through 3×3 group convolution, and further enhancing the expressive power of frequency domain features with modified linear unit activation; wherein the activation function is calculated as: Sigmoid(x)=1 / (1+e x ).

8. The method according to claim 1, characterized in that, The process of fusing and reconstructing spatial and frequency domain features of images processed by spatial and frequency domain feature coding to obtain a denoised image specifically includes: For images processed by frequency domain feature encoding, bilinear interpolation is used for upsampling to align their resolution with the spatial domain features. The aligned frequency domain features and spatial domain features are then concatenated along the channel dimension to form a fused feature map. The fused feature map and the corresponding spatial coding features are reconstructed using multiple decoding units.

9. The method according to claim 8, characterized in that, Both the spatial and frequency domain feature fusion and reconstruction adopt skip connection processing. The image transformed by intermediate features in the spatial domain is fused with the image transformed by intermediate features in the frequency domain in the first stage; the image fused in the first stage and the image in the second stage of spatial coding are then decoded and reconstructed by the first decoding unit. The image reconstructed by the first decoding unit is fused with the image from frequency domain coding stage 2 in the second stage. The image fused in the second stage and the image from the spatial coding stage 1 are decoded and reconstructed by the second decoding unit; The image decoded and reconstructed by the second decoding unit is fused with the image from frequency domain coding stage 1 in the third stage; the image fused in the third stage is then decoded and reconstructed with the image initialized in the spatial domain through the third decoding unit. The image decoded and reconstructed by the third decoding unit is processed by two layers of convolutional mapping and then added to the pixel-by-pixel residual of the preprocessed image to generate a four-channel Bayer format denoised image with the same size as the input image.

10. An image noise reduction device, characterized in that, include: The module includes a preprocessing module, a spatial domain coding module, a frequency domain coding module, and a fusion and reconstruction module. The preprocessing module is used to preprocess the original Bayer image; The spatial domain coding processing module is used to perform spatial domain feature coding processing on the preprocessed image; The frequency domain coding processing module is used to perform frequency domain feature coding processing on the preprocessed image; the fusion and reconstruction module is used to perform spatial and frequency domain feature fusion and reconstruction on the image output by the spatial domain coding processing module and the frequency domain coding processing module to obtain the denoised image.

Images