EBAPS-oriented low-light image denoising method and system and data set construction method

Abstract

The invention discloses an EBAPS-oriented low-light image denoising method and system and a data set construction method, which are used for solving the problem of nonlinear noise of an electron bombardment active pixel sensor (EBAPS) under an extremely low-light condition. According to the data set construction method, clear-noisy image pairs of pixel-level alignment are obtained by adjusting the aperture of a light source instead of changing exposure at a fixed geometric position. The denoising method adopts a two-stage neural network, and the core of the denoising method is a frequency domain-based noise estimation module (FANS). The FANS module generates a noise priori parameter accurately corresponding to illumination by calculating an under-curve area (AUC) of a normalized radial power spectrum of an input image. According to the network, by using the parameter, through low-rank guided decomposition (LROD), two-way adaptive spatial fusion (DASF) and other modules, denoising is carried out on a noisy image. According to the method, EBAPS noise under 10 <-5 > lx limit illumination can be effectively removed, and image details are reserved.

Description

A method, system, and dataset construction method for low-light image denoising for EBAPS Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a method, system, and dataset construction method for low-light image denoising for EBPS. Background Technology

[0002] Electron Bombardment Active Pixel Sensors (EBAPS) are sensors capable of operating in extremely low light conditions (e.g., below 10 ppm). -1 lx, down to 10 -5 A device that enables real-time imaging under cloudy, starlight conditions (lx). In these photon-scarce scenarios, traditional commercial CMOS sensors, under the same exposure settings, achieve less than 10 lx of image quality. -1 When using lx, it becomes impossible to capture usable images.

[0003] EBAPS achieves its superior low-light imaging capabilities through a vacuum-solid hybrid architecture that utilizes voltage-regulated (e.g., 0-1500V) electron bombardment. However, this high-voltage electron bombardment mechanism, while achieving high gain, introduces significant noise. More importantly, this noise increases dramatically as ambient light decreases, leading to a severe degradation in image quality.

[0004] The fundamental challenge faced by EBAPS stems from its dynamic noise behavior. This noise characteristic is illumination-dependent, and its distribution contradicts the static Gaussian or Poisson noise assumptions embedded in traditional commercial image signal processing pipelines. Therefore, traditional denoising algorithms based on fixed, simplified noise priors (such as Gaussian / Poisson assumptions), or deep learning denoisers trained on ordinary consumer-grade CMOS data, struggle to handle the unique, non-stationary noise in EBAPS images and perform poorly when applied to EBAPS systems.

[0005] Developing denoising algorithms specifically for EBAPS currently faces two main challenges: data scarcity and model inadequacy.

[0006] Regarding data, existing publicly available low-light datasets have serious limitations. First, these datasets are typically built for commercial CMOS / CCD sensors, whose noise distribution assumptions (such as the Poisson-Gaussian mixture model) are fundamentally different from the physical characteristics of EBAPS (such as electron avalanche noise). Second, these datasets also do not cover the extreme operating range of EBAPS (e.g., 10). - 1 lx to 10 -5Currently, no existing dataset specifically addresses the unique voltage-modulated noise characteristics of EBAPS, and there is a lack of sufficient paired clear-noisy samples for supervised training.

[0007] In summary, existing technologies lack a pixel-aligned "noisy-noise" paired dataset that accurately reflects the physical processes and noise characteristics of EBAPS images, covers their entire operating illumination range, and is pixel-aligned. Therefore, there is an urgent need in this field for a novel denoising algorithm specifically designed for EBAPS images, capable of adapting to the nonlinear noise characteristics under different illumination levels, and a corresponding dataset. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a method, system and dataset construction method for low-light image denoising for EBPS.

[0009] A low-light image denoising system for EBAPS includes a LuxRA-Net model with a cascaded architecture, comprising a first-stage bi-branch generator and a second-stage multi-scale thinner.

[0010] The dual-branch generator includes a perceptual denoising branch and an illumination enhancement branch, and outputs a clean grayscale image and illumination coefficients respectively. After fusion, it outputs a coarse-enhanced image. The multi-scale thinner receives the coarse-enhanced image and the clean grayscale image generated by the dual-branch generator, restores details through multi-scale feature fusion, and outputs the final image.

[0011] Preferably, the perceptual denoising branch adopts a multi-layer cascaded architecture, consisting of a front-end noise estimation module and a subsequent deep feature encoder. The processing flow includes: Step 11: Noise estimation of the input image. First, the noise estimation module at the front end performs a Fast Fourier Transform on the image, calculates the normalized radial power spectrum R(f), and then calculates the area under the curve (AUC). Subsequently, the noise prior parameter σ is generated through a linear mapping.

[0012]

[0013] Where f represents the radial spatial frequency; α and β are learnable linear parameters; Step 12: Multi-layer cascaded deep feature encoding and denoising generates noise parameters σ and input image The input image is an improved grayscale encoder, which is composed of N layers of identical denoising modules cascaded and stacked. First, the initial features are mapped through a convolutional layer. Subsequently, The encoder sequentially passes through the denoising modules from layer 1 to layer N; after layer-by-layer processing through N modules, the encoder finally generates a set of multi-scale feature pyramids containing different resolutions and semantic depths. Step 13: Multi-scale Feature Decoding and Reconstruction. The multi-scale feature pyramid set generated in step 12 is fed into the residual decoder for image reconstruction. This decoder consists of 5 cascaded convolutional decoding modules, and its detailed processing flow is as follows: Multi-scale input and skip connections: The decoder not only receives the output of the last layer of the encoder, but also... As initial input, intermediate feature maps generated by the encoder at different scales are received sequentially via skip connections. Layer-by-layer upsampling and fusion: The decoder consists of four decoding blocks and one output block. In each decoding block, the feature map from the previous level is first upsampled through deconvolution to increase resolution. Subsequently, the feature maps corresponding to the encoder's scale are concatenated along the channel dimension. Finally, the concatenated features are fused and smoothed again through a residual convolution module. Final output: After being upsampled four times to restore the original resolution, the feature map enters the last output block. This block maps high-dimensional features to single-channel data through a convolutional layer and processes them using the Tanh activation function, ultimately outputting a denoised, clean grayscale image. .

[0014] Preferably, step 12 specifically includes: (i) hierarchical noise embedding: the global noise parameter σ is input to the noise embedding module of this layer, which is mapped by a multilayer perceptron to generate a noise condition vector adapted to the feature dimension of the current i-th layer. (ii) Feature extraction and FiLM dynamic modulation: the input features F of the current layer in First, feature extraction is performed in the convolutional unit; before the convolution operation, the noise conditional vector generated in step (i) is used. The features are dynamically calibrated through a feature linear modulation layer, and the modulation formula is as follows:

[0015] In the formula: The input feature map is denoted by Norm, which is a group normalization operation. γ and δ are scaling and translation factors, respectively, which are derived from the noise condition vector C. i The result is calculated through a conditional mapping network consisting of a linear layer, SiLU activation, and a linear layer, which allows the response weights of the convolution kernel to be adaptively adjusted according to the current physical noise level; (iii) Low-rank guided decomposition: The features after convolution and modulation enter the LORD module, which utilizes the non-local self-similarity of the image to remove high-frequency random noise through matrix decomposition. The calculation formula is as follows:

[0016] In the formula: U is the basis matrix, representing the low-rank structural basis of the image, generated by the input features through the first 1×1 convolution and reshape operation; V is the coefficient matrix, representing the linear combination coefficients of the structural basis, generated by the input features through the second 1×1 convolution and reshape operation; F low (iv) Module output and cascaded propagation: The reconstructed low-rank feature is filtered out by high-rank random noise components; Input features By performing residual phase extraction and preserving necessary feature information, the output features of the current i-th denoising module are obtained. : .

[0017] Preferably, the specific processing flow of the illumination enhancement branch includes: input image Features are extracted using the MobileNetV2 backbone, and then fused using global and local paths. Subsequently, iterative guided slicing is employed to output an illumination adjustment coefficient map with the same resolution as the original image through multiple iterations. Specifically, the illuminance adjustment coefficient diagram. The following iterative formula is used to calculate the cumulative result:

[0018] In the formula: B t S is the cumulative illuminance coefficient after the t-th iteration; t The slice coefficients for the current step are obtained by sampling in the bilateral grid using the guide graph in the t-th iteration. This indicates element-wise multiplication; after multiple iterations, the final cumulative result is the output illuminance adjustment coefficient diagram.

[0019] Preferably, the dual-branch generator includes a perceptual denoising branch and an illumination enhancement branch, and outputs a clean grayscale image and illumination coefficients respectively. After fusion, it outputs a coarse-enhanced image, specifically including the following fusion formula:

[0020] In the formula: To produce a clean grayscale image output by the perception denoising branch; The illuminance adjustment coefficient for the output of the illuminance enhancement branch; This indicates element-wise multiplication; λ is the illuminance gain scaling factor, used to control the magnitude of the brightness increase.

[0021] Preferably, the multi-scale thinner employs a dual-channel interactive encoder-decoder network, specifically including the following process: Step 21: Dual-channel feature interactive encoding of the input image. and First, the encoder enters the encoding stage; the encoder consists of multiple cascaded thinning modules, each maintaining two parallel feature streams, as follows: Independent feature extraction: In the i-th thinning module, Branch characteristics and The features of each branch are first processed through their respective independent residual convolutional blocks to extract coarse enhancement features at the current scale. and denoising guidance features DASF module fusion: extracted and Then, the system enters the dual-path adaptive spatial fusion (DASF) module for interaction and fusion; Step 22: Specific implementation of the DASF module. The DASF module fusion formula is as follows:

[0022] In the formula, GELU represents the Gaussian error linear unit activation function, which is used to introduce nonlinear transformation and optimize gradient propagation; Conv represents a 1×1 convolutional layer, which is used to perform channel dimensionality reduction and information fusion on the concatenated features; [ , ] represents feature concatenation along the channel dimension; The residual term is added to the output to maintain the structural stability of the denoised features; the fused features in this formula consist of the following two components: the SFT component, which uses the denoised features to guide the distribution correction of the coarse features:

[0023] In the formula: The coarsening enhancement feature corresponds to the coarsening enhancement image output from stage one. Features extracted from X; d The denoising guide feature corresponds to the clean grayscale image output from stage one. Features extracted from; S(X) d ) and T(X d ) are respectively based on the denoising feature X d The generated zoom and pan maps are used to utilize Clear structural correction Feature distribution; SBE components, utilizing attention mechanisms to select effective information:

[0024] In the formula: Norm is the normalization operation; A c and A d For spatial attention weights, they are obtained by... and The concatenated feature map, after convolution and sigmoid activation, is used to adaptively select effective information from the two feature paths; Step 23: Each refiner in the multi-scale decoding and output encoding stage outputs a fused feature map. The feature map F fuse That is, in step 22, using the formula DASF(X) c , X d The final result obtained from the calculation; will be obtained through Skip connections are passed to subsequent decoders; the decoder adopts a U-Net architecture and contains multiple decoding blocks; each decoding block receives upsampled features from the previous level and is compared with those generated by the encoder at the corresponding level. The images are then stitched together. After progressive decoding and reconstruction, the model finally outputs the image. .

[0025] A denoising method based on the above denoising system includes: constructing a low-light image dataset for EBPS, obtaining a dataset containing multiple... noisy ,I clean >The training dataset for image pairs; where I noisy Indicates a noisy image, I clean Represents a clear reference image; the LuxRA-Net model is trained based on the training dataset, where I noisy As model input, I clean As the true label, by minimizing the model output and I clean The loss function is used to optimize the model parameters; for a noisy image frame as external input, it is fed into the trained LuxRA-Net model, and a clear image is output in real time. .

[0026] A dataset construction method for the above-mentioned denoising method includes: S1, acquiring a clear reference image: for any scene s, firstly, the aperture level of the programmable light source is set to the maximum, at which point the illuminance meter displays the first illuminance as a reference. After the readings stabilize, acquire a clear reference image with a high signal-to-noise ratio. S2. Acquire noisy images: Keep the geometric positions of the camera and scene absolutely fixed, and iterate through a preset list of low-light illumination levels; for each target illumination level in the low-light illumination level list... (a) Adjusting the aperture level of the LED light source remotely via software to adjust the lux meter reading. Approaching target illumination .

[0027] (b) pending Stable at (c) Collect the current scene s in Noisy images under illumination​ S3. Generate paired data: Repeat step S2 until all preset scenes and low-light illumination level lists have been traversed; finally, pixel-level aligned data is obtained. Image pairs.

[0028] Preferably, the list of low-light illuminance levels is as follows: .

[0029] Preferably, the dataset is obtained in an optical darkroom with walls coated with black carbon to eliminate ambient light interference; images are captured using an EBAPS camera; the camera is a 9B-3 EBAPS camera from North Night Vision Technologies, featuring a resolution of 1280×1024 and a pixel pitch of 9.76µm; illumination is provided by a programmable light source, employing a 124cm×64cm LED panel with a color temperature of 2856K; this light source consists of eight independently adjustable sub-LEDs, supporting software-controlled aperture adjustment in 1-1000 levels for fine illuminance control; illuminance is obtained using a lux meter: a KWZ-2000 wide-range microlux meter, with a measurement range covering 10 3 Up to 10 -7 lx, accuracy of ±5%; acquisition scenarios: 800 different scenarios were constructed, including a resolution board and indoor still objects; during acquisition, the target object was placed 1.5m away from the LED panel, the EBAPS camera was flexibly positioned to simulate the real perspective, and the illuminance meter probe was fixed next to the target object to ensure direct measurement; all camera parameters remained fixed during the acquisition process.

[0030] The present invention has the following beneficial effects: The dataset construction method of the present invention, by fixing the geometric position and controlling the aperture of the light source, for the first time constructs a pixel-level aligned dataset for EBAPS that covers the entire working illuminance range (10). -2 lx to 10 -5 The real "noisy-noise" paired dataset (lx) solves the data missing problem in EBAPS denoising research in this field.

[0031] The denoising method of this invention innovatively proposes a noise estimation method (S5) based on the frequency domain radial spectrum AUC. This method can physically and accurately quantify the illuminance-related EBAPS noise, providing an accurate and illuminance-adaptive noise prior for the denoising model.

[0032] The denoising method of this invention (LuxRA-Net), through two-stage decoupling and the synergistic effect of modules such as FANS, LORD, and DASF, significantly outperforms existing technologies in denoising performance (such as PSNR and SSIM indices) under extremely low illumination, achieving stronger noise suppression and better preservation of edge and texture details. Attached Figure Description

[0033] Figure 1 is a schematic diagram of the overall process of a low-light noise reduction method provided in an embodiment of the present invention.

[0034] Figure 2 shows a scene in the MILD dataset constructed in this embodiment of the invention under a "clean" reference illuminance (4×10⁻⁶). -1 lx) and six different low light intensity levels (4×10) -2 lx to 5×10 -5 Image examples of lx.

[0035] Figure 3 is a schematic diagram of the network architecture of an EBAPS image denoising model (LuxRA-Net) provided in an embodiment of the present invention.

[0036] Figure 4 is a comparison of the visual effects of the denoising method (Our model) and the comparison method under different illumination levels according to the embodiment of the present invention. Detailed Implementation

[0037] The purpose of this invention is to overcome the shortcomings of existing technologies, such as the lack of a real-world paired dataset suitable for EBAPS and the lack of a denoising model capable of handling the nonlinear noise that varies with illumination unique to EBAPS, which leads to problems with EBAPS images under extremely low light conditions (e.g., 10 lux). -5 The LX (Local Denoising) technology suffers from poor noise reduction and insufficient detail retention.

[0038] Example 1: Construction Method of Low-Light Paired Dataset (MILD) As shown in Figure 1, this example provides a method for constructing a low-light paired dataset, aiming to create a pixel-aligned "noisy-free" image dataset covering the entire operating illumination range of the EBAPS sensor. The acquisition process in this example is performed in a controlled darkroom environment, as shown in the system schematic diagram in Figure 2.

[0039] (1) Acquisition System Configuration The acquisition system in this embodiment is built in a 5m×5m×3m optical darkroom, with walls covered in a black carbon coating to eliminate ambient light interference. The system mainly includes: EBAPS camera: a 9B-3 EBAPS camera from North Night Vision Technologies (NNVT). This camera has a resolution of 1280×1024 and a pixel pitch of 9.76µm.

[0040] Programmable light source: Employs a 124cm × 64cm LED panel with a color temperature of 2856K (matching CIE standard A light source). This light source consists of 8 independently adjustable sub-LEDs and supports software-controlled aperture adjustment in 1-1000 levels to achieve precise illuminance control.

[0041] Illuminance meter: A KWZ-2000 wide-range micro illuminance meter is used, with a measurement range covering 10... 3 Up to 10 -7 lx, with an accuracy of ±5%.

[0042] Data collection scenarios: 800 different scenarios including resolution panels and indoor still life.

[0043] During data acquisition, the target object was placed 1.5m away from the LED panel. The EBAPS camera was flexibly positioned to simulate a real-world perspective, while the illuminance meter probe was fixed next to the target object to ensure direct measurement. All camera parameters (ISO 100, exposure time 40ms, aperture f / 1.4-f / 22) remained constant throughout the acquisition process.

[0044] (2) Data Acquisition Protocol The acquisition in this embodiment strictly follows a standardized protocol to ensure that the scene geometry remains unchanged and the illumination is changed only by adjusting the aperture of the light source, thereby achieving perfect pixel-level alignment.

[0045] S1. Acquire a clear reference image: For any scene s, first set the aperture level of the programmable light source to the maximum. At this time, the illuminance meter displays the first illuminance as the reference. (greater than or equal to 10) -1 (lx). After the readings stabilize, acquire a clear reference image with a high signal-to-noise ratio. .

[0046] S2. Acquire noisy images: Keep the geometric positions of the camera and scene absolutely fixed. Iterate through a preset list of low-light illumination levels. .

[0047] For each target illuminance (a) Adjusting the aperture level of the LED light source remotely via software (e.g., decreasing the distance step from 50 to 0) to adjust the lux meter reading. Approaching .

[0048] (b) pending t Stable at (e.g., |L) current -L t (≤5%), and wait an additional 10-second stabilization interval (Δt) to eliminate transient fluctuations.

[0049] (c) Collect the current scene s in Noisy images under illumination .

[0050] S3. Generate Pairing Data: Repeat the above steps until all 800 scenes and 6 low-light illumination levels have been traversed. This will ultimately yield 4800 pixel-level aligned images. Image pairs (12-bit RAW and JPG formats).

[0051] Feature analysis of the dataset (MILD) constructed in this embodiment shows (as shown in Figure 2) that the PSNR / SSIM of the image decreases sharply with decreasing illumination, especially at 10 lux. -3 The SSIM "collapse" occurs near lx. This demonstrates that the noise of EBAPS (under high-voltage acceleration) has strong nonlinearity and signal dependence, providing a physical basis for the illumination adaptive design of the denoising model in subsequent embodiments.

[0052] Example 2: Implementation of LuxRA-Net denoising model (1) Training configuration hardware and platform: The model training in this example is carried out on a computing device equipped with NVIDIA GeForce RTX 3090 GPU, using the PyTorch 1.12 deep learning framework.

[0053] Training data: The MILD dataset constructed in Example 1 was used. The 800 scenes were divided into training, validation, and test sets in a ratio of 70%:15%:15% to ensure scene-level separation and avoid data leakage.

[0054] Training parameters: The Adam optimizer was used, with an initial learning rate of 3e-4 and a batch size of 16.

[0055] Loss function: A combination of L1 loss and structural similarity (SSIM) loss is used to simultaneously optimize pixel-level accuracy and structure preservation.

[0056] Training cycle: The model is trained for a total of 100 epochs, and an early stopping mechanism is set. If the PSNR on the validation set does not improve for 20 consecutive epochs, the training is terminated.

[0057] (2) The overall network architecture is shown in Figure 3. The LuxRA-Net model in this embodiment adopts a cascaded architecture, which mainly consists of the first stage: dual-branch generator and the second stage: multi-scale refiner.

[0058] The first stage (generator) employs a parallel and decoupled design. It includes a perceptual denoising branch (integrating the FANS module for noise perception) and an illumination enhancement branch. The two branches output a clean grayscale image and illumination coefficients, respectively, which are then fused to output a coarsely enhanced image.

[0059] The second stage (thinner): receives the coarse enhanced image from the first stage and the clean grayscale image, restores details through multi-scale feature fusion, and outputs the final image.

[0060] (3) Stage 1: Input Image of Dual-Branch Decoupling Process Simultaneously, it enters the following two parallel branches: (a) the perceptual denoising branch, and (b) the illumination enhancement branch.

[0061] (a) Perceptual Denoising Branch: This branch aims to extract the clean grayscale image after noise removal. It employs a multi-stage cascaded architecture, consisting of a front-end noise estimation module and a subsequent deep feature encoder. The processing flow is as follows: Step 11: FANS module noise estimation of the input image. First, the module proceeds to the FANS (Frequency-Aware Noise Scaling) module at the front end of the branch. This module performs a Fast Fourier Transform (FFT) on the image, calculates the normalized radial power spectrum R(f), and then calculates the area under the curve (AUC). Subsequently, it generates the noise prior parameter σ through linear mapping:

[0062]

[0063] Where f represents the radial spatial frequency (i.e., the normalized distance from a point in the frequency domain to the center of the spectrum); α and β are learnable linear parameters.

[0064] Step 12: Multi-layer cascaded deep feature encoding and denoising generate noise parameters σ and input image Enter the improved grayscale encoder. This encoder is composed of N (e.g., 6) identical denoising blocks cascaded and stacked.

[0065] Input image First, the initial features are mapped through a convolutional layer. Subsequently, The data processing proceeds sequentially through the 1st to the Nth denoising modules. For the i-th denoising module (1≤i≤N), the data processing includes the following four sub-steps: (i) Layer-wise Noise Embedding: The global noise parameter σ is input to the noise embedding module of this layer. This module is mapped by a multilayer perceptron (MLP) to generate a noise condition vector C adapted to the feature dimension of the current i-th layer.i .

[0066] (ii) Feature extraction and FiLM dynamic modulation: the input features F of the current layer in First, feature extraction is performed in the convolution unit (composed of residual blocks). Before the convolution operation, the noise conditional vector C generated in step (i) is used. i The features are dynamically calibrated using a Feature Linear Modulation (FiLM) layer. The modulation formula is:

[0067] In the formula: The input feature map is represented by Norm, which is a group normalization operation. γ and δ are scaling and translation factors, respectively. They are derived from the noise condition vector C. i The response weights of the convolutional kernel are calculated through a conditional mapper network consisting of a linear layer, SiLU activation, and a linear layer, so that the response weights of the convolutional kernel can be adaptively adjusted according to the current physical noise level.

[0068] (iii) Low-Rank Guided Decomposition (LORD): The convolutional and modulated features are fed into the LORD module, which utilizes the non-local self-similarity of the image to remove high-frequency random noise through matrix decomposition. Its calculation formula is as follows:

[0069] In the formula: U is the basis matrix, representing the low-rank structural basis of the image, generated by the input features through the first 1×1 convolution and reshape operation; V is the coefficient matrix, representing the linear combination coefficients of the structural basis, generated by the input features through the second 1×1 convolution and reshape operation; F low The reconstructed low-rank feature is filtered out by high-rank random noise components.

[0070] (iv) Module output and cascading: reconstructing low-rank features Input features By performing residual phase extraction and preserving necessary feature information, the output features of the current i-th denoising module are obtained. :

[0071] This output feature It has a dual function: 1) Vertical transmission: It is directly used as the input of the next denoising module (the i+1th layer) to extract deeper semantic features; 2) Lateral connection: It is preserved as a multi-scale feature and passed to the subsequent decoder through skip connections to supplement spatial detail information.

[0072] After processing through N modules layer by layer, the encoder finally generates a set of multi-scale feature pyramids containing different resolutions and semantic depths. .

[0073] Step 13: Multi-scale Feature Decoding and Reconstruction The multi-scale feature pyramid set generated in Step 12 is fed into the residual decoder for image reconstruction. To fully utilize the detailed information of the image at different scales, this decoder consists of five cascaded convolutional decoding modules (corresponding to Convolution in Figure 3). Its detailed processing flow is as follows: Multi-scale input and skip connections: The decoder not only receives the output of the last layer of the encoder... As the initial input, it also receives intermediate feature maps generated by the encoder at different scales (such as...) through skip connections (shown by dashed lines in the figure). to ).

[0074] Layer-by-layer upsampling and fusion: The decoder consists of 4 decoder blocks and 1 output block. In each decoder block, the feature map from the previous level is first upsampled by deconvolution (ConvTranspose2d) to increase the resolution; then, the feature maps at the corresponding scale of the encoder are concatenated along the channel dimension; finally, the concatenated features are again fused and smoothed by the residual convolution module.

[0075] Final Output: After being upsampled four times to restore the original resolution, the feature map enters the final output block. This block uses a convolutional layer to map the high-dimensional features into single-channel data, and then processes it through the Tanh activation function, finally outputting a denoised, clean grayscale image. .

[0076] (b) Illumination Enhancement Branch: This branch runs in parallel with the noise reduction branch.

[0077] Input image Features are extracted using the MobileNetV2 backbone and fused using both global and local paths. Then, iterative guided slicing is employed, updating the image multiple times (e.g., 9 times) to output an illumination adjustment coefficient map with the same resolution as the original image. Specifically, the illuminance adjustment coefficient diagram. The following iterative formula is used to calculate the cumulative result:

[0078] In the formula: B t is the cumulative illuminance coefficient after the t-th iteration (initially, B0 is a zero matrix); St is the current step slice coefficient obtained by sampling in the bilateral grid using the guide graph in the t-th iteration. This indicates element-wise multiplication.

[0079] After N iterations (N=9 in this example), the final cumulative result is the output illuminance adjustment coefficient diagram: .

[0080] (4) The stage one fusion generator will output the I from the denoising branch. gray B with the output of the illuminance enhancement branch map The image is fused to obtain a coarsely enhanced image I. pre .

[0081] The fusion formula is as follows:

[0082] In the formula: To produce a clean grayscale image output by the perception denoising branch; The illuminance adjustment coefficient for the output of the illuminance enhancement branch; λ represents element-wise multiplication; λ is the illuminance gain scaling factor (0.5 in this embodiment), used to control the increase in brightness.

[0083] (5) Stage Two: Multi-scale Refinement Stage. Stage Two aims to utilize pure grayscale images. Clear structural information guides coarse enhancement of images This stage restores high-frequency texture details and eliminates artifacts. It employs a dual-path interactive encoder-decoder architecture, with the detailed processing flow as follows: Step 21: Dual-path feature interactive encoding of the input image. and The first step is the encoding stage of the refiner. This encoder consists of N (e.g., 6) cascaded refinement blocks. Unlike traditional single-path networks, each refinement block maintains two parallel feature streams, and its internal processing logic is as follows: Independent feature extraction: In the i-th refinement block, Branch characteristics and The features of each branch are first processed through their own independent residual convolutional blocks to extract coarse enhancement features at the current scale. and denoising guidance features .

[0084] DASF module fusion: extracted and Then it enters the Dual-Path Adaptive Spatial Fusion (DASF) module for interaction and fusion.

[0085] Step 22: Specific Implementation of the DASF Module The DASF module is the core for achieving complementary features from two sources. Its fusion formula is as follows:

[0086] The symbols in the formula are defined as follows: GELU represents the Gaussian Error Linear Unit activation function, which is used to introduce nonlinear transformation and optimize gradient propagation; Conv represents a 1×1 convolutional layer, which is used to perform channel dimensionality reduction and information fusion on the concatenated features; [ , ] represents feature concatenation along the channel dimension. It is added to the output as a residual term to maintain the structural stability of the denoising features.

[0087] The fusion feature in this formula consists of the following two components: ①SFT component (spatial feature transformation): using denoising features to guide the distribution correction of coarse features.

[0088]

[0089] In the formula: The coarsening enhancement feature corresponds to the coarsening enhancement image output from stage one. Features extracted from X; d The denoising guide feature corresponds to the clean grayscale image output from stage one. Features extracted from; S(X) d ) and T(X d ) are respectively based on the denoising feature X d The generated scale and shift plots are used to utilize... Clear structural correction Distribution of features.

[0090] ②SBE component (Spatial Attention Fusion): Utilizes attention mechanisms to select effective information.

[0091]

[0092] In the formula: Norm is the normalization operation; A c and A d These are spatial attention weights. They are obtained by... and The concatenated data is then processed through convolution and Sigmoid activation to adaptively select effective information from the two feature streams.

[0093] Step 23: In the multi-scale decoding and output encoding stage, each refinement module outputs a fused feature map. The feature map F fuse That is, in step 2, using the formula DASF(X) c , X d The final result obtained through computation. These fused feature maps not only contain corrected lighting information and restored texture details, but are also passed to the subsequent decoder via skip connections. The decoder adopts a U-Net-style architecture, containing multiple decoding blocks. Each decoding block receives upsampled features from the previous level and is compared with those generated by the encoder at the corresponding level. The images are then stitched together. After progressive decoding and reconstruction, the model ultimately outputs a high-quality, detailed, and noise-free image. .

[0094] Example 3: This example provides a practical application of the denoising method based on the LuxRA-Net model described above. In practical applications, the model containing the training weights described above is deployed on a computing device. When a noisy image frame is input to the EBAPS sensor, the system sequentially executes the above-described stage one (dual-branch parallel processing → fusion) and stage two (thinning), outputting a clear image in real time. .

[0095] This also includes training using the training dataset, and using I noisy As model input, I clean As the true label, by minimizing the model output and I clean The model parameters are optimized using loss functions (such as a combination of L1 loss and SSIM loss).

[0096] Example 4: System Implementation and Verification of Beneficial Effects This example provides a specific implementation of the fourth aspect of the "Summary of the Invention", namely, an image denoising system, and provides experimental data to verify the beneficial effects of the present invention.

[0097] (1) The denoising method of the present invention (as described in Example 2) can be deployed in an image denoising system. This system can be a dedicated night vision imaging device or a general-purpose computing device.

[0098] In one specific implementation, the system includes: a processor, such as one or more central processing units (CPUs), graphics processing units (GPUs) (such as the NVIDIA RTX3090 5), or field-programmable gate arrays (FPGAs).

[0099] Memory: such as RAM, ROM, or hard disk, used to store executable instructions for a computer.

[0100] During runtime, the memory stores the LuxRA-Net model parameters trained according to the method described in Example 2. The processor loads the model parameters and is configured to perform the following operation: receive a noisy image I. noisy (For example, from an EBAPS sensor connected to it); perform all the computational steps described in Examples 2(3) and (4) (including FANS noise estimation, LROD low-rank denoising, illumination enhancement, DASF thinning, etc.); output a frame of denoised image. .

[0101] (2) Validation of beneficial effects As shown in Figure 4, in order to verify the beneficial effects of Embodiment 1 (MILD dataset) and Embodiment 2 (LuxRA-Net model) of the present invention, the denoising method of the present invention (labeled as "LuxRA-Net(Ours)") was compared with various existing denoising methods (such as DnCNN, MPRNet, Restormer, SUNet, CTNet) on the MILD test set constructed in Embodiment 1.

[0102] All comparison models were retrained using the training set of Example 1 under the same conditions as Example 2(1) to ensure fair comparison.

[0103] Objective metrics validation: Peak signal-to-noise ratio (PSNR, higher is better) and structural similarity (SSIM, higher is better) were used as evaluation metrics. In 10... -2 lx to 10 -5Tests were conducted at six different illuminance levels (lx), and the results showed that the method of this invention significantly outperformed all comparable existing technologies in terms of PSNR and SSIM at all illuminance levels. This was especially true at 10 lx. -5 Under extreme low-light conditions of lx, the PSNR of the method of this invention is significantly improved by 2.1 dB compared with the second-best performing Restormer.

[0104] Subjective effect verification: As shown in Figure 4, the visual comparison results show that at 10 -3 lx to 10 -5 Under low illumination (lx), existing methods (comparison methods) retain significant noise and exhibit severe oversmoothing artifacts in some areas, leading to a loss of texture detail. In contrast, the method of this invention (LuxRA-Net) achieves this even at 10 lx. -5 Even under extreme conditions, complete noise suppression can be achieved while clearly preserving the outline of objects and high-frequency details.

[0105] The aforementioned objective data and subjective effects jointly verify the "beneficial effects" of the present invention: the present invention, through the dataset construction method of Example 1 and the illumination adaptive denoising model (especially the FANS module) of Example 2, successfully solves the technical problem of EBAPS nonlinear noise being difficult to remove and details being unable to be preserved in the background art.

[0106] In summary, the above are merely preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A low-light image denoising system for EBAPS, characterized in that, The LuxRA-Net model employs a cascaded architecture, comprising a first-stage dual-branch generator and a second-stage multi-scale refiner. The dual-branch generator includes a perceptual denoising branch and an illumination enhancement branch, which respectively output a clean grayscale image and illumination coefficients, and after fusion, output a coarse-enhanced image. The multi-scale refiner receives a coarse enhanced image and a clean grayscale image generated by the dual-branch generator, restores details through multi-scale feature fusion, and outputs the final image.

2. The denoising system for EBAPS images as described in claim 1, characterized in that, The perceptual denoising branch adopts a multi-layer cascaded architecture, consisting of a front-end noise estimation module and a subsequent deep feature encoder. The processing flow includes: Step 11: Noise estimation of the input image. First, the noise estimation module at the front end performs a Fast Fourier Transform on the image, calculates the normalized radial power spectrum R(f), and then calculates the area under the curve (AUC). Subsequently, the noise prior parameter σ is generated through a linear mapping. Where f represents the radial spatial frequency; α and β are learnable linear parameters; Step 12: Multi-layer cascaded deep feature encoding and denoising generates noise parameters σ and input image The input image is an improved grayscale encoder, which is composed of N layers of identical denoising modules cascaded and stacked. First, the initial features are mapped through a convolutional layer. Subsequently, The encoder sequentially passes through the denoising modules from layer 1 to layer N; after layer-by-layer processing through N modules, the encoder finally generates a set of multi-scale feature pyramids containing different resolutions and semantic depths. Step 13: Multi-scale Feature Decoding and Reconstruction. The multi-scale feature pyramid set generated in step 12 is fed into the residual decoder for image reconstruction. This decoder consists of 5 cascaded convolutional decoding modules, and its detailed processing flow is as follows: Multi-scale input and skip connections: The decoder not only receives the output of the last layer of the encoder, but also... As initial input, intermediate feature maps generated by the encoder at different scales are received sequentially via skip connections. Layer-by-layer upsampling and fusion: The decoder consists of four decoding blocks and one output block. In each decoding block, the feature map from the previous level is first upsampled through deconvolution to increase resolution. Subsequently, the feature maps corresponding to the encoder's scale are concatenated along the channel dimension. Finally, the concatenated features are fused and smoothed again through a residual convolution module. Final output: After being upsampled four times to restore the original resolution, the feature map enters the last output block. This block maps high-dimensional features to single-channel data through a convolutional layer and processes them using the Tanh activation function, ultimately outputting a denoised, clean grayscale image. 。 3. The denoising system for EBAPS images as described in claim 2, characterized in that, Step 12 specifically includes: (i) Hierarchical noise embedding: The global noise parameter σ is input to the noise embedding module of this layer. This module is mapped by a multilayer perceptron to generate a noise condition vector adapted to the feature dimension of the current i-th layer. (ii) Feature extraction and FiLM dynamic modulation: the input features F of the current layer in First, feature extraction is performed in the convolutional unit; before the convolution operation, the noise conditional vector generated in step (i) is used. The features are dynamically calibrated through a feature linear modulation layer, and the modulation formula is as follows: In the formula: The input feature map is denoted by Norm, which is a group normalization operation. γ and δ are scaling and translation factors, respectively, which are derived from the noise condition vector C. i The result is calculated through a conditional mapping network consisting of a linear layer, SiLU activation, and a linear layer, allowing the response weights of the convolutional kernel to be adaptively adjusted according to the current physical noise level; (iii) Low-rank guided decomposition: The features after convolution and modulation enter the LORD module, which utilizes the non-local self-similarity of the image to remove high-frequency random noise through matrix decomposition. The calculation formula is as follows: In the formula: U is the basis matrix, representing the low-rank structural basis of the image, generated by the input features through the first 1×1 convolution and reshape operation; V is the coefficient matrix, representing the linear combination coefficients of the structural basis, generated by the input features through the second 1×1 convolution and reshape operation; F low (iv) Module output and cascaded propagation: The reconstructed low-rank feature is filtered out by high-rank random noise components; Input features By performing residual phase extraction and preserving necessary feature information, the output features of the current i-th denoising module are obtained. ： 。 4. The denoising system for EBAPS images as described in claim 1, characterized in that, The specific processing flow of the illumination enhancement branch includes: input image Features are extracted using the MobileNetV2 backbone, and then fused using global and local paths. Subsequently, iterative guided slicing is employed to output an illumination adjustment coefficient map with the same resolution as the original image through multiple iterations. Specifically, the illuminance adjustment coefficient diagram. The following iterative formula is used to calculate the cumulative result: In the formula: B t S is the cumulative illuminance coefficient after the t-th iteration; t The slice coefficients for the current step are obtained by sampling in the bilateral grid using the guide graph in the t-th iteration. This indicates element-wise multiplication; after multiple iterations, the final cumulative result is the output illuminance adjustment coefficient diagram.

5. The denoising system for EBAPS images as described in claim 1, characterized in that, The dual-branch generator includes a perceptual denoising branch and an illumination enhancement branch, which respectively output a clean grayscale image and illumination coefficients. After fusion, a coarsely enhanced image is output, specifically including the following fusion formula: In the formula: To produce a clean grayscale image output by the perception denoising branch; The illuminance adjustment coefficient for the output of the illuminance enhancement branch; This indicates element-wise multiplication; λ is the illuminance gain scaling factor, used to control the magnitude of the brightness increase.

6. The denoising system for EBAPS images as described in claim 1, characterized in that, The multi-scale thinner employs a dual-channel interactive encoder-decoder network, and the specific process includes: Step 21: Dual-channel feature interactive encoding of the input image. and First, the encoder enters the encoding stage; the encoder consists of multiple cascaded thinning modules, each maintaining two parallel feature streams, as follows: Independent feature extraction: In the i-th thinning module, Branch characteristics and The features of each branch are first processed through their respective independent residual convolutional blocks to extract coarse enhancement features at the current scale. and denoising guidance features DASF module fusion: extracted and Then, the system enters the dual-path adaptive spatial fusion (DASF) module for interaction and fusion; Step 22: Specific implementation of the DASF module. The DASF module fusion formula is as follows: In the formula, GELU represents the Gaussian error linear unit activation function, which is used to introduce nonlinear transformation and optimize gradient propagation; Conv represents a 1×1 convolutional layer, which is used to perform channel dimensionality reduction and information fusion on the concatenated features; [ , ] represents feature concatenation along the channel dimension; The residual term is added to the output to maintain the structural stability of the denoised features; the fused features in this formula consist of the following two components: the SFT component, which uses the denoised features to guide the distribution correction of the coarse features: In the formula: The coarsening enhancement feature corresponds to the coarsening enhancement image output from stage one. Features extracted from X; d The denoising guide feature corresponds to the clean grayscale image output from stage one. Features extracted from; S(X) d ) and T(X d ) are respectively based on the denoising feature X d The generated zoom and pan maps are used to utilize Clear structural correction Feature distribution; SBE components, utilizing attention mechanisms to select effective information: In the formula: Norm is the normalization operation; A c and A d For spatial attention weights, they are obtained by... and The concatenated feature map, after convolution and sigmoid activation, is used to adaptively select effective information from the two feature paths; Step 23: Each refiner in the multi-scale decoding and output encoding stage outputs a fused feature map. The feature map F fuse That is, in step 22, using the formula DASF(X) c , X d The final result obtained from the calculation; will be obtained through Skip connections are passed to subsequent decoders; the decoder adopts a U-Net architecture and contains multiple decoding blocks; each decoding block receives upsampled features from the previous level and is compared with those generated by the encoder at the corresponding level. The images are then stitched together. After progressive decoding and reconstruction, the model finally outputs the image. 。 7. A denoising method based on the denoising system of claim 1, characterized in that, include: Construct a low-light image dataset for EBPS to obtain a dataset containing multiple noisy ,I clean >Training dataset for image pairs;​ Among them, I noisy Indicates a noisy image, I clean Represents a clear reference image; the LuxRA-Net model is trained based on the training dataset, where I noisy As model input, I clean As the true label, by minimizing the model output and I clean The loss function is used to optimize the model parameters; for a noisy image frame as external input, it is fed into the trained LuxRA-Net model, and a clear image is output in real time. 。 8. A dataset construction method for the denoising system of claim 7, characterized in that, include: S1. Acquire a clear reference image: For any scene s, first set the aperture level of the programmable light source to the maximum. At this time, the illuminance meter displays the first illuminance as the reference. After the readings stabilize, acquire a clear reference image with a high signal-to-noise ratio. S2. Acquire noisy images: Keep the geometric positions of the camera and scene absolutely fixed, and iterate through a preset list of low-light illumination levels; for each target illumination level in the low-light illumination level list... (a) Adjusting the aperture level of the LED light source remotely via software to adjust the lux meter reading. Approaching target illumination (b) To be continued Stable at ； (c) Collect the current scene s in Noisy images under illumination S3. Generate paired data: Repeat step S2 until all preset scenes and low-light illumination level lists have been traversed; finally, pixel-level aligned data is obtained. Image pairs.

9. The data construction method of the denoising method as described in claim 8, characterized in that, The list of low light intensity levels is as follows: 。 10. The data construction method of the denoising method as described in claim 8, characterized in that, The dataset was obtained in an optical darkroom with walls coated with black carbon to eliminate ambient light interference. Images were captured using an EBAPS camera, specifically a 9B-3 EBAPS camera from North Night Vision Technologies, featuring a resolution of 1280 × 1024 and a pixel pitch of 9.76 µm. Illumination was provided by a programmable light source consisting of a 124 cm × 64 cm LED panel with a color temperature of 2856 K. This light source comprises eight independently adjustable sub-LEDs and supports software-controlled aperture adjustment in 1-1000 levels for precise illuminance control. Illuminance was measured using a KWZ-2000 wide-range microilluminance meter, with a measurement range covering 10... 3 Up to 10 -7 lx, with an accuracy of ±5%; Data Acquisition Scenarios: 800 different scenarios were constructed, including a resolution board and indoor still objects. During data acquisition, the target object was placed 1.5m away from the LED panel, the EBAPS camera was flexibly positioned to simulate the real perspective, and the illuminance meter probe was fixed next to the target object to ensure direct measurement. All camera parameters remained constant during the data acquisition process.

Images