Raw domain image noise reduction method with controllable noise reduction intensity

Abstract

The invention discloses a Raw domain image noise reduction method with controllable noise reduction intensity. The method comprises the following steps: carrying out image correction on an input original Raw image, and calculating the normalized global noise intensity of the original Raw image; inputting the preprocessed Raw image and the normalized global noise intensity into a control noise reduction intensity sub-network, and generating a noise reduction control feature map with the same resolution as the preprocessed Raw image by the control noise reduction intensity sub-network; and inputting the preprocessed Raw image and the noise reduction control feature map into a noise reduction main network, wherein each layer of the noise reduction main network adjusts the noise reduction intensity and the like of each region of the Raw image according to the adaptability of the noise reduction control feature map. The method can solve the problems that in an existing Raw domain image noise reduction technology, the noise reduction intensity is fixed, flexible adjustment cannot be carried out according to image noise distribution and user requirements, and consequently details are lost due to excessive noise reduction or noise is remained due to insufficient noise reduction.

Description

Technical Field

[0001] This invention relates to the field of image denoising technology, and in particular to a Raw domain image denoising method with controllable denoising intensity. Background Technology

[0002] Raw domain images are the original image data directly output by the image sensor without post-processing such as white balance, gamma correction, and color interpolation. They retain the most complete light intensity information and noise characteristics of the image and are the foundation for achieving high-quality image optimization. With the improvement of the shooting capabilities of consumer electronic devices (such as mobile phones and mirrorless cameras), users have increasingly higher demands for image quality in low-light and high dynamic range scenes. However, raw images in these scenes are prone to significant noise (such as mixed noise of Gaussian noise, Poisson noise, and salt-and-pepper noise) due to insufficient sensor light sensitivity and electronic noise interference, which seriously affects the subsequent image processing effect and the final image quality. Therefore, raw domain image noise reduction has become one of the core technologies in the fields of computer vision and image processing.

[0003] In recent years, deep learning-based Raw domain image denoising schemes have gradually replaced traditional denoising algorithms (such as BM3D and nonlocal means), achieving significant breakthroughs in noise suppression performance thanks to their powerful noise modeling and feature learning capabilities. Most current mainstream deep learning denoising schemes are based on network architectures such as U-Net and Transformer, learning the mapping relationship between "noisy Raw images → clean Raw images" through end-to-end training. Examples include Google's RAFT-Denoiser and Samsung's DeepRaw, which have already been implemented in professional image processing equipment.

[0004] Existing technological shortcomings:

[0005] 1. Uncontrollable denoising intensity and poor adaptability: Currently, most Raw domain denoising models adopt a single input mode, using only the noisy image as input. This keeps the denoising intensity fixed during processing. During training, the model optimizes only for specific noise types in the training data, while during inference, a uniform denoising strategy is applied to all input images. However, in real-world applications, images often contain textures similar to noise or weak textures masked by noise. When using a fixed denoising intensity, these details are easily blurred or eliminated. Furthermore, in heavily noisy environments, continuing to use a fixed denoising intensity will result in two undesirable outcomes: on the one hand, if heavy noise is completely removed, the image will appear blurry due to a lack of texture information; on the other hand, if some noise is retained to maintain image sharpness, noticeable noise residue will appear when processing lightly noisy images. It is evident that a fixed noise reduction level is difficult to adapt to complex scenes with varying noise intensities simultaneously, and is prone to problems such as "over-denoising of low-noise images (resulting in weak textures and blurred edge details)" or "insufficient noise reduction of high-noise images (leaving a large amount of residual noise)," failing to meet the processing needs of diverse noise scenes.

[0006] 2. Imbalance between detail and noise: In pursuit of noise suppression, traditional methods often over-smooth image features, resulting in the loss of key details in the Raw image (such as hair strands, textures, and subtle changes in light and shadow). These details exist in the Raw domain as slight differences in light intensity and are easily misjudged as noise filtering during the noise reduction process. On the other hand, some methods that emphasize detail preservation may amplify noise during subsequent color interpolation and white balance processing due to incomplete noise suppression, thus affecting the final image quality.

[0007] 3. Lack of user interaction and scene customization capabilities: Existing solutions are mostly "black box" processing, and users cannot adjust the noise reduction intensity according to their actual needs (such as professional photographers who want to retain more details for post-editing, and ordinary users who want to quickly obtain clean images). At the same time, existing technologies cannot meet the customized noise reduction needs for specific scenarios (such as preserving star details in night scene starry sky shooting, and preserving skin texture in portrait shooting), which limits flexibility and practicality. Summary of the Invention

[0008] In view of this, the purpose of this invention is to propose a Raw domain image denoising method with controllable denoising intensity, which can solve the problems of fixed denoising intensity in existing Raw domain image denoising technologies, which cannot be flexibly adjusted according to the image noise distribution and user needs, resulting in "loss of detail due to over-denoising" or "residual noise due to under-denoising". It can achieve precise control of denoising intensity under different noise scenarios (such as Raw images shot in low light and high ISO), while taking into account both image detail preservation and noise suppression effect. It is suitable for devices and scenarios that need to process Raw images, such as smartphones, professional cameras, security monitoring, and image editing software.

[0009] According to one aspect of the present invention, a Raw domain image denoising method with controllable denoising intensity is provided, the method comprising:

[0010] Image correction is performed on the input raw Raw image, and the normalized global noise intensity of the raw Raw image is calculated.

[0011] The preprocessed Raw image and the normalized global noise intensity are input into a noise reduction intensity control sub-network, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image.

[0012] The preprocessed Raw image and the noise reduction control feature map are input into a noise reduction main network. Each layer of the noise reduction main network adjusts the noise reduction intensity of each region of the Raw image according to the adaptability of the noise reduction control feature map. The noise reduction main network outputs a preliminary noise-reduced Raw image.

[0013] The initially denoised Raw image is post-processed to output the final denoised clean Raw image.

[0014] The core contribution of the above technical solution lies in introducing normalized global noise intensity and noise reduction control feature maps into the noise reduction process, constructing a complete control link from noise estimation to region-adaptive noise reduction. This method addresses three problems: "fixed noise reduction intensity," "imbalance between details and noise," and "lack of interactivity." Through calculable noise intensity input and feature-level adjustment, it achieves flexible adaptation of noise reduction intensity under different noise scenarios.

[0015] To address the issues of "uncontrollable noise reduction intensity and poor adaptability," this solution calculates the normalized global noise intensity of the input Raw image and uses it as input to the sub-network controlling the noise reduction intensity. This allows the noise reduction process to be initialized and adjusted based on the actual noise level of the image. The generation of the noise reduction control feature map further enables spatially differentiated noise reduction, allowing the network to strengthen suppression in high-noise regions and weaken noise reduction in low-noise or textured regions. This avoids the extreme problems of "over-smoothing" or "noise residue," improving its adaptability to diverse noise scenarios.

[0016] To address the imbalance between detail and noise, each layer of the denoising main network adaptively adjusts the noise reduction intensity for different regions of the Raw image based on the denoising control feature map. In regions with significant texture and rich detail, the denoising intensity is reduced while preserving high-frequency information such as hair strands and faint light and shadow; in flat or high-noise regions, the denoising intensity is increased to effectively suppress noise. This mechanism achieves discriminative processing of noise and detail at the feature level, helping to maintain key light intensity gradients and structural information in the Raw image while suppressing noise.

[0017] To address the issue of "lack of user interaction and scene customization capabilities," a control architecture based on normalized noise intensity provides a technical foundation for subsequent interaction design. This noise intensity parameter can be extended to serve as a carrier for user input or scene labels, allowing users to adjust noise reduction tendencies according to photographic intent (such as preserving details of the starry sky or skin texture in portraits) or device scenarios (such as mobile phone night scene mode or low-light security mode), thereby enhancing the system's customizability and practicality.

[0018] In summary, this solution constructs a Raw domain denoising system that can dynamically adapt to different noise distributions and intensities through three stages: noise intensity quantization, control feature generation, and network hierarchical adjustment. Its advantage lies in transforming the originally fixed denoising mapping into an adaptive process guided by noise features. This process suppresses noise while preserving key details of the Raw image and provides a scalable architectural foundation for user-adjustable denoising strategies, thereby improving the overall robustness and practicality of the denoising method in real-world scenarios.

[0019] In some embodiments, image correction is performed on the input raw image, which includes image normalization and at least one of the following methods:

[0020] Defective pixel repair: The neighborhood median replacement method is used to traverse the image pixels. If the difference between a pixel value and its 8 neighboring pixels exceeds a preset threshold, it is determined to be a defective pixel and the median of the neighboring pixels is used to replace the defective pixel.

[0021] Black level correction: Read the black level value from the sensor metadata, perform pixel value-black level operation on the Raw image tensor to eliminate dark noise generated by dark current;

[0022] Shadow correction: Loads the lens shadow correction matrix pre-stored in the device to compensate for the brightness attenuation in the edge areas of the image.

[0023] In the above technical solution, the image correction process includes normalization, bad pixel repair, black level correction, and shadow correction. Its core advantage lies in improving the standardization and signal purity of the raw data from the input source. This step addresses the inherent noise and distortion introduced by sensor non-ideals and optical system defects, laying the foundation for subsequent accurate estimation of "real scene noise" and implementation of controllable noise reduction.

[0024] To provide standardized input for noise intensity estimation, the raw image is normalized, scaling pixel values ​​to a standard range to eliminate overall brightness differences caused by variations in exposure or ISO. This allows the subsequently calculated "normalized global noise intensity" to more accurately reflect the noise level of the image itself, rather than brightness differences, enhancing the cross-scene consistency of noise assessment and serving as a prerequisite for controllable noise reduction.

[0025] Repairing inherent sensor defects and avoiding noise misjudgment and amplification: Dead pixel repair and black level correction suppress sensor-specific noise. Dead pixels (hot pixels, dead spots) are particularly noticeable under low light or long exposure conditions. If their outliers are not repaired, they are easily misjudged as extremely strong noise in subsequent noise reduction, leading to over-smoothing of that area or interference with the estimation of global noise intensity. Black level correction eliminates fixed bias noise (dark noise) in the image by subtracting the reference signal generated by the sensor's dark current, preventing this scene-independent noise from interfering with the subsequent evaluation and processing of real noise such as photon shot noise.

[0026] Compensating for optical attenuation ensures spatial consistency in noise assessment: Lens vignetting (shadowing) causes brightness reduction at image edges, resulting in a lower signal-to-noise ratio (SNR) at the edges compared to the center of the same uniform scene. Without vignetting correction, subsequent algorithms may misinterpret the low SNR at edges as higher noise intensity, leading to spatial bias in the noise reduction control feature map. After vignetting correction, the brightness baseline across different image regions becomes more consistent, facilitating subsequent algorithms' assessment and processing of real-world scene noise at different spatial locations.

[0027] In summary, this image correction stage is a key signal conditioning process targeting the characteristics of Raw domain data. By repairing sensor defects, eliminating system static noise, and compensating for optical inhomogeneities, it effectively removes some easily confused "non-scene noise," allowing the image input to the controllable denoising core network to more purely reflect the mixture of scene light signals and real noise. This improves the accuracy of global noise intensity estimation and provides a reliable guarantee that the denoising control feature map can more accurately guide the adjustment of regional denoising intensity.

[0028] In some embodiments, calculating the normalized global noise intensity of the original Raw image includes:

[0029] The ISO field in the metadata of the original Raw image is parsed. If it is in automatic ISO mode, the equivalent ISO value is calculated by back-calculating the exposure time of the image.

[0030] Obtain the noise parameters k and b from the sensor calibration of the captured raw image, and calculate the global noise intensity σ. ISO The formula is as follows: σ ISO = k*ISO+b;

[0031] Obtain the ISO range of the sensor that captured the raw image, and the global noise intensity σ. ISO Mapped to a normalized coefficient σ between 0 and 1 normal Above; the formula for calculating this coefficient is σ normal =σ ISO / σ MAX In the formula, σ MAXThis is the upper limit of the ISO of the sensor;

[0032] If the global noise intensity σ ISO If the ISO range of the sensor is exceeded, the upper limit of the normalization coefficient is set to 1.0 and the lower limit is set to 0.001, and the normalized global noise intensity is output.

[0033] The advantage of the above technical solution lies in establishing a computable link from shooting parameters (ISO) to noise intensity estimation, and then to the normalized control signal. It abandons the fixed or implicit noise intensity estimation method in traditional solutions, and provides a stable and interpretable intensity adjustment benchmark for the entire noise reduction system through an explicit and quantitative model.

[0034] To address the issue of "fixed noise reduction intensity, unable to adapt to different noise scenarios": This solution utilizes the ISO value (sensitivity) and the formula σ ISO The ISO value is estimated using k*ISO+b, allowing the denoising system to perceive the approximate noise level of the input image (high ISO typically corresponds to low-light, high-noise scenes). This fundamentally changes the traditional model's "one-size-fits-all" approach to processing all inputs, enabling the system to pre-adjust its denoising tendency based on noise intensity, providing prior information to resolve the contradiction between "over-denoising" and "under-denoising".

[0035] To achieve device independence and model universality for noise intensity: the solution obtains specific parameters k and b through sensor calibration and normalizes them using the sensor's ISO range (σ). normal =σ ISO / σ MaX This design maps the noise intensity of different sensor models (with different noise characteristic curves) to a standard range of [0,1]. This allows the subsequent "noise reduction intensity control sub-network" and "noise reduction main network" to be trained and inferred using the same set of parameters, eliminating the need to train a separate model for each sensor, significantly improving the algorithm's adaptability and deployment flexibility.

[0036] To ensure the robustness and numerical stability of the control signal: the scheme incorporates boundary handling for abnormal situations (such as σ). ISO When it exceeds the range, σ normal The values ​​are set to [0.001, 1.0]. This ensures that the control signals input to the deep learning network are always within a stable and reasonable range, avoiding extreme values ​​that could lead to network instability or gradient anomalies, thus enhancing the robustness of the entire system in practical applications.

[0037] In summary, the calculation of normalized global noise intensity is a noise quantization interface that integrates device calibration, physical models, and normalization processing. It transforms the key external factors (ISOs) affecting noise into standardized, continuous control variables, enabling subsequent neural networks to dynamically adjust the intensity of their noise reduction behavior based on this. This design addresses the issues of "uncontrollability" and "lack of scene adaptability" in the noise reduction process, laying a data foundation for achieving differentiated and adaptive noise reduction processing.

[0038] In some embodiments, the preprocessed Raw image and the normalized global noise intensity are input into a noise reduction intensity control subnetwork, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image, including:

[0039] The pre-processed Raw image and the normalized global noise intensity are used as dual inputs. A noise reduction intensity control sub-network is used to learn the relationship between noise intensity and image features, and generate a noise reduction control feature map to control the noise reduction intensity. The noise reduction control feature map with the same resolution as the pre-processed Raw image can simultaneously adjust different global and local noise reduction intensities.

[0040] The noise reduction control sub-network adopts a neural network architecture based on the Self-Attention mechanism, and is guided by the normalized global noise intensity to output a noise reduction control feature map with the same resolution as the preprocessed Raw image.

[0041] In the above technical solution, the core advantage of the design of the noise reduction intensity subnetwork lies in constructing a content-aware, intensity-controllable feature modulator. Through dual-input fusion and a self-attention mechanism, the global noise prior (σ) is incorporated. normal By combining local image content features, a spatial control map is generated that can simultaneously encode global noise reduction tendencies and local differential adjustment requirements, providing operational instructions for the subsequent main network to achieve pixel-level or region-level adaptive noise reduction.

[0042] To address the issue of "fixed denoising intensity, unable to achieve spatial adaptation," this sub-network uses the pre-processed Raw image and normalized global noise intensity as dual inputs. The Raw image input provides complete spatial content information, while the normalized noise intensity provides a priori global noise level for the scene. By learning the relationship between these two, the network can generate a denoising control feature map with the same resolution as the input. This means that the denoising intensity is no longer a globally uniform scalar, but a matrix that can vary with image position, thus achieving spatially differentiated processing of "applying high-intensity denoising to flat, high-noise regions and weak denoising to textured regions," fundamentally improving the method's adaptability to complex scenes.

[0043] To address the imbalance between detail and noise, a network architecture based on the Self-Attention mechanism is crucial. Self-Attention captures long-range dependencies in an image, effectively distinguishing between structural textures (such as hair strands and edges) and random noise. Under this mechanism, the network learns that even under high global noise levels, strong responses in certain regions may represent structural details that need to be preserved, rather than noise. Therefore, the generated control feature maps can more accurately label lower noise reduction in detail regions and higher noise reduction in noise-dominant regions, thus providing the subsequent main network with guidance on balancing detail preservation and noise suppression at the feature level.

[0044] A scalable control interface is provided for "user interaction and scene customization": the normalized global noise intensity, as an explicit guiding input, is itself a naturally continuous and adjustable parameter. While the current solution calculates this value automatically, this architecture allows this input to be extended to a carrier of user input or advanced scene labels (such as "portrait mode" or "starry sky mode"). Users or the system can indirectly but effectively influence the generation tendency of the entire control feature map by adjusting this input value, thereby achieving different styles of noise reduction effects and enhancing the system's flexibility and practicality.

[0045] In summary, the noise reduction intensity control subnetwork is a joint noise-content parser and decision-maker. Leveraging the powerful relational modeling capabilities of the self-attention mechanism, it deeply fuses quantitative noise priors with qualitative image content, outputting a soft control signal that precisely guides spatially differentiated noise reduction. This design addresses the issue of detail blurring or noise residue caused by "one-size-fits-all" noise reduction by achieving fine-grained, content-aware intensity control, providing core regulatory capabilities for achieving high-quality, high-fidelity Raw image noise reduction. Simultaneously, its architecture also reserves interfaces for higher-level interaction and control.

[0046] In some embodiments, the noise reduction intensity control subnetwork also receives an externally input user noise reduction intensity command, which, together with the normalized global noise intensity, serves as a control signal to generate the final noise reduction control feature map.

[0047] In the above technical solution, the noise reduction intensity control subnetwork is allowed to receive user noise reduction intensity commands from external input. Its core advantage lies in constructing a human-machine collaborative decision-making closed loop. It integrates objective noise estimation (normalized global noise intensity) based on a physical model with user preferences (user commands) based on subjective intent to jointly generate the final control feature map. This technically achieves subjective adjustability of noise reduction intensity based on objective adaptation, solving the problem of existing technologies' "black box" processing and inability to meet diverse needs.

[0048] To address the deficiency of "lack of user interaction and scenario customization capabilities," this design explicitly provides users with an input interface that allows them to intervene in the noise reduction process. Users (or upper-layer applications) can input commands based on specific intentions (such as needing more original details in professional post-production or requiring a cleaner image in the direct preview), which will directly affect the generation of the control feature map. This allows the same noisy image to produce differentiated noise reduction results based on different user commands. The system is no longer a fixed "one-size-fits-all" process, but rather has the ability to respond to personalized needs, significantly enhancing its practicality.

[0049] Achieving a balance between objective scenarios and subjective intentions: User commands and the system's automatically calculated normalized global noise intensity are used together as control signals. This ensures that the system, during processing, respects both the objective noise level determined by parameters such as ISO (preventing users from making extreme and unreasonable demands without understanding the noise mechanism) and the user's subjective preferences. For example, in high-noise scenarios, even if the user selects "weak noise reduction" to preserve details, the system will still use a higher σ... normal The value maintains a certain level of noise reduction to avoid uncontrolled noise in the output image due to completely following user instructions.

[0050] Providing a technical implementation path for advanced scene customization: This external input interface provides direct technical support for the implementation of predefined scene modes. Device manufacturers or software can map these modes to specific user command values, thereby driving the generation of matching noise reduction control feature maps. For example, the "starry sky mode" can correspond to a command that tends to preserve weak star points (considered as details) rather than erasing them all (considered as noise), thus guiding the network to generate corresponding control features. This enables the solution to have customized noise reduction capabilities for vertical scenes.

[0051] In summary, by introducing the user's noise reduction intensity command as an external input and integrating objective evaluation and subjective preference at the control signal layer, not only is the pain point of users being unable to participate in the control directly addressed, but the "co-control" design also ensures the rationality of the interaction and the robustness of the system.

[0052] In some embodiments, the preprocessed Raw image and the denoising control feature map are input into a denoising main network. Each layer of the denoising main network adaptively adjusts the denoising intensity of each region of the Raw image according to the denoising control feature map. The denoising main network outputs a pre-denoised Raw image, including:

[0053] The denoising main network employs a neural network based on an improved U-Net architecture, integrating a Self-Attention module in its encoder and / or decoder. During the forward propagation of the denoising main network, layers of different depths reference and apply information from the denoising control feature map based on their own feature map resolution and receptive field. Specifically, the high-resolution features in the shallow layers of the main network focus on local detail regions in the denoising control feature map to preserve texture; while the low-resolution features in the deeper layers focus on the overall statistical characteristics of regions in the denoising control feature map to enhance denoising in flat areas.

[0054] This improved neural network based on the U-Net architecture uses a noise prediction loss to train the main network. The loss function is the L1 loss of the predicted image and the label image, and a gradient constraint on the noise reduction control feature map is added.

[0055] In the above technical solution, the core advantage of the denoising main network design lies in constructing a multi-scale, adjustable noise filtering actuator. It achieves a deep understanding of image content through an improved U-Net architecture and a Self-Attention module, and uses the denoising control feature map as a cross-layer modulation signal to guide the network to implement differentiated denoising intensities in different regions and feature levels, thus integrating the concept of "controllability" into every feature transformation step of the denoising process.

[0056] To address the imbalance between detail and noise, multi-scale content-aware processing is implemented: the network employs an improved U-Net architecture, whose encoder-decoder structure inherently possesses multi-scale feature extraction and fusion capabilities. The key design lies in the fact that each layer of the network references the noise reduction control feature map differently based on its own resolution and receptive field. Shallow, high-resolution features focus on local detail regions in the control map, thus tending to preserve texture and edges in pixel-level operations; deep, low-resolution features focus on the overall statistical characteristics of regions in the control map, thus applying stronger smoothing to flat or homogeneous high-noise regions in semantic-level, large receptive field operations. This multi-scale guidance mechanism enables the network to more accurately distinguish between structural details that should be preserved and random noise that should be removed, fundamentally optimizing the detail-noise trade-off.

[0057] To address the issues of "uncontrollable noise reduction intensity and poor adaptability," spatial control signals are deeply integrated into the network's forward propagation. The noise reduction control feature map is not used only once at the network entry point but is continuously referenced and applied by each layer during the forward propagation process. This means that the control signal can dynamically adjust the feature activation and transmission process within the network, achieving region-adaptive noise reduction from lower to higher layers throughout the entire process. This deep integration ensures that the control intent generated based on noise estimation and user commands is accurately and consistently executed in the final noise reduction output, preventing the control signal from failing or attenuating in deep networks, thus guaranteeing the effectiveness of global and local noise reduction intensity adjustment.

[0058] The stability and effectiveness of the control mechanism are ensured through a specialized loss function design: the network directly optimizes the denoising effect using noise prediction loss (L1 loss), while incorporating gradient constraints on the denoising control feature map. This loss term design prevents the control feature map itself from generating overly drastic or unreasonable spatial variations (i.e., excessive non-smoothness), thereby guiding the control sub-network to generate control maps that are more in line with physical intuition (noise distribution usually has a certain degree of spatial continuity) and visual requirements. This ensures the robustness of the control mechanism at the optimization objective level, enabling the main network to reliably adjust based on a high-quality control signal.

[0059] In summary, the main denoising network is not an independent end-to-end black box, but a controlled feature processor with spatial and scale awareness. Through a multi-scale architecture, attention mechanisms, and hierarchical utilization of control feature maps, the unified denoising task is decomposed into a series of finely tuned local operations. This design not only effectively resolves the contradiction between "over-denoising" and "under-denoising" caused by a fixed denoising intensity, but also achieves a high level of balance between detail preservation and noise suppression through content awareness. Combined with the preceding noise estimation and control map generation stages, this main network ultimately transforms "controllable denoising" from a concept into high-quality pixel-level output.

[0060] According to another aspect of the present invention, a Raw domain image denoising apparatus with controllable denoising intensity is provided, the apparatus comprising:

[0061] The preprocessing module is used to perform image correction on the input raw Raw image and calculate the normalized global noise intensity of the raw Raw image.

[0062] The feature map extraction module inputs the preprocessed Raw image and the normalized global noise intensity into a noise reduction intensity control sub-network, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image.

[0063] The noise reduction module is used to input the preprocessed Raw image and the noise reduction control feature map into a noise reduction main network. Each layer of the noise reduction main network adjusts the noise reduction intensity of each region of the Raw image according to the adaptability of the noise reduction control feature map, and the noise reduction main network outputs the initially denoised Raw image.

[0064] The post-processing module is used to perform post-processing on the initially denoised Raw image and output the final denoised clean Raw image.

[0065] In order to better utilize the above method, this application proposes a Raw domain image denoising device with controllable denoising intensity. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0066] According to another aspect of the present invention, a Raw domain image denoising device with controllable denoising intensity is provided, comprising:

[0067] At least one processor and a memory communicatively connected to said at least one processor;

[0068] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0069] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0070] According to another aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method.

[0071] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0072] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0073] Figure 1This is a flowchart illustrating an embodiment of a Raw domain image denoising method with controllable denoising intensity according to the present invention.

[0074] Figure 2 This is a schematic diagram of the architecture of an embodiment of a Raw domain image denoising method with controllable denoising intensity according to the present invention;

[0075] Figure 3 This is a schematic diagram of an embodiment of a Raw domain image denoising device with controllable denoising intensity according to the present invention. Detailed Implementation

[0076] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0077] This invention proposes a Raw domain image denoising method with controllable denoising intensity, aiming to solve three core problems of existing technologies:

[0078] 1. Achieve "dynamically adjustable" noise reduction intensity for Raw domain images, supporting adaptive adjustment of noise reduction parameters based on global noise intensity and local noise distribution of the image. At the same time, allow users or downstream systems to input custom noise reduction intensity commands to adapt to diverse noise scenarios and needs.

[0079] 2. While flexibly adjusting the noise reduction intensity, we take into account both the noise suppression effect and the ability to preserve details, avoiding image quality loss caused by "over-noise reduction" or "under-noise reduction";

[0080] 3. Establish an end-to-end process of "noise analysis - intensity decision - adaptive noise reduction", which can complete accurate noise reduction without manual intervention, while retaining user-customized interfaces to improve the practicality of the solution.

[0081] Example 1

[0082] Please see Figure 1 , Figure 2A controllable noise reduction method for Raw domain images is proposed. The core logic of this method is "preprocessing for feature extraction → noise intensity parameter setting → sub-network generating control features → main network dynamic noise reduction → post-processing output". The method includes: the Raw image undergoes basic correction via a preprocessing module; the ISO parameters of the Raw image are simultaneously acquired, and the global noise intensity is calculated by combining it with device noise calibration parameters, then normalized and used as the control basis; the normalized noise intensity and the preprocessed Raw image are input into a "noise reduction intensity control sub-network" to generate a feature map for controlling the noise reduction intensity. During inference, the user can set noise reduction intensity parameters to control the noise reduction intensity; the preprocessed Raw image and feature map are input into the main noise reduction network, dynamically adjusting the noise reduction intensity of each layer and adjusting the global and local noise reduction intensity; the main network outputs a preliminarily clean Raw image, which is then restored to its original data format by a post-processing module, finally outputting a reusable clean Raw image. The specific steps are as follows:

[0083] S1. Perform image correction on the input raw Raw image and calculate the normalized global noise intensity of the raw Raw image;

[0084] In this embodiment, image correction is performed on the input raw image. This image correction includes image normalization and at least one of the following methods:

[0085] Defective pixel repair: The neighborhood median replacement method is used to traverse the image pixels. If the difference between a pixel value and its 8 neighboring pixels exceeds a preset threshold, it is determined to be a defective pixel and the median of the neighboring pixels is used to replace the defective pixel.

[0086] Black level correction: Read the black level value from the sensor metadata, perform pixel value-black level operation on the Raw image tensor to eliminate dark noise generated by dark current;

[0087] Shadow correction: Loads the lens shadow correction matrix pre-stored in the device to compensate for the brightness attenuation in the edge areas of the image.

[0088] For example, a basic correction is performed on the input Raw image tensor to eliminate inherent sensor noise and hardware interference, outputting a standardized preprocessed Raw image to provide high-quality input for subsequent network modules. The implementation method is as follows:

[0089] Defective pixel repair: The “neighborhood median replacement method” is used to traverse the image pixels. If the difference between a pixel value and its 8 neighboring pixels exceeds a preset threshold (determined by sensor calibration parameters), it is determined to be a defective pixel and the median value of the neighboring pixels is used to replace the defective pixel.

[0090] Black level correction: Read the black level value from the sensor metadata (e.g., the black level of 16-bit Raw is 256), perform "pixel value - black level" operation on the Raw image tensor to eliminate dark noise generated by dark current;

[0091] Lens shadow correction: Loads the lens shadow correction matrix pre-stored in the device (defined by the manufacturer) to compensate for the brightness attenuation in the edge areas of the image (e.g., multiplying edge pixels by a gain coefficient of 1.2-1.5) to ensure the uniformity of image brightness;

[0092] Data normalization: The corrected Raw data is mapped to the [0,1] interval (formula: normalized value = (original value - black level) / (sensor full trap capacity - black level)), and the preprocessed Raw image tensor (dimension [4,H / 2,W / 2]) is output.

[0093] In this embodiment, calculating the normalized global noise intensity of the original Raw image includes:

[0094] The ISO field in the metadata of the original Raw image is parsed. If it is in automatic ISO mode, the equivalent ISO value is calculated by back-calculating the exposure time of the image.

[0095] Obtain the noise parameters k and b from the sensor calibration of the captured raw image, and calculate the global noise intensity σ. ISO The formula is as follows: σ ISO =K*ISO+b; It should be noted that actual noise is also affected by factors such as exposure time, sensor temperature, and pixel size. However, the reason for still using a linear model in this case is that different exposure times are reflected in the ISO variable. The statistics of k and b are relatively complex. k is approximated using a Gaussian distribution to account for signal-related noise such as electronic component noise, temperature noise, and quantization noise. Therefore, the influence of sensor temperature is considered in k and can be basically ignored. b is the noise of the light signal entering the sensor, called signal-related noise, which is approximated using a Poisson distribution. Therefore, the statistics of k and b in this linear model are not simply calculated, and this linear model will not have a large deviation. Pixel size will not affect the noise estimation. On the contrary, the introduction of the linear model ensures the validity of the calculation results while reducing the computational complexity.

[0096] Obtain the ISO range of the sensor that captured the raw image, and the global noise intensity σ. ISO Mapped to a normalized coefficient σ between 0 and 1 normal Above; the formula for calculating this coefficient is σ normal =σ ISo / σ MAX In the formula, σ MAXThis represents the upper limit of the sensor's ISO. It should be noted that only the global noise intensity is considered here. Local noise is removed by subsequent sub-networks and the main network. Externally input parameters do not need to consider local noise, which will not be elaborated here.

[0097] If the global noise intensity σ ISO If the ISO range of the sensor is exceeded, the upper limit of the normalization coefficient is set to 1.0 and the lower limit is set to 0.001, and the normalized global noise intensity is output.

[0098] For example, the ISO sensitivity parameter is parsed from the raw image metadata, the overall image noise intensity is calculated based on the calibrated noise parameters, and then normalized. The implementation method is as follows:

[0099] Parse the ISO field in the EXIF ​​metadata. If it is in automatic ISO mode (such as ISO Auto), then infer the equivalent ISO value based on the image's exposure time (such as an exposure time of 1 / 30s corresponding to ISO 800).

[0100] Calculate noise intensity σ ISO Obtain the noise parameters k and b (provided by the manufacturer or self-calibrated) for sensor calibration, and the noise intensity σ. ISO The calculation formula for σ is: ISO = k*ISO+b;

[0101] Normalize noise intensity, preset the ISO range supported by the device (e.g., the lowest ISO 50 corresponds to noise intensity 0, the highest ISO 6400 corresponds to noise intensity 1, and set the σ corresponding to ISO 6400). ISO For σ MAX ), σ normal =σ ISO / σ MAX If the global noise intensity exceeds the preset range (such as ISO 12800), the upper limit of the normalization coefficient is set to 1.0 (representing the highest noise reduction intensity), and the lower limit is set to 0.001 (representing the lowest noise reduction intensity), and the normalized noise intensity coefficient is output.

[0102] S2. Input the preprocessed Raw image and the normalized global noise intensity into a noise reduction intensity control sub-network, and generate a noise reduction control feature map with the same resolution as the preprocessed Raw image by the noise reduction intensity control sub-network.

[0103] In this embodiment, the preprocessed Raw image and the normalized global noise intensity are input into a noise reduction intensity control sub-network. The noise reduction intensity control sub-network generates a noise reduction control feature map with the same resolution as the preprocessed Raw image, including:

[0104] The pre-processed Raw image and the normalized global noise intensity are used as dual inputs. A noise reduction intensity control sub-network is used to learn the relationship between noise intensity and image features, and generate a noise reduction control feature map to control the noise reduction intensity. The noise reduction control feature map with the same resolution as the pre-processed Raw image can simultaneously adjust different global and local noise reduction intensities.

[0105] The noise reduction control sub-network adopts a neural network architecture based on the Self-Attention mechanism, and is guided by the normalized global noise intensity to output a noise reduction control feature map with the same resolution as the preprocessed Raw image.

[0106] The subnetwork controlling the noise reduction intensity employs a non-unet neural network architecture based on the Self-Attention mechanism (the introduction of the Self-Attention mechanism helps improve the model's sensitivity to details, thereby enabling the model to focus on local noise and details), and increases the influence of the noise level coefficient (it should be noted that the aforementioned global noise coefficient from 0 to 1 is the main control factor for noise reduction intensity, i.e., the noise level coefficient; a level of 0 results in the weakest noise reduction intensity and the most detail preservation, while a level of 1 results in the strongest noise reduction and the least detail preservation). This effect is mainly achieved during the training process, for example, during training... When the noise coefficient is 0, the ground truth (gt) is only slightly cleaner than the input image; when the coefficient is 1, the gt is the cleanest image. Training in this manner allows the network's denoising strength to be controlled by the aforementioned normalized global noise intensity. The noise coefficient, through normalization and numerous extreme cases (0 or 1) during training, avoids instability and overfitting. The output is a denoising control feature map with the same resolution as the pre-processed Raw image. This denoising control feature map guides the shallow layers of the main network to output low-intensity control values ​​for edges and textured regions, preventing detail blurring; it also guides the deeper layers of the main network to output high-intensity control values ​​for flat, dark regions, strengthening noise filtering. It's important to note that sub-networks can output feature maps based on prior information about the global noise parameters. If the noise parameters are large, the feature map output by the sub-network "tells" the main network that strong denoising is needed, thus avoiding denoising residue; if the noise parameters are small, the feature map tells the main network that the pre-processed Raw image has weak noise, so weak denoising is used to preserve more details. This control is mainly completed during training.

[0107] In this embodiment, the noise reduction intensity control subnetwork also receives an externally input user noise reduction intensity command. This command, together with the normalized global noise intensity, serves as a control signal to generate the final noise reduction control feature map. As one method, the user can actively set global noise parameters to replace the automatically calculated global noise parameters. Specifically, a slider interface is used, with a value range set from 0.01 to 1, a minimum adjustment unit of 0.01, and manual input of actual values ​​is supported. This user-set value directly replaces the automatically calculated normalized global noise intensity as the input parameter of the subnetwork. Simultaneously, the output of this stage is only a single-scale feature map with the same size as the original image, rather than a multi-scale feature map, thus simplifying the data processing flow. This design clearly defines the numerical range, dimensions (normalized values), and adjustment granularity of the user command, providing an intuitive and precise control method and enhancing the user's ability to customize the noise reduction intensity. By transforming ambiguous commands into specific, quantifiable parameters and ensuring output consistency, this scheme effectively solves the problem of unclear parameter definitions and optimizes the adjustability of the noise reduction process.

[0108] S3. Input the preprocessed Raw image and the denoising control feature map into a denoising main network (using a Neural Network architecture based on the Self-Attention mechanism). Each layer of the denoising main network adjusts the denoising intensity of each region of the Raw image according to the adaptiveness of the denoising control feature map, and the denoising main network outputs the initially denoised Raw image.

[0109] In this embodiment, the preprocessed Raw image and the denoising control feature map are input into a denoising main network. Each layer of the denoising main network adaptively adjusts the denoising intensity of each region of the Raw image according to the denoising control feature map. The denoising main network outputs a preliminarily denoised Raw image, including:

[0110] The main denoising network employs a neural network based on an improved U-Net architecture, and integrates a Self-Attention module in its encoder and / or decoder. During the forward propagation of the main denoising network, layers of different depths reference and apply information from the denoising control feature map based on the resolution and receptive field of their own feature maps. Specifically, the high-resolution features of the shallow layers of the main network focus on local detail regions in the denoising control feature map to preserve texture, while the low-resolution features of the deeper layers focus on the overall statistical characteristics of the regions in the denoising control feature map to enhance denoising in flat areas.

[0111] It is important to note that, firstly, the overall framework adopts the U-net network, which has a well-defined structure in the field of image processing. Its encoder-decoder structure and skip connections provide a clear foundation for multi-scale feature extraction and reconstruction. Building upon this, the core improvement of the scheme lies in the systematic integration of an attention mechanism into the feature processing module (block) of each layer of U-net. This improvement adaptively weights feature channels or spatial locations through the attention mechanism, enabling the network to dynamically focus on important information (such as complex textures or noisy regions) while suppressing secondary information. This achieves more intelligent feature selection and fusion within the inherent multi-scale information flow of U-net. This architectural design inherits the advantages of U-net in efficiently processing image context, and through global attention embedding, significantly enhances the model's adaptive modeling ability and feature representation ability for complex content in denoising tasks.

[0112] This improved neural network based on the U-Net architecture uses a noise prediction loss to train the main network. The loss function is the L1 loss of the predicted image and the label image, and a gradient constraint on the noise reduction control feature map is added.

[0113] Based on this dynamic adjustment coefficient, the denoising parameters of each layer are adaptively adjusted. Specifically, denoising control feature maps with the same resolution as the pre-processed Raw image are input into the main network together with the pre-processed Raw image. The effect of the feature map is mainly constrained during training by the training data (e.g., global noise parameter is 1, gt is a noise-free image, and input is a high-noise image). The feature map only provides prior information to the main network (e.g., global noise is prior information of the sub-network). During inference, when the input is a heavily noisy image and the global noise coefficient is 0, the main network still has a denoising effect, but it will be much weaker than that of a normally trained denoising network. This avoids situations where the currently processed Raw image is heavily noisy, the input global noise coefficient is 0, but the denoising result is very clean or has a heavy smearing effect.

[0114] This improved neural network based on the U-Net architecture trains the main network using a noise prediction loss function, which is the L1 loss between the predicted and labeled images. Gradient constraints on the denoising control feature maps are also incorporated. These gradient constraints also use L1 loss, implemented by denoising the noise-free image during training to obtain feature maps `gt_feature` from different layers of the denoised main network. These denoised feature maps are then compared with the feature maps from different layers of the preprocessed raw image using L1 loss constraints, but with smaller weights for the loss calculated on the feature maps, such as 0.3, 0.5, etc.

[0115] It is important to note that image denoising schemes based on a dual-module cascaded network of a control denoising sub-network and a main denoising network face core challenges such as training stability, loss function parameter tuning, and clean label dependency. Specifically, since the two networks require end-to-end joint training, the feature map output by the control sub-network directly influences the behavior of the main network, leading to a complex gradient backpropagation path that may cause training instability or getting stuck in local optima. Simultaneously, the strength coefficient of the "gradient constraint" applied to the control feature map in the loss function needs fine-tuning; too strong a constraint weakens the spatial adaptability of the feature map, while too weak a constraint fails to provide effective control, resulting in meaningless fluctuations in the control map. Furthermore, training relies on "noise-clean" image pairs, but obtaining absolutely clean raw images is extremely difficult, and the "optimal" denoising effect is subjective and scene-dependent, potentially causing the controllable mapping learned by the network to deviate from the actual requirements.

[0116] To address training stability issues, this solution employs a progressive training strategy, residual structures within network layers, and a learning rate with varying cosine values, combined with a small batch size. This effectively mitigates the instability caused by complex gradient paths, avoids local optima, and ensures smooth convergence during training. In terms of loss function design, the solution introduces a phased training process: first, a control sub-network is trained independently, taking noise maps and noise intensity values ​​as input, and forcing the output feature map to approximate the noise values ​​calculated based on clean labels; then, the main network is trained with fixed sub-network parameters; finally, the two networks are jointly trained, incorporating the sub-network's loss function. In this process, gradient constraints, acting as supervision of the feature map, are integrated into the loss function. By balancing the constraint strength, sub-network training divergence is prevented while preserving the spatial adaptability of the feature map. The actual output is a three-channel noise value map with the same size as the input. Regarding the dependence on and ambiguity of clean labels, the network behavior can be calibrated through repeated image testing and iterative optimization using positive and negative samples, making the noise reduction effect more closely meet diverse user needs. This scheme effectively solves the stability and parameter tuning problems in the joint training of dual networks through structural optimization, phased training and iterative calibration. At the same time, it alleviates the inherent ambiguity of clean labels through a practice-oriented labeling strategy, thereby improving the robustness and practicality of the noise reduction system.

[0117] It's important to note that in the main network, only inputting feature maps with the same resolution as the pre-processed raw image is beneficial for clean denoising and detail preservation in local areas. This also allows for control of the denoising intensity of the main network via extrinsic parameters (e.g., through global noise parameters), and makes the main network training easier to converge. Inputting intermediate layer feature maps from sub-networks into the main network can strengthen the influence of 'prior information' on the main network. However, since these sub-networks only need to provide local noise information to the main network, only feature maps at the same level as the input resolution should be input. Including intermediate layer features increases the convergence difficulty during training. Furthermore, inputting intermediate layer feature maps into the main network when deploying to mobile devices increases memory consumption. If the goal is simply to interactively control the denoising intensity, only global noise parameters can be input into the main network, but this does not improve local noise and detail.

[0118] It is important to note that in the design of the image denoising scheme, the control sub-network and the main denoising network are run sequentially, and both employ computationally intensive Self-Attention modules. This leads to significant memory consumption and computational latency issues when processing high-resolution (e.g., tens of millions of pixels) Raw images. For devices such as smartphones and cameras that require real-time preview or rapid burst shooting, such a complex processing flow struggles to meet stringent power consumption and latency requirements. To address this issue, the solution optimizes network deployment through model compression and quantization inference. Model compression techniques (such as pruning, distillation, or structural simplification) reduce the number of network parameters and computational complexity, while quantization inference converts model parameters and activation values ​​from high-precision floating-point numbers to low-precision representations (e.g., 8-bit integers), thereby significantly reducing memory consumption and computational overhead and improving inference speed. These methods effectively alleviate resource bottlenecks while maintaining essentially unchanged denoising performance, making low-latency, low-power real-time operation possible in high-resolution Raw image processing. Therefore, this solution balances algorithm complexity and device constraints through efficient optimization strategies, ensuring feasibility and practicality in embedded environments.

[0119] For example, the controllable denoising main network takes the preprocessed Raw image and feature map as input, calculates the dynamic adjustment coefficient of the denoising intensity of each layer, and adaptively adjusts the denoising parameters of each layer based on the coefficient, thereby achieving dynamic optimization of the denoising intensity, and finally outputting the denoised Raw image tensor. The implementation method is as follows:

[0120] The controllable denoising network based on U-Net includes an encoder (4 layers) and a decoder (4 layers), and incorporates self attention to enhance the influence between long-distance pixels. In addition, a feature map of the sub-network controlling the denoising intensity is added. The training strategy adopts "noise prediction loss" to train the main network, that is, the input is "noisy preprocessed raw image" and the label is "noisy raw image". The loss function is the L1 loss of the predicted image and the label image (to avoid the L2 loss from over-smoothing details). Gradient constraints of the feature map are also added (to ensure the stability of the control parameters).

[0121] S4. Perform post-processing on the initially denoised Raw image to output the final denoised clean Raw image.

[0122] For example, the initial denoised Raw image output by the main network is subjected to inverse normalization and format restoration to eliminate the correction bias introduced by preprocessing, and output a clean image that conforms to the original Raw format standard. The implementation method is as follows:

[0123] Inverse normalization: Maps the initial denoised image tensor from the [0,1] interval back to the original Raw data range (formula: original value = normalized value × (sensor full well capacity - black level) + black level). For example, if the full well capacity of the 16-bit Raw image is 65535, then the original value = normalized value × (65535 - 256) + 256.

[0124] It is important to note that excessive preservation of high-frequency information or overly strong noise reduction during the noise reduction stage may reduce the image's dynamic range in subsequent post-processing workflows, thus shrinking the post-processing space. To address this potential issue, the core of the solution lies in inverse normalization and format restoration during the post-processing workflow. Inverse normalization linearly transforms the normalized data from the [0, 1] range back to the typical range of the original sensor data [0, 65535], restoring the image's dynamic range. Format restoration is a standardized operation that ensures the integrity of the Raw file structure, containing metadata such as exposure parameters and image dimensions, as well as the image data. The noise reduction algorithm only processes the data portion of the Raw data locally, then re-embeds the processed result into the Raw format without affecting the metadata. This approach avoids the loss of overall image information through precise manipulation of the data layer, while the inverse normalization process ensures the restoration of the numerical range, thus maintaining the flexibility for adjustments in post-processing. Therefore, this solution improves the image signal-to-noise ratio while protecting the inherent high dynamic range of the Raw format through the data processing workflow, effectively mitigating the potential reduction in post-processing space.

[0125] Based on the above embodiments, the present invention has the following advantages:

[0126] 1. Flexible and controllable noise reduction intensity, adaptable to diverse scenarios: Compared with existing fixed-intensity noise reduction schemes, this invention achieves dynamic adjustment of the noise reduction intensity of the main network through a noise reduction intensity controllable sub-network. It can both adaptively optimize according to the actual noise intensity of the raw image (such as the difference between ISO 100 and ISO 12800) and receive user instructions to customize the intensity, solving the compatibility problem of "over-noise reduction" and "under-noise reduction". It is applicable to various shooting scenarios such as low light, high ISO, night scene, and portrait.

[0127] 2. Significantly improved ability to balance details and noise: Existing U-Net-based Raw noise reduction main networks rely solely on local convolutions to extract features, resulting in poor suppression of continuous noise across regions (such as noise in large dark areas) and a tendency to lose long-distance related details (such as hair strands and fabric textures). This invention adds long-distance Self-Attention modules to the encoder and decoder of the main network: In the encoder stage, Self-Attention captures the global noise distribution pattern of the image, avoiding the "missed" noise from local convolutions; In the decoder stage, Self-Attention associates the detailed features stored in the encoder, restoring weak textures (such as skin pore details under ISO 3200) that are obscured by noise while reducing noise.

[0128] 3. User-friendly interface, meeting customized needs: Compared with the existing "black box" noise reduction solutions, this invention provides a noise reduction intensity command interface, which allows users to adjust the noise reduction effect according to actual needs (such as professional photo editing, quick preview), taking into account the usage scenarios of professional users and ordinary users. Its practicality and flexibility are significantly better than existing technologies.

[0129] Example 2

[0130] Please see Figure 3 A raw domain image denoising device with controllable denoising intensity, based on the method described in one embodiment, the device comprising:

[0131] The preprocessing module is used to perform image correction on the input raw Raw image and calculate the normalized global noise intensity of the raw Raw image.

[0132] The feature map extraction module inputs the preprocessed Raw image and the normalized global noise intensity into a noise reduction intensity control sub-network, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image.

[0133] The noise reduction module is used to input the preprocessed Raw image and the noise reduction control feature map into a noise reduction main network. Each layer of the noise reduction main network adjusts the noise reduction intensity of each region of the Raw image according to the adaptability of the noise reduction control feature map, and the noise reduction main network outputs the initially denoised Raw image.

[0134] The post-processing module is used to perform post-processing on the initially denoised Raw image and output the final denoised clean Raw image.

[0135] In order to better utilize the method described in one of the embodiments, this application proposes a Raw domain image denoising device with controllable denoising intensity. Each module corresponds to each step of the above method, and its specific principle has been described above and will not be repeated here.

[0136] Example 3

[0137] A Raw domain image denoising device with controllable denoising intensity includes:

[0138] At least one processor and a memory communicatively connected to said at least one processor;

[0139] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method as described in one of the embodiments.

[0140] In the above technical solution, in order to better operate and process the method described in one of the embodiments, the method is stored in a memory, and the stored method is executed by a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0141] Example 4

[0142] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in one of the embodiments.

[0143] In the above technical solution, to better operate and use the method described in one of the embodiments, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0144] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A Raw domain image denoising method with controllable denoising intensity, characterized in that, The method includes: Image correction is performed on the input raw Raw image, and the normalized global noise intensity of the raw Raw image is calculated. The preprocessed Raw image and the normalized global noise intensity are input into a noise reduction intensity control sub-network, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image. The preprocessed Raw image and the noise reduction control feature map are input into a noise reduction main network. Each layer of the noise reduction main network adjusts the noise reduction intensity of each region of the Raw image according to the adaptability of the noise reduction control feature map. The noise reduction main network outputs a preliminary noise-reduced Raw image. The initially denoised Raw image is post-processed to output the final denoised clean Raw image.

2. The Raw domain image denoising method with controllable denoising intensity as described in claim 1, characterized in that, Image correction is performed on the input raw image, which includes image normalization and at least one of the following methods: Defective pixel repair: The neighborhood median replacement method is used to traverse the image pixels. If the difference between a pixel value and its 8 neighboring pixels exceeds a preset threshold, it is determined to be a defective pixel and the median of the neighboring pixels is used to replace the defective pixel. Black level correction: Read the black level value from the sensor metadata, perform pixel value-black level operation on the Raw image tensor to eliminate dark noise generated by dark current; Shadow correction: Loads the lens shadow correction matrix pre-stored in the device to compensate for the brightness attenuation in the edge areas of the image.

3. The Raw domain image denoising method with controllable denoising intensity as described in claim 1, characterized in that, Calculating the normalized global noise intensity of the original Raw image includes: The ISO field in the metadata of the original Raw image is parsed. If it is in automatic ISO mode, the equivalent ISO value is calculated by back-calculating the exposure time of the image. Obtain the noise parameters k and b from the sensor calibration of the captured raw image, and calculate the global noise intensity σ. IsO The formula is as follows: σ ISO =K*ISO+b; Obtain the ISO range of the sensor that captured the raw image, and the global noise intensity σ. ISO Mapped to a normalized coefficient σ between 0 and 1 normal Above; the formula for calculating this coefficient is σ normal =σ ISO / σ MAX In the formula, σ MAX This is the upper limit of the ISO of the sensor; If the global noise intensity σ ISO If the ISO range of the sensor is exceeded, the upper limit of the normalization coefficient is set to 1.0 and the lower limit is set to 0.001, and the normalized global noise intensity is output.

4. The Raw domain image denoising method with controllable denoising intensity as described in claim 1, characterized in that, The preprocessed Raw image and the normalized global noise intensity are input into a noise reduction intensity control sub-network. This sub-network generates a noise reduction control feature map with the same resolution as the preprocessed Raw image, including: The pre-processed Raw image and the normalized global noise intensity are used as dual inputs. A noise reduction intensity control sub-network is used to learn the relationship between noise intensity and image features, and generate a noise reduction control feature map to control the noise reduction intensity. The noise reduction control feature map with the same resolution as the pre-processed Raw image can simultaneously adjust different global and local noise reduction intensities. The noise reduction control sub-network adopts a neural network architecture based on the Self-Attention mechanism, and is guided by the normalized global noise intensity to output a noise reduction control feature map with the same resolution as the preprocessed Raw image.

5. A Raw domain image denoising method with controllable denoising intensity as described in any one of claims 1 or 4, characterized in that, The noise reduction intensity control subnetwork also receives an externally input user noise reduction intensity command, which, together with the normalized global noise intensity, serves as a control signal to generate the final noise reduction control feature map.

6. The Raw domain image denoising method with controllable denoising intensity as described in claim 1, characterized in that, The preprocessed Raw image and the denoising control feature map are input into a denoising main network. Each layer of the denoising main network adaptively adjusts the denoising intensity of each region of the Raw image according to the denoising control feature map. The denoising main network outputs a preliminarily denoised Raw image, including: The denoising main network employs a neural network based on an improved U-Net architecture, integrating a Self-Attention module in its encoder and / or decoder. During the forward propagation of the denoising main network, layers of different depths reference and apply information from the denoising control feature map based on their own feature map resolution and receptive field. Specifically, the high-resolution features in the shallow layers of the main network focus on local detail regions in the denoising control feature map to preserve texture; while the low-resolution features in the deeper layers focus on the overall statistical characteristics of regions in the denoising control feature map to enhance denoising in flat areas. This improved neural network based on the U-Net architecture uses a noise prediction loss to train the main network. The loss function is the L1 loss of the predicted image and the label image, and a gradient constraint on the noise reduction control feature map is added.

7. A Raw domain image denoising device with controllable denoising intensity, characterized in that, Based on the method according to any one of claims 1-6, the apparatus comprises: The preprocessing module is used to perform image correction on the input raw Raw image and calculate the normalized global noise intensity of the raw Raw image. The feature map extraction module inputs the preprocessed Raw image and the normalized global noise intensity into a noise reduction intensity control sub-network, which generates a noise reduction control feature map with the same resolution as the preprocessed Raw image. The noise reduction module is used to input the preprocessed Raw image and the noise reduction control feature map into a noise reduction main network. Each layer of the noise reduction main network adjusts the noise reduction intensity of each region of the Raw image according to the adaptability of the noise reduction control feature map, and the noise reduction main network outputs the initially denoised Raw image. The post-processing module is used to perform post-processing on the initially denoised Raw image and output the final denoised clean Raw image.

8. A Raw domain image denoising device with controllable denoising intensity, characterized in that, include: At least one processor and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images