A new raw domain denoising method and system
By constructing an adaptive filtering kernel and high-frequency compensation processing, the problems of noise adaptability and structural information preservation in raw domain denoising methods are solved, achieving a balance between noise suppression and detail preservation in the raw domain and improving imaging quality.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- 深圳森云智能科技有限公司
- Filing Date
- 2026-04-13
- Publication Date
- 2026-07-10
Smart Images
Figure CN122367787A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of digital image processing technology, and more specifically to a novel raw domain denoising method and system. Background Technology
[0002] With the development of image acquisition technology, image sensors have been widely used in visual fields such as mobile phone photography, security monitoring, automotive vision, and industrial imaging. Currently, most mainstream image sensors employ CMOS technology, which offers advantages such as small size, low power consumption, and high integration. During operation, an image sensor receives light signals, generates a photoelectric response, and then sequentially converts the light signals into electrical, analog, and digital signals before outputting them to a system-on-a-chip (SoC) for subsequent image signal processing. In this process, the sensor typically first performs exposure control and then amplifies the signal amplitude through gain amplification.
[0003] In low-light or low-light scenarios, sensors often need to apply high gain to obtain sufficient effective signals, which leads to the synchronous amplification of sensor noise. At this time, the output data is still in the raw domain. If noise is not effectively suppressed, it will be further amplified in subsequent ISP processes such as demosaicing and color processing, affecting image quality. Therefore, noise reduction processing in the raw domain is necessary.
[0004] Among existing raw domain denoising methods, a common approach is to use Gaussian filtering, which employs a weighted average of the pixel neighborhood using a fixed filter kernel to achieve noise reduction. However, this method primarily focuses on the spatial relationship between pixels, neglecting the frequency characteristics between pixels, which can easily lead to the loss of high-frequency detail information. Another improved approach introduces an edge detection mechanism on top of Gaussian denoising, adjusting the filter weights by detecting gradient information in the horizontal and vertical directions. However, this method has limited adaptability to non-horizontal and non-vertical structures, and still suffers from over-smoothing in the processing of detail textures in non-edge regions. In summary, existing raw domain denoising methods still have shortcomings in terms of noise adaptability and multi-directional structure preservation, making it difficult to balance noise suppression and detail preservation under complex imaging conditions. Summary of the Invention
[0005] The purpose of this invention is to provide a novel raw domain denoising method and system, so as to at least solve the problems of existing raw domain denoising methods in terms of noise adaptive processing and structural information preservation.
[0006] To achieve the above objectives, the first aspect of the present invention provides a novel raw domain denoising method, the method comprising: acquiring raw image data to be processed and performing black level correction and normalization processing to obtain normalized raw image data and corresponding noise intensity parameters; calculating the noise normalization difference at each pixel position based on the noise intensity parameters and determining the filter kernel size parameter corresponding to each pixel position accordingly, while performing multi-directional structural analysis on the normalized raw image data to generate direction parameters and direction confidence parameters; constructing a direction adaptive filter kernel based on the filter kernel size parameter, direction parameters, and direction confidence parameters, and performing filtering processing on the normalized raw image data to generate low-frequency denoised image data; calculating high-frequency residual information based on the normalized raw image data and the low-frequency denoised image data, and determining high-frequency compensation parameters based on the direction confidence parameters and the noise intensity parameters; performing high-frequency component compensation on the low-frequency denoised image data according to the high-frequency compensation parameters to generate denoised raw image data, and performing inverse normalization to output the raw domain denoising result.
[0007] Optionally, the process involves acquiring raw image data to be processed and performing black level correction and normalization to obtain normalized raw image data and corresponding noise intensity parameters. This includes: acquiring sensor black level parameters and saturation level parameters recorded in the raw image data to be processed, and performing black level correction on the raw image data based on the black level parameters; after completing black level correction, performing amplitude normalization on the corrected raw image data based on the saturation level parameters to map the values of each pixel to a uniform numerical range, thereby generating the normalized raw image data; and based on the normalized raw image data, and combined with preset raw domain noise modeling rules, constructing a noise parameter model to characterize the relationship between pixel signal amplitude and noise fluctuation, and calculating the noise intensity parameters corresponding to each pixel position based on the noise parameter model.
[0008] Optionally, calculating the noise normalization difference for each pixel location based on the noise intensity parameter and determining the corresponding filter kernel size parameter for each pixel location accordingly includes: obtaining the noise intensity parameter corresponding to each pixel location in the normalized raw image data, and constructing a preset neighborhood range around each corresponding pixel location to calculate the local statistical value of the pixels within the neighborhood range; performing noise normalization processing on the difference between the pixel value of each pixel location in the normalized raw image data and the local statistical value within the corresponding neighborhood range, and the corresponding noise intensity parameter, to generate the noise normalization difference for each pixel location; comparing the noise normalization difference with a preset difference interval, and dividing the corresponding pixel location into different noise saliency levels according to the comparison result; and determining the filter kernel size parameter corresponding to each level based on the noise saliency level.
[0009] Optionally, performing multi-directional structural analysis on normalized raw image data to generate directional parameters and directional confidence parameters includes: constructing a preset set of directions for each pixel position based on the normalized raw image data, and performing local structural response calculation on the normalized raw image data along the preset set of directions to obtain structural response values corresponding to each direction; determining the principal structural direction for each pixel position based on the structural response values corresponding to each direction, and determining the principal structural direction as the directional parameter corresponding to the pixel position; and calculating the directional confidence parameter used to characterize the stability and consistency of the principal structural direction based on the degree of difference between the structural response value corresponding to the principal structural direction and the structural response value corresponding to non-principal structural directions.
[0010] Optionally, an adaptive direction filter kernel is constructed based on the filter kernel size parameter, direction parameter, and direction confidence parameter. This includes: for each pixel position in the normalized raw image data, reading the filter kernel size parameter corresponding to each pixel position to determine the spatial range of the adaptive direction filter kernel at each pixel position; determining the main action direction of the adaptive direction filter kernel at each pixel position based on the direction parameter corresponding to each pixel position, and constraining the spatial distribution of the adaptive direction filter kernel at each pixel position based on the main action direction; adjusting the expansion degree of the adaptive direction filter kernel at each pixel position in the main action direction and its perpendicular direction based on the direction confidence parameter corresponding to each pixel position, so that the action weight of the adaptive direction filter kernel at each pixel position changes with the direction confidence parameter in different directions; after determining the spatial range, main action direction, and direction-related expansion degree corresponding to each pixel position, an adaptive direction filter kernel corresponding to each pixel position is generated.
[0011] Optionally, filtering is performed on the normalized raw image data to generate low-frequency denoised image data, including: performing pixel-by-pixel filtering operation on the normalized raw image data based on the direction adaptive filtering kernel corresponding to each pixel position to generate a filter output value that corresponds one-to-one with each pixel position; and collecting the filter output values corresponding to each pixel position to form low-frequency denoised image data.
[0012] Optionally, based on the normalized raw image data and the low-frequency denoised image data, high-frequency residual information is calculated, and high-frequency compensation parameters are determined based on the orientation confidence parameter and the noise intensity parameter. This includes: calculating high-frequency residual information for each pixel position to characterize the high-frequency components based on the difference relationship between the normalized raw image data and the low-frequency denoised image data; and performing weight adjustment processing on the high-frequency residual information based on the orientation confidence parameter and noise intensity parameter corresponding to each pixel position to determine the high-frequency compensation parameters corresponding to each pixel position.
[0013] Optionally, high-frequency component compensation is performed on the low-frequency denoised image data according to the high-frequency compensation parameters to generate denoised raw image data, and inverse normalization is performed to output the raw domain denoising result. This includes: adjusting the high-frequency components of each pixel position in the low-frequency denoised image data based on the high-frequency compensation parameters corresponding to each pixel position to generate denoised raw image data; and performing inverse normalization processing corresponding to the normalization processing based on the denoised raw image data to map the pixel values back to the numerical range of the original raw image data and output the raw domain denoising result.
[0014] A second aspect of the present invention provides a novel raw domain denoising system, the system comprising: an acquisition unit for acquiring raw image data to be processed and performing black level correction and normalization processing to obtain normalized raw image data and corresponding noise intensity parameters; a processing unit for calculating the noise normalization difference at each pixel position based on the noise intensity parameters and determining the filter kernel size parameter corresponding to each pixel position accordingly, and simultaneously performing multi-directional structural analysis on the normalized raw image data to generate direction parameters and direction confidence parameters; a filtering unit for constructing a direction adaptive filter kernel based on the filter kernel size parameter, direction parameters, and direction confidence parameters, and performing filtering processing on the normalized raw image data to generate low-frequency denoised image data; a scheme generation unit for calculating high-frequency residual information based on the normalized raw image data and the low-frequency denoised image data, and determining high-frequency compensation parameters based on the direction confidence parameters and the noise intensity parameters; and a compensation unit for performing high-frequency component compensation on the low-frequency denoised image data according to the high-frequency compensation parameters to generate denoised raw image data, and performing inverse normalization to output the raw domain denoising result.
[0015] On the other hand, the present invention provides a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the novel raw domain denoising method described above.
[0016] Through the above technical solution, this invention performs joint processing of image noise and structural information in the raw domain, enabling the denoising process to adaptively adjust the processing strategy according to changes in sensor noise levels. By performing black level correction and normalization in the raw domain and obtaining noise intensity parameters, the subsequent denoising process has a basis for adjustment under different noise conditions. By introducing the noise normalization difference and determining the filter kernel size parameters accordingly, the filter intensity can be dynamically adjusted according to changes in pixel noise significance, avoiding the use of the same processing method in high-noise and low-noise regions. Simultaneously, by obtaining direction parameters and direction confidence parameters through multi-directional structural analysis, the filtering process can perceive the directional characteristics of local structures, thereby adaptively constraining the filter kernel in different directions and improving the ability to preserve non-horizontal and non-vertical structures. Based on this, by adjusting the high-frequency residual information between the low-frequency denoising result and the original raw data, and combining noise intensity and structural reliability to determine high-frequency compensation parameters, selective compensation of effective structural high-frequency components is achieved, reducing the risk of noise amplification. Thus, a balance between noise suppression and structural detail preservation is achieved in the raw domain, which is beneficial to improving the overall imaging stability in the subsequent ISP processing flow.
[0017] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description
[0018] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of the steps of a novel raw domain denoising method provided in one embodiment of the present invention; Figure 2 This is a schematic diagram comparing the processing results of different raw domain denoising methods provided in one embodiment of the present invention; Figure 3 This is a system structure diagram of a novel raw domain denoising system provided in one embodiment of the present invention; Figure 4 This is an internal structural diagram of a computer device provided in one embodiment of the present invention. Detailed Implementation
[0019] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.
[0020] like Figure 1 As shown, embodiments of the present invention provide a novel raw domain denoising method, the method comprising: Step S10: Obtain the raw image data to be processed and perform black level correction and normalization processing to obtain normalized raw image data and corresponding noise intensity parameters.
[0021] Specifically, the black level parameters and saturation level parameters of the sensor recorded in the raw image data to be processed are obtained, and black level correction processing is performed on the raw image data based on the black level parameters. After the black level correction is completed, amplitude normalization processing is performed on the corrected raw image data based on the saturation level parameters to map the values of each pixel to a uniform numerical range, generating the normalized raw image data. Based on the normalized raw image data and combined with preset raw domain noise modeling rules, a noise parameter model is constructed to characterize the relationship between pixel signal amplitude and noise fluctuation, and the noise intensity parameter corresponding to each pixel position is calculated based on the noise parameter model.
[0022] In this embodiment of the invention, basic correction and normalization are performed on the raw image data to be processed, providing a unified data foundation for subsequent noise modeling and denoising. Raw image data is the original pixel data directly output by the image sensor after exposure and gain adjustments. While reflecting true illumination information, this data inevitably includes noise components introduced by the sensor circuitry, readout process, and environmental conditions. Differences in manufacturing processes and operating modes of different image sensors can lead to fixed biases and inconsistent amplitude ranges in the raw image data. Therefore, it is necessary to perform unified preprocessing on the raw image data before performing denoising.
[0023] Specifically, the sensor black-level parameters and saturation-level parameters corresponding to the image acquisition process are read from the raw image data to be processed. The black-level parameter characterizes the reference bias value output by the sensor under no-light conditions, which mainly originates from dark current and fixed-mode noise; the saturation-level parameter characterizes the maximum effective pixel value that the sensor can output under the current operating mode. Based on the black-level parameters, black-level correction processing is performed on the pixel values in the raw image data. By subtracting the black-level parameters from the original pixel values, the influence of fixed bias on the pixel signal is eliminated, allowing the corrected pixel values to more accurately reflect the effective signal amplitude after photoelectric conversion.
[0024] After black level correction, amplitude normalization is performed on the corrected raw image data based on the saturation level parameter. By scaling the corrected pixel values proportionally, the pixel amplitudes are mapped to a uniform numerical range, thereby generating normalized raw image data. Through this normalization process, raw image data obtained under different exposure conditions, different gain configurations, and different bit depth settings can be converted to a consistent data scale, avoiding the impact of numerical range differences on subsequent noise modeling and parameter calculation processes.
[0025] After obtaining the normalized raw image data, the relationship between pixel signal amplitude and noise fluctuation is modeled to characterize the noise intensity characteristics at different pixel locations. Considering that image sensor noise typically consists of shot noise related to signal amplitude and readout noise independent of signal amplitude, in one possible implementation, pixel noise intensity is described using the following relationship: in, Represents the pixel position in normalized raw image data The corresponding noise intensity parameter is used to characterize the variance level of pixel signal fluctuation at that location; Represents the normalized raw image data at pixel positions Pixel signal amplitude at the location; parameters This parameter describes the intensity of the noise component related to the pixel signal amplitude; it is typically related to sensor gain settings and exposure conditions. This model describes the fundamental noise components that are not directly related to the pixel signal amplitude, primarily reflecting the influence of readout circuit noise and other fixed noise factors. The above noise parameter model is merely an example; those skilled in the art can employ other noise modeling methods based on specific sensor characteristics.
[0026] Based on the aforementioned noise parameter model, the corresponding noise intensity parameter is calculated for each pixel position in the normalized raw image data, and a correlation is established between the noise intensity parameter and the corresponding pixel position. This processing enables adaptive adjustment for noise level differences at different pixel positions during subsequent denoising, providing basic input conditions for noise normalization difference calculation, filter kernel size determination, and high-frequency compensation parameter calculation.
[0027] Step S20: Calculate the noise normalization difference at each pixel location based on the noise intensity parameter and determine the filter kernel size parameter corresponding to each pixel location accordingly. At the same time, perform multi-directional structural analysis on the normalized raw image data to generate directional parameters and directional confidence parameters.
[0028] Specifically, the method of calculating the noise normalization difference for each pixel location based on the noise intensity parameter and determining the corresponding filter kernel size parameter for each pixel location includes: obtaining the noise intensity parameter corresponding to each pixel location in the normalized raw image data, and constructing a preset neighborhood range around each pixel location to calculate the local statistical value of the pixels within the neighborhood range; performing noise normalization processing on the difference between the pixel value of each pixel location in the normalized raw image data and the local statistical value within the corresponding neighborhood range, and the corresponding noise intensity parameter, to generate the noise normalization difference for each pixel location; comparing the noise normalization difference with a preset difference interval, and dividing the corresponding pixel location into different noise saliency levels based on the comparison result; and determining the filter kernel size parameter corresponding to each noise saliency level based on the noise saliency level.
[0029] Furthermore, multi-directional structural analysis is performed on the normalized raw image data to generate directional parameters and directional confidence parameters, including: constructing a preset set of directions for each pixel position based on the normalized raw image data, and performing local structural response calculation on the normalized raw image data along the preset set of directions to obtain the structural response values corresponding to each direction; determining the principal structural direction for each pixel position based on the structural response values corresponding to each direction, and determining the principal structural direction as the directional parameter corresponding to the pixel position; and calculating the directional confidence parameter used to characterize the stability and consistency of the principal structural direction based on the degree of difference between the structural response value corresponding to the principal structural direction and the structural response value corresponding to the non-principal structural direction.
[0030] In this embodiment of the invention, after obtaining the normalized raw image data and noise intensity parameters, this embodiment further quantifies and analyzes the noise significance at different pixel locations based on the noise intensity parameters, and adaptively determines the size and orientation attributes of the subsequent filter kernel in combination with the local structural features of the image, so as to enhance the stability and consistency of the denoising process under complex imaging conditions.
[0031] To address the issue of varying noise levels at different pixel locations in normalized raw image data, this implementation first calculates the noise normalization difference for each pixel location based on the noise intensity parameter, and then determines the filter kernel size parameter accordingly. Specifically, for each pixel location in the normalized raw image data, the noise intensity parameter corresponding to that pixel location is read, and a preset neighborhood range is constructed around that pixel location. The shape of the neighborhood range can be set to a rectangular window or an approximately circular window, depending on implementation needs, and its size is used to characterize the set of pixels participating in local statistical calculations at the current scale. By performing statistical analysis on the pixels within the neighborhood range, local statistical values of the pixel signals within the neighborhood are calculated to reflect the overall trend of pixel signal changes within that region.
[0032] In one implementation, local statistical values within the neighborhood can be represented using a weighted mean, weighted median, or other robust statistical measures to reduce the interference of local extreme values on the statistical results. After completing the neighborhood statistics, for each pixel location, the difference between the pixel value at that location in the normalized raw image data and the corresponding local statistical value within the neighborhood is calculated to obtain a difference quantity reflecting the degree of deviation between that pixel location and its local background. Since the random fluctuation amplitude of pixel values may increase significantly under high noise conditions, it is difficult to distinguish between real structural changes and noise disturbances by relying solely on the difference quantity itself. Therefore, this implementation further combines a noise intensity parameter to perform noise normalization processing on the difference quantity.
[0033] In one exemplary implementation, the noise normalization difference can be calculated in the following form: in, Indicates pixel position The corresponding noise normalization difference; Represents the normalized raw image data at pixel positions The pixel value at that location; Represented by pixel position Local statistical values of pixel signals within the neighborhood of the center; Indicates pixel position The noise intensity parameter at that location; This is a preset stability factor used to prevent the denominator from being zero. Through the above noise normalization process, the pixel difference under different noise levels can be mapped to a uniform comparison scale, thereby avoiding the misjudgment of noise fluctuations as structural changes in high-noise areas.
[0034] After obtaining the noise normalization difference for each pixel location, the noise normalization difference is compared with a preset difference range. The preset difference range is used to divide different noise saliency levels. Each noise saliency level corresponds to a type of pixel noise state, reflecting the degree of noise significance at that pixel location relative to the local background. Through the above comparison and judgment process, each pixel location is divided into different noise saliency levels. Based on the noise saliency levels, the filter kernel size parameters corresponding to each level are further determined, so that a relatively large filter kernel size is used in pixel areas with high noise saliency to enhance noise suppression capability, while a relatively small filter kernel size is used in pixel areas with low noise saliency or obvious structural changes to reduce over-smoothing of detailed structures.
[0035] On the other hand, while determining the filter kernel size parameters based on noise intensity parameters, this embodiment also performs multi-directional structural analysis on the normalized raw image data to obtain the direction parameters and direction confidence parameters at each pixel location. Multi-directional structural analysis is used to characterize the main structural orientation and the stability of structural directions within a local region of the image, providing a basis for subsequently constructing an adaptive direction filter kernel. Specifically, for each pixel location in the normalized raw image data, a preset set of directions is constructed. The direction set represents multiple candidate directions participating in the structural analysis in the current implementation; the number of directions and the direction interval in the direction set can be set according to actual application requirements.
[0036] After constructing the direction set, local structural response calculations are performed on the normalized raw image data along each direction in the direction set. The local structural response describes the gray-level variation characteristics of the image in the corresponding direction, and its calculation can be based on directional gradients, directional differences, or other direction-sensitive operators. By calculating the local structural responses in each direction of the direction set, a set of structural response values corresponding to different directions can be obtained.
[0037] After obtaining the structural response values corresponding to each direction, the structural response values are compared and analyzed to determine the principal structural direction at the pixel location. The principal structural direction is used to characterize the direction in which the image grayscale change is most significant or most stable within the region surrounding the current pixel location, and this direction is determined as the direction parameter. To further evaluate the reliability of the principal structural direction, this embodiment also calculates a direction confidence parameter based on the degree of difference between the structural response values corresponding to the principal structural direction and those corresponding to non-principal structural directions. The direction confidence parameter is used to reflect the stability and consistency of the principal structural direction within a local region. When the response difference between the principal structural direction and other directions is large, the direction confidence parameter takes a high value, indicating that the structural direction is clear; when the response difference between different directions is small, the direction confidence parameter takes a low value, indicating that the structural direction is not obvious or the structure within the region is relatively complex.
[0038] In one possible implementation, the orientation confidence parameter can be calculated in the following form: in, Indicates pixel position The direction confidence parameter corresponding to the location; This represents the maximum structural response value corresponding to the main structural direction in the direction set; This represents the average value of the structural response in the non-principal structural direction; These are preset parameters used for numerical stabilization. Through the above calculation method, the direction confidence parameter can comprehensively reflect the degree of dominance of the main structure's direction relative to other directions.
[0039] Through the above processing, each pixel location can obtain its corresponding filter kernel size parameter, orientation parameter, and orientation confidence parameter. The filter kernel size parameter controls the size of the filtering range, the orientation parameter indicates the main orientation of the local structure, and the orientation confidence parameter characterizes the reliability of the structural orientation. These parameters collectively serve as input conditions for subsequently constructing the orientation-adaptive filter kernel and performing filtering processing. This allows the denoising process to not only adaptively adjust according to changes in noise level but also to differentiate the filtering behavior in different directions by incorporating the multi-directional structural features of the image.
[0040] Step S30: Construct an adaptive direction filter kernel based on the filter kernel size parameter, direction parameter, and direction confidence parameter, and perform filtering processing on the normalized raw image data to generate low-frequency denoised image data.
[0041] Specifically, an adaptive direction filter kernel is constructed based on filter kernel size parameters, direction parameters, and direction confidence parameters. This includes: for each pixel position in the normalized raw image data, reading the filter kernel size parameters corresponding to each pixel position to determine the spatial range of the adaptive direction filter kernel at each pixel position; determining the main action direction of the adaptive direction filter kernel at each pixel position based on the direction parameters corresponding to each pixel position, and constraining the spatial distribution of the adaptive direction filter kernel at each pixel position based on the main action direction; adjusting the extent of the adaptive direction filter kernel at each pixel position in the main action direction and its perpendicular direction based on the direction confidence parameters corresponding to each pixel position, so that the weight of the adaptive direction filter kernel at each pixel position varies with the direction confidence parameters in different directions; after determining the spatial range, main action direction, and direction-related extent of the adaptive direction filter kernel at each pixel position, an adaptive direction filter kernel corresponding to each pixel position is generated.
[0042] Furthermore, filtering is performed on the normalized raw image data to generate low-frequency denoised image data, including: performing pixel-by-pixel filtering operations on the normalized raw image data based on the direction adaptive filtering kernel corresponding to each pixel position to generate a filter output value that corresponds one-to-one with each pixel position; and collecting the filter output values corresponding to each pixel position to form low-frequency denoised image data.
[0043] In this embodiment of the invention, after obtaining the filter kernel size parameters, orientation parameters, and orientation confidence parameters corresponding to each pixel position, this embodiment further constructs an orientation adaptive filter kernel based on the above parameters, and uses the constructed orientation adaptive filter kernel to perform filtering processing on the normalized raw image data to generate low-frequency denoised image data. This step is used to smooth the pixel signal in the raw domain, while minimizing the destruction of image structural information, providing a stable low-frequency reference for subsequent high-frequency residual analysis and compensation processing.
[0044] In constructing the orientation-adaptive filter kernel, for each pixel location in the normalized raw image data, the corresponding filter kernel size parameter is first read. The filter kernel size parameter determines the spatial range of the orientation-adaptive filter kernel, and its value reflects the relationship between the noise saliency and smoothing requirements at the current pixel location. By introducing the filter kernel size parameter, the effective range of the filter kernel at different pixel locations can be differentiated according to the noise state, thus avoiding the use of a uniform fixed kernel size across the entire image.
[0045] After determining the spatial range of the filter kernel, the main operating direction of the orientation-adaptive filter kernel is further determined based on the orientation parameters. The orientation parameters characterize the main structural orientation within the region surrounding the pixel location and are derived from the aforementioned multi-directional structural analysis process. When constructing the orientation-adaptive filter kernel, the direction corresponding to the orientation parameters is used as the principal axis direction of the filter kernel, ensuring that the weight distribution of the filter kernel in this direction aligns with the image structural orientation. In this way, the filtering process can smooth along the direction of image structural extension while suppressing excessive diffusion in directions perpendicular to the structural orientation, thereby reducing the risk of blurring across structural boundaries.
[0046] After determining the primary action direction, the expansion degree of the direction-adaptive filter kernel in different directions is further adjusted based on the direction confidence parameter. The direction confidence parameter reflects the stability and consistency of the primary action direction within a local region. When the direction confidence parameter is high, it indicates that there is a relatively clear primary structural direction around the pixel location. In this case, the extension capability of the filter kernel in the primary action direction can be appropriately enhanced, while limiting the expansion of the filter kernel in the vertical direction. When the direction confidence parameter is low, it indicates that the structural direction in the local region is not obvious or there is multi-directional texture. In this case, the expansion degree of the filter kernel in each direction tends to be balanced to avoid over-constraining unstable directions.
[0047] In one possible implementation, the spatial weight distribution of the orientation adaptive filter kernel can be constructed using an anisotropic distance metric, for example, as follows: in, This represents the spatial distance metric within the filter kernel relative to the center pixel position. This represents the projection component of the pixel offset in the main action direction; Represents the projection component of the pixel offset perpendicular to the main action direction; parameter The scaling factor used to control the filter kernel in the main action direction can be determined by the filter kernel size parameter and the direction confidence parameter; Parameter The value of the filter kernel, which controls its vertical expansion scale, is adjusted according to the direction confidence parameter. In this way, a direction-adaptive filter kernel shape that matches the orientation of the local structure can be formed spatially.
[0048] After determining the spatial range of action, the main direction of action, and the degree of directional extension, a corresponding directional adaptive filter kernel is generated for each pixel location. The directional adaptive filter kernels corresponding to different pixel locations can differ in size, orientation, and weight distribution, thus enabling the filtering process to have pixel-by-pixel adaptive capability.
[0049] After obtaining the orientation adaptive filter kernel, filtering is performed on the normalized raw image data to generate low-frequency denoised image data. Specifically, for each pixel position in the normalized raw image data, a pixel-by-pixel filtering operation is performed on the pixel signals in its neighborhood using the orientation adaptive filter kernel corresponding to that pixel position. The filtering operation obtains the filtered output value at the current pixel position by weighted summing of the neighboring pixel values and the orientation adaptive filter kernel weights. In an exemplary implementation, the pixel-by-pixel filtering operation can be expressed as: in, Indicates pixel position The filtered output value at that point; Represented by pixel position The neighborhood range centered on the filter kernel size parameter; Represents the position of normalized raw image data in neighboring pixels. The pixel value at that location; This indicates the position of the orientation adaptive filter kernel in the neighboring pixel area. The corresponding weight values are then used. This weighted filtering process can suppress random noise fluctuations while reducing information loss along the structural direction.
[0050] After performing pixel-by-pixel filtering on all pixels in the normalized raw image data, the filtered output values corresponding to each pixel are collected in their original spatial order to form low-frequency denoised image data. This low-frequency denoised image data primarily reflects the trend of the smoothed pixel signals, with significantly reduced noise fluctuations compared to the original normalized raw image data, while still retaining the main structural contour information of the image. This low-frequency denoised image data serves as the foundational input for subsequent high-frequency residual information calculation and high-frequency component compensation processing, providing stable data support for further balancing noise suppression and detail preservation.
[0051] Step S40: Based on the normalized raw image data and the low-frequency denoised image data, calculate the high-frequency residual information, and determine the high-frequency compensation parameters based on the direction confidence parameter and the noise intensity parameter.
[0052] Specifically, based on the difference between normalized raw image data and low-frequency denoised image data, high-frequency residual information for characterizing high-frequency components is calculated for each pixel position; based on the orientation confidence parameter and noise intensity parameter corresponding to each pixel position, weight adjustment processing is performed on the high-frequency residual information to determine the high-frequency compensation parameter corresponding to each pixel position.
[0053] In this embodiment of the invention, after obtaining the low-frequency denoised image data and the direction confidence parameters and noise intensity parameters corresponding to each pixel position, this embodiment further analyzes and adjusts the high-frequency information in the image to refine and correct the denoising result in the raw domain. The main purpose of this step is to distinguish between high-frequency fluctuations caused by noise and high-frequency components caused by real structure or texture, and to selectively compensate the high-frequency components based on this.
[0054] Based on the difference between normalized raw image data and low-frequency denoised image data, high-frequency residual information is calculated for each pixel location to characterize high-frequency components. Normalized raw image data contains all frequency components of the original pixel signal, while low-frequency denoised image data mainly reflects the low-frequency trend information after smoothing. Therefore, the difference between the two can be used to characterize the high-frequency components weakened or removed by the filtering process. By calculating the difference between the normalized raw image data and the low-frequency denoised image data at corresponding pixel locations, high-frequency residual information can be obtained for each pixel location, reflecting the amplitude and direction of local high-frequency changes at that location.
[0055] In one possible implementation, the high-frequency residual information can be calculated using a pixel-by-pixel difference method, for example, as follows: in, Indicates pixel position High-frequency residual information at the location; Represents the normalized raw image data at pixel positions The pixel value at that location; This indicates the pixel location of the low-frequency denoised image data. The pixel values at that location. Through the above calculations, a high-frequency residual containing positive and negative variation information can be obtained, which is used to describe the degree of deviation of the original signal from the low-frequency reference.
[0056] After obtaining the high-frequency residual information, it is not directly added back to the low-frequency denoised image data. Instead, it is further weighted by combining the orientation confidence parameter and the noise intensity parameter to determine the high-frequency compensation parameters corresponding to each pixel position. The orientation confidence parameter characterizes the stability of the orientation of the structure around the pixel position. When the orientation confidence is high, the high-frequency residual information is more likely to originate from a stable structure or real texture; when the orientation confidence is low, the high-frequency residual information is more likely to contain noise fluctuations. The noise intensity parameter reflects the noise level at the current pixel position. When the noise intensity is high, the proportion of noise components in the high-frequency residual information is usually high.
[0057] Based on the above considerations, this embodiment applies position-dependent adjustment weights to the high-frequency residual information by jointly utilizing the orientation confidence parameter and the noise intensity parameter. Specifically, at pixel locations with high orientation confidence and low noise intensity, the adjustment weights are set to relatively large values, ensuring that the corresponding high-frequency residual information is adequately compensated in subsequent processing; at pixel locations with low orientation confidence or high noise intensity, the adjustment weights are set to relatively small values to suppress the impact of noise-dominated high-frequency fluctuations on the final result. Through the above weight adjustment process, a corresponding high-frequency compensation parameter can be determined for each pixel location.
[0058] High-frequency compensation parameters characterize the extent to which high-frequency residual information participates in correcting low-frequency denoising results during subsequent compensation processes. By establishing a correlation between high-frequency compensation parameters and high-frequency residual information, the compensation process for high-frequency components no longer employs a uniform fixed ratio but rather adaptively adjusts based on local structural reliability and noise levels. This processing method provides a fine-grained control over the subsequent generation of denoised raw image data, helping to achieve a balance between noise suppression and structure preservation in the raw domain.
[0059] Step S50: Perform high-frequency component compensation on the low-frequency denoised image data according to the high-frequency compensation parameters to generate denoised raw image data, and perform inverse normalization to output the raw domain denoising result.
[0060] Specifically, based on the high-frequency compensation parameters corresponding to each pixel position, the high-frequency components of each pixel position in the low-frequency denoised image data are adjusted to generate denoised raw image data; based on the denoised raw image data, the inverse normalization process corresponding to the normalization process is performed to map the pixel values back to the numerical range of the original raw image data, and the raw domain denoising result is output.
[0061] In this embodiment of the invention, after determining the high-frequency compensation parameters corresponding to each pixel position, this implementation further performs high-frequency component compensation processing on the low-frequency denoised image data based on the high-frequency compensation parameters to generate denoised raw image data. Then, it performs inverse normalization processing on this data to output the final raw domain denoising result. This step is used to controllably recover the effective high-frequency information weakened by the filtering process while maintaining the stability of the low-frequency denoising result, and to prevent the reintroduction of noise components.
[0062] Specifically, for each pixel location in the low-frequency denoised image data, the high-frequency compensation parameters corresponding to that pixel location are read. These high-frequency compensation parameters are determined jointly by the direction confidence parameter and noise intensity parameter in the previous steps, and their values characterize the degree to which the high-frequency residual information at the current pixel location participates in the compensation process. When performing high-frequency component compensation, the low-frequency denoised image data is not processed uniformly as a whole; instead, the high-frequency components at each pixel location are adjusted pixel-by-pixel according to the high-frequency compensation parameters corresponding to that pixel location.
[0063] In one possible implementation, the high-frequency component compensation process can be understood as introducing adjusted high-frequency information onto the low-frequency denoised image data. Specifically, for each pixel location, the high-frequency residual information obtained in the previous steps is proportionally adjusted according to the high-frequency compensation parameters corresponding to that pixel location. The adjusted high-frequency component is then combined with the low-frequency denoised image data at the same pixel location to generate the denoised raw image data. This pixel-by-pixel, high-frequency compensation parameter control method avoids over-recovery of high-frequency components in structurally unstable or high-noise-level regions, while preserving necessary high-frequency details in structurally clear regions with low-noise-level regions.
[0064] After completing the high-frequency component compensation processing for all pixel locations, the denoised raw image data is obtained. This denoised raw image data is still within the normalized numerical range, and its pixel numerical scale is consistent with the previously normalized raw image data. To ensure that the output result matches the numerical representation of the original raw image data and meets the requirements for subsequent storage or transmission, this embodiment further performs inverse normalization processing on the denoised raw image data.
[0065] The denormalization process corresponds to the preceding normalization process in terms of numerical mapping. Specifically, in the denormalization process, based on the saturation level parameter obtained during the image acquisition stage, an inverse proportional mapping is performed on the pixel values in the denoised raw image data, restoring the pixel values from the normalized range to the range corresponding to the original raw image data. Simultaneously, during the denormalization process, necessary bias recovery processing can be performed on the pixel values in conjunction with the black level parameter to ensure that the denormalized pixel values maintain numerical consistency with the raw data format output by the sensor.
[0066] After inverse normalization, the resulting pixel values are output in their original spatial order to form the raw domain denoising result. This raw domain denoising result maintains the same numerical format as the original raw image data and can be directly used in subsequent demosaicing, color correction, or other image signal processing workflows without requiring additional data format conversion. Through these processing steps, a complete data loop is achieved from the low-frequency denoising result to the final raw domain denoising output, providing concluding support for the entire raw domain denoising method.
[0067] In another possible implementation, after calculating the high-frequency residual information, pixel-by-pixel compensation is not immediately performed. Instead, a high-frequency participation window corresponding to each pixel position is first determined based on the high-frequency compensation parameters. The high-frequency participation window defines the neighborhood range for participating in the high-frequency compensation calculation at the current pixel position, and its size varies with the high-frequency compensation parameters. When the high-frequency compensation parameters are large, the coverage area of the high-frequency participation window is relatively small, allowing only the high-frequency residual information to participate in compensation within a local area; when the high-frequency compensation parameters are small, the coverage area of the high-frequency participation window is relatively large, allowing the high-frequency residual information to be smoothly fused within a larger neighborhood. Subsequently, within the determined high-frequency participation window, spatial weighted aggregation is performed on the high-frequency residual information, and the aggregation result is combined with the low-frequency denoised image data at the corresponding pixel position to generate the denoised raw image data. In this way, the high-frequency compensation process introduces spatial constraints in addition to amplitude control, so that the recovery method of high-frequency information is simultaneously affected by structural reliability and spatial consistency.
[0068] In one specific implementation, a set of raw image data collected in an outdoor natural scene is selected as the processing object. This scene includes intricate structures such as tree branches and leaves, as well as large low-frequency areas such as building facades. Furthermore, a certain degree of gain amplification occurred during image acquisition, resulting in noticeable sensor noise superimposed on the raw data. For this raw image data, both traditional raw domain denoising methods and the raw domain denoising method proposed in this invention are used for processing, and the processing results are compared and analyzed.
[0069] Specifically, Figure 2 (a) shows the processing results obtained using the traditional raw domain denoising method. In this result, although the overall noise level is suppressed to some extent, in high-frequency structural regions such as leaves and twigs, local details are significantly smoothed, some leaf and branch edges are blurred, and fine structures in different directions show inconsistent performance after denoising. Figure 2 (b) shows the denoising results obtained using the method of the present invention. In this result, by combining the noise intensity parameters with the multi-directional structural analysis results in the raw domain, the filter kernel and the high-frequency compensation process are adaptively adjusted so that while maintaining the smoothness of the low-frequency region, the multi-directional structures such as branches and leaves still retain clear morphological features, and the continuity of the high-frequency structure is well preserved.
[0070] As can be seen from the above embodiments, the method of the present invention provides more precise control over the relationship between noise suppression and structural information preservation in the raw domain, resulting in more stable consistency of the denoising process in complex natural scenes.
[0071] like Figure 3 As shown, this invention provides a novel raw domain denoising system. The system includes: an acquisition unit for acquiring raw image data to be processed and performing black level correction and normalization processing to obtain normalized raw image data and corresponding noise intensity parameters; a processing unit for calculating the noise normalization difference at each pixel position based on the noise intensity parameters and determining the filter kernel size parameters corresponding to each pixel position, while performing multi-directional structural analysis on the normalized raw image data to generate direction parameters and direction confidence parameters; a filtering unit for constructing a direction adaptive filter kernel based on the filter kernel size parameters, direction parameters, and direction confidence parameters, and performing filtering processing on the normalized raw image data to generate low-frequency denoised image data; a scheme generation unit for calculating high-frequency residual information based on the normalized raw image data and the low-frequency denoised image data, and determining high-frequency compensation parameters based on the direction confidence parameters and the noise intensity parameters; and a compensation unit for performing high-frequency component compensation on the low-frequency denoised image data according to the high-frequency compensation parameters to generate denoised raw image data, and performing inverse normalization to output the raw domain denoising result.
[0072] The present invention also provides a computer-readable storage medium storing instructions which, when executed on a computer, cause the computer to perform the novel raw domain denoising method described above.
[0073] In one embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 4As shown, the computer device includes a processor A01, a network interface A02, memory (not shown), and a database (not shown) connected via a system bus. The processor A01 provides computing and control capabilities. The memory includes internal memory A03 and a non-volatile storage medium A04. The non-volatile storage medium A04 stores an operating system B01, a computer program B02, and a database (not shown). The internal memory A03 provides an environment for the operation of the operating system B01 and the computer program B02 stored in the non-volatile storage medium A04. The network interface A02 is used for communication with external terminals via a network connection. When the computer program B02 is executed by the processor A01, it implements a novel raw domain denoising method.
[0074] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.
[0075] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.
[0076] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.
Claims
1. A novel raw domain denoising method, characterized in that, The method includes: Acquire the raw image data to be processed and perform black level correction and normalization to obtain normalized raw image data and corresponding noise intensity parameters; Based on the noise intensity parameter, the noise normalization difference at each pixel position is calculated, and the filter kernel size parameter corresponding to each pixel position is determined accordingly. At the same time, multi-directional structural analysis is performed on the normalized raw image data to generate directional parameters and directional confidence parameters. An adaptive direction filter kernel is constructed based on the filter kernel size parameter, direction parameter, and direction confidence parameter, and the normalized raw image data is filtered to generate low-frequency denoised image data. Based on the normalized raw image data and the low-frequency denoised image data, high-frequency residual information is calculated, and high-frequency compensation parameters are determined based on the direction confidence parameter and the noise intensity parameter. The low-frequency denoised image data is compensated for high-frequency components according to the high-frequency compensation parameters to generate denoised raw image data, and inverse normalization is performed to output the raw domain denoising result.
2. The novel raw domain denoising method according to claim 1, characterized in that, The process involves acquiring the raw image data to be processed and performing black level correction and normalization to obtain normalized raw image data and corresponding noise intensity parameters, including: Obtain the sensor black level parameters and saturation level parameters recorded in the raw image data to be processed, and perform black level correction processing on the raw image data based on the black level parameters; After completing the black level correction, the amplitude normalization process is performed on the corrected raw image data based on the saturation level parameter, mapping the values of each pixel to a uniform value range to generate the normalized raw image data. Based on the normalized raw image data and combined with the preset raw domain noise modeling rules, a noise parameter model is constructed to characterize the relationship between pixel signal amplitude and noise fluctuation, and the noise intensity parameter corresponding to each pixel position is calculated based on the noise parameter model.
3. The novel raw domain denoising method according to claim 1, characterized in that, Based on the noise intensity parameter, the normalized noise difference at each pixel location is calculated, and the corresponding filter kernel size parameter at each pixel location is determined accordingly, including: Obtain the noise intensity parameters corresponding to each pixel position in the normalized raw image data, and construct a preset neighborhood range around each corresponding pixel position to calculate the local statistical value of the pixels within the neighborhood range; Based on the degree of difference between the pixel value at each pixel position in the normalized raw image data and the local statistical value in the corresponding neighborhood range, and the corresponding noise intensity parameter, noise normalization processing is performed on the degree of difference to generate the noise normalization difference corresponding to each pixel position. The noise normalized difference is compared with the preset difference range, and the corresponding pixel position is divided into different noise significance levels based on the comparison result. Based on the noise saliency level, the filter kernel size parameters corresponding to each level are determined.
4. The novel raw domain denoising method according to claim 1, characterized in that, Perform multi-directional structural analysis on normalized raw image data to generate orientation parameters and orientation confidence parameters, including: Based on normalized raw image data, a preset direction set is constructed for each pixel position, and local structural response calculation is performed on the normalized raw image data along the preset direction set to obtain the structural response value corresponding to each direction; Based on the structural response values corresponding to each direction, the main structural direction of each pixel position is determined, and the main structural direction is determined as the direction parameter corresponding to the pixel position. Based on the degree of difference between the structural response values corresponding to the main structural direction and the structural response values corresponding to the non-main structural directions, a direction confidence parameter is calculated to characterize the stability and consistency of the main structural direction.
5. The novel raw domain denoising method according to claim 1, characterized in that, An adaptive direction filter kernel is constructed based on the filter kernel size parameter, direction parameter, and direction confidence parameter, including: For each pixel position in the normalized raw image data, the filter kernel size parameter corresponding to each pixel position is read to determine the spatial range of the directional adaptive filter kernel at each pixel position. Based on the orientation parameters corresponding to each pixel position, the main action direction of the orientation adaptive filter kernel at each pixel position is determined, and the spatial distribution of the orientation adaptive filter kernel at each pixel position is constrained based on the main action direction at each pixel position. Based on the direction confidence parameter corresponding to each pixel position, the expansion degree of the direction adaptive filter kernel corresponding to each pixel position in the main action direction and its perpendicular direction is adjusted so that the action weight of the direction adaptive filter kernel corresponding to each pixel position in different directions changes with the direction confidence parameter. After determining the spatial range of action, the main direction of action, and the degree of directional extension corresponding to each pixel position, an adaptive directional filter kernel corresponding to each pixel position is generated.
6. The novel raw domain denoising method according to claim 5, characterized in that, Perform filtering on normalized raw image data to generate low-frequency denoised image data, including: Based on the orientation adaptive filtering kernel corresponding to each pixel position, a pixel-by-pixel filtering operation is performed on the normalized raw image data to generate a filtered output value that corresponds one-to-one with each pixel position. The filtered output values corresponding to each pixel position are collected to form low-frequency denoised image data.
7. The novel raw domain denoising method according to claim 1, characterized in that, Based on the normalized raw image data and the low-frequency denoised image data, high-frequency residual information is calculated, and high-frequency compensation parameters are determined based on the direction confidence parameter and the noise intensity parameter, including: Based on the difference between normalized raw image data and low-frequency denoised image data, high-frequency residual information for characterizing high-frequency components is calculated for each pixel position. Based on the orientation confidence parameter and noise intensity parameter corresponding to each pixel position, weight adjustment processing is performed on the high-frequency residual information to determine the high-frequency compensation parameter corresponding to each pixel position.
8. The novel raw domain denoising method according to claim 1, characterized in that, Based on the high-frequency compensation parameters, high-frequency component compensation is performed on the low-frequency denoised image data to generate denoised raw image data, and inverse normalization is performed to output the raw domain denoising result, including: Based on the high-frequency compensation parameters corresponding to each pixel position, the high-frequency components of each pixel position in the low-frequency denoised image data are adjusted to generate denoised raw image data. Based on the denoised raw image data, an inverse normalization process corresponding to the normalization process is performed to map the pixel values back to the numerical range of the original raw image data, and the raw domain denoising result is output.
9. A novel raw domain denoising system, characterized in that, The system includes: The acquisition unit is used to acquire raw image data to be processed and perform black level correction and normalization processing to obtain normalized raw image data and corresponding noise intensity parameters. The processing unit is used to calculate the noise normalization difference at each pixel position based on the noise intensity parameter and determine the filter kernel size parameter corresponding to each pixel position accordingly. At the same time, it performs multi-directional structural analysis on the normalized raw image data to generate directional parameters and directional confidence parameters. The filtering unit is used to construct an adaptive filtering kernel based on the filter kernel size parameter, orientation parameter, and orientation confidence parameter, and to perform filtering processing on the normalized raw image data to generate low-frequency denoised image data. The scheme generation unit is used to calculate high-frequency residual information based on the normalized raw image data and the low-frequency denoised image data, and to determine high-frequency compensation parameters based on the direction confidence parameter and the noise intensity parameter. The compensation unit is used to perform high-frequency component compensation on the low-frequency denoised image data according to the high-frequency compensation parameters, generate denoised raw image data, and perform inverse normalization to output the raw domain denoising result.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the novel raw domain denoising method according to any one of claims 1-8.