Lightweight RAW-to-sRGB image mapping method and system based on single-phase opto-color

Abstract

The invention discloses a lightweight RAW-to-sRGB image mapping method and system based on single-phase opto-color, and relates to the technical field of computer vision and image processing, and the method comprises the steps: taking RAW image data as input, carrying out preprocessing to suppress noise, generating clean RAW data, and enabling four channels of the RAW image data to correspond to denoised Bayer data; extracting edge features and illumination features from the clean RAW data through an IEEM module, and outputting an illumination attention map from the illumination features; processing the image data into opposite colors, extracting multi-feeling features of opposite color channels, and calculating to obtain single-phase opposite color features; edge constraint is carried out on the single-phase color features by using the edge features to obtain color features after attention constraint and residual fusion; on the basis of color features, an illumination attention map is introduced for pixel-by-pixel weighting, and brightness adaptive adjustment is carried out; and finally generating a feature map. The feature map is processed through a plurality of convolution layers and an up-sampling layer, and the feature map is converted into a final sRGB image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision and image processing, and particularly relates to a lightweight RAW-to-sRGB image mapping method and system based on a single pair of color. BACKGROUND

[0002] In the image acquisition process of mobile devices and digital cameras, an image signal processor (ISP) converts RAW data collected by an image sensor into high-quality sRGB images for viewing. A traditional standard ISP processing chain usually includes a series of artificially designed sub-modules that work together, such as denoising, demosaicing, and automatic white balance (AWB), to optimize RAW data through multi-stage processing. Although the existing traditional ISP scheme in stages has achieved good results in conventional scenarios, it has the technical problem of relying on tedious parameter calibration to adapt to diverse scenes.

[0003] For each functional module in the traditional ISP, a large number of studies have been conducted to replace and optimize it using deep learning schemes. Among them, the data-driven end-to-end deep learning ISP scheme has excellent image generation performance and has become a research hotspot in recent years. This method directly learns the complex mapping relationship from RAW data to high-quality images. To solve the inevitable spatial misalignment problem when using this scheme, a pre-trained optical flow network is used for misalignment compensation, and there are studies on a recovery architecture based on a diffusion model, which uses its inherent iterative denoising and structure recovery capabilities to generate high-quality sRGB images.

[0004] However, the above methods generally face the following two technical problems: 1. Data recovery is performed in the RGB color space, and there is strong coupling between color components and luminance components in the color space, which leads to color distortion, overflow, and other problems in the RAW-to-sRGB mapping process; 2. To address the problem of color distortion in the final recovered image, existing technologies rely on designing more complex models to achieve the RAW-to-sRGB mapping task. Complex models are generally accompanied by a large number of parameters, and the significant increase in model parameters significantly restricts their deployment and application in actual scenarios.

[0005] Therefore, there is a need for a lightweight mapping method and system that can alleviate the distortion problem caused by color-luminance coupling while maintaining high computational efficiency. SUMMARY

[0006] The present application provides a lightweight RAW-to-sRGB image mapping method and system based on a single opponent color to solve the color distortion and edge overflow phenomenon caused by the strong coupling of color and brightness in the current RAW-to-sRGB mapping method, and the complex model structure and large number of parameters caused by solving this problem, which restricts the application in actual scenarios.

[0007] The present application is implemented by adopting the following technologies: The present application provides a lightweight RAW-to-sRGB image mapping method based on a single opponent color, comprising the following steps: S1: Taking a RAW image data with a dimension of × ×4 as input; preprocessing is performed to suppress noise, so as to avoid the noise components introduced by factors such as light fluctuation, sensor electronic device thermal noise, etc. in the acquisition process of RAW data, and to avoid noise interference with the accuracy of subsequent demosaicing, color correction, etc. Specifically, the UNet structure subnetwork is used to realize preliminary noise removal of the RAW data, to generate clean RAW data, and the 4 channels of the RAW image data correspond to the denoised Bayer data.

[0008] S2: Extracting edge features and illumination features from the clean RAW data through an IEEM (intensity and edge estimation) module, and outputting an illumination attention map from the illumination features, which is used to enhance the structure preservation and brightness adjustment capability in the subsequent image reconstruction process.

[0009] The IEEM module processing includes the following steps: S21: To obtain the brightness information consistent with human visual perception, the brightness feature BF is calculated based on the clean RAW data, and in the RGB three-channel features of the clean RAW data, R and B correspond to the red and blue channels of the Bayer array respectively, and G is the average of the two green channels, i.e. G=(G1+G2) / 2, so that the brightness feature BF conforms to the following formula:

[0010] In the formula, α1, α2, α3 are learnable parameters, and the initial weights are set to 0.299, 0.587, and 0.114 according to the human eye's perception sensitivity to different waveband light.

[0011] It is more consistent with the human visual system's perception law of environmental brightness, and provides a reliable brightness reference for subsequent feature extraction; at the same time, the learnable characteristic enhances the robustness of brightness feature extraction, so that it can better adapt to the data distribution of different scenes.

[0012] S22: On the basis of the brightness feature BF, the edge branch uses the Sobel operator to extract the edge response, and adjusts the feature dimension through a 1x1 convolution layer to obtain the edge feature EF, which conforms to the following formula:

[0013] S23: The brightness feature BF is respectively subjected to channel maximum pooling (Cmax) and channel average pooling (Cmin), the two features obtained after splicing are fused through a 7x7 convolution layer, and finally the illumination attention feature IF is generated through the Sigmoid activation function, the process is as follows:

[0014] Among them, the edge feature (EF) is used to constrain the edge information in the subsequent color optimization, and the illumination spatial attention feature (IF) is used to adjust the features of the region sensitive to the brightness change of the image, and both of them provide accurate constraints for the generation of high-quality sRGB images.

[0015] S3: Based on the single-opponent color encoding mechanism, following the "center-peripheral antagonism" logic of biological vision, the image data is processed into opponent colors, and multi-receptive features of the opponent color channels are extracted to obtain single-opponent color features.

[0016] In step S3, the following steps are specifically included: S31: For the paired opponent color channels, fixed-size convolution kernels of different scales are used to extract multi-receptive field features. One channel is used as the center excitation area to extract local features through 3x3, 5x5 and 7x7 convolution kernels, and the other color channel is used as the peripheral inhibition area to extract global features through 5x5, 7x7 and 9x9 convolution kernels. The scale of the peripheral receptive field is always larger than that of the center receptive field, which is consistent with the physiological mechanism of retinal single-opponent color encoding; S32: Subtract the center features and the peripheral features at the same scale element by element to generate antagonistic features of three scales, forming a multi-scale single-opponent color feature set; S33: The antagonistic features in the multi-scale single-opponent color feature set are spliced along the channel dimension, and the feature fusion is completed through the convolution layer to finally obtain the complete single-opponent color feature.

[0017] Through data-driven adaptive learning of the optimal receptive field scale in different scenes, the limitations of relying on artificial fixed σ and ρ in the DoG model are overcome, and more accurate opponent color feature modeling is achieved, providing high-fidelity feature input for the subsequent color optimization link, and further ensuring the color restoration accuracy of sRGB images.

[0018] S4: Although the single-opponent color coding can guarantee the accuracy of color restoration, it has weak constraint ability on color edges, which can easily lead to color overflow and edge blur in the generated image. Therefore, after obtaining the multi-scale single-opponent color features, a color cross-attention module based on residual is used to constrain the edge of the single-opponent color features in S3 by the edge features in S2, so as to enhance the clarity of edge details and obtain the color features after attention constraint and residual fusion.

[0019] The color feature CF satisfies the following formula:

[0020] In the formula: SO color represents the features after splicing of different single-opponent colors output by the multi-scale single-opponent color coding link; Q EF , K SO , and V SO are respectively composed of Query, Key, and Value of the token of the attention mechanism converted from the edge features EF, the single-opponent color features SO color , and the color features CF, wherein Q EF is used to capture edge position information, K SO and V SO correspond to the information of the color features; · represents matrix multiplication, which is used to calculate the similarity of Query and Key, generate attention weights, and then act on Value; CF represents the final color features after attention constraint and residual fusion.

[0021] S5: In order to further realize the output of sRGB image with balanced brightness and clear details, the illumination attention map in S2 is introduced to perform pixel-by-pixel weighting on the basis of the color features CF obtained in S4, and the brightness is adaptively adjusted. The brightness contribution weight of the color features is improved in the area with insufficient brightness (such as the shadow area), and the detail visibility is enhanced. The weight is reduced in the over-bright area (such as the highlight area), and the brightness redundancy and detail loss are suppressed. Finally, the feature map with balanced brightness distribution and accurate color restoration is generated.

[0022] S6: The feature map generated in S5 is processed through a plurality of convolution layers (used for feature fine-tuning and dimension adaptation) and an up-sampling layer (used for resolution recovery), so as to convert the feature map into a high-quality sRGB image with a final dimension of HxWx3.

[0023] The present invention also provides a lightweight RAW-to-sRGB image mapping system based on a single contrasting color, for implementing the above-mentioned lightweight RAW-to-sRGB image mapping method based on a single contrasting color; the system includes: a preprocessing module (RAWData Pre-processing Module, RDPM), configured to perform noise suppression processing on RAW image data.

[0024] The IEEM module (Illumination & Edge Estimation Module) is configured to extract edge features and illumination features.

[0025] The Single Opponent Color Optimization Module (SOCO) is configured to extract multi-sensory features of opposite colors to obtain single opposite color features. The core mechanism of this module is: firstly, edge color enhancement. Because edge features EF have a strong response to image edge regions, Q is calculated... EF With K SO The similarity can be used to accurately filter out SOs. color First, it assigns higher weights to feature regions associated with the edges, thereby achieving targeted enhancement of edge color details; second, it preserves non-edge information through residual connections (i.e., connections with the original SO). color Adding these elements together preserves the original color features of non-edge regions, avoiding excessive suppression of non-edge region information by the attention mechanism, thus balancing the feature representation of edge and non-edge regions. Furthermore, through the aforementioned cross-attention operation, SO can be further forced... color By aligning with edge features in the spatial dimension, the spatial distribution of color features closely matches the object's edge contour, effectively alleviating problems such as color blurring at the edges and color "overflowing" from the object's interior to the background, ultimately improving the accuracy of color restoration.

[0026] The brightness adjustment module is configured to adjust the brightness of color features using a spatial attention map of illumination.

[0027] The output module outputs sRGB images.

[0028] Through the collaborative work between modules, the model can efficiently achieve end-to-end mapping from low-quality RAW data to high-quality sRGB data. The specific functions and implementation details of each module will be elaborated in subsequent sections.

[0029] The inspiration and principles of this invention are as follows: Humans perceive the surrounding environment through the visual system, in which the retina is the core light-sensing structure, responsible for converting external light signals into neural signals that can be recognized by the nervous system and transmitted to the cerebral cortex. The signal processing of the retina relies on two types of cell pathways: one is the longitudinal pathway, composed of photoreceptor cells (cone cells and rod cells), bipolar cells and ganglion cells, which undertakes the hierarchical transmission function of signals; the other is the horizontal inhibition pathway, composed of horizontal cells and amacrine cells, which regulates signal transmission through lateral interaction. The two types of pathways work together to encode the light signals received by the photoreceptor cells into a single pair of opposition, and then transmitted to the visual cortex of the brain by the ganglion cells, completing the environmental perception process.

[0030] Based on the opponent color coding mechanism of the retina, some studies have extended it to multiple fields, such as color constancy preservation, underwater image enhancement, etc., verifying its effectiveness in image processing tasks. However, relevant studies have pointed out that single-opponent color coding is sensitive to the overall color atmosphere of the scene, but has weak ability to depict color edges, making it difficult to accurately capture boundary details.

[0031] Based on the above findings, the present application proposes a residual-based cross-attention, which optimizes the opponent color coding features through luminance edge information, and achieves good performance in the RAW-to-sRGB mapping task.

[0032] Based on the single-opponent color coding mechanism of the retina color perception, the present application proposes a color and luminance decoupled RAW-to-sRGB mapping model, namely a multi-scale single-opponent color ISP model (Multi-scale Single Opponent Color ISP, MSOCISP). Unlike existing technologies, this model does not directly operate on the RGB three channels, but draws on the opponent color mechanism of the retina to encode image information into red-green (R-G), blue-yellow (B-Y) and luminance (Luminance, L) three forms of opposition to achieve high-quality RAW-to-sRGB mapping.

[0033] Compared with the prior art, the present application has the following beneficial effects: The lightweight RAW-to-sRGB image mapping method and system based on single-opponent color designed by the present application is inspired by the single-opponent color coding mechanism of the human retina, and effectively decouples color and luminance through multi-module cooperation, finally generating high-quality sRGB images. Under the condition of realizing accurate recovery of the image, the parameter amount is also significantly reduced.

[0034] This invention upgrades the traditional Gaussian difference-based single-opposite-color calculation to a multi-scale learnable architecture. It simulates the retinal single-opposite-color mechanism to decouple brightness and color information, significantly improving the accuracy of color feature extraction and restoration. Simultaneously, it combines cross-attention mechanisms and edge feature constraints to suppress color overflow. A brightness enhancement module generates high-quality sRGB images. Experiments show that the proposed algorithm achieves optimal performance in both SSIMc and MS-SWD metrics under ZRR, demonstrating good performance in color edge restoration and confirming its accuracy and stability in color restoration. Compared to existing mainstream solutions, the MSOCISP model achieves significant performance gains in color evaluation metrics of the final generated sRGB image, and its PSNR and SSIM metrics also outperform most existing advanced solutions. Furthermore, it achieves state-of-the-art performance in terms of model parameter count and inference speed.

[0035] To implement the image mapping method described above, the included model modules need to be trained. During supervised training, a combined objective function incorporating L1 loss, wavelet loss, and color consistency loss can be used for optimization. Specifically, L1 loss reduces the overall image error, ensuring that the pixel-level difference between the restored image and the real image is minimized; wavelet loss further optimizes the ability to restore details and preserve structure; and color consistency loss constrains the consistency of color distribution between the restored image and the label.

[0036] This invention employs a convolutional structure that is easy to deploy and migrate, successfully achieving a good balance between the number of model parameters and the image restoration effect. While maintaining a higher peak signal-to-noise ratio (PSNR) than other comparative schemes, this model achieves state-of-the-art performance in both parameter size and inference speed. Attached Figure Description

[0037] Figure 1 This is a diagram showing the overall architecture of the model of the present invention.

[0038] Figure 2 It is a multi-scale monochromatic computing architecture.

[0039] Figure 3 The SSIMC and MS-SWD indices of this invention and the prior art are shown.

[0040] Figure 4 This is a comparison of the image generation effects of the present invention and existing technologies. Detailed Implementation

[0041] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings. Example 1

[0042] A lightweight RAW-to-sRGB image mapping method based on a single contrasting color, such asFigures 1-4 As shown, it includes the following steps: S1: Based on dimensions × The input consists of 4×4 RAW image data. Preprocessing is performed to suppress noise, so as to avoid noise components introduced during the acquisition of RAW data due to factors such as light fluctuations and thermal noise of sensor electronic devices. This prevents noise from interfering with the accuracy of subsequent steps such as de-mosaicing and color correction. Specifically, the initial noise removal of RAW data is achieved through a UNet subnetwork to generate clean RAW data. The 4 channels of the RAW image data correspond to the denoised Bayer data.

[0043] S2: Extract edge features and illumination features from clean RAW data using the IEEM (Luminance and Edge Estimation) module. The illumination features output an illumination attention map to enhance the ability to preserve structure and adjust brightness in subsequent image reconstruction processes.

[0044] The IEEM module processing includes the following steps: S21: The luminance feature BF is calculated based on clean RAW data. In the RGB three-channel features of clean RAW data, R and B correspond to the red and blue channels of the Bayer array, respectively, and G is the mean of the two green channels, i.e., G=(G1+G2) / 2. The luminance feature BF conforms to the following formula:

[0045] In the formula: α1, α2, and α3 are learnable parameters, and the initial weights are set to 0.299, 0.587, and 0.114, respectively, based on the sensitivity of the human eye to light of different wavelengths.

[0046] It better aligns with the human visual system's perception of ambient brightness, providing a reliable brightness benchmark for subsequent feature extraction; at the same time, the learnable nature enhances the robustness of brightness feature extraction, enabling it to better adapt to data distributions in different scenarios.

[0047] S22: Based on the brightness feature BF, the edge branch uses the Sobel operator to extract the edge response, and adjusts the feature dimension through a 1×1 convolutional layer to obtain the edge feature EF, which conforms to the following formula:

[0048] S23: Perform channel max pooling (Cmax) and channel average pooling (Cmin) on the luminance feature BF, concatenate the two features, and then fuse them through a 7×7 convolutional layer. Finally, generate the illumination attention feature IF through the sigmoid activation function. The process is as follows:

[0049] Among them, edge features (EF) are used to constrain edge information in subsequent color optimization, while illumination spatial attention features (IF) are used to adjust the features of areas sensitive to changes in image brightness. Together, they provide precise constraints for the generation of high-quality sRGB images.

[0050] S3: Based on the single-opposite-color coding mechanism, the image data is processed into opposite colors, the multi-sensory features of the opposite color channels are extracted, and the single-opposite-color features are calculated.

[0051] like Figure 2 Step S3 specifically includes the following steps: S31: For the paired yellow (Y) and blue (B) color channels, fixed-size convolutional kernels of different scales are used to extract multi-receptive field features. The blue channel, as the central excitation area, uses 3×3, 5×5, and 7×7 convolutional kernels to extract local features. The yellow channel, as the peripheral inhibition area, uses 5×5, 7×7, and 9×9 larger convolutional kernels to extract global features. The peripheral receptive field scale is always larger than the central receptive field; this is consistent with the physiological mechanism of single-opposite color coding in the retina. S32: Subtract the blue central feature from the yellow peripheral feature at the same scale element by element to simulate the antagonistic effect of "central excitation - peripheral inhibition", thereby generating a multi-scale mono-opposite color feature MCSO. B+Y- (Multi-scale SingleOpponent feature for B + Y - Its formal expression is shown in the following formula:

[0052] In the formula: Convk(·) represents a k×k convolution operation, This indicates a feature concatenation operation along the channel dimension. The three subtraction terms correspond to antagonistic features at small, medium, and large scales, respectively, and together constitute a multi-scale single-opposite color feature set.

[0053] S33: Introducing a channel attention mechanism for MCSO B+Y- Adaptive weighting is performed by learning the importance weights of channels at different scales to highlight the scale features that are more critical to the current image scene and weaken the interference of redundant scales, ultimately obtaining an accurate SO. B+Y- =CA(MCSO B+Y- Encoding features (CA represents the channel attention module). Similarly, SO can be calculated. Y+B- feature.

[0054] SO B+Y- with SO Y+B-The features are stitched together along the channel dimension and fused through a convolutional layer to obtain the complete blue-yellow single-color contrast features.

[0055] By adaptively learning the optimal receptive field scale in different scenarios through a data-driven approach, the DoG model overcomes the limitations of manually fixing σ and ρ, achieving more accurate modeling of contrasting color features. This provides high-fidelity feature input for subsequent color optimization, thereby ensuring the color reproduction accuracy of sRGB images.

[0056] S4: A residual-based color cross-attention module is adopted. The edge features in S2 are used to constrain the single opposite color features in S3, thereby enhancing the clarity of edge details and obtaining the color features after attention constraint and residual fusion.

[0057] Color characteristics (CF) conform to the following formula:

[0058] In the formula: SO color This represents the features of different single-opposite colors output from the multi-scale single-opposite color coding stage after splicing. Q EF K SO V SO Composed of edge features EF and single contrasting color features SO respectively color The transformed attention mechanism tokens constitute the Query, Key, and Value, where Q... EF K is used to capture edge location information. SO With V SO Information corresponding to color characteristics; • Represents matrix multiplication, used to calculate the similarity between the Query and the Key, and then apply the generated attention weights to the Value; CF represents the final color characteristics after attention constraint and residual fusion.

[0059] S5: To further achieve balanced brightness and clear details in sRGB image output, based on the color feature CF obtained in S4, the illumination attention map in S2 is introduced for pixel-by-pixel weighting to adaptively adjust brightness; IF is used to weight CF pixel-by-pixel, increasing the brightness contribution weight of color features in areas with insufficient brightness (such as shadow areas) to enhance detail visibility; and reducing the weight in overly bright areas (such as highlight areas) to suppress brightness redundancy and detail loss; finally, a feature map with balanced brightness distribution and accurate color reproduction is generated.

[0060] S6: The feature map generated by S5 is processed through several convolutional layers (for feature fine-tuning and dimension adaptation) and upsampling layers (for resolution restoration) to convert the feature map into a high-quality sRGB image with a final dimension of H×W×3. Example 2

[0061] A lightweight RAW-to-sRGB image mapping system based on a single contrasting color, the system comprising: The preprocessing module (RDPM) is configured to perform noise suppression processing on RAW image data.

[0062] The IEEM module (Luminance and Edge Estimation Module) is configured to extract edge features and illumination features.

[0063] The Single Opposite Color Processing Module (SOCO) is configured to extract multi-sensory features of opposite colors to obtain single opposite color features.

[0064] The brightness adjustment module is configured to adjust the brightness of color features using a spatial attention map of illumination.

[0065] The output module outputs sRGB images.

[0066] Through the collaborative work between modules, the model can efficiently achieve end-to-end mapping from low-quality RAW data to high-quality sRGB data. The specific functions and implementation details of each module will be elaborated in subsequent sections.

[0067] Effect verification: Training and testing were performed on the publicly available RAW-to-sRGB datasets ZRR and MAI. The ZRR dataset contains 46.8k training RAW-sRGB image pairs and 1.2k evaluation pairs. The RAW images were captured by a Huawei P20 (10-bit), and the sRGB images were captured by a Canon camera. The MAI dataset focuses on the mapping of RAW images (12-bit) captured by a Sony IMX586 to the sRGB distribution of a Fuji camera. Since there is no ground truth (GT) in the test set, the training set was divided into 21.7k training pairs and 2.4k evaluation pairs in a 9:1 ratio.

[0068] During training, random 256×256 pixel cropping was used, along with the Adam optimizer and a learning rate of 10. -4 The experiment was optimized using a combined objective function that includes L1 loss, wavelet loss, and color consistency loss, and trained for 500 epochs. The experiment was implemented using the PyTorch framework and the hardware platform was an NVIDIA RTX 4090 GPU.

[0069] Performance comparison with comparable models: On the ZRR (Huawei P20 RAW → Canon sRGB) and MAI (Sony IMX586 RAW → Fujifilm sRGB) datasets, MSOCISP comprehensively outperforms or surpasses most comparable models in core objective quality and efficiency metrics. In the original ground truth (GT) settings of the ZRR dataset, its PSNR reaches 21.79, higher than PyNet (21.19), AWNet-D (21.53), and even slightly exceeding ISPDiffuser (21.77); after GT alignment, the PSNR further increases to 24.33, significantly leading LiteISPNet (23.76) and FourierISP (23.93); on the MAI dataset, the PSNR is 25.72, leading ISPDiffuser (25.64) and FourierISP (25.37). In terms of efficiency, it has a more prominent advantage, with only 4.8M parameters, which is nearly 40% lower than the lightest existing FourierISP (7.6M). Its inference time is 20.5ms, which is much faster than ISPDiffuser (398.6ms) and AWNet-D (109.4ms). It is the only model that balances high recovery quality and extreme lightweight design.

[0070] MSOCISP performs exceptionally well in terms of color restoration accuracy and qualitative visual effects. In ZRR-aligned ground truth (GT) scenarios, its color-related metrics (SSIMc, MS-SWD) are optimal. SSIMc is higher than models such as AWNet-R / D and LiteISPNet, while MS-SWD (quantized color difference) is significantly lower, confirming the high consistency between its color structure and the real image (GT). Qualitative error maps show that in large areas of monochromatic regions that are difficult to restore (e.g., the sky) where its generated images are susceptible to luminance-color coupling interference, the difference from GT is much smaller than that of PyNet and MW-ISNet. In terms of detail texture, although SSIM is on par with FourierISP and LiteISPNet in some scenes, it does not sacrifice detail integrity due to its lightweight design. Furthermore, while LPIPS (perceptual loss) is not optimal, it is only slightly different from top models such as ISPDiffuser and FourierISP, achieving an overall balance between color accuracy, detail fidelity, and deployment efficiency.

[0071] The comparison examples include: seven mainstream and advanced solutions in the field, specifically: PyNet proposed by Ignatov et al. in 2020, which is the first end-to-end model focusing on "mobile phone RAW image → SLR RGB image", trained on large-scale paired data, but failed to solve the spatial misalignment problem of dual-device shooting; AWNet-R / AWNet-D, an excellent solution for the AIM Challenge proposed by Dai et al. in 2020, which extracts multi-scale information through wavelet transform and optimizes color mapping by combining global context blocks, with R / D being different variants and AWNet-D having a longer inference time; MW-ISNet, a solution from the AIM2020 Challenge, which focuses on multi-scale feature fusion and is mainly optimized by engineering based on existing technologies, showing stable performance in the task; and LiteISPN proposed by Zhang et al. in 2021. et al. designed a global color mapping module to solve the color inconsistency problem and introduced optical flow to align the output with the real image, which was one of the lighter models before. FourierISP proposed by He et al. in 2024 decoupled the image into structure (phase) and style (amplitude) by Fourier transform and reduced the RGB space coupling interference, which was once a representative of lightweight models. DiffRAW proposed by Yi et al. in 2024 was the first solution to use the diffusion model for this task. It uses RAW images as diffusion conditions to preserve contour texture, but the source code is not public and the number of parameters and inference time cannot be obtained. And ISPDiffuser proposed by Ren et al. in 2025 uses the diffusion model to generate texture and a dedicated module to generate color. The image quality is excellent, but the iterative characteristics of the diffusion model result in an extremely long inference time (398.6ms).

[0072] Quantitative Results: The method of this invention was compared with the current mainstream solutions on the same test set. It should be noted that due to the slight spatial misalignment between the label image and the input RAW data in the ZRR dataset, the optical flow estimation model PWCNet was used to align the output with the label. The results under this setting are referred to as ZRR (Align GT with RAW) in the subsequent experimental results.

[0073] like Figure 3 SSIMc is a structural similarity index that considers color information and is used to quantify the structural consistency of color-information components. MS-SWD is a color consistency evaluation index, which can robustly evaluate the color restoration effect without relying on precise image alignment. A higher SSIMC value is better, while a higher MS-SWD value is better. It is evident that this invention achieves the best performance in both of these metrics, demonstrating the excellent performance of the proposed algorithm in color edge restoration and confirming its accuracy and stability in color restoration.

[0074] To further verify the effectiveness of the proposed MSOCISP algorithm, Tables 1 and 2 present the multi-dimensional quantitative evaluation results of the present invention and the comparative examples on the two public datasets ZRR and MAI.

[0075]

[0076]

[0077] As can be seen, under the Original GT setting of the ZRR dataset, the peak signal-to-noise ratio (PSNR) of this invention is the best. Although affected by the misalignment problem of the original label space, its structural similarity index (SSIM) and perceptual loss (LPIPS) do not reach the optimal level, the difference with the comparative examples is small, and it still remains competitive. The evaluation results of the ZRR (Align GT with RAW) and MAI datasets show that this invention achieves the best performance in terms of PSNR. This result shows that MSOCISP has outstanding performance in terms of consistency of image restoration, which strongly confirms the effectiveness and reliability of this invention. In addition, the SSIM and LPIPS of this invention are better than most comparative examples, which fully demonstrates that the restored image achieves an excellent level in terms of both structural integrity and subjective perceptual quality.

[0078] It is worth noting that, as shown in Table 1, the number of parameters in this invention is reduced by nearly 40% compared to the smallest model in the comparative example, while achieving optimal computation time. This indicates that by mimicking the human eye's monochromatic color mechanism to decouple brightness and color, the model complexity can be significantly reduced while maintaining superior performance.

[0079] Qualitative results: such as Figure 4 As shown in the figure, the error map represents the absolute difference between the generated image and the ground truth (GT) image. The greater the difference, the higher the pixel value and the brighter the corresponding area. Combining the quantized differences in the error map, it can be seen that the error in overall image color consistency of this invention is significantly smaller than that of the comparative model. Especially in large monochromatic areas such as the sky, these areas are typical difficult-to-recover scenes in RAW-to-sRGB mapping tasks due to their uniform color distribution and susceptibility to brightness coupling effects. However, the image generated by this invention still maintains a high degree of color consistency with the GT, fully demonstrating the robustness of the model. Furthermore, observation also shows that this invention is no less detailed than the comparative model.

[0080] Module necessity verification Ablation experiments were conducted using controlled variable experiments to verify the contributions of each core module. The specific analysis is as follows: The traditional Difference-of-Gaussian (DOB) model was replaced with two additional DOB-based single-opposite color coding models. One was a fixed-parameter model, with the Gaussian convolution kernel parameters σ fixed at 1.5, ρ fixed at 2, and a kernel size of 3×3. The other was a learnable-parameter model, with initial values ​​of σ and ρ set to 1.5 and 2 respectively, both used as learnable parameters, and a kernel size of 6σ (consistent with the energy coverage characteristics of a Gaussian distribution). The comparative experimental results are shown in Table 3.

[0081]

[0082] The results show that the evaluation metrics of the learnable parameter model are superior to those of the fixed parameter model, confirming the effectiveness of the data-driven approach in neural model modeling; however, compared with the multi-scale model proposed in this invention, the learnable parameter model still has a performance gap. This further verifies that by fusing features from different scales, this invention can more accurately simulate the human eye's color coding mechanism, and its performance is superior to single-scale Gaussian difference simulation.

[0083] In the SOCO module, a residual-based color cross-attention module is used for subsequent processes. A comparative experiment was designed to remove this module, simply concatenating the edge features (EF) with the color features before proceeding to the next step (corresponding to the "w / o CO" configuration in Table 3). The experiment shows that after removal, all metrics decreased, and the edge regions of the generated image exhibited color blurring and overflow. The SSIM metric was significantly reduced, indicating that this module effectively constrains color edges, enhances the texture details of the restored result, and is crucial for improving image quality.

[0084] To verify the necessity of each loss term, three sets of comparative experiments were designed: using only L1 loss, L1 + wavelet loss (Lwave), and L1 + color consistency loss (L... Color The results are shown in Table 3.

[0085] Experimental results show that using only L1 loss yields the worst performance. Adding wavelet loss improves all metrics, confirming its positive effect on image restoration. Adding color consistency loss also improves color-related evaluation metrics, indicating that this loss effectively constrains the color distribution differences between the generated image and the label, accelerating metric convergence within the same training epochs. In summary, each loss term plays a unique role in model training, and their combined use achieves optimal performance.

[0086] The scope of protection claimed by this invention is not limited to the specific embodiments described above. Moreover, for those skilled in the art, this invention can have various modifications and alterations. Any modifications, improvements, and equivalent substitutions made within the concept and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A lightweight RAW-to-sRGB image mapping method based on a single contrasting color, characterized in that, Includes the following steps: S1: Using RAW image data as input, preprocessing is performed to suppress noise and generate clean RAW data, wherein the 4 channels of the RAW image data correspond to the denoised Bayer data; S2: Extract edge features and illumination features from clean RAW data using the IEEM module, and output the illumination features as an illumination attention map; S3: Based on the single-opposite-color coding mechanism, the image data is processed into opposite colors, the multi-sensory features of the opposite color channels are extracted, and the single-opposite-color features are calculated. S4: Use the edge features in S2 to perform edge constraints on the single contrasting color features in S3 to obtain the color features after attention constraint and residual fusion; S5: Based on the color feature CF obtained in S4, the illumination attention map in S2 is introduced for pixel-by-pixel weighting and adaptive brightness adjustment; finally, a feature map is generated. S6: The feature map generated by S5 is processed through several convolutional layers and upsampling layers to convert the feature map into the final sRGB image.

2. The lightweight RAW-to-sRGB image mapping method based on a single contrasting color according to claim 1, characterized in that: The IEEM module processing includes the following steps: S21: The luminance feature BF is calculated based on clean RAW data. In the RGB three-channel features of clean RAW data, R and B correspond to the red and blue channels of the Bayer array, respectively, and G is the mean of the two green channels. The luminance feature BF conforms to the following formula: ； In the formula: α1, α2, α3 are learnable parameters; S22: Based on the brightness feature BF, the edge branch uses the Sobel operator to extract the edge response, and adjusts the feature dimension through a 1×1 convolutional layer to obtain the edge feature EF, which conforms to the following formula: ； S23: Perform channel max pooling and channel average pooling on the luminance feature BF respectively. Concatenate the two features and fuse them through a 7×7 convolutional layer. Finally, generate the illumination attention feature IF through the sigmoid activation function. The process is as follows: 。 3. The lightweight RAW-to-sRGB image mapping method based on a single contrasting color according to claim 1, characterized in that: Step S3 specifically includes the following steps: S31: For paired opposing color channels, fixed-size convolutional kernels of different scales are used to extract multi-receptive field features. One channel is used as the central excitation area to extract local features through three types of convolutional kernels: 3×3, 5×5, and 7×7. The other color channel is used as the peripheral inhibition area, and three types of convolutional kernels of one level larger, namely 5×5, 7×7, and 9×9, are used to extract global features. The scale of the peripheral receptive field is always larger than that of the central receptive field. S32: Subtract the central features from the surrounding features at the same scale element by element to generate antagonistic features at three scales, forming a multi-scale single-opposite color feature set; S33: The antagonistic features in the multi-scale single-opposite color feature set are spliced ​​along the channel dimension, and the feature fusion is completed through the convolutional layer to finally obtain the complete single-opposite color features.

4. The lightweight RAW-to-sRGB image mapping method based on a single contrasting color according to claim 1, characterized in that: In step S4, the color feature CF conforms to the following formula: ； In the formula: SO color This represents the features of different single-opposite colors output from the multi-scale single-opposite color coding stage after splicing. Q EF K SO V SO Composed of edge features EF and single contrasting color features SO respectively color The transformed attention mechanism tokens constitute the Query, Key, and Value, where Q... EF K is used to capture edge location information. SO With V SO Information corresponding to color characteristics; • Represents matrix multiplication, used to calculate the similarity between the Query and the Key, and then apply the generated attention weights to the Value; CF represents the final color characteristics after attention constraint and residual fusion.

5. The lightweight RAW-to-sRGB image mapping method based on a single contrasting color according to claim 1, characterized in that: In step S5, the brightness contribution weight of color features is increased in areas with insufficient brightness to enhance the visibility of details; Reduce the weight of overly bright areas to suppress brightness redundancy and loss of detail.

6. A lightweight RAW-to-sRGB image mapping system based on a single contrasting color, characterized in that: Used to implement the lightweight RAW-to-sRGB image mapping method based on a single contrasting color as described in any one of claims 1 to 5; The system includes: The preprocessing module is configured to perform noise suppression processing on RAW image data; The IEEM module is configured to extract edge features and illumination features; The single-opposite-color processing module is configured to extract multi-sensory features of opposite colors to obtain single-opposite-color features. The brightness adjustment module is configured to adjust the brightness of color features using the illumination spatial attention map. The output module outputs sRGB images.

Images