Image processing method, electronic device, and computer-readable storage medium

Abstract

This application provides an image processing method, an electronic device, and a computer-readable storage medium. Relating to the field of computer vision technology, the method includes: processing multiple sub-images included in a target image to obtain noise distribution features corresponding to each sub-image; adjusting the feature extraction intensity of the initial image features corresponding to each sub-image based on the noise distribution features corresponding to each sub-image to obtain noise fusion features corresponding to each sub-image; decoupling the noise fusion features corresponding to each sub-image into illumination features and reflection features; fusing the illumination features and reflection features corresponding to each sub-image to obtain illumination fusion features corresponding to each sub-image; and performing feature processing at different scales on the illumination fusion features corresponding to each sub-image to obtain a processed target image. In the technical solution of this application embodiment, this application solves the technical problem that the related technologies have insufficient processing capabilities for low-light images, resulting in unsatisfactory image quality.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and in particular to an image processing method, an electronic device, and a computer-readable storage medium. Background Technology

[0002] In extremely low-light environments, image acquisition faces significant inherent challenges, leading to a severe degradation in image quality and consequently negatively impacting various computer vision tasks. Due to insufficient photons, image sensors capture light with a high degree of graininess, random artifacts, and blurring, making previously sharp details difficult to discern or completely lost. This severe degradation in image quality directly affects the performance of subsequent advanced computer vision tasks.

[0003] Currently, image enhancement evaluation in related technologies often focuses on visual perception quality, such as brightness, contrast, and color reproduction. These technologies frequently retain or even amplify input noise, leading to detail loss in subsequent image processing and resulting in suboptimal post-processing image quality. Summary of the Invention

[0004] This application provides an image processing method, an electronic device, and a computer-readable storage medium to alleviate or solve the technical problem that the related technologies have insufficient processing capabilities for low-light images, resulting in unsatisfactory image quality.

[0005] In a first aspect, embodiments of this application provide an image processing method applied to a target model, comprising:

[0006] The target image is processed separately for each of its multiple sub-images to obtain the noise distribution features corresponding to each of the multiple sub-images;

[0007] Based on the noise distribution characteristics corresponding to each sub-image, the feature extraction intensity of the initial image features corresponding to each sub-image is adjusted to obtain the noise fusion features corresponding to each sub-image.

[0008] The noise fusion features corresponding to each sub-image are decoupled into illumination features and reflection features. The illumination features are used to represent the brightness distribution in the corresponding sub-image, and the reflection features are used to represent the texture features in the corresponding sub-image.

[0009] The illumination features and reflection features corresponding to each sub-image are fused to obtain the illumination fusion features corresponding to each sub-image.

[0010] The illumination fusion features corresponding to each sub-image are subjected to feature processing at different scales to obtain the processed target image.

[0011] Secondly, embodiments of this application provide an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the methods of embodiments of this application when executing the computer program.

[0012] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the method of any one of the embodiments of this application.

[0013] Based on the image method described in the first aspect above, this application has at least the following beneficial effects or advantages:

[0014] Based on noise distribution characteristics, the extraction intensity of initial image features for each sub-image is adjusted to effectively suppress the impact of noise on the image. Separating and processing illumination and reflection features separately better preserves image details and textures. Multi-scale feature processing captures different levels of detail in the image, improving the overall image quality. This application aims to overcome the limitations of existing low-light image enhancement methods in noise suppression, detail preservation, color fidelity, and generalization ability, especially under extremely low-light conditions. Robust enhancement of extremely low-light images is achieved through a lightweight and efficient end-to-end network. This method does not rely on large-scale paired datasets but instead significantly improves its applicability to downstream computer vision tasks while ensuring image visual quality through an innovative unsupervised / self-supervised loss function.

[0015] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application, it can be implemented according to the contents of the specification. In order to make the above and other objects, features and advantages of this application more obvious and understandable, specific embodiments of this application are given below. Attached Figure Description

[0016] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments according to this application and should not be construed as limiting the scope of this application.

[0017] Figure 1 A flowchart illustrating an image processing method according to an embodiment of this application is shown;

[0018] Figure 2 A schematic block diagram of an image processing method according to an embodiment of this application is shown;

[0019] Figure 3 A schematic diagram of an image processing apparatus according to an embodiment of this application is shown;

[0020] Figure 4 A block diagram of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0021] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the concept or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0022] To facilitate understanding of the technical solutions of the embodiments of this application, the relevant technologies of the embodiments of this application are described below. The following relevant technologies are optional solutions and can be combined with the technical solutions of the embodiments of this application in any way, and all of them fall within the protection scope of the embodiments of this application.

[0023] The following terms will be used in the following text:

[0024] The U-Net network is a deep learning architecture consisting of an encoder and a decoder, arranged in a U-shape. The encoder extracts image features and reduces spatial dimensions through convolution and pooling operations, while the decoder restores the image resolution through upsampling and convolution operations, and performs precise processing by combining the feature information from the encoder.

[0025] Attention mechanisms are model design methods that simulate human visual attention, helping neural networks focus on important parts when processing images. They dynamically assign different weights to different parts of the input data, allowing the model to focus on key information relevant to the current task while ignoring less important parts. For example, in image recognition, attention mechanisms allow the model to concentrate on regions in the image related to the target object, improving model performance and efficiency while making the model's decision-making process more interpretable.

[0026] Image acquisition in extremely low-light environments faces significant inherent challenges, leading to a severe decline in image quality and adversely affecting various computer vision tasks. These challenges are mainly manifested in the following aspects: Severe noise and low signal-to-noise ratio are the core problems of images in extremely low light. Due to the insufficient number of photons, the internal noise (such as shot noise and readout noise) of the image sensor becomes exceptionally prominent when capturing light. This results in a large amount of graininess, random artifacts, and blurring in the image, making originally clear details difficult to discern or completely lost. For example, in scenarios such as surveillance, night driving, or smartphone photography, even weak light is insufficient to effectively activate the sensor, resulting in high-noise, low-signal images.

[0027] Images in extremely low light conditions generally suffer from very low contrast and insufficient brightness. The dynamic range of the image is severely compressed, and most pixel values ​​are concentrated at the low end of the brightness range, resulting in an overall dark and gray visual effect. Under these conditions, the brightness differences between different objects or areas in the image are negligible, making them visually difficult to distinguish, and a large amount of information is submerged in darkness.

[0028] Color distortion and loss of detail are also common in images produced in extremely low light. Due to insufficient light, color information in an image is easily distorted or completely lost, resulting in a monotonous and unnatural tone. Simultaneously, due to the aforementioned noise and contrast issues, textures, edges, and fine structural details in the image are severely blurred or even completely lost, further reducing the image's information content and visual quality.

[0029] Severe image quality degradation directly impacts the performance of subsequent advanced computer vision tasks. For example, in low-light environments, the false positive rate of object detection algorithms increases significantly, and the false negative rate also rises; the accuracy of image segmentation and classification algorithms drops sharply; and tasks such as face recognition and vehicle recognition face significant challenges due to insufficient image information. This indicates that relying solely on the robustness of existing computer vision algorithms is insufficient to effectively handle images in extremely low light conditions, and a method for high-quality image enhancement is urgently needed.

[0030] It is worth noting that traditionally, image enhancement evaluations have often focused on visual perception quality, such as brightness, contrast, and color fidelity. However, existing research has found that high-perceptual-quality images do not always effectively improve the performance of downstream computer vision tasks, and sometimes there can even be contradictions. For example, supervised learning methods, while capable of generating images with high perceptual quality, often offer limited improvement to downstream vision tasks; conversely, zero-shot learning methods may score lower on image quality metrics but more consistently improve performance across various vision tasks. This indicates a disconnect between image quality evaluation metrics and the actual needs of machine vision tasks. Therefore, a "practical" image enhancement method not only needs to make images look better visually, but more importantly, it must be able to generate enhanced images that are more compatible with machine vision algorithms to effectively improve their accuracy and reliability in practical applications.

[0031] In view of the limitations of the prior art, this application aims to overcome the following specific technical problems:

[0032] (1) Under extremely low light conditions, it effectively suppresses the inherent severe noise in the image while preserving the texture details and edge information of the image to the maximum extent, avoiding the loss of details caused by excessive smoothing.

[0033] (2) Adaptive enhancement and correction of image brightness, contrast and color to avoid overexposure or underexposure, restore the natural color balance of the image and eliminate color deviation.

[0034] (3) The model has high generalization ability and robustness to diverse and complex real-world low-light scenes, reducing the dependence on large-scale paired training data, and enabling it to work stably and efficiently in unseen environments.

[0035] (4) The gap between image perception quality and the performance of downstream computer vision tasks is eliminated, ensuring that the enhanced image not only performs well visually, but also effectively improves the accuracy and reliability of machine vision algorithms such as target detection, image segmentation, target recognition and classification.

[0036] The technical solution of this application and how it solves the aforementioned technical problems are described in detail below with specific embodiments. The listed specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0037] Figure 1 A flowchart illustrating an embodiment of the image processing method of this application is shown, such as... Figure 1 As shown, the method may include steps S101 to S105.

[0038] Step S101: Process the multiple sub-images included in the target image separately to obtain the noise distribution features corresponding to each sub-image;

[0039] Step S102: Based on the noise distribution characteristics corresponding to each sub-image, adjust the feature extraction intensity of the initial image features corresponding to each sub-image to obtain the noise fusion features corresponding to each sub-image;

[0040] Step S103: Decouple the noise fusion features corresponding to each sub-image into illumination features and reflection features. Illumination features are used to represent the brightness distribution in the corresponding sub-image, and reflection features are used to represent the texture features in the corresponding sub-image.

[0041] Step S104: Fuse the illumination features and reflection features corresponding to each sub-image to obtain the illumination fusion features corresponding to each sub-image;

[0042] Step S105: Perform feature processing at different scales on the illumination fusion features corresponding to each sub-image to obtain the processed target image.

[0043] For example, the execution subject of this application embodiment is a target model, which can be embedded in various devices or systems to adapt to different application scenarios. The following description uses various application scenarios such as surveillance, autonomous driving, and smartphone photography as examples. The target model can be embedded in the hardware system of a surveillance camera or deployed on the backend server of the surveillance system, enabling the surveillance camera to clearly capture details of the target object even under low light, backlight, or complex lighting conditions.

[0044] The target model can be embedded in the onboard computer system of an autonomous vehicle or integrated into the vehicle's camera module. In autonomous driving, the vehicle needs to process a large amount of visual information in real time to identify roads, traffic signs, pedestrians and other vehicles. Through noise suppression and feature enhancement, the clarity and accuracy of images in adverse weather conditions such as rain or fog can be improved, thereby enhancing the safety and reliability of the autonomous driving system.

[0045] The target model can be embedded in the image processing chip of a smartphone or integrated as a software algorithm into the phone's photography application. When shooting night scenes, optimizing lighting features can improve the brightness and color performance of the photo, while enhancing reflection features can preserve the texture and details of the subject.

[0046] The target model can be embedded in the vision inspection system on the industrial production line or deployed on the back-end server of industrial quality control. Image quality directly affects the accuracy and efficiency of defect detection. Effectively suppressing noise and enhancing the illumination and reflection characteristics of the image can more accurately identify and locate defects on the product surface.

[0047] In the embodiments provided in this application, each sub-image is processed separately to extract its noise distribution features, thereby understanding the noise type and intensity in each sub-image. Based on the noise distribution features, the initial image feature extraction intensity of each sub-image is dynamically adjusted to generate a noise fusion feature that combines noise and image features. The noise distribution features are used to guide the adjustment of the image feature extraction intensity. Traditional methods often fail to accurately distinguish between noise and the true features of an image when processing images, resulting in the loss of important details or the presence of noise interference in the processed image. However, this application, by analyzing the noise distribution features of each sub-image and dynamically adjusting the feature extraction intensity, can effectively suppress the impact of noise on image quality while preserving key image information. In real images, illumination and reflection features are usually mixed together. Changes in illumination affect the presentation of reflection features, and reflection features are also affected by illumination conditions. This mixture makes it difficult to distinguish between noise and true features in an image.

[0048] The noise fusion features are further decoupled into illumination features and reflection features. By separating these two features, noise can be identified and processed more clearly, which helps in processing the brightness and texture information of the image. The decoupled illumination features and reflection features can be processed independently. This separation makes the processing more targeted; specialized illumination correction algorithms can be used for illumination features, and texture enhancement algorithms can be used for reflection features. Targeted optimization can more effectively improve image quality. The decoupled illumination features and reflection features are then fused to obtain the illumination fusion features. Multi-scale processing is performed on the illumination fusion features to capture image details at different levels, ultimately generating a high-quality processed target image.

[0049] It should be noted that noise is one of the most fundamental forms of degradation in extremely low-light images and a common pain point faced by existing methods. While the Zero-DCE series of methods performs well in brightness adjustment and color correction, their primary focus is not on noise suppression. Therefore, the enhancement results often retain input noise, and may even amplify it. Other related techniques, such as Retinex or histogram equalization, often amplify noise or cause loss of detail while enhancing brightness. The core difference between the embodiments of this application and related technologies lies in using noise distribution characteristics to guide the intensity of image feature extraction. This avoids over-extracting invalid features in areas with high noise levels and also prevents under-extracting features in areas with low noise levels, thereby making the obtained noise fusion features more accurately reflect the effective information of the sub-image.

[0050] The embodiments provided in this application, through a pre-constructed noise-aware feature extraction module, proactively model and suppress noise during the feature extraction stage. Combined with depthwise separable convolution and attention mechanisms, this enables more refined noise reduction while preserving image texture details and edge information to the maximum extent. This avoids the shortcomings of Zero-DCE in noise processing and the over-smoothing defects of traditional methods. This design, which elevates noise suppression to the core module level, represents a deeper understanding and a more fundamental solution to low-light problems.

[0051] According to some embodiments provided in this application, the target model includes a noise perception network layer. The noise perception network includes classification heads corresponding to multiple predetermined noise types. Multiple sub-images included in the target image are processed to obtain noise distribution features corresponding to each sub-image, including:

[0052] Based on the sub-images and the device data for acquiring the target image, the modulation vectors corresponding to multiple noise types are determined.

[0053] Based on the modulation vectors corresponding to multiple noise types, the feature extraction intensity of the corresponding classification head is controlled to obtain the noise component features and confidence levels corresponding to multiple noise types.

[0054] Based on confidence levels, the noise component features corresponding to multiple noise types are fused to obtain the noise distribution features corresponding to each sub-graph.

[0055] In the embodiments provided in this application, based on the various sub-images contained in the target image and the device data for acquiring the target image, modulation vectors corresponding to multiple noise types are determined. These modulation vectors reflect the characteristics of different noise types and the impact of device data on the noise. Based on the modulation vectors corresponding to each of the multiple noise types, the feature extraction intensity of the corresponding classification head is controlled to obtain the noise component features and confidence levels corresponding to each noise type. This makes feature extraction for each noise type more targeted, where the confidence level reflects the reliability of the noise component feature. Based on the confidence levels of each noise type, the corresponding noise component features are fused, effectively highlighting the role of high-confidence features and suppressing the interference of low-confidence features, thereby obtaining the noise distribution features corresponding to each sub-image, making the fusion result more closely match the actual noise situation of the sub-image.

[0056] For example, the Noise-Aware Feature Extraction Module (NAFEM) provided in this application explicitly models and suppresses noise during the feature extraction stage, rather than simply smoothing it. This module combines depthwise separable convolutions with attention mechanisms (such as Squeeze-and-Excitation blocks or SimAM attention mechanisms) to effectively denoise while preserving texture details, thus addressing the shortcomings of existing methods in noise processing.

[0057] For example, an adaptive learning strategy can be adopted in the feature extraction intensity control step of the classification head. By introducing a deep learning network, the classification head can autonomously learn and adjust the feature extraction intensity based on the matching degree between the noise component features in historical processed data and the actual noise distribution. For instance, when the confidence level of a certain noise type's noise component features does not match the actual noise impact in multiple processing iterations, the network can automatically correct the corresponding control parameters, improving the targeting of feature extraction.

[0058] For example, a multi-dimensional weight evaluation mechanism is introduced in the noise component feature fusion stage, such as the correlation between noise component features and basic features such as sub-image texture and brightness. If a certain noise component feature has a strong correlation with the basic features of the sub-image, it means that it has a greater impact on the overall noise distribution and can be given a higher weight during fusion, so that the obtained noise distribution features are more comprehensive and reliable.

[0059] According to some embodiments provided in this application, based on the target image and the device data for acquiring the target image, modulation vectors corresponding to multiple noise types are determined, including:

[0060] For pixel types in multiple noise types, feature extraction is performed based on each sub-image to obtain noise pattern information, texture distortion information and pixel value statistics. The pixel value statistics are used to describe the statistical data of variance and / or skewness of local pixels in each sub-image.

[0061] Feature fusion is performed based on noise pattern information, texture distortion information, and pixel value statistics to obtain a pixel-type modulation vector.

[0062] In some embodiments provided in this application, among multiple noise types, multi-dimensional information is extracted for pixel-type noise. Noise pattern information reflects the distribution pattern of pixel noise, texture distortion information reflects the degree of damage noise causes to image texture, and pixel value statistics quantify the discreteness of local pixels from a numerical perspective. The combination of these three allows the extracted features to more comprehensively cover the characteristics of pixel noise. The modulation vector obtained by fusing these multi-dimensional information can more accurately characterize the essence of pixel-type noise, providing a more reliable basis for subsequently controlling the feature extraction intensity of the classification head, thereby improving the accuracy and effectiveness of the entire noise distribution feature processing flow.

[0063] For example, the variance and / or skewness statistics of local pixels in each sub-image are a quantitative description of the distribution characteristics of local pixel values. Variance measures the degree of dispersion between the pixel values ​​in a local area and the average pixel value in that area. The larger the variance, the more dispersed the distribution of local pixel values, and the more obvious the noise interference may be. Skewness reflects the asymmetry of the local pixel value distribution. A positive skewness indicates that the distribution is biased towards the side with larger values, and a negative skewness indicates that the distribution is biased towards the side with smaller values. Skewness can be used to understand the offset of pixel value distribution and help to judge the influence of noise on the distribution shape of pixel values.

[0064] For example, to extract noise pattern information, each sub-image can first be divided into multiple non-overlapping local windows. A pre-defined noise pattern template library, containing typical patterns of common pixel noise such as salt-and-pepper noise and Gaussian noise, is used. A sliding window is used to calculate the matching degree between each local region in the sub-image and various noise patterns in the template library. For instance, if a local window contains a large number of isolated abnormal pixels, and the grayscale values ​​of these pixels differ significantly from the surrounding pixels, then the matching degree with the salt-and-pepper noise pattern is high. The noise pattern with the highest matching degree and its corresponding confidence level are taken as the noise pattern information of that local region. Then, the information from all local regions is integrated to obtain the noise pattern information of the entire sub-image.

[0065] For texture distortion information extraction, Gabor filters or LBP (Local Binary Pattern) operators can be used to extract the original texture features of the sub-image, including texture direction, texture thickness, and local pixel grayscale variation patterns. The extracted original texture features are then compared with noise-free reference texture features. These noise-free reference features can be obtained by statistically modeling the texture features of similar noise-free images. By calculating the Euclidean distance or cosine similarity between texture feature vectors, the blurring, breakage, or abnormal changes in the texture of the sub-image caused by noise interference are quantified. These quantization results constitute the texture distortion information of the sub-image.

[0066] For pixel value statistics, it is optional to traverse each local region of the subimage (such as an 8×8 pixel window) and calculate the variance and skewness of all pixel values ​​within that region. Variance is calculated by taking the average of the average pixel value within the region and the square of the difference between each pixel value and the average, reflecting the dispersion of pixel values ​​around the average. Skewness is calculated as the ratio of the third central moment to the cube of the standard deviation, measuring the degree of asymmetry in the pixel value distribution. The variance and skewness values ​​of each local region are then summarized to form the pixel value statistics of the subimage, fully presenting the statistical characteristics of the local pixel value distribution.

[0067] According to some embodiments provided in this application, based on the target image and the device data for acquiring the target image, modulation vectors corresponding to multiple noise types are determined, including:

[0068] For physical device noise among multiple noise types, dark current data is determined based on the device type and device temperature included in the device data. Dark current data represents the electronic noise generated by physical devices due to the influence of device temperature.

[0069] Based on the exposure time and dark current data included in the device data, feature embedding is performed to obtain the modulation vector corresponding to the physical device noise.

[0070] In some embodiments provided in this application, by combining device type and device temperature to determine dark current data, the electronic noise characteristics of physical devices caused by temperature changes can be captured. The device type determines the inherent law of dark current change with temperature, making the determination of dark current data more targeted and accurate. By embedding exposure time with dark current data as features, the influence of exposure time on the accumulation of noise in physical devices is fully considered, and the resulting modulation vector can comprehensively reflect the impact of physical device noise on image quality.

[0071] Exemplarily, the above device data has good adaptability to highly sensitive image acquisition devices. Taking astronomical CCD images as an example, due to their special shooting environment, the dark current noise has an obvious impact on the edge area of the target image. The above dark current is the electronic noise generated by the thermal effect of the sensor under lightless conditions, and its intensity is positively correlated with temperature, pixel position, and exposure time. For each sub-image, a dark current distribution map D(x, y) that can reflect the dark current differences at different pixel positions is constructed. Since the edge pixels of the sensor are close to the circuit, their dark current intensity is higher than that of the central pixels, and the corresponding dark current is stronger. After the temperature parameter T is obtained through sensor metadata or environmental information, the influence of temperature on the dark current is quantified. The higher the temperature, the more electrons are thermally excited, and the greater the dark current intensity. In terms of feature extraction adjustment, by taking the dark current map D(x, y) as a new input channel and splicing it with the original RGB image to form input features, and then inputting them into the convolutional layer of the noise estimation sub-network, the spatial correlation pattern between the dark current and the actual noise is automatically learned, and the distribution law of the dark current noise at different positions and temperatures is captured.

[0072] Exemplarily, the noise decomposition with exposure adaptability is a precise processing method that dynamically adjusts according to the differences in noise characteristics under different exposure conditions. When the exposure is insufficient (such as low ISO, short exposure time), the noise is mainly photon shot noise and follows the Poisson distribution. When the exposure is excessive, the readout noise and quantization noise become dominant and show a Gaussian distribution. In terms of the noise type switching mechanism, when the input image is obtained, exposure parameters such as ISO and exposure time t are synchronously acquired, and a threshold t_threshold is set based on the exposure time t to dynamically select the corresponding noise model. When t < t_threshold, that is, in the case of insufficient exposure, the noise N(x, y) follows the Poisson distribution, where the parameters of the Poisson distribution are related to the image pixel values and quantum efficiency, reflecting the random fluctuation characteristics of the photon number; when t ≥ t_threshold, that is, when the exposure is excessive, the noise N(x, y) follows the Gaussian distribution, reflecting that the noise is more generated by the electronic component reading and signal quantization processes at this time. It is also possible to add an exposure conditioning branch to the residual block in the feature extraction network, so that the output feature is equal to the input feature plus the convolution transformation related to the exposure time, enabling the network to adaptively adjust the feature extraction method according to different exposure times t and better adapt to the noise characteristics under different exposure conditions.

[0073] According to some embodiments provided in this application, the feature extraction intensity includes the feature extraction amplitude and the feature extraction bias. In step S102: based on the noise distribution characteristics corresponding to each sub-image, the feature extraction intensity of the initial image features corresponding to each sub-image is adjusted to obtain the noise fusion features corresponding to each sub-image, which may include the following steps:

[0074] For any first sub-image among multiple sub-images, the initial image features of the first sub-image are processed using multiple feature channels to obtain the initial mean and initial variance of the initial image features in each feature channel;

[0075] Based on the initial mean and initial variance of each feature channel, the normalized result of the first sub-image is obtained;

[0076] The variance of the noise distribution features corresponding to the first sub-graph is used as the feature extraction amplitude, which is used to control the sensitivity of the feature response based on the noise distribution features.

[0077] The average value of the noise distribution features corresponding to the first sub-image is used as the feature extraction bias. The feature extraction bias is used to compensate the normalization results based on the noise distribution features.

[0078] Based on the feature extraction amplitude, feature extraction bias, and normalization results, the noise fusion features of the first sub-image are obtained.

[0079] In the embodiments provided in this application, the feature extraction intensity includes feature extraction amplitude and bias. For any first sub-image among multiple sub-images, the initial image features are processed using multiple feature channels to obtain the initial mean and initial variance of the initial image features in each feature channel. Based on these initial mean and initial variance, the normalization result of the first sub-image is obtained. The variance of the noise distribution features corresponding to the first sub-image is used as the feature extraction amplitude, which is used to control the sensitivity of the feature response based on the noise distribution features. The average value of the noise distribution features corresponding to the first sub-image is used as the feature extraction bias, which is used to compensate for the normalization result based on the noise distribution features. Finally, the noise fusion features of the first sub-image are obtained by combining the feature extraction amplitude, feature extraction bias, and normalization result.

[0080] By calculating the initial mean and variance of each feature channel and then normalizing them, the dimensional differences between different feature channels can be eliminated, making the initial image features more comparable. Using the variance and mean of the noise distribution features as the feature extraction amplitude and feature extraction bias, respectively, allows the feature extraction process to be controlled by noise characteristics. The feature extraction amplitude can adjust the sensitivity of the feature response according to the noise strength; the weaker the noise, the higher the sensitivity to capture more effective information, and the stronger the noise, the lower the sensitivity to avoid over-extracting noise. The feature extraction bias can specifically compensate for the normalization results, correcting the feature shift caused by noise. The resulting noise fusion features can integrate effective image features with noise distribution information, improving the reliability and discriminative power of the features and providing a high-quality feature foundation for subsequent image processing.

[0081] For example, the feature extraction magnitude and feature extraction bias described above are used to achieve instance normalization, and targeted optimizations are performed based on these biases and noise distribution features. The core of instance normalization is to calculate the mean and variance for each feature channel of a single sub-image, and then normalize the features using scaling and offset factors. The aim is to stabilize the training process and improve feature representation capabilities.

[0082] In this embodiment, for the first sub-graph, the initial mean and initial variance are calculated through multiple feature channels, corresponding to the solution process of the mean μ and variance σ in instance normalization. Using the variance of the noise distribution features as the feature extraction amplitude binds the scaling factor to the noise characteristics. When the variance of the noise distribution features is large (strong noise), the amplitude value decreases, which reduces the sensitivity to the feature response, avoids extracting too many noise features, and prevents excessive amplification of noise. When the noise is weak, the amplitude value increases. Using the average value of the noise distribution features as the feature extraction bias links the offset compensation to the noise distribution. For example, when the overall noise shows a positive offset, the bias value can be adjusted accordingly to offset the feature mean offset caused by noise, making the normalized features closer to the true distribution.

[0083] By binding the variance of noise distribution features to the feature extraction amplitude, the extraction intensity is lowered when noise is high. The lower the sensitivity of the feature extraction process to noise, the less noise features are introduced. The feature extraction bias can compensate for the normalization result according to the overall distribution trend of noise, avoiding the loss of effective signals due to excessive noise suppression. This allows the noise fusion features to remove redundant noise while improving adaptability to complex noise scenes, preserving the key information of the image to the greatest extent.

[0084] For example, embodiments of this application employ a target model for image processing to achieve an end-to-end deep learning network architecture. The input layer receives the original extremely low-light image as input to the network. The Noise-Aware Feature Extraction (NAFEM) module is located in the early stages of the network, and its main responsibility is to extract robust and denoised features from the low-light image containing severe noise.

[0085] The unprocessed target image, as the raw image, is input into the initial convolutional layer for preliminary feature extraction. The preliminary feature maps (i.e., each sub-image) then enter the attention mechanism unit, which can be such as Squeeze-and-Excitation blocks or the SimAM attention mechanism. Squeeze-and-Excitation (SE) blocks compress the spatial information of each channel into a single channel descriptor through Global Average Pooling (GAP), and learn the non-linear dependencies between channels through two fully connected layers, generating a dynamic scaling factor for each channel. This scaling factor is used for channel-wise multiplication to dynamically weight and enhance meaningful image feature channels while suppressing noisy channels. The SimAM attention mechanism effectively denoises while preserving texture details by directly evaluating the importance of each neuron, calculating its linear separability in the local context, and generating spatial and channel attention weights. Through image feature extraction of the target image using convolution and attention units, the initial image features corresponding to each sub-image can be obtained.

[0086] The input image, rendered in extremely low light, serves as the target image. Lightweight convolutional layers extract features of noise type, intensity, and spatial distribution from the image. These lightweight convolutional layers can, for example, use 3x3 or 5x5 kernels, combined with group convolutions or depthwise separable convolutions to reduce computation. The convolutional layers are trained to identify high-frequency typical noise patterns (i.e., noise pattern information), texture distortion (texture distortion information), and pixel value statistics (i.e., pixel value statistics) anomalies in the image. Specific filters are learned to capture the periodicity or randomness of noise, and local pixel statistics are combined to statistically model noise characteristics, with particular attention to the characteristics of dark and high-noise regions. The filters used to capture noise can be, for example, Gabor filters, and the local pixel statistics can include statistical analysis of local variance and skewness of pixels. A lightweight sub-network (i.e., a noise-aware network) for noise feature extraction explicitly estimates pixel-level noise levels in parallel from the original input image (i.e., the target image), generating noise distribution features, which can be represented as a noise map. This subnetwork consists of a series of downsampling convolutional layers, residual blocks, and upsampling layers, and is trained to regress the noise intensity value or noise distribution parameters of each pixel. For example, for Gaussian noise, it predicts its mean and variance.

[0087] The aforementioned attention mechanism unit and noise perception network are two parallel sub-networks, each undergoing convolution and processing before being merged. The noise map, serving as auxiliary information, is fed into the main branch for processing, specifically used to adjust the initial image feature flow corresponding to each sub-map after processing by the attention mechanism unit. Based on the noise distribution characteristics, the intensity and direction of feature extraction are dynamically adjusted by the feature modulation layer in the noise-perception feature extraction module. For example, the sensitivity of the feature response is reduced in areas with severe noise, or feature extraction is enhanced in areas with rich detail, thereby enabling more effective perception and processing of noise and maximizing the preservation of texture details. The aforementioned feature modulation layer can employ methods such as Adaptive Instance Normalization (AdaIN) to fuse noise distribution characteristics and initial image features.

[0088] According to the embodiments provided in this application, in step S103: the noise fusion features corresponding to each sub-image are decoupled into illumination features and reflection features. The illumination features are used to represent the brightness distribution in the corresponding sub-image, and the reflection features are used to represent the texture features in the corresponding sub-image.

[0089] For example, the target model may include a Dynamic Illumination-Reflection Decoupling Module (DIRDM). Based on the noise fusion features output by NAFEM, this module is responsible for decoupling image features into illumination-related features and reflection-related features, and performing independent and collaborative enhancements. This module achieves dynamic decoupling and fusion through the following steps: (1) Separation of illumination and reflection features: The illumination estimation branch receives the noise fusion features output by NAFEM and predicts pixel-level or region-level illumination maps through a series of convolutional layers, downsampling operations, and upsampling operations. The downsampling operation is preferably average pooling, and the upsampling operation is preferably transposed convolution. This branch is designed to learn a smooth, low-frequency illumination component, assuming that illumination changes slowly in space. The prediction process is achieved by a neural network learning a nonlinear mapping from input features to illumination maps.

[0090] The reflection estimation branch also receives the noise fusion features from the NAFEM output and, through another series of convolutional layers and residual blocks, predicts the intrinsic content of the image, i.e., texture, edge, structure, and color information independent of illumination. This branch is trained to extract high-frequency details, assuming that the reflection components contain the fine structure of the image. The prediction process is also a neural network learning a non-linear mapping from input features to the reflection map.

[0091] According to the embodiments provided in this application, in step S104: the illumination features and reflection features corresponding to each sub-image are fused to obtain the illumination fusion features corresponding to each sub-image, including:

[0092] The fusion weights of each subgraph are determined based on the noise distribution characteristics of each subgraph and the predetermined steepness used to control the weight changes.

[0093] By employing fusion weights, the illumination features and reflection features corresponding to each sub-image are fused to obtain the illumination fusion features corresponding to each sub-image.

[0094] In the embodiments provided in this application, the fusion ratio of illumination and reflection features is dynamically adjusted through noise distribution characteristics, and feature selection is guided by the physical properties of noise to achieve an optimal balance between noise suppression and detail preservation. A predefined kurtosis parameter is used to control the sensitivity of weights to changes in noise intensity. By combining noise distribution characteristics with the kurtosis parameter, fusion weights are dynamically generated. Image features in reflection features that easily carry noise but contain high-frequency components have their base weights reduced in high-noise areas due to the larger amount of noise included, thereby reducing the proportion of reflection features. Conversely, in low-noise areas, the base weights are increased to reserve space for reflection features. Illumination consistency correction ensures that special noise-sensitive areas such as extremely dark areas can preferentially retain illumination features, which conforms to the physical coupling characteristics of noise and brightness. The predetermined kurtosis parameter controls the smoothness of weight transitions, avoiding abrupt changes in the fusion boundary. By weighted integration of illumination and reflection features through fusion weights, the effective features of low-noise sub-images occupy a higher proportion in the fusion result, while suppressing the interfering features of high-noise sub-images. The resulting illumination fusion features retain key visual information while reducing distortion caused by noise.

[0095] For example, it is necessary to analyze the noise distribution characteristics of each sub-image. These characteristics include not only overall statistical information about noise intensity, such as average noise value and noise variance, but also the spatial distribution pattern of the noise. Some sub-images may exhibit noise concentration in edge regions, while others may show noise randomly distributed in areas with complex textures. By extracting the noise intensity value of each pixel in the sub-image and constructing a pixel-level noise distribution matrix, the spatial variation trend of noise within the sub-image can be clearly presented. Based on the analysis results of the noise distribution characteristics, a predetermined kurtosis parameter is set. The value of the kurtosis parameter must be determined based on the overall fluctuation range of the noise distribution. As mentioned earlier, the core function of the kurtosis parameter is to control the sensitivity of the weights to changes in noise intensity; its setting must be closely matched with the fluctuation characteristics of the noise distribution. When the noise distribution characteristics of a subimage show that the noise intensity changes drastically in a local area, the steepness parameter should be set to be greater than a predetermined threshold to avoid feature fusion confusion caused by weight ambiguity in the abrupt change area. This ensures that the fusion weights can quickly switch from emphasizing reflection features to emphasizing illumination features. If the noise distribution characteristics show that the noise intensity changes gradually within the subimage, the steepness parameter should be set to be less than or equal to the predetermined threshold to make the weights change more gently with the noise intensity, ensuring the continuity of the fusion weights in the transition area and reducing boundary artifacts caused by abrupt weight changes.

[0096] For example, the kurtosis parameter is adjusted in conjunction with the noise type. For low-light images dominated by photon shot noise, the noise distribution exhibits Poisson characteristics. Since noise intensity is strongly correlated with the signal itself, the kurtosis parameter needs to be appropriately reduced to avoid excessive weight jumps due to signal fluctuations. However, for device imaging containing fixed-pattern noise, the noise distribution is closer to Gaussian characteristics, and the kurtosis parameter can be increased to ensure a rapid switch to illumination characteristics dominance in areas of abrupt noise changes, thereby enhancing noise suppression.

[0097] For example, the aforementioned dynamic illumination-reflection decoupling module includes a dynamic fusion unit containing a small neural network. Its input consists of illumination and reflection features output from illumination and reflection estimation branches. This neural network includes multiple convolutional layers and activation functions. This small neural network adaptively calculates a set of pixel-level or channel-level fusion weights by analyzing the contextual information and local characteristics of the input illumination and reflection features. The fusion weights are used to perform a weighted summation or more complex nonlinear combination of the illumination and reflection features, thereby generating initially enhanced features. This fusion method avoids the limitations of the traditional Retinex model, which simply multiplies or adds the illumination and reflection components. It can dynamically adjust the contributions of both components based on the local content of the image, thus achieving a more refined balance between brightness and detail, preventing detail loss and color distortion. For example, it can emphasize the illumination component in dark areas to enhance brightness, and emphasize the reflection component in bright areas to preserve detail.

[0098] For example, the aforementioned nonlinear combination method can be a gating mechanism or a feature selection mechanism. The gating mechanism dynamically adjusts the fusion ratio of illumination features and reflection features by learning a set of gating signals. The convolutional layer of the neural network first encodes the illumination features (L) and reflection features (R) to obtain intermediate features with dimension matching. Illumination gating (g_L) and reflection gating (g_R) are generated through two parallel convolutional branches. In dark scenes, since illumination features better reflect the brightness enhancement requirements, the network automatically increases g_L and decreases g_R, prioritizing the preservation of brightness information in the illumination features. In bright areas or areas with dense texture, g_R is significantly increased, allowing the detail information of the reflection features to dominate, avoiding overexposure or blurring of details caused by excessive illumination enhancement, and adaptively switching the feature contribution based on the brightness and texture characteristics of local content.

[0099] The feature selection mechanism can extract the contextual association between illumination and reflection features through convolutional layers. This contextual information can include the average brightness and texture gradient of the local region, generating a pixel-level mask. For each pixel location, the mask assigns weights to each channel of the illumination and reflection features; higher weights indicate that the feature of that channel is more critical. For example, in smooth areas of dark regions, the weights of low-frequency channels representing brightness trends in illumination features are increased, while the weights of high-frequency channels in reflection features that are susceptible to noise interference are decreased. At texture edges in bright regions, the weights of channels representing edge details in reflection features are increased, while the weights of channels in illumination features that cause edge blurring are suppressed.

[0100] According to the embodiments provided in this application, the target model includes a multi-scale enhancement network layer, which includes an encoder and a decoder. In step S105: the illumination fusion features corresponding to each sub-image are subjected to feature processing at different scales to obtain the processed target image, which may include the following steps:

[0101] For any second subgraph among multiple subgraphs, an encoder is used to perform downsampling processing on multiple predetermined scales from large to small to determine the downsampling features of the second subgraph at each scale.

[0102] Input the smallest downsampled feature among multiple scales into the decoder;

[0103] A decoder is used to perform upsampling processing on multiple scales from small to large to obtain intermediate features at each scale;

[0104] Based on the residual features and intermediate features at each scale, the upsampled features of the second sub-graph at each scale are obtained. The residual features are the features obtained by connecting the encoder and decoder at each scale through the corresponding residual blocks.

[0105] The downsampled and upsampled features of the second subgraph at each scale are fused to obtain the scale fusion features of the second subgraph.

[0106] The scale fusion features corresponding to multiple sub-images are stitched together to obtain the processed target image.

[0107] In the embodiments provided in this application, an encoder is used to perform downsampling processing based on multiple predetermined scales from large to small to determine the downsampled features of the sub-image at each scale; this allows for the gradual extraction of global and detailed features of the sub-image at different scales. Key information at each scale is preserved during the downsampling process.

[0108] The smallest downsampled feature among multiple scales is input into the decoder. The decoder then performs upsampling at each scale from smallest to largest, obtaining intermediate features at each scale. The decoder's upsampling, combined with residual features, restores feature resolution while compensating for information loss caused by upsampling, ensuring the integrity of features at each scale. The encoder and decoder obtain residual features at each scale by concatenating corresponding residual blocks. Based on the residual features and intermediate features at each scale, the upsampled features of the second sub-image at each scale are obtained. The fusion of downsampled and upsampled features achieves complementarity between features at different scales, preserving global structural information at large scales while incorporating local details at small scales. The downsampled and upsampled features of this sub-image at each scale are fused to obtain its scale-fused features. By stitching together the scale-fused features of each sub-image, the target image achieves improved global consistency and local detail richness, effectively enhancing the overall image quality and information representation capability.

[0109] For example, the target model also includes an Adaptive Multi-Scale Enhancement Module (AMSEM), which is an encoder-decoder structure used to perform multi-scale processing and refinement enhancement of the illumination fusion features output by the DIRDM. The encoder progressively extracts multi-scale contextual features through a series of convolutional layers and downsampling operations to capture global information of the image. The downsampling operations are preferably max pooling or stride convolution. The decoder progressively restores the spatial resolution through upsampling operations and performs refinement enhancement based on the features passed from the encoder. The upsampling operations are preferably transposed convolution or bilinear interpolation followed by convolution.

[0110] The adaptive fusion unit, located on the skip connections between the encoder and decoder, differs from the simple splicing in related techniques. This unit comprises a lightweight subnetwork whose workflow is as follows: First, it receives low-level (high-resolution) detail features from the encoder and high-level (semantically rich) features from the decoder. Then, this subnetwork dynamically generates a set of fusion weights by analyzing the statistical information and spatial relationships of the input features. Using these dynamically generated weights, the detail features from the encoder and the high-level features from the decoder are fused through weighted summation or gating mechanisms. To address the noise introduced by skip connections in existing U-Net-like networks, this unit integrates instance normalization (IN) during the fusion process. By processing the features of each sample independently, the mean and variance of each feature map are normalized, effectively eliminating noise and artifacts that may be carried by skip connections and enabling the model to better adapt to specific lighting conditions and noise levels of different images.

[0111] To ensure the processing capability of the target model provided in this embodiment, a multi-task, no-reference / self-supervised composite loss function is employed to implicitly optimize and enhance image quality during training, while also considering its friendliness to downstream vision tasks. This loss function design enables this embodiment not only to generate visually better images but also to ensure that these images are effective for subsequent machine vision tasks.

[0112] According to the embodiments provided in this application, the following steps may also be included: determining the perceptual loss representing semantic similarity based on the image semantic deviation between the target image and the processed target image;

[0113] Based on the noise distribution characteristics of each subgraph, the perceptual loss is weighted and adjusted to obtain the adjusted perceptual loss.

[0114] Based on at least one of the exposure control loss function, spatial consistency loss function, and color constancy loss function, as well as perceptual loss, a total loss function is obtained. The exposure control loss function is used to represent the difference between the average pixel intensity in the corresponding sub-image and the preset target exposure value. The spatial consistency loss function is used to control the difference in gradient change of the target image before and after processing. The color constancy loss function is used to control the color difference of the target image before and after processing.

[0115] The target model is iteratively processed using the total loss function.

[0116] In the embodiments provided in this application, a perceptual loss representing semantic similarity is determined based on the semantic deviation between the target image and the processed target image. This perceptual loss captures the semantic similarity of the images, ensuring that the processed image is semantically similar to the original image at a higher level. Figure 1 By weighting the perceptual loss using noise distribution characteristics, the model can prioritize semantic fidelity in noisy sub-image regions, improving processing targeting. Exposure control loss, spatial consistency loss, and color constrain the model output from multiple dimensions, including exposure, spatial structure, and color, avoiding issues such as exposure anomalies, structural distortion, or color shifts. The synergistic effect of multiple loss functions allows the total loss function to comprehensively evaluate model performance. Iterative optimization drives the target model to generate superior processing results in terms of semantics, exposure, structure, and color, significantly improving the model's robustness and processing accuracy.

[0117] For example, the exposure control loss function described above may include calculating the L1 distance between the average pixel intensity of a local region and a preset target exposure value, which can promote uniform and moderate overall image brightness. The local region can preferably be obtained through a sliding window or block-based method, and the target exposure value is determined based on historical exposure values, preferably 0.6.

[0118] For example, the spatial consistency loss function measures the gradient difference between adjacent pixels (or patches) before and after enhancement. It encourages consistency in local structure by calculating the L1 norm of the pixel value differences between the enhanced and original images in the horizontal and vertical directions. Unnatural local variations (i.e., excessive gradient differences) in the enhanced image are penalized by directly increasing the training loss value, thus preserving the image's natural texture and edge information.

[0119] For example, the color constancy loss function, based on the gray-world assumption or white balance principle, constrains the mean values ​​of the RGB channels of the enhanced image to tend to be consistent, or to maintain consistency with the relative color relationships of the input image. This helps eliminate color distortion caused by low light, making the enhanced image colors more natural and realistic. Each color channel in the RGB channels needs to remain within a predetermined difference range from the original image, and the relative channel relationships between the RGB channels of the processed target image also need to remain within a predetermined difference range from the relative channel relationships between the RGB channels of the original image.

[0120] For example, the noise-perceptual loss function (i.e., the perceptual loss described above) uses a pre-trained feature extractor to extract deep features from the enhanced image and the original low-light image and calculates the distance between them. Perceptual loss measures the semantic similarity of images, rather than just pixel-level differences, thus more accurately reflecting the understanding of image quality by both human and machine vision. Furthermore, embodiments of this application introduce a noise-perceptual weighting mechanism, enabling the network to prioritize feature robustness rather than simple visual smoothing when dealing with heavily noisy regions. Based on the noise map output by the noise estimation sub-network (integrated in NAFEM), the feature loss for different regions is dynamically weighted. For example, for noisy regions, a relatively small weight is assigned to the loss function to avoid overfitting the model to the noise, instead encouraging the network to generate more stable and discriminative feature representations in those regions, thereby improving effectiveness for subsequent machine vision tasks.

[0121] Based on the above embodiments and example embodiments, this application also provides an optional implementation method, such as... Figure 2 As shown, an end-to-end deep learning network architecture is proposed, which consists of three main modules: a noise-perception feature extraction module, a dynamic illumination-reflection decoupling module, and an adaptive multi-scale enhancement module.

[0122] The noise-aware feature extraction module takes the original low-quality image as input, randomly samples the input image to enhance data diversity, and simulates different noise / scene distributions. f(x) represents the image matrix of the target image, which may contain noise, underexposure, etc., and f'(x) represents the noise features included therein. By superimposing the image features with the noise distribution features, the features of the input dynamic illumination-reflection decoupled DIRDM network are obtained. The noise-aware feature extraction module introduces a dedicated module in this embodiment, explicitly modeling and suppressing noise during the feature extraction stage, rather than simply smoothing it. This module effectively denoises while preserving texture details by combining depthwise separable convolution and attention mechanisms. This addresses the shortcomings of existing methods in noise processing.

[0123] Dynamic Illumination-Reflection Decoupling and Enhancement (DIRDE) can adaptively separate the illumination and reflection components of an image and enhance them independently and collaboratively, thereby avoiding the loss of detail and color distortion problems of the traditional Retinex model. It obtains the illumination fusion feature after fusing the illumination and reflection features.

[0124] An adaptive multi-scale fusion network constructs a lightweight encoder-decoder structure, similar to a U-Net network. The input is illumination fusion features. By introducing adaptive fusion units, skip connections are optimized, addressing the noise introduction issue that may arise from skip connections in existing U-Net-like networks. This ensures effective fusion of features at different scales, avoids the introduction of noise and redundant features, and preserves rich contextual information and spatial details.

[0125] This application's embodiments design a task-driven, no-reference loss function that not only focuses on image brightness, contrast, and color consistency, but also introduces perceptual losses for downstream visual tasks, such as edge sharpness and texture fidelity, to bridge the gap between image quality and task performance. By combining these various losses and obtaining a weighted summation of the total loss function, the weights of each loss can be adjusted during training according to actual needs to balance different optimization objectives. This allows this application's embodiments to generate enhanced images that are more compatible with machine vision algorithms.

[0126] Figure 3 The diagram illustrates an image processing apparatus according to an embodiment of this application. Corresponding to the application scenarios and methods provided in the embodiments of this application, this application also provides an image processing apparatus, including:

[0127] The noise extraction module 301 is used to process multiple sub-images included in the target image respectively to obtain the noise distribution features corresponding to the multiple sub-images respectively;

[0128] The noise perception module 302 is used to adjust the feature extraction intensity of the initial image features corresponding to each sub-image based on the noise distribution characteristics corresponding to each sub-image, so as to obtain the noise fusion features corresponding to each sub-image.

[0129] The decoupling module 303 is used to decouple the noise fusion features corresponding to each sub-image into illumination features and reflection features. The illumination features are used to represent the brightness distribution in the corresponding sub-image, and the reflection features are used to represent the texture features in the corresponding sub-image.

[0130] The illumination fusion module 304 is used to fuse the illumination features and reflection features corresponding to each sub-image to obtain the illumination fusion features corresponding to each sub-image.

[0131] The multi-scale processing module 305 is used to perform feature processing at different scales on the illumination fusion features corresponding to each sub-image to obtain the processed target image.

[0132] The functions of each module in each device in the embodiments of this application can be found in the corresponding description in the above method, and they have corresponding beneficial effects, which will not be repeated here.

[0133] Figure 4This is a block diagram of an electronic device used to implement embodiments of this application. For example... Figure 4 As shown, the electronic device includes a memory 401 and a processor 402. The memory 401 stores a computer program that can run on the processor 402. When the processor 402 executes the computer program, it implements the method described in the above embodiments. The number of memories 401 and processors 402 can be one or more. In a specific implementation, the electronic device may also include a communication interface 403 for communicating with external devices and performing data exchange and transmission.

[0134] In practical implementation, if the memory 401, processor 402, and communication interface 403 are implemented independently, they can be interconnected via a bus to communicate with each other. This bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0135] Optionally, in a specific implementation, if the memory 401, processor 402 and communication interface 403 are integrated on a single chip, the memory 401, processor 402 and communication interface 403 can communicate with each other through an internal interface.

[0136] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method provided in this application.

[0137] This application provides a computer program product, including a computer program that, when executed by a processor, implements the method provided in this application.

[0138] This application also provides a chip including a processor for calling and executing instructions stored in a memory, causing a communication device with the chip installed to perform the method provided in this application.

[0139] This application also provides a chip, including: an input interface, an output interface, a processor, and a memory. The input interface, output interface, processor, and memory are connected through an internal connection path. The processor is used to execute code in the memory. When the code is executed, the processor is used to execute the method provided in the application embodiment.

[0140] It should be understood that the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. General-purpose processors can be microprocessors or any conventional processor. It is worth noting that the processor can be a processor supporting Advanced Reduced Instruction Set Machines (ARM) architecture.

[0141] Further, optionally, the aforementioned memory may include read-only memory and random access memory. The memory may be volatile memory or non-volatile memory, or may include both. Non-volatile memory may include read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory may include random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available. Examples include Static Random Access Memory (SRAM), Dynamic Random Access Memory (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Sync Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0142] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions according to this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another.

[0143] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0144] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0145] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process. Furthermore, the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functionality involved.

[0146] The logic and / or steps described in the flowchart or otherwise herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus or device (such as a computer-based system, a processor-included system or other system that can fetch and execute instructions from, an instruction execution system, apparatus or device).

[0147] It should be understood that various parts of this application can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. All or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware, the program being stored in a computer-readable storage medium, which, when executed, includes one or a combination of the steps of the method embodiments.

[0148] Furthermore, the functional units in the various embodiments of this application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. This storage medium can be a read-only memory, a disk, or an optical disk, etc.

[0149] The above are merely exemplary embodiments of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various variations or substitutions within the technical scope described in this application, and these should all be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An image processing method, characterized in that, Applied to the target model, including: The target image is processed separately for each of its multiple sub-images to obtain the noise distribution features corresponding to each of the multiple sub-images; Based on the noise distribution characteristics corresponding to each sub-image, the feature extraction intensity of the initial image features corresponding to each sub-image is adjusted to obtain the noise fusion features corresponding to each sub-image. The noise fusion features corresponding to each sub-image are decoupled into illumination features and reflection features. The illumination features are used to represent the brightness distribution in the corresponding sub-image, and the reflection features are used to represent the texture features in the corresponding sub-image. The illumination features and reflection features corresponding to each sub-image are fused to obtain the illumination fusion features corresponding to each sub-image. The illumination fusion features corresponding to each sub-image are subjected to feature processing at different scales to obtain the processed target image; The feature extraction intensity includes feature extraction amplitude and feature extraction bias. The step of adjusting the feature extraction intensity of the initial image features corresponding to each sub-image based on the noise distribution features corresponding to each sub-image to obtain the noise fusion features corresponding to each sub-image includes: For any first sub-image among the multiple sub-images, the initial image features of the first sub-image are processed based on multiple feature channels to obtain the initial mean and initial variance of the initial image features in each feature channel; Based on the initial mean and initial variance of each feature channel, the normalized result of the first sub-image is obtained. The variance of the noise distribution features corresponding to the first sub-graph is used as the feature extraction amplitude, which is used to control the sensitivity of the feature response based on the noise distribution features. The average value of the noise distribution features corresponding to the first sub-image is used as the feature extraction bias, and the feature extraction bias is used to compensate the normalization result based on the noise distribution features. Based on the feature extraction amplitude, the feature extraction bias, and the normalization result, the noise fusion features of the first sub-image are obtained.

2. The method according to claim 1, characterized in that, The target model includes a noise-aware network layer, which includes classification heads corresponding to multiple predetermined noise types. The process of processing multiple sub-images within the target image to obtain noise distribution features corresponding to each sub-image includes: Based on the sub-images and the device data for acquiring the target image, the modulation vectors corresponding to the multiple noise types are determined respectively; Based on the modulation vectors corresponding to the multiple noise types, the feature extraction intensity of the corresponding classification head is controlled to obtain the noise component features and confidence levels corresponding to the multiple noise types. Based on the confidence level, the noise component features corresponding to the multiple noise types are fused to obtain the noise distribution features corresponding to each sub-graph.

3. The method according to claim 2, characterized in that, The step of determining the modulation vectors corresponding to the multiple noise types based on the target image and the device data used to acquire the target image includes: For each of the multiple noise types, feature extraction is performed on each sub-image to obtain noise pattern information, texture distortion information, and pixel value statistics. The pixel value statistics are used to describe the statistical data of variance and / or skewness of local pixels in each sub-image. Based on the noise pattern information, the texture distortion information, and the pixel value statistics, feature fusion is performed to obtain the modulation vector of the pixel type.

4. The method according to claim 2, characterized in that, The step of determining the modulation vectors corresponding to the multiple noise types based on the target image and the device data used to acquire the target image includes: For physical device noise among the multiple noise types, dark current data is determined based on the device type and device temperature included in the device data. The dark current data represents the electronic noise generated by the physical device due to the influence of the device temperature. Based on the exposure time and dark current data included in the device data, feature embedding is performed to obtain the modulation vector corresponding to the noise of the physical device.

5. The method according to claim 1, characterized in that, The process of fusing the illumination features and reflection features corresponding to each sub-image to obtain the illumination fusion features corresponding to each sub-image includes: Based on the noise distribution characteristics of each subgraph and a predetermined steepness for controlling weight changes, the fusion weights of each subgraph are determined. Using the fusion weight, the illumination features and reflection features corresponding to each sub-image are fused to obtain the illumination fusion features corresponding to each sub-image.

6. The method according to claim 1, characterized in that, The target model includes a multi-scale enhancement network layer, which comprises an encoder and a decoder. The step of performing feature processing at different scales on the illumination fusion features corresponding to each sub-image to obtain the processed target image includes: For any second subgraph among the multiple subgraphs, the encoder is used to perform downsampling processing based on multiple predetermined scales from large to small to determine the downsampling features of the second subgraph at each scale; The smallest downsampled feature among the multiple scales is input into the decoder; Using the decoder, upsampling is performed on the multiple scales from small to large to obtain intermediate features at each scale; Based on the residual features and intermediate features at each scale, the upsampled features of the second sub-graph at each scale are obtained. The residual features are the features obtained by connecting the encoder and the decoder at each scale through corresponding residual blocks. The downsampled features and upsampled features of the second sub-image at each scale are fused to obtain the scale fusion features of the second sub-image; The scale fusion features corresponding to the multiple sub-images are stitched together to obtain the processed target image.

7. The method according to any one of claims 1 to 6, characterized in that, The method further includes: Based on the image semantic deviation between the target image and the processed target image, a perceptual loss representing semantic similarity is determined. Based on the noise distribution characteristics of each subgraph, the perceptual loss is weighted and adjusted to obtain the adjusted perceptual loss; Based on at least one of the exposure control loss function, spatial consistency loss function, and color constancy loss function, and the perceptual loss, a total loss function is obtained. The exposure control loss function is used to represent the difference between the average pixel intensity in the corresponding sub-image and the preset target exposure value. The spatial consistency loss function is used to control the gradient change difference of the target image before and after processing. The color constancy loss function is used to control the color difference of the target image before and after processing. The target model is iteratively processed using the total loss function.

8. An electronic device, characterized in that, It includes a memory, a processor, and a computer program stored in the memory, wherein the processor, when executing the computer program, implements the method of any one of claims 1 to 7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, implements the method of any one of claims 1 to 7.

Images