A weak light image enhancement method of image color migration network fusing language features

Abstract

The application discloses a weak light image enhancement method based on an image color migration network with language features, and comprises the following steps: S1, based on rectification theory, modeling low light content images and reference style images as three-dimensional probability distribution in RGB space, realizing color distribution smooth migration through ordinary differential equation and linear interpolation; S2, decomposing the low light image based on Retinex theory, respectively extracting brightness details and color information through a double-branch encoder, and generating a super-resolution image; S3, analyzing user text instructions, decomposing regional descriptors and intensity coefficients, generating an enhancement mask, and realizing local area optimization; S4, pre-training related modules in stages, and through end-to-end fine-tuning of a composite loss function, the output image color distribution, structural details and style reference are consistent. Compared with traditional methods, the method can realize personalized enhancement, improve weak light image quality and style migration effect, and has the advantages of flexible input, high-fidelity, no artifact, strong detail preservation and noise suppression capacity.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, and particularly relates to an image color transfer network low-light image enhancement method fusing language features. BACKGROUND

[0002] Low-light image enhancement (LLIE) plays a crucial role in realizing downstream computer vision tasks and has attracted extensive attention in the academic community. However, LLIE is still a highly ill-posed problem because there is a complex and ambiguous mapping relationship between low-light images and well-exposed images. At the same time, traditional color style transfer techniques commonly used for photo editing are prone to artifacts, low image fidelity, and limited interpretability.

[0003] Low-light image enhancement (LLIE) aims to improve the visibility, contrast, and perceptual quality of images captured under dim lighting conditions while preserving natural details and color fidelity. LLIE is a key preprocessing step for various downstream computer vision applications such as image recognition, object detection, traffic monitoring, semantic segmentation, etc. Therefore, LLIE has become a cross-disciplinary research hotspot and has attracted extensive attention.

[0004] Traditional distribution mapping-based LLIE methods include histogram equalization (HE) and s-curve-based methods. However, histogram equalization (HE)-based methods are only effective in images with uniform lighting, while s-curve-based methods often result in color distortion. Model-based optimization methods based on the Retinex theory decompose images into reflectance and illumination components. Although these methods use regularization terms to guide the decomposition, they are prone to artifacts, blurred details, and unsaturated colors due to reliance on pre-determined design priors. In recent years, with the advent of deep learning, end-to-end LLIE methods have achieved remarkable success in image restoration and denoising (such as FEUNet, DRANet, PhotoWCT, RetinexNet, KinD++, etc.). To improve representation capacity, a series of ar architectures have been proposed that utilize deep convolutional neural networks (cnn), warping, Mamba, generative adversarial networks (gan), and diffusion models. In addition, LLIE still has inherent ill-posedness because the mapping from low-light to well-exposed images is one-to-many. Many existing methods over-rely on pixel-level reconstruction loss and fail to capture the complex conditional distribution of well-exposed images, leading to overexposure, color abnormalities, and structural artifacts.

[0005] Both supervised and unsupervised paradigms attempt to incorporate structural priors to improve augmentation quality. In unsupervised learning, models like CUT leverage contrastive learning operations on raw image patches and achieve one-sided translation in unpaired image-to-image settings. DRBN proposes a semi-supervised approach to LLIE that benefits from both paired and unpaired data. Recently, Yang et al. proposed DKP, an unsupervised kernel estimation model for super-resolution based on Markov Chain Monte Carlo sampling. However, these methods still rely on hand-designed loss functions and lack consistency with human perception priors, resulting in limited interpretability and suboptimal visual fidelity.

[0006] Color style transfer, which matches a reference style by altering the emotional and perceptual tonalities of an image, is widely used in photo editing applications to correct lighting, unify the aesthetics of an image set, and emphasize key content. Commercial solutions often rely on image filters or lookup tables (LUTs). For example, Gatys et al. proposed a pioneering method that iteratively optimizes based on content and style reconstruction losses. Later, feedforward networks were introduced to further reduce computational costs. However, their encoder-decoder architecture often introduces artifacts and compromises overall image quality. Moreover, these methods lack human interpretability and often neglect image denoising or super-resolution tasks, which are crucial for obtaining high-fidelity results in low-light image enhancement scenarios.

[0007] Overall, existing methods have the following problems:

[0008] 1. Ill-posedness problem of traditional LLIE: Existing LLIE methods have a complex and ambiguous mapping relationship between low-light images and well-exposed images, making it a highly ill-posed problem. This causes many existing methods to rely excessively on pixel-level reconstruction loss, failing to capture the complex conditional distribution of well-exposed images, which easily leads to improper exposure, color abnormalities, and structural artifacts.

[0009] 2. Inherent defects of traditional color style transfer techniques: Conventional color style transfer techniques, although commonly used in photo editing, are prone to artifacts, low image fidelity, and limited interpretability. Their encoder-decoder architecture often introduces artifacts and compromises overall image quality. Moreover, these methods lack human interpretability and often neglect image denoising or super-resolution tasks, which are crucial for obtaining high-fidelity results in LLIE scenarios.

[0010] 3. Lack of personalization and semantic perception: Existing LLIE methods and color style transfer techniques are usually global, making it difficult to achieve fine control over local regions of an image and adjust according to users' specific intentions. This limits users' ability to customize enhancements according to their personal preferences and aesthetic goals.

[0011] 4. Color extraction process is susceptible to noise: In low light conditions, the inherent soft color features of the image and the fine details that are obscured due to insufficient lighting make the color extraction process extremely susceptible to the introduction of noise, which can adversely affect the final style transfer performance.

[0012] It should be noted that the information disclosed in the above background section is only for understanding the background of the present application, and therefore can include information that does not constitute prior art known to those of ordinary skill in the art. SUMMARY

[0013] The main purpose of the present application is to overcome the defects existing in the background art, and to provide a weak light image enhancement method of image color migration network fused with language features.

[0014] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0015] A weak light image enhancement method of image color migration network fused with language features, comprising the following steps:

[0016] S1, color distribution modeling and transmission: based on the rectification theory, the input low light content image and the reference style image are modeled as three-dimensional probability distribution in RGB space, the transmission process between the distributions is described by ordinary differential equation, the intermediate state is constructed by linear interpolation and the transmission mapping function is fitted, realizing the smooth migration of color distribution from content image to style image;

[0017] S2, super-resolution color feature extraction: based on Retinex theory, the low light image is decomposed into illumination component and reflectivity component, and double-branch encoder is used to extract brightness details and color information respectively; wherein the reflectivity branch integrates U-Net structure and bilateral filter for noise suppression and edge preservation, the illumination branch adopts residual convolution network to enhance structural details, and generates super-resolution image through cross-branch feature fusion;

[0018] S3, text-driven regional enhancement: analyze user text instructions, and decompose regional descriptors and intensity coefficients through language model; combine reflectivity component and regional descriptors to generate spatially accurate enhancement mask, control rectification transmission process based on mask and intensity coefficient, realize iterative optimization and adaptive mixing of local region;

[0019] S4, modular joint training: pre-train Retinex decomposition module, super-resolution module and rectification encoder in stages, and fine-tune end-to-end through composite loss function, so that the color distribution, structural details and style reference of the output image are consistent.

[0020] Further, step S1 specifically includes the following process:

[0021] The pixel samples of the content image and the style image are respectively regarded as corresponding random variables;

[0022] An intermediate state variable between the two is constructed by linear interpolation in the time dimension;

[0023] A differential equation describing the distribution transmission process is defined, and the transmission function is optimized by minimizing the fitting loss between the transmission function and the intermediate state variable;

[0024] A latent distribution subject to a uniform distribution on a uniform interval is introduced to bridge the content image distribution and the style image distribution, and the color distribution transmission from the content image to the style image is realized through the combination of forward sampling of the content image distribution and backward sampling of the style image distribution.

[0025] Further, the transmission mapping and its inverse mapping are realized by integration of the transmission function:

[0026] The forward transmission mapping is obtained by forward integration of the transmission function along the time domain, and is used for transmission from the content image distribution to the intermediate latent distribution;

[0027] The backward transmission mapping is obtained by reverse integration of the transmission function along the time domain, and is used for transmission from the intermediate latent distribution to the style image distribution.

[0028] Further, in step S2, the specific operation process of the double-branch encoder includes:

[0029] Reflectance branch: U-Net structure is adopted for downsampling and upsampling operations, and an edge-preserving denoising module is inserted between each downsampling layer and the corresponding upsampling layer to suppress the noise introduced in the downsampling process;

[0030] Illuminance branch: a residual network group composed of multiple layers of convolution and activation function is adopted to extract high-frequency brightness detail features from the illuminance image;

[0031] In the first layer and the last layer of the U-Net, an element-wise multiplication operation of the illuminance branch output and the reflectance branch output is introduced, so that the reflectance branch can adaptively focus on the chrominance information;

[0032] The outputs of the double-branch are fused by element-wise multiplication, and then weighted summed with the input image to generate a super-resolution image after tail processing.

[0033] Further, the edge-preserving denoising module adopts a bilateral filter to bridge each downsampling- upsampling pair of the U-Net, while suppressing the noise introduced in the downsampling process and maintaining the sharpness of the image edges.

[0034] Further, in step S2, the super-resolution image is mapped to the weight parameters of the rectified transfer function through a convolutional neural network, establishing the association between the super-resolution image features and the rectified transfer process.

[0035] Further, step S3 specifically includes the following processing process:

[0036] The language model is used to analyze the user input text instruction, and output the intensity parameter for regulating the enhancement degree and the semantic region description for positioning the target region;

[0037] The segmentation model combines the reflectance component obtained from the Retinex encoder, and generates a spatially accurate binary mask according to the semantic region description, to identify the local region that needs to be enhanced;

[0038] For the region covered by the mask, the rectified transfer process under the intensity parameter control is applied to realize the style transfer and enhancement of the region;

[0039] By iteratively optimizing the rectified transfer parameters, the enhancement effect at the mask boundary is smoothly transitioned, avoiding obvious region segmentation traces.

[0040] Further, the step S4 of the phased pre-training specifically includes the following stages in sequence:

[0041] First stage: With the decomposition loss of reflectance and illumination components as the target, the dual-branch encoder is trained to make the decomposed reflectance and illumination components accurately reconstruct the input low-light image;

[0042] Second stage: The training image is down-sampled and then up-sampled to the original size to simulate blurred input. With the multi-constrained reconstruction loss as the target, the super-resolution module is trained to make the module reconstruct a high-resolution image from the blurred input;

[0043] Third stage: The pre-trained super-resolution module is combined with the rectified encoder to integrate a composite loss function with multiple losses as the target for end-to-end fine-tuning to optimize the overall network parameters.

[0044] Further, the composite loss function specifically includes the following components:

[0045] Encoding loss: By constraining the difference between the predicted value and the true value of the transfer function, the fitting accuracy of the transfer function is ensured;

[0046] Style loss: Using the features extracted by the deep neural network, the consistency of the generated image and the reference style image in style is constrained by matching the deep features of the two images;

[0047] Content loss: By constraining the pixel-level difference between the generated image and the content image, the structural content consistency of the image is maintained;

[0048] Reconstruction loss: The constraints on image gradient details, the regularization constraints on color space and the measurement constraints on structural similarity ensure the details and quality of the generated image.

[0049] A computer program product comprises a computer program which, when executed by a processor, implements the image color transfer network weak light image enhancement method fusing language features.

[0050] The present application has the following beneficial effects:

[0051] The present application proposes an image color transfer network weak light image enhancement method fusing language features to solve the problems of complex blur mapping relationship between low light images and well-exposed images in low light image enhancement (LLIE), and the problems of artifacts, low image fidelity and limited interpretability in traditional color style transfer technology, and constructs a text-driven color style transfer network (TCST-N).

[0052] The method is based on text-driven and rectified flow theory, guides local low light enhancement and global color style transfer through natural language description, realizes color distribution smooth transmission by regarding image as RGB space probability distribution, using ordinary differential equation and linear interpolation, and can realize precise local area control by combining LISA model (an image segmentation model), effectively achieving personalized and semantic-aware image enhancement. Meanwhile, based on Retinex theory and deep learning, the method extracts color details by using U-Net, bilateral denoising module and residual convolutional neural network, combines VGG-based style loss function and modified flow coding loss, realizes more accurate and detailed color extraction and noise suppression, and improves the realism and consistency of style transfer; by constructing a multi-exposure gradient (MEG) dataset, using a modular training paradigm and a composite loss function, the superior performance and generalization ability in multiple challenging scenarios are verified.

[0053] The important feature of the present application is that low light image enhancement (LLIE) and color style transfer (CST) are deeply fused instead of being processed independently; the rectified flow theory is used to model the color distribution flow conversion from low light images to well-exposed images, to establish the continuous mapping between images; the Retinex theory is used in the encoder to decompose the low light image into illumination and reflectance components, to realize independent color information extraction and brightness detail enhancement; the whole system is based on deep convolutional neural network (CNNs), U-Net and other deep learning architectures for image feature extraction, conversion and generation.

[0054] Compared with the prior art, the present application has the following advantages: high input flexibility, supporting various inputs such as text instructions and reference style images, and users can guide local low-light enhancement through natural language description; high-fidelity and artifact-free color style transfer can be provided, a continuous mapping is established with the help of rectilinear flow theory, and a related loss function is combined to ensure that the RGB distribution of the output image is consistent with the style image; excellent detail preservation and noise suppression capability, the super-resolution module combines multiple structures to accurately extract color details, and the bilateral filter effectively reduces the noise of down-sampling and maintains the edge definition.

[0055] In addition to low-light enhancement, the method of the present application has been widely verified in many challenging scenarios such as cultural relic coloring, artistic style transfer, infrared to visible spectrum conversion and underwater image enhancement, and has shown good performance. Its application scenarios are wide, in the monitoring and security field, it can significantly enhance the visibility, contrast and perceptual quality of the monitoring image, while preserving the natural details and color fidelity; in the e-commerce field, it allows users to achieve local low-light enhancement and global lighting adjustment through natural language instructions, providing a highly customized experience; in the field of digitalization of cultural relics, it can greatly reduce the dependence on hardware equipment and manual operation for obtaining low-light images.

[0056] Other benefits of the embodiments of the present application will be further described below. BRIEF DESCRIPTION OF DRAWINGS

[0057] Figure 1 The figure is the architecture diagram of the text-driven color style transfer network (TCSTNet) of the embodiments of the present application.

[0058] Figure 2 The figure is the low-light enhancement result of three content images and nine style images of the embodiments of the present application.

[0059] Figure 3 The figure is the low-light enhancement effect diagram of a single text command at 20%, 40%, 60%, 80% intensity of the embodiments of the present application.

[0060] Figure 4 The figure is the color transfer result diagram compared with the style transfer method of the embodiments of the present application.

[0061] Figure 5 The figure is the low-light enhancement effect of the rain scene compared with the style conversion method of the embodiments of the present application.

[0062] Figure 6 The figure is the low-light enhancement effect of the night scene compared with the style conversion method of the embodiments of the present application.

[0063] Figure 7 The figure is the low-light enhancement result compared with the style transfer method in the backlit scene of the embodiments of the present application.

[0064] Figure 8 Result graph for the embodiment of the present application in a sunny scene compared with the low-light enhancement method.

[0065] Figure 9 Result graph for the embodiment of the present application in a rainy scene compared with the low-light enhancement method.

[0066] Figure 10 Result graph for the embodiment of the present application in a cloudy scene compared with the low-light enhancement method.

[0067] Figure 11 Result graph for the embodiment of the present application in a cloudy scene compared with the low-light enhancement method.

[0068] Figure 12 Result graph for the embodiment of the present application in a cloudy scene compared with the low-light enhancement method.

[0069] Figure 13 Overall flow chart of the weak light image enhancement method of the image color transfer network fusing language features of the present application. DETAILED DESCRIPTION

[0070] The embodiments of the present application are described in detail below. It should be emphasized that the following description is merely exemplary and is not intended to limit the scope of the present application and its applications.

[0071] The present application aims to solve the problems of complex and fuzzy mapping relationship in low-light image enhancement and the existence of artifacts and low fidelity in traditional color style transfer technology, and proposes a weak light image enhancement method of image color transfer network fusing language features, constructs a text-driven color style transfer network (referred to as TCST-N), which is based on rectified flow and Retinex theory, and realizes personalized enhancement combined with deep learning, has the advantages of flexible input, high fidelity without artifacts, strong ability of detail preservation and noise suppression, etc.

[0072] The embodiment of the present application provides a weak light image enhancement method of image color transfer network fusing language features (the overall flow is shown in Figure 13 ), which comprises the following steps:

[0073] Step S1, color distribution modeling and transmission: based on the rectified theory, the input low-light content image and the reference style image are modeled as three-dimensional probability distribution in RGB space, the transmission process between the distributions is described by ordinary differential equation, the intermediate state is constructed by linear interpolation and the transmission mapping function is fitted, realizing the smooth transfer of color distribution from content image to style image.

[0074] In some embodiments, step S1 specifically comprises the following process: regarding the pixel samples of the content image and the style image as corresponding random variables respectively; constructing an intermediate state variable between the two through linear interpolation in the time dimension; defining an ordinary differential equation describing the distribution transmission process, and optimizing the transmission function by minimizing the fitting loss between the transmission function and the intermediate state variable; introducing a latent distribution that obeys a uniform distribution on a uniform interval to bridge the content image distribution and the style image distribution, and realizing the color distribution transmission from the content image to the style image through the combination of forward sampling of the content image distribution and backward sampling of the style image distribution.

[0075] In some embodiments, the transmission mapping and its inverse mapping are realized by integration of the transmission function: the forward transmission mapping is obtained by forward integration of the transmission function along the time domain, for transmission from the content image distribution to the intermediate latent distribution; the backward transmission mapping is obtained by reverse integration of the transmission function along the time domain, for transmission from the intermediate latent distribution to the style image distribution.

[0076] Step S2, super-resolution color feature extraction: based on the Retinex theory, the low-light image is decomposed into an illumination component and a reflectance component, and a double-branch encoder is used to extract brightness details and color information respectively; wherein the reflectance branch integrates a U-Net structure and a bilateral filter for noise suppression and edge preservation, and the illumination branch uses a residual convolutional network to enhance structural details and generates a super-resolution image through cross-branch feature fusion.

[0077] In some embodiments, in step S2, the specific operation process of the double-branch encoder comprises: reflectance branch: U-Net structure is used for downsampling and upsampling operations, and an edge-preserving denoising module is inserted between each downsampling layer and the corresponding upsampling layer to suppress the noise introduced in the downsampling process; illumination branch: a residual network group composed of multiple convolutional layers and activation functions is used to extract high-frequency brightness detail features from the illumination image; at the same time, element-wise multiplication operation is introduced between the output of the illumination branch and the output of the reflectance branch at the first layer and the last layer of the U-Net, so that the reflectance branch can adaptively focus on the chrominance information; the outputs of the double-branch are fused through element-wise multiplication, and then weighted summation is performed with the input image to generate a super-resolution image after tail processing.

[0078] In some embodiments, the edge-preserving denoising module uses a bilateral filter to bridge each downsample-upsample pair of the U-Net, while suppressing the noise introduced in the downsampling process and preserving the sharpness of the image edges.

[0079] In some embodiments, the super-resolution image is mapped to the weight parameters of the rectified transmission function through a convolutional neural network, establishing the association between the super-resolution image features and the rectified transmission process.

[0080] Step S3, text-driven region enhancement: parsing user text instructions, decomposing region descriptors and intensity coefficients through language models; generating a spatially accurate enhancement mask combining reflectance components and region descriptors, controlling rectified transmission process based on the mask and intensity coefficients, realizing iterative optimization and adaptive mixing of local regions.

[0081] In some embodiments, step S3 specifically includes the following processing process: parsing the user input text instructions using a language model, outputting intensity parameters for regulating enhancement degree and semantic region descriptions for positioning target regions; segmenting the model to combine the reflectance components obtained from the Retinex encoder, and generating a spatially accurate binary mask according to the semantic region descriptions to identify local regions that need to be enhanced; applying a rectified transmission process under the control of the intensity parameters to the region covered by the mask to realize style transfer and enhancement of the region; and iteratively optimizing the rectified transmission parameters to smooth the transition of the enhancement effect at the mask boundary, avoiding obvious region segmentation traces.

[0082] Step S4, modular joint training: pre-training Retinex decomposition module, super-resolution module and rectified encoder in stages, and end-to-end fine-tuning through a composite loss function to make the color distribution, structural details and style of the output image consistent with the reference.

[0083] In some embodiments, the staged pre-training of step S4 specifically includes the following stages in sequence:

[0084] First stage: train the dual-branch encoder with the decomposition loss of reflectance and illumination components as the target, so that the decomposed reflectance and illumination components can accurately reconstruct the input low-light image;

[0085] Second stage: downsample the training image and then upscale it to the original size to simulate blurred input, train the super-resolution module with multi-constrained reconstruction loss as the target, so that the module can reconstruct a high-resolution image from the blurred input;

[0086] Third stage: joint the pre-trained super-resolution module and rectified encoder to perform end-to-end fine-tuning with a composite loss function that integrates multiple losses as the target to optimize the overall network parameters.

[0087] In some embodiments, the composite loss function specifically includes the following components: an encoding loss: by constraining the difference between the predicted value and the true value of the transmission function, the fitting accuracy of the transmission function is ensured; a style loss: by matching the deep features of the generated image and the reference style image, the consistency of the two in style is constrained by using the features extracted by the deep neural network; a content loss: by constraining the difference between the generated image and the content image at the pixel level, the structural content consistency of the image is maintained; and a reconstruction loss: by integrating the constraints on image gradient details, the regularization constraints on color space, and the measurement constraints on structural similarity, the details and quality of the generated image are ensured.

[0088] The method of the present application solves the ill-posed mapping problem in low-light image enhancement and the artifact defect in color style transfer by fusing rectilinear flow theory, Retinex decomposition and text-driven mechanism. Specifically, first, the RGB space probability distribution modeling is adopted to describe the continuous transmission mapping between low-light images and normally exposed images by ordinary differential equations, which overcomes the exposure abnormalities and structural artifacts caused by fuzzy mapping in traditional methods; second, a double-branch encoder design based on Retinex theory is used to extract noise-robust reflectance features using U-Net and bilateral filter, and to enhance the illumination details by combining residual networks, which significantly improves color fidelity and edge clarity while suppressing noise; third, a text instruction driven local enhancement mechanism is created, which analyzes semantic region description and intensity parameters through a language model, and combines reflectance components to generate a spatially accurate mask, realizing user customizable regional adaptive optimization and breaking through the limitations of global processing lacking personalized control; fourth, a modular training paradigm and a composite loss function are optimized together to integrate transmission encoding loss, deep feature style matching, pixel-level content constraints and multi-scale reconstruction regularization, ensuring the high consistency of the output image in color distribution, structural details and style reference. Finally, the input flexibility (supporting natural language instructions), image quality extraction (high fidelity without artifacts), controllability (semantic level regional adjustment) and diversity (multi-scene generalization) are comprehensively improved, which is suitable for surveillance security, e-commerce, digital cultural relics and other fields that require high-precision low-light enhancement.

[0089] The specific embodiments of the present application, algorithm examples and experimental verification are further described below.

[0090] The present application proposes an image color transfer network low-light image enhancement method combined with language features. In order to overcome the limitations and deficiencies of existing methods, the method of the present application constructs a text-driven color style transfer network TCSTNet, which is a text-driven color style transfer network specially designed for LLIE. Figure 1The text-driven color style transfer network TCSTNet is demonstrated. The TCSTNet realizes local low-light enhancement through natural language input, realizes global illumination adjustment through reference-based color style transfer, and embodies the vision of "natural language guided image adjustment". The method of the present application is evaluated on multiple benchmark datasets, including a newly constructed multi-exposure gradient (MEG) dataset (which is composed of aligned images of different exposures of the same scene), and compared with several state-of-the-art (SOTA) baseline methods. In addition, the ablation study further verifies the contribution of each proposed module. On the basis of the foregoing, the important innovative contributions of the present application include:

[0091] New task of text-driven low-light image enhancement: the present application introduces a personalized LLIE task that uses natural language description to guide local low-light enhancement and combines it with global color style transfer to meet different user preferences and aesthetic goals.

[0092] Enhanced color style transfer model based on rectified flow: based on the rectified flow theory, the present application establishes a continuous mapping between the content image, the latent image, the style image and the stylized image. On the basis of the encoding loss, the model of the present application combines the style and content loss to ensure that the RGB distribution of the output image is closely consistent with the RGB distribution of the style image, thereby improving the realism and consistency.

[0093] Color extraction encoder based on super-resolution: the present application designs an encoder that combines U-Net with a bilateral denoising module and a residual convolutional neural network (CNN) to preserve image details. The transformation parameters for rectified flow are finally predicted using EfficientNet. This design can effectively suppress noise while achieving more accurate and detailed color extraction.

[0094] Low-light enhancement pipeline based on rectification

[0095] Under the guidance of the concept of color style transfer, the low-light enhancement task aims to produce a stylized result, allowing users to achieve customized enhancement effects according to the preferred visual style. The rectified flow is introduced to solve the color transfer problem and good results are obtained. According to the definition of rectified flow, the image is regarded as a continuous three-dimensional probability distribution in RGB space. Then, the random variable X 0 ~ π 0 And X 1 ~ π1 refers to the pixel samples taken from the content image and the target image, so the definition of the style transfer problem can be given as

[0096]

[0097] where, T is the transport map between the two distributions, J T (x) is the map acquired at sample point x T The optimal distribution transport problem can be solved by rectification. A constant differentiable equation (ODE) can be used to describe the flow on the time zone t ∈[0,1], in the style transfer problem, the step size is a factor of numerical stability control, so the size of the training gradient is used to balance the stability and convergence speed of the training process. The step size can be adjusted inversely proportional to the size of the gradient, ensuring that a smaller step size is used in the large gradient area and a larger step size is used in the small gradient area.

[0098] where is the norm of the current gradient, is a small constant to prevent division by zero error, and α is a constant value of 10 -2 , control the adjustment range.

[0099]

[0100] It completes the transport from to Z1 as linearly as possible through the path in the direction of Z 0 Therefore, linear interpolation between X 0 and X 1 is introduced, and the transfer process is represented as

[0101]

[0102] where, Xt is the intermediate state, t is the interpolation step. Then, the transport v ( Z t , t) can be fitted by a neural network to minimize the following loss function

[0103]

[0104] The weights of the neural network v ψ ( Z t , t) are represented as e ψ。 The encoder Enc (X ψ )= e ψDefinition of image X ψ function weight e ψ This will be discussed in the next subsection. The rectified flow has the non-crossing property, which means that the ODE: Z t = v ( Z t , t)dt has a unique solution and achieves the minimum cost in transportation. From this, one can present the transportation graph T

[0105]

[0106] In addition, the objective in equation (4) is time-symmetric, so the backward sampling can be computed as , denoted as

[0107]

[0108] Inspired by the time-symmetric property of the rectified flow, a general latent distribution U is introduced to bridge the content image and the style image. U The existence of can be given as: if the random variable X ~ π, X∈R has a monotonically increasing cumulative distribution function CDF

[0109]

[0110] then the transformed variable U = F(X) obeys a uniform distribution on [0,1]. Therefore, the transportation from the color distribution X 0 and X 1 to U is denoted as T 0 and T 1 respectively. Thus, the color transfer process can be denoted as

[0111]

[0112] where T 0 is the content image distribution X 0 Forward sampling of , is the style image distribution X 1 Backward sampling of and the stylized image distribution is X 01。

[0113] Super-resolution encoder based on Retinex theory

[0114] Encoder according to the rectification pipeline Enc (X ψ ) must accurately extract the color distribution of the low-light image before rectification. Due to the soft color feature inherent to low-light conditions, and the fact that fine details are easily occluded by insufficient lighting, the color extraction process is prone to the introduction of noise, which can adversely affect the final style transfer performance.

[0115] Therefore, the Retinex theory is combined to decompose the low-light image into an illumination and a reflectance component. This separation allows the independent extraction of color information (reflectance) and brightness details (illumination), thus facilitating more effective detail enhancement in the subsequent processing stages. The theoretical basis of Retinex is the trichromatic theory and color constancy. This means that the color of an object is determined by its reflective properties of the three colors red, green, and blue, not by the absolute intensity of the light reflected. At the same time, the color of an object remains constant and is not affected by uneven lighting. The Retinex model uses this color constancy to achieve a balance between dynamic range compression, edge enhancement, and color constancy. Therefore, it can adaptively enhance various types of images, unlike traditional methods that can only enhance specific features of an image.

[0116] However, the classic Retinex operation causes color distortion in the RGB space, which uses the maximum value of the RGB channels to obtain the initial illumination L 0 Then the reflectance is calculated by element-wise division R 0 .

[0117] Therefore, a convolutional neural network module composed of a full CNN is introduced to learn the illumination and reflectance simultaneously. This module consists of three Conv+LeakyReLU layers followed by a Conv+ReLU layer. To train the Retinex network, the loss function is defined as:

[0118]

[0119] where is the output image of the network, is the pixel value of the input image in color channel c, R 0 and L 0 are the reflectance and illumination components in the Retinex network, respectively, and µ is a hyperparameter, c ∈{R,G,B} is the RGB channel, and ||·||1 and ||·||​F denotes l 1 norm and F-norm.

[0120] For Figure 1 In the illumination branch, after passing through a head convolutional network, a residual network group containing 8 Conv+ReLU is proposed to extract detailed information from the illumination image. The reflection branch is composed of a head convolutional layer and a U-Net layer, which enhances the extraction of color information of the weak light image. The input channel size of the down-sampling layer of the U-Net is 64, 128, 256 and 512, while the corresponding up-sampling layer is opposite to this channel configuration (512, 256, 128, 64). The bilateral filter bridges each down-sampling-up-sampling pair, which weakens the down-sampling noise before feature reconstruction, and its denoising performance is better than other methods such as Gaussian filter, which realizes denoising while maintaining edge sharpness. In the first layer and the last layer of the U-Net, element-wise multiplication with the illumination branch is introduced, which enables the reflection branch to adaptively focus on the chrominance information under different lighting conditions.

[0121] The two branches are merged by element-wise multiplication, then weighted with the input image, and then a tail Conv+ReLU, to generate the final super-resolution image with enhanced structural details. Then the Efficient-Net finally establishes the mapping relationship between the super-resolution image and the rectification parameter eψ.

[0122] Text-based regional image enhancement module

[0123] Recognizing the subjectivity of weak light enhancement preferences and the need for local style control, the framework of the present invention introduces a text command-based style transfer mechanism that supports specific region conversion. GPT parses the input text command and decomposes the style transfer parameters into: (i) an intensity coefficient for rectification adjustment, (ii) a target region descriptor for the LISA segmentation model (the LISA model is pre-trained based on the CLIP architecture, aligning text instructions with image region features through contrastive learning; the high-frequency edge information of the reflectance component is input into LISA as a spatial prior, constraining the mask generation to cover only the connected regions consistent in chrominance). Notably, LISA also processes the reflection mapping from the Retinex encoder to generate a spatially accurate mask.

[0124] For example, the user inputs "please enhance the left building a little bit", the "left building" segment will be processed in the LISA model and generate a mask, and the "a little bit" segment will be converted to an enhancement intensity of 10% or 20% by GPT.

[0125] If the user further instructs "make the right building a little brighter like the reference", the system can adjust the enhancement parameters through iterative rectification optimization and region-aware blending, and finally generate a personalized low-light enhancement image that meets the complex style requirements.

[0126] Training method and loss design

[0127] Considering the inherent complexity of the model, a modular training paradigm is adopted, where each component is pre-trained separately before the final end-to-end fine-tuning. Compared with joint training from scratch, this staged approach significantly speeds up the convergence. First, the Retinex module is trained to obtain the reflectance and illumination images of the origin, and its loss function is shown in equation (9).

[0128] Then, the super-resolution module based on Retinex theory is trained. The training images are compressed to 25% of the resolution (4 times compression), and then upgraded to the original size to generate artificially blurred inputs. The super-resolution capability of the network is quantitatively verified by comparing the output with the corresponding high-resolution ground truth image using a combination of indicators.

[0129] The loss function is:

[0130]

[0131] Where:

[0132] The weights are determined by grid search to find the optimal value.

[0133] is l 1 The norm loss, which enforces pixel-level content consistency between the super-resolution image and the ground truth image.

[0134] is the gradient magnitude loss, which preserves high-frequency edge details by using the Sobel operator.

[0135] is the color space loss, which uses LAB space regularization to prevent color distortion in the enhanced output.

[0136] is the structural similarity (SSIM) loss, which maintains global image statistics and contrast relationships.

[0137] λ L1 , λ grad , λ color and λ SSIM are the weights of each loss term.

[0138] The expression of the loss function is:

[0139]

[0140] where C, H, W are the channel size, height size and width size, respectively, I output c h w denote the pixel values of the output image at channel c height h width w I target c h w denote the pixel values of the target image at channel c height h width w

[0141]

[0142] ▽ x I output and ▽ y I output denote the gradients of the output image I output in the x and y directions, respectively, x I target and ▽ y I target denote the gradients of the target image I target in the x and y directions, respectively, x and ▽ y denote the gradient operators in the x and y directions, respectively, L out a out b out denote the L, a, b channel values of the output image in the Lab color space, L tgt a tgt b tgt denote the L, a, b channel values of the target image in the Lab color space, L denote the lightness of the image,​​​​​​​​​​​​R 、 G 、 B represent the value of the corresponding color channel in the RGB color space, respectively, L SSIM is a loss function based on structural similarity (SSIM),

[0143] where is the mean, σ x is the standard deviation, and σ y is the covariance, xy C 1 and C 2 is a stabilization constant, whose optimal value is determined by experiments to achieve a balance between noise suppression and edge preservation.

[0144] The pre-trained Retinex-based super-resolution network is jointly fine-tuned with EfficientNet to optimize the final encoder parameters. This stage has a self-adaptive weighted compound loss function formula as follows:

[0145]

[0146] where is the RetiConv loss, is the rectified encoding loss, is the VGG-style loss, is the l 1 norm content loss, λ RetiConv , λ enc , λ style , and λ content are the weights of each loss term. The specific expressions are as follows:

[0147]

[0148] where v ψ ( X t , t) is the ODE vector field predicted by the encoder, v true is true value.

[0149]

[0150] where φ VGG represents the features from the fourth layer of VGG19, I style is the reference style image, I output ​is the final output image. This term enforces high-level style transfer through deep feature matching.

[0151] Ground truth for each image v true A two-layer MLP with 1024 hidden neurons and tanh activation function is trained to generate the image parameter pair for each sample.

[0152] Experiments

[0153] Datasets and metrics

[0154] Training and validation were performed on three datasets: two widely used low-light datasets (LOL and LOL-V2 datasets) and a multi-exposure gradient (MEG) dataset collected by the inventors. The LOL dataset contains 500 pairs of images, while the LOL-V2 dataset contains 1500 pairs of images. These two datasets cover different scenarios, including indoor, outdoor, daytime, and nighttime environments, with each pair of images including one low-light image and a corresponding high-light image. The inventors' custom MEG dataset includes 1000 groups of images, each containing 5 images at different exposure levels. To accommodate different training and testing needs, 1000 groups of images with post-processing exposure adjustment are provided, as well as 1000 groups of images taken at different exposure levels through an in-camera rig.

[0155] It is worth noting that the post-processed images maintain perfect content alignment, while the photos in the brackets exhibit slight content changes during the capture process due to environmental fluctuations. This dataset covers multiple university campuses, different urban scenes, and various weather conditions.

[0156] The training set in this paper is constructed by randomly selecting 40% of the images from the three datasets mentioned above, resulting in a final training set containing 1200 pairs of low-light / normal-light image pairs. For the MEG dataset, images are specifically paired with the second and fourth exposure levels to create data pairs. From the remaining 40% of the images, the normal-light images are selected as style references and randomly matched for the style transfer process. The remaining 20% of the images are used as the validation set.

[0157] The evaluation metrics in the experiment are as follows: BRISQUE: measures image distortion and artifacts; NIQE: assesses the naturalness of generated images and their compliance with human visual perception; PIQE: assesses block-based artifacts and noise in images; Gram Metric: quantifies the style similarity between generated images and reference style images using the Gram matrix; Color Histogram Similarity: calculates the similarity between the color distribution histogram of the generated image and the style reference image. PSNR: quantifies the peak signal-to-noise ratio between the generated image and the reference image. SSIM: measures the structural similarity between images by comparing brightness, contrast, and structure. LPIPS: uses a deep neural network to assess perceptual similarity, consistent with human judgment.

[0158] Workflow

[0159] Contrastive Experiments. Figure 2 The enhancement effect obtained by applying a positive light image with different color tones to low light input is shown at a transformation intensity of 100%. Figure 3 The local low light enhancement effect achieved by single command control is shown. The enhancement results of multiple intensity levels (20%, 40%, 60%, 80%) are visually compared, revealing progressive lighting improvement while retaining structural details in shadow areas.

[0160] First, a series of representative images are tested. Not limited to low light enhancement scenarios, the evaluation of the present invention covers various brightness levels, color tones, saturation values, and various light characteristics including sunroofs, water reflections, urban lighting, and crystal refraction. This comprehensive test framework thoroughly verifies the style transfer performance of the algorithm of the present invention.

[0161] Figure 4 Color transfer results compared with style transfer methods are shown. It can be observed from Figure 4 that the proposed method generally demonstrates robust performance without showing obvious artifacts or excessive noise. Moreover, it maintains reasonable color distribution characteristics after style transfer. To further verify the effectiveness of the algorithm in low light enhancement, several typical scenes are selected, including sunny, overcast, rainy, night, backlight, and high saturation conditions. Comparative experiments are conducted using various style transfer algorithms for weak light enhancement.

[0162] For weak light images with sunlight scene characteristics, Modflow, CAP-VSTnet, WCT2, and TCSTNet (the present invention) all exhibit good foreground LOL-V2 performance. However, other comparative methods exhibit obvious artifacts in the darkest foreground regions. In terms of sky region processing, the CAP-VSTnet result shows abnormal square textures, and the WCT2 output contains obvious noise.

[0163] Low-light enhancement and style transfer methods were compared under cloudy scenarios. Under cloudy conditions, the overall image presents high gray values. Obvious artifacts were observed in large areas of both PhotoWCT and PCA-KD results, while the DeepPreset method could not maintain effective LOL-V2 capability under weak light conditions. Although other comparative methods showed relatively better performance, careful examination found that WCT2 and Modflow outputs maintained a high gray level. CAP-VSTnet produced superior tone reproduction, but introduced a noticeable blue tone in the shadowed forest. Although the TCSTNet method of the present invention achieved satisfactory processing results overall, slight color bleeding was observed in the red flower area.

[0164] Comparative example data under rainy conditions are shown in Figure 5 It can be observed that, due to the higher exposure rate of this input image compared to the previous two groups, no obvious artifacts appear in the results of the algorithms. However, by observing the ground reflection, it can be seen that there is color distortion in the red light reflection in the PCA-KD method.

[0165] Figure 6 The experimental results under extreme night conditions. As shown in Figure 6 DeepPreset basically cannot function effectively, while CAP-VSTnet shows local overexposure in the road surface and sky area, accompanied by a large number of square noise artifacts. At the same time, WCT2, phoptowct and PCA-KD all produce extensive artifacts in the dark area.

[0166] It is worth noting that only the present invention and Modflow achieved satisfactory results. However, since the method of the present invention has a dedicated detail color extraction module, it successfully recovers more accurate color information in key areas such as lane markings and road signs, showing outstanding performance in color restoration. Under backlit conditions, the present invention still shows strong performance, as shown in Figure 7 Specifically, this method neither produces block artifacts nor produces abnormal brightness artifacts observed in the building-sky boundary in the CAP-VSTnet and PCA-KD results.

[0167] Similarly, for high-saturation images, the present invention effectively avoids generating blocking artifacts while demonstrating superior color detail extraction capability. Detailed quantitative metrics (e.g., Histo metric, etc.) are listed in Table 1, which gives the validation results on three different data sets, respectively. In addition to comparing with style transfer methods, the present invention also conducts comparative analysis with a series of low-light enhancement techniques. To ensure a fair comparison, when running the algorithm of the present invention, the low-light image is used as the content image, and its corresponding well-illuminated version is used as the style image. Then the results are compared with those obtained by other methods.

[0168] Table 1: Comparative experiments of different style transfer methods

[0169]

[0170] Four scenarios are chosen to demonstrate the results: sunny, rainy, cloudy, and indoor conditions, as shown in Figure 8 to Figure 10 The quantitative results on the MEG and LOL data sets are shown in Table 2.

[0171] Table 2: Comparative experiments of different low-light enhancement methods

[0172]

[0173] Ablation study

[0174] The ablation study is conducted on the lol-v2 data set. To validate the functions of each sub-module in the TCSTNet framework of the present invention and study the impact of adjustable parameters on the algorithm, the following parts are studied for ablation: style loss, super-resolution color extraction module, and encoding step in rectification. The results of the proposed super-resolution module are shown in Figure 11 The visualization results of the ablation study are shown in Figure 12

[0175] The results show that each designed module has a significant contribution to the overall performance. On the encoding step, increasing the number of steps by 22 generally improves the results, and a diminishing return of more than 15-25 steps is observed. This trade-off between performance gain and computational efficiency indicates that for most cases, the optimal range of 15-25 encoding steps is sufficient, as further increases: greatly extend the running time, only produce marginal quality improvements. The quantitative results of the ablation study are shown in Table 3.

[0176] Table 3: Ablation study

[0177]

[0178] ​In addition, on the dark light segmentation task, the night street view segmentation mIOU (mean intersection over union) on the public dataset Dark Zurich is 64.8%, as shown in Table 4.

[0179] Table 4 Comparison of night street view segmentation performance on dataset Dark Zurich

[0180]

[0181] Other applications

[0182] In addition to low light enhancement, the method of the present application has been widely verified in multiple challenging scenarios, including artifact coloring, artistic style transfer, infrared to visible spectrum conversion, and underwater image enhancement, showing good performance.

[0183] The advantages of the present application compared with existing methods are shown in Table 5 below:

[0184] Table 5 Comparison of performance of the present application and existing methods

[0185]

[0186] 1. High input flexibility: The present application supports multiple inputs, such as text-driven instructions, reference style images, etc., rather than being limited to fixed structure or bubble chart input. Users can use natural language descriptions to guide local low light enhancement.

[0187] 2. Provide high-fidelity, artifact-free color style transfer: The present application uses rectified flow theory to simulate the color distribution flow conversion from low light to well-exposed images, establishing a continuous mapping between images. Combined with VGG-based style loss and modified flow encoding loss, it ensures that the RGB distribution of the output image is closely consistent with the style image, improving realism and consistency.

[0188] 3. Excellent detail preservation and noise suppression capability: The present application designs a super-resolution module combining U-Net, bilateral denoising module and residual convolutional neural network (CNN) to accurately extract color details from the original image. The bilateral filter effectively reduces down-sampling noise while maintaining edge sharpness.

[0189] In summary, the present application proposes a weak light image enhancement method of image color transfer network fused with language features to address the complex and fuzzy mapping relationship between low light images and well-exposed images in LLIE, as well as the situation that traditional color style transfer techniques are prone to artifacts, low image fidelity, and limited explainability. A text-driven color style transfer network (TCST-N) is constructed.

[0190] The method of the present application is based on text-driven and rectified flow theory, which can effectively realize personalized and semantically aware image enhancement by fully utilizing natural language descriptions to guide local low-light enhancement and global color style transfer, and establishing the characteristics of color distribution flow conversion relationship. This is due to its viewing the image as a probability distribution in RGB space, achieving smooth transmission of color distribution through ordinary differential equations and linear interpolation, and combining LISA model to realize precise local area control.

[0191] The method of the present application is based on Rnx theory and deep learning, which can effectively realize more accurate and detailed color extraction and suppress noise, while improving the realism and consistency of style transfer, by fully utilizing U-N, bilateral denoising module and residual convolutional neural network for color detail extraction, and VGG-basd style loss function and modified flow encoding loss characteristics. In addition, by constructing a multi-exposure gradient (MEG) dataset and using a modular training paradigm and composite loss function, the superior performance and generalization ability of the method in multiple challenging scenarios are verified.

[0192] The important features of the present application include: deep fusion of low-light image enhancement (LLIE) and color style transfer (CST) instead of independent processing. Using rectified flow theory, the color distribution flow conversion from low-light images to well-exposed images is modeled, and continuous mapping between images is established. In the encoder, the Retinex theory is used to decompose the low-light image into illumination and reflectance components for independent color information extraction and brightness detail enhancement. The entire system is based on deep convolutional neural networks (CNNs), U-Net and other deep learning architectures for image feature extraction, conversion and generation.

[0193] Alternative embodiments:

[0194] 3D scene reconstruction and stylization: extending rectified flow theory and semantic segmentation to 3D point cloud or mesh data, realizing lighting reconstruction and artistic style transfer of weak light 3D scenes.

[0195] Reverse style transfer and style learning: developing an inverse model that allows users to extract the underlying text description or style parameters from enhanced or stylized images for new creation or personalized recommendations.

[0196] Interactive editing of style transfer: developing a user interface that allows users to make real-time, fine-grained local adjustments to the effects of style transfer through brushes, region selection, etc. based on text instructions.

[0197] Scene-aware enhancement combined with embodied intelligence: integrating the model into robots or AR / VR devices so that it can automatically adjust visual output based on real-world environmental lighting conditions and semantic information, providing a more immersive and adaptive experience.

[0198] The present application has the diversity of input modalities: in addition to text instructions, other modalities (such as audio, hand-drawn sketches, emotional labels) can be introduced to drive style transfer.

[0199] The platform and form of model deployment can be different deployment forms such as cloud services, local software, SDK / API of embedded devices (such as smart phones, surveillance cameras).

[0200] Applications

[0201] The method of the present application has a wide range of application scenarios. The specific applications are not limited to:

[0202] Monitoring and security field: TCSTN can significantly enhance the visibility, contrast and perceptual quality of monitoring images while preserving natural details and color fidelity.

[0203] E-commerce field: TCSTN allows users to achieve local low-light enhancement and global lighting adjustment through natural language instructions, providing users with a highly customized experience.

[0204] Cultural relics digitization and other fields: greatly reduce the dependence on hardware equipment and manual operation for obtaining low-light images.

[0205] The above are only examples of application fields, and the application of the text-driven color style transfer network in the actual application can also involve more fields.

[0206] The embodiment of the present application also provides a storage medium for storing a computer program, which is executed to perform at least the method described above.

[0207] The above is a further detailed description of the present application in combination with specific preferred embodiments, and the specific implementation of the present application cannot be limited to these descriptions. For those skilled in the art to which the present application belongs, without departing from the concept of the present application, a number of equivalent alternatives or obvious variations can be made, and the performance or use is the same, which should be regarded as belonging to the protection scope of the present application.

Claims

1. A low-light image enhancement method using an image color transfer network that integrates language features, characterized in that, Includes the following steps: S1. Color distribution modeling and transmission: Based on rectification theory, the input low-light content image and reference style image are modeled as three-dimensional probability distributions in RGB space. The transmission process between the distributions is described by ordinary differential equations. The intermediate state is constructed by linear interpolation and the transmission mapping function is fitted to achieve smooth migration of color distribution from content image to style image. S2. Super-resolution color feature extraction: Based on Retinex theory, low-light images are decomposed into illuminance and reflectance components. A dual-branch encoder is used to extract luminance details and color information respectively. The reflectance branch integrates a U-Net structure and a bilateral filter for noise suppression and edge preservation, while the illuminance branch uses a residual convolutional network to enhance structural details. A super-resolution image is generated by cross-branch feature fusion. S3. Text-driven region enhancement: Parse user text commands and decompose region descriptors and intensity coefficients through a language model; combine reflectivity components and region descriptors to generate spatially accurate enhancement masks; control the rectification and transmission process based on the mask and intensity coefficients to achieve iterative optimization and adaptive mixing of local regions. S4. Modular Joint Training: The Retinex decomposition module, super-resolution module, and rectifier encoder are pre-trained in stages, and end-to-end fine-tuning is performed through a composite loss function to ensure that the color distribution, structural details, and style reference of the output image are consistent.

2. The method as described in claim 1, characterized in that, Step S1 specifically includes the following processes: The pixel samples of the content image and the style image are treated as corresponding random variables; An intermediate state variable between the two is constructed through linear interpolation in the time dimension; Define an ordinary differential equation describing the distributed transport process, and optimize the transport function by minimizing the fitting loss between the transport function and the intermediate state variables; A latent distribution that follows a uniform distribution over a uniform interval is introduced to bridge the content image distribution and the style image distribution. Color distribution transfer from the content image to the style image is achieved by combining forward sampling of the content image distribution and backward sampling of the style image distribution.

3. The method as described in claim 2, characterized in that, The transport mapping and its inverse mapping are achieved by integrating the transport function: Forward transfer mapping is obtained by integrating the transfer function in the forward direction of the time domain and is used to transfer from the content image distribution to the intermediate latent distribution; Backward transfer mapping is obtained by inversely integrating the transfer function along the time domain and is used to transfer from the intermediate latent distribution to the style image distribution.

4. The method according to any one of claims 1 to 3, characterized in that, In step S2, the specific operation process of the dual-branch encoder includes: Reflectivity branch: U-Net structure is used for downsampling and upsampling operations. An edge-preserving denoising module is inserted between each downsampling layer and the corresponding upsampling layer to suppress noise introduced during downsampling. Illumination branch: A residual network consisting of multiple convolutional layers and activation functions is used to extract high-frequency brightness detail features from the illumination image; In the first and last layers of U-Net, element-wise multiplication operations are introduced between the output of the illuminance branch and the output of the reflectance branch, so that the reflectance branch can adaptively focus on chromaticity information. The outputs of the two branches are fused by element-wise multiplication, then weighted and summed with the input image, and finally processed at the end to generate a super-resolution image.

5. The method as described in claim 4, characterized in that, The edge-preserving denoising module employs a dual-sided filter that bridges each downsampling-upsampling pair of the U-Net, suppressing noise introduced during downsampling while maintaining the sharpness of image edges.

6. The method according to any one of claims 1 to 3, characterized in that, In step S2, the super-resolution image is mapped to the weight parameters of the rectified transfer function through a convolutional neural network, thereby establishing the correlation between the super-resolution image features and the rectified transfer process.

7. The method according to any one of claims 1 to 3, characterized in that, Step S3 specifically includes the following processing procedures: The language model is used to parse the text commands input by the user and output strength parameters for adjusting the enhancement level and semantic region descriptions for locating the target region. The segmentation model combines reflectance components obtained from the Retinex encoder to generate spatially accurate binary masks based on semantic region descriptions to identify local regions that need enhancement. For the area covered by the mask, a rectification and transmission process under intensity parameter control is applied to achieve style transfer and enhancement in that area; By iteratively optimizing the rectification and transmission parameters, the enhancement effect at the mask boundary is smoothly transitioned, avoiding obvious regional segmentation marks.

8. The method according to any one of claims 1 to 3, characterized in that, Step S4, the phased pre-training, specifically includes the following stages performed sequentially: The first stage: Using the decomposition loss of reflectance and illuminance components as the target, a dual-branch encoder is trained so that the decomposed reflectance and illuminance components can accurately reconstruct the input low-light image. The second stage involves downsampling the training images and then upsampling them back to their original size to simulate a blurred input. The super-resolution module is then trained using a multi-constraint reconstruction loss as the target, enabling the super-resolution module to reconstruct a high-resolution image from the blurred input. The third stage involves combining the pre-trained super-resolution module with the rectifier encoder, and performing end-to-end fine-tuning with the goal of integrating a composite loss function that incorporates multiple losses to optimize the overall network parameters.

9. The method according to any one of claims 1 to 3, characterized in that, The composite loss function specifically includes the following components: Encoding loss: By constraining the difference between the predicted and true values ​​of the transfer function, the fitting accuracy of the transfer function is ensured; Style loss: By using features extracted by deep neural networks, the style consistency between the generated image and the reference style image is constrained by matching the deep features of the generated image and the reference style image. Content loss: By constraining the differences between the generated image and the content image at the pixel level, the structural content consistency of the image is maintained; Reconstruction loss: Integrates constraints on image gradient details, color space regularization constraints, and structural similarity metric constraints to ensure the detail and quality of the generated image.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the low-light image enhancement method of the image color transfer network that integrates language features as described in any one of claims 1 to 9.

Images