Adaptive learning method and system for low-light face super-resolution

Abstract

The application belongs to the technical field of image processing, and discloses a self-adaptive learning method and system for low-light face super-resolution, extracts illumination features of an image, obtains illumination prior based on the illumination features, operates on latent features of a stable diffusion model by using the illumination prior, obtains first adaptive features, performs Fourier transform on a low-light low-resolution image, generates an illumination enhanced image and extracts features, uses the illumination enhanced image features and the first adaptive features to obtain second adaptive features, corrects the illumination features by using the second adaptive features, obtains corrected illumination features, splices the corrected illumination features and the illumination enhanced image features, operates on the spliced features, and generates a high-definition image with normal illumination. The application integrates the advantages of a diffusion model, designs a unified architecture to realize super-resolution reconstruction and illumination enhancement of low-light low-spatial-resolution face images, and simultaneously solves the dual challenges of illumination enhancement and resolution improvement.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of image processing, in particular to a self-adaptive learning method and system for low-light face super-resolution. BACKGROUND

[0002] The statements in this section merely provide background information related to the present application and do not necessarily constitute the prior art.

[0003] Face super-resolution (FSR) as a classic low-level computer vision task aims to improve the perceptual clarity and fidelity of low spatial resolution (LR) face images, and reconstruct high-resolution (HR) images with photo-realistic quality. In low-light or night-time environments, face images are not only limited by inherent spatial resolution loss, but also further degraded by reduced visibility, contrast attenuation, and color distortion. This double degradation seriously damages the structural integrity and texture authenticity of the image, and significantly restricts the application of practical vision systems such as surveillance security and biometric recognition. Such a complex degradation phenomenon poses a severe challenge to the reconstruction algorithm, requiring both brightness enhancement and spatial resolution enhancement while ensuring high-fidelity reconstruction.

[0004] Current mainstream methods for low-light-low spatial resolution compound degradation problems show a polarized research situation: on the one hand, low-light image enhancement (LLIE) aims to recover normal light images from low-light inputs by modeling the mapping relationship of color, brightness, and contrast. Although the LLIE framework has achieved remarkable results in brightness recovery, it often ignores the inherent spatial degradation characteristics of LR images, resulting in poor detail fidelity of the enhanced results. On the other hand, FSR technology has undergone significant development, from solving ideal down-sampling problems to expanding to modeling complex unknown degradation such as noise, blur, and compression artifacts. However, existing FSR methods perform well in good lighting conditions, but are difficult to effectively cope with the unique challenges of low-light scenes. It can be seen that the method system that treats LLIE and FSR as independent tasks has obvious limitations and cannot solve the double degradation problem at the same time.

[0005] To address the above dilemma, an intuitive solution is to serially process the two tasks in a cascaded manner (i.e., LLIE→FSR or FSR→LLIE). However, the essential complexity difference between single degradation and compound degradation severely hinders the effectiveness of such methods. The first-stage processing (either LLIE or FSR) is difficult to effectively decouple the intertwined degradation patterns, imposing excessive requirements on the result quality. This processing paradigm inevitably leads to the risk of error propagation, such as artifact amplification or erroneous structure priors, which misleads the execution of subsequent tasks and ultimately degrades the reconstruction quality. The above defects highlight the urgent need to establish a unified framework to jointly model low-resolution and low-light scenes. Although there have been a few exploratory works recently attempting to study the FSR problem under low-light conditions, there are still obvious deficiencies in color consistency and texture fidelity.

[0006] Existing methods mainly focus on degraded scenes under normal lighting conditions and lack effective modeling of compound degradation problems such as brightness attenuation and noise interference specific to low-light environments. Some researchers have proposed the IC-FSRDENet framework, which decouples the task into structure recovery and detail enhancement sub-tasks, but the step-by-step processing framework is prone to suboptimal solutions. SUMMARY

[0007] To solve the above problems, the present application proposes an adaptive learning method and system for low-light face super-resolution, which integrates the advantages of diffusion models and designs a unified architecture to achieve super-resolution reconstruction and illumination enhancement of low-light low spatial resolution face images, simultaneously solving the dual challenges of illumination enhancement and resolution improvement.

[0008] To achieve the above purpose, the present application adopts the following technical solutions:

[0009] In a first aspect, the present application provides an adaptive learning method for low-light face super-resolution, comprising the following steps:

[0010] Extract the illumination features and image features of the low-light low-resolution image, use the image features as guide information, generate the illumination coefficient based on the illumination features, and obtain the illumination prior based on the illumination coefficient;

[0011] Input the latent features in the stable diffusion model into the UNet encoder for downsampling to obtain intermediate layer features, and perform pixel-by-pixel curve adjustment on the intermediate layer features using the illumination prior to obtain first adaptive features;

[0012] Perform Fourier transform on the low-light low-resolution image to generate an illumination-enhanced image, extract the features of the illumination-enhanced image, and modulate the first adaptive features using the illumination-enhanced image features to obtain second adaptive features;

[0013] The second adaptive feature is used to correct the illumination feature, to obtain a corrected illumination feature, the corrected illumination feature and the illumination enhanced image feature are spliced, the spliced features are input into a control network, the features generated by the control network are input into an Unet decoder in stable diffusion for upsampling, and the latent representation after upsampling is decoded to generate a high-definition face image with normal illumination and realistic texture.

[0014] As an alternative embodiment, the image features of the low-illumination low-resolution image are extracted, specifically:

[0015] The bicubic interpolation method is used to interpolate and enlarge the low-illumination low-resolution image, and the features of the interpolated and enlarged image are extracted as the image features of the low-illumination low-resolution image.

[0016] As an alternative embodiment, the illumination coefficient is generated based on the illumination feature, specifically:

[0017] Based on the illumination feature, a bilateral mesh algorithm is used to construct an illumination coefficient mesh, the image features of the low-illumination low-resolution image are used as guide information, the illumination coefficient mesh is projected to a three-dimensional mesh space, then the three-dimensional mesh is smoothed using Gaussian blur, and finally the illumination coefficient is obtained through three-dimensional slicing.

[0018] As an alternative embodiment, the low-illumination low-resolution image is subjected to Fourier transform to generate an illumination enhanced image, specifically:

[0019] The low-illumination low-resolution image is subjected to Fourier transform to generate an amplitude component and a phase component, a scaling factor is introduced to amplify the amplitude, and the amplified amplitude is combined with the phase to generate an illumination enhanced image through inverse fast Fourier transform.

[0020] As an alternative embodiment, the second adaptive feature is used to correct the illumination feature, specifically:

[0021] The illumination feature and the second adaptive feature are processed through three linear layers respectively to generate corresponding query vectors, key vectors and value vectors, cross-attention weights are calculated, an attention map is generated, and the attention map is input into a feedforward network to generate the corrected illumination feature.

[0022] As an alternative embodiment, a loss function is defined to optimize the trainable parameters in the adaptive learning method, and the loss function uses a latent diffusion model denoising target and a latent space reconstruction target to jointly optimize.

[0023] In a second aspect, the present application provides an adaptive learning system for low-illumination face super-resolution, comprising:

[0024] The lightness prior calculation module is configured to extract lightness features and image features of the low-light low-resolution image, take the image features as guide information, generate lightness coefficients based on the lightness features, and obtain lightness priors based on the lightness coefficients.

[0025] The first adaptive feature calculation module is configured to input the latent features in the stable diffusion model into a UNet encoder for down-sampling to obtain intermediate layer features, perform pixel-by-pixel curve adjustment on the intermediate layer features using the lightness priors, and obtain first adaptive features.

[0026] The second adaptive feature calculation module is configured to perform Fourier transform on the low-light low-resolution image to generate a lightness-enhanced image, extract features of the lightness-enhanced image, modulate the first adaptive features using the lightness-enhanced image features, and obtain second adaptive features.

[0027] The image output module is configured to correct the lightness features using the second adaptive features to obtain corrected lightness features, perform a splicing operation on the corrected lightness features and the lightness-enhanced image features, input the spliced features into a control network, input features generated by the control network into a Unet decoder in the stable diffusion for up-sampling, perform decoding operation on the up-sampled latent representation, and generate a high-definition human face image with normal lightness and realistic texture.

[0028] In a third aspect, the present application provides an electronic device, comprising a memory and a processor, and computer instructions stored in the memory and running on the processor, when the computer instructions are run by the processor, the method of the first aspect is completed.

[0029] In a fourth aspect, the present application provides a computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the method of the first aspect is completed.

[0030] In a fifth aspect, the present application provides a computer program product comprising a computer program, when the computer program is executed by the processor, the method of the first aspect is completed.

[0031] Compared with the prior art, the present application has the following advantages:

[0032] The disclosure proposes an adaptive learning method and system for low-light face super-resolution, which initiates a DiffLLFace collaborative optimization framework, realizes robust face super-resolution reconstruction in low-light environment by fusing the generation ability of diffusion model and light perception trajectory modeling, proposes an alternating light-diffusion adaptive mechanism, realizes bidirectional collaborative enhancement of feature expression along the diffusion trajectory through dynamic coordination of the interaction process of latent representation and light information, and effectively strengthens the controllability of the generation process. A non-parametric Fourier enhancement module is designed, which fuses frequency domain structure representation and alternating adaptive mechanism to build dual constraints of texture detail enhancement and color consistency maintenance, and significantly improves visual fidelity. Through multi-scene verification, DiffLLFace exhibits excellent generalization ability in complex natural scenes, and maintains excellent reconstruction performance under dynamic low-light conditions.

[0033] The advantages of the additional aspects of the present application will be partially given in the following description, partially become obvious from the following description, or be known by the practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0034] The accompanying drawings, which form a part of the present application, are intended to provide further understanding of the present application, and the illustrative embodiments of the present application and their description serve the purpose of explaining the present application, and do not constitute improper limitation of the present application.

[0035] Figure 1 The overall architecture diagram of the DiffLLFace network provided for embodiment 1 of the present application is shown in the figure.

[0036] Figure 2 The comparison result graph of image processing using the method of the present application and using other advanced methods on the CelebAMask-HQ dataset is shown in the figure. DETAILED DESCRIPTION

[0037] The present application will be further described below in combination with the drawings and embodiments.

[0038] It should be pointed out that the following detailed description is exemplary and is intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0039] It is to be understood that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting, as the scope of the exemplary embodiments of this disclosure is limited only by the appended claims. Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Also, unless otherwise indicated herein, the materials described herein can be used in a variety of applications.

[0040] The embodiments in the present disclosure and the features in the embodiments can be combined with each other without conflict.

[0041] Term explanation:

[0042] Low-light face super-resolution is a task of restoring high-resolution clear details for blurred, low-resolution face images collected in an insufficiently light environment.

[0043] Alternating light-diffusion adaptation is a core optimization mechanism, which realizes image enhancement through the alternation of light correction and diffusion models.

[0044] Adaptive learning method refers to a model that can dynamically adjust according to the characteristics of input data or task requirements to improve performance and generalization ability.

[0045] Embodiment 1

[0046] As shown in Figure 1 The present embodiment provides an adaptive learning method for low-light face super-resolution, comprising the following steps:

[0047] Extract the light features and image features of the low-light low-resolution image, use the image features as guide information, generate light coefficients based on the light features, and obtain light priors based on the light coefficients;

[0048] Input the latent features in the stable diffusion model into the UNet encoder for down-sampling to obtain intermediate layer features, and use the light priors to adjust the intermediate layer features pixel by pixel to obtain first adaptive features;

[0049] Perform Fourier transform on the low-light low-resolution image to generate a light-enhanced image, extract the features of the light-enhanced image, and use the light-enhanced image features to modulate the first adaptive features to obtain second adaptive features;

[0050] The second adaptive feature is used to correct the illumination feature, a corrected illumination feature is obtained, the corrected illumination feature and the illumination enhanced image feature are spliced, the spliced features are input into the control network, the features generated by the control network are input into the Unet decoder in the stable diffusion to perform upsampling, and the latent representation after upsampling is decoded to generate a high-definition face image with normal illumination and realistic texture.

[0051] The application proposes a framework DiffLLFace based on a diffusion model (DM), which can realize super-resolution reconstruction and illumination enhancement of a low-light low spatial resolution (LLR) face image in a unified framework.

[0052] The application proposes a DiffLLFace based on a diffusion model (DM), which can realize super-resolution reconstruction and illumination enhancement of a low-light low spatial resolution (LLR) face image in a unified framework.

[0053] The specific implementation technical solutions of the application are as follows:

[0054] DiffLLFace is summarized as follows:

[0055] For a low-light low-resolution input image, the application proposes to realize the joint optimization of low-light enhancement and super-resolution in a unified framework, and finally generate a high-definition face image with normal illumination and realistic texture. Figure 1As shown, the DiffLLFace framework controls the sampling trajectory of the stable diffusion model by learning the lighting condition prior based on the control network architecture. Compared with existing methods that simply rely on one-way condition control of pre-trained models, this work has two major innovations: First, through an alternating lighting-diffusion adaptive mechanism, the latent diffusion representation is coordinated with the lighting prior extracted from the input image in both directions, thereby achieving precise regulation of the denoising process. Second, by combining Fourier-enhanced features with corrected lighting priors, the representation ability of ControlNet is enhanced to improve reconstruction fidelity.

[0056] During the training phase, only the trainable parameters of the alternating adaptive module and the ControlNet branch are updated to maintain the basic knowledge of the pre-trained model. Our DiffLLFace uses a latent diffusion model denoising target and a latent space reconstruction target to optimize jointly, which can be mathematically expressed as follows:

[0057]

[0058] where, denotes the total loss function, which consists of and two parts, is the difference between the predicted noise and the true noise in the denoising process, where is the noise prediction model learned by the denoising U-Net, is the original latent representation, is the variance accumulated at time step t. The reconstruction loss is the difference between the denoised latent representation and the original latent representation .

[0059] Alternating lighting-diffusion adaptive adjustment:

[0060] The lighting information extracted from low-light low-resolution images is crucial for modeling the degradation pattern, and this information can directly guide the controllable generation process as a condition. However, due to insufficient structural information, the color consistency problem has not been effectively solved. To this end, we propose an alternating lighting-diffusion adaptive mechanism that synergistically utilizes the advantages of lighting conditions and latent features to achieve more precise control capabilities. This mechanism consists of three key steps: lighting estimation, lighting perception adaptation, and lighting correction.

[0061] 1) Lighting Estimation

[0062] First, we use a shallow network to extract lighting features Based on the bilateral grid algorithm, we construct a lighting coefficient grid Meanwhile, high-resolution features are extracted from the up-scaled image by convolution operation, which are used as guidance information . The projection is performed on a three-dimensional grid space composed of pixel coordinates and image density. Then, Gaussian blur is used to smooth the three-dimensional grid, and finally, the illumination coefficient is obtained by three-dimensional slicing operation.

[0063]

[0064] where represents the slicing operation by tri-linear interpolation.

[0065] 2) Illumination-aware adaptive

[0066] At the t-th iteration, based on the estimated illumination prior , we perform pixel-wise curve adjustment on the latent feature output by the previous module: is the feature obtained by Unet encoder down-sampling of the latent feature at t time step in the stable diffusion process.

[0067]

[0068] where has been normalized to the interval [0, 1], is a trainable curve parameter, and is obtained by convolution layer and Sigmoid function operation. represents the adjusted feature. It is worth noting that for clarity, the operation related to the Fourier prior is omitted here, which will be explained in the "Combination with Fourier-based enhancement" section.

[0069] 3) Illumination correction

[0070] Due to the lack of image structure information, directly using the estimated illumination parameters as control signals is difficult to achieve ideal results. Therefore, we propose to use the deep knowledge contained in the second adaptive feature obtained after combining with Fourier-based enhancement to correct the illumination feature , thereby enhancing the control accuracy. In specific implementation, the illumination feature and the second adaptive feature are processed by three linear layers respectively to generate corresponding query vectors , key vectors and value vectors (all from the second adaptive feature ), and the cross-attention weight is calculated by the following formula:

[0071]

[0072] where, denotes the attention map, and then generates the corrected illumination feature through a feed-forward network as the final control signal. Experiments show that after the model training is completed, our DiffLLFace can dynamically couple the illumination prior and latent features in the image denoising process, and optimize the information flow of the control network through a bidirectional enhancement mechanism.

[0073] Combined with Fourier-based enhancement:

[0074] Fourier-domain techniques have achieved remarkable success in both LLIE and SR tasks. On the one hand, by operating in the Fourier domain, the model can capture the global receptive field of the image, overcoming the locality limitation of spatial domain methods; on the other hand, the luminance information is mainly contained in the amplitude component, while the structural information exists in the phase component. This indicates that we can handle the low-light and noise problems separately in the Fourier domain.

[0075] Therefore, this work also uses Fourier-based enhancement methods to deal with the composite degradation problem of brightness, structure, and details. In the Fourier branch, we perform a Fast Fourier Transform (FFT) on the input LLR image to generate the amplitude component and the phase component :

[0076]

[0077] Subsequently, a scaling factor is directly introduced to amplify the amplitude, and the result is combined with the phase to generate an illumination-enhanced image :

[0078]

[0079] where, denotes the amplification factor, which is empirically set to 2.5 in this work. This strategy can significantly improve the visibility of the input image and is expected to provide the model with structural appearance information.

[0080] After that, we integrate the Fourier enhancement information into the alternating adaptive module. First, it passes through multi-head self-attention (MHSA) and feed-forward network (FFN) to capture global dependencies and extract appearance features. Then, AdaIN is used to modulate the illumination first adaptive feature in the latent space:

[0081]

[0082] where, and denote the affine transformation parameters learned by two independent 1x1 convolutional layers for adjusting . In addition, in the ControlNet branch, we simply concatenate with the corrected illumination features along the channel dimension to update the control signal.

[0083] The present application proposes a unified framework DiffLLFace, which cooperatively utilizes the generative ability of diffusion model and illumination-aware trajectory, realizing robust face super-resolution reconstruction under low-light conditions. The core innovation lies in the alternating illumination-diffusion adaptive mechanism, which effectively strengthens the control flow by coordinating the bidirectional enhancement of latent representation and illumination information along the diffusion path. In addition, the non-parametric Fourier enhancement module introduced by us significantly improves the visibility of the input image, thereby providing reliable structural appearance information for the model. This module works in conjunction with the alternating adaptive mechanism, effectively maintaining texture details and color consistency. Experiments show that DiffLLFace not only surpasses existing optimal methods on facial images, but also exhibits excellent generalization ability on complex natural scenes.

[0084] Figure 2 The generation result visualization of each comparative method is shown. We selected five state-of-the-art face super-resolution methods (SISN, SCTANet, SFMNet, PGDiff, DR2) and five advanced low-light enhancement methods (FECNet, LLFormer, LEDNet, FourierDiff, Quadprior). The joint processing method uses the same IC-FSRDENet as our research task for low-light face image super-resolution. It can be seen that the high-resolution images reconstructed by the cascade method generally have serious color and texture distortion. Even if we use diffusion models with strong generative ability in both stages (such as DR2→FourierDiff and FourierDiff→PGDiff), the results still have obvious blurring. IC-FSRDENet can generate super-resolution images with clearer texture, but there are still problems of identity information deviation and color inconsistency. In contrast, our DiffLLFace exhibits more natural and accurate detail restoration ability, and its superior visual quality has stronger perceptual appeal.

[0085] Figure 2 ​The first column lists the low-resolution input, columns 2-5 list the results of the "super-resolution first and then enhancement" (FSR→LLIE) method, columns 6-9 list the results of the "enhancement first and then super-resolution" (LLIE→FSR) method, columns 10-11 list the results of the joint processing method, and columns 12-13 list the real high-definition images. The method is superior to the current optimal method in terms of face texture fidelity, image detail restoration, color accuracy, and the like.

[0086] Embodiment 2

[0087] The embodiment provides an adaptive learning system for low-light face super-resolution, comprising:

[0088] The illumination prior calculation module is configured to extract illumination features and image features of the low-light low-resolution image, use the image features as guide information, generate illumination coefficients based on the illumination features, and obtain the illumination prior based on the illumination coefficients.

[0089] The first adaptive feature calculation module is configured to input the latent features in the stable diffusion model into a UNet encoder for down-sampling to obtain intermediate layer features, perform pixel-by-pixel curve adjustment on the intermediate layer features using the illumination prior, and obtain the first adaptive features.

[0090] The second adaptive feature calculation module is configured to perform Fourier transform on the low-light low-resolution image to generate an illumination-enhanced image, extract features of the illumination-enhanced image, modulate the first adaptive features using the illumination-enhanced image features, and obtain the second adaptive features.

[0091] The image output module is configured to correct the illumination features using the second adaptive features to obtain corrected illumination features, perform a splicing operation on the corrected illumination features and the illumination-enhanced image features, input the spliced features into a control network, input the features generated by the control network into a Unet decoder in the stable diffusion for up-sampling, perform decoding operation on the up-sampled latent representation, and generate a high-definition face image with normal illumination and realistic texture.

[0092] It should be noted that the above modules correspond to the steps described in Embodiment 1, and the above modules have the same examples and application scenarios as the corresponding steps, but are not limited to the content disclosed in Embodiment 1. It should be noted that the above modules, as part of the system, can be executed in a computer system such as a set of computer executable instructions.

[0093] In more embodiments, there are also provided:

[0094] An electronic device includes a memory and a processor and computer instructions stored on the memory and running on the processor, when the computer instructions are run by the processor, the method described in embodiment 1 is completed. For the sake of brevity, it will not be repeated here.

[0095] It should be understood that in the embodiments, the processor can be a central processing unit CPU, and the processor can also be other general-purpose processors, digital signal processors DSPs, application-specific integrated circuits ASICs, ready-to-program gate arrays FPGA or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0096] The memory can include read-only memory and random access memory, and provide instructions and data to the processor, and a part of the memory can also include non-volatile random access memory. For example, the memory can also store device type information.

[0097] A computer readable storage medium for storing computer instructions, when the computer instructions are executed by the processor, the method described in embodiment 1 is completed.

[0098] The method in embodiment 1 can be directly embodied as a hardware processor to complete, or be executed by a combination of hardware and software modules in the processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory, and combines the hardware to complete the steps of the above method. To avoid repetition, it will not be described in detail here.

[0099] A computer program product includes a computer program, which is executed by the processor to realize the method described in embodiment 1.

[0100] The present application also provides at least one computer program product tangibly stored on a non-transitory computer readable storage medium. The computer program product includes computer executable instructions, such as instructions included in program modules, which are executed in devices on real or virtual processors of targets to perform processes / methods as described above. Generally, program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform specific tasks or implement specific abstract data types. In various embodiments, the functions of the program modules can be combined or divided as needed among the program modules. Machine executable instructions for program modules can be executed within local or distributed devices. In distributed devices, program modules can be located in local and remote storage media.

[0101] Computer program code for carrying out operations of the present application can be written in any combination of one or more programming languages. The computer program code can execute entirely on a computer, a special purpose computer, or other programmable apparatus to produce the functions / acts specified in the flow diagrams and / or block diagrams. The program code can execute entirely on a computer, a special purpose computer, or other programmable apparatus, as a stand-alone software package, partly on the computer and partly on a remote computer, or entirely on the remote computer or server.

[0102] In the context of the present application, the computer program code or related data can be carried by any suitable carrier to enable the device, apparatus or processor to perform the various processes and operations described above. Examples of carriers include signals, computer readable media, and the like. Examples of signals can include electrical, optical, radio, sound or other forms of propagated signals, such as carrier waves, infrared signals, and the like.

[0103] Those skilled in the art can realize that the units and algorithm steps of the examples described in conjunction with the present embodiments can be realized in electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to realize the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0104] The above describes the specific embodiments of the present application in conjunction with the accompanying drawings, but is not a limitation on the scope of protection of the present application. Those skilled in the art should understand that various modifications or variations made by those skilled in the art on the basis of the technical solutions of the present application without inventive labor are still within the scope of protection of the present application.

Claims

1. An adaptive learning method for low-light face super-resolution, characterized in that, The method comprises the following steps: extracting the illumination feature and the image feature of the low-illumination low-resolution image, taking the image feature as the guide information, generating the illumination coefficient based on the illumination feature, and obtaining the illumination prior based on the illumination coefficient; inputting the latent feature in the stable diffusion model into a UNet encoder for down-sampling to obtain intermediate layer features, performing pixel-by-pixel curve adjustment on the intermediate layer features by using the illumination prior to obtain first adaptive features; performing Fourier transform on the low-illumination low-resolution image to generate an illumination-enhanced image, extracting the feature of the illumination-enhanced image, and modulating the first adaptive features by using the feature of the illumination-enhanced image to obtain second adaptive features; correcting the illumination feature by using the second adaptive features to obtain a corrected illumination feature, performing a splicing operation on the corrected illumination feature and the feature of the illumination-enhanced image, inputting the spliced feature into a control network, inputting the feature generated by the control network into a Unet decoder in the stable diffusion for up-sampling, and performing decoding operation on the up-sampled latent representation to generate a high-definition human face image with normal illumination and realistic texture; extracting the image feature of the low-illumination low-resolution image, specifically: using a bicubic interpolation method to interpolate and enlarge the low-illumination low-resolution image, and extracting the feature of the interpolated and enlarged image as the image feature of the low-illumination low-resolution image; the generating of the illumination coefficient based on the illumination feature, specifically: based on the illumination feature, using a bilateral grid algorithm to construct an illumination coefficient grid, taking the image feature of the low-illumination low-resolution image as the guide information, projecting the illumination coefficient grid to a three-dimensional grid space, then performing smoothing processing on the three-dimensional grid using Gaussian blur, and finally obtaining the illumination coefficient through three-dimensional slicing operation.

2. The adaptive learning method for low-light human face super-resolution of claim 1, wherein, performing Fourier transform on the low-illumination low-resolution image to generate an illumination-enhanced image, specifically: performing Fourier transform on the low-illumination low-resolution image to generate an amplitude component and a phase component, introducing a scaling factor to amplify the amplitude, combining the amplified amplitude with the phase, and generating an illumination-enhanced image through inverse fast Fourier transform.

3. The adaptive learning method for low-light human face super-resolution of claim 1, wherein, correcting the illumination feature by using the second adaptive features, specifically: processing the illumination feature and the second adaptive features through three linear layers respectively to generate corresponding query vectors, key vectors and value vectors, calculating cross-attention weights, generating an attention map, and inputting the attention map into a feedforward network to generate the corrected illumination feature.

4. The adaptive learning method for low-light human face super-resolution of claim 1, wherein, It also includes defining a loss function to optimize the trainable parameters in the adaptive learning method, and the loss function adopts joint optimization of the latent diffusion model denoising target and the latent space reconstruction target.

5. An adaptive learning system for low-light face super-resolution, characterized in that, It comprises: The illumination prior calculation module is configured to extract the illumination feature and the image feature of the low-illumination low-resolution image, take the image feature as the guide information, generate the illumination coefficient based on the illumination feature, and obtain the illumination prior based on the illumination coefficient; The first adaptive feature calculation module is configured to input the latent feature in the stable diffusion model into a UNet encoder for down-sampling to obtain intermediate layer features, perform pixel-by-pixel curve adjustment on the intermediate layer features by using the illumination prior to obtain first adaptive features; The second adaptive feature calculation module is configured to perform Fourier transform on the low-illumination low-resolution image to generate an illumination-enhanced image, extract features of the illumination-enhanced image, modulate the first adaptive feature by using the features of the illumination-enhanced image, and obtain the second adaptive feature; The image output module is configured to correct the illumination features by using the second adaptive feature, obtain corrected illumination features, perform a splicing operation on the corrected illumination features and the features of the illumination-enhanced image, input the spliced features into a control network, input features generated by the control network into an Unet decoder in stable diffusion for upsampling, perform a decoding operation on the upsampled latent representation, and generate a high-definition human face image with normal illumination and realistic texture. The image feature of the low-illumination low-resolution image is extracted by performing bicubic interpolation on the low-illumination low-resolution image to enlarge the image, and extracting features of the enlarged image as the image feature of the low-illumination low-resolution image. The illumination coefficient is generated based on the illumination features, specifically, a bilateral grid algorithm is used to construct an illumination coefficient grid, the image feature of the low-illumination low-resolution image is used as guide information, the illumination coefficient grid is projected to a three-dimensional grid space, then the three-dimensional grid is smoothed by using Gaussian blur, and finally the illumination coefficient is obtained by three-dimensional slicing.

6. An electronic device, comprising: The computer program product comprises a memory and a processor, and computer instructions stored in the memory and run on the processor, and when the computer instructions are run by the processor, the method of any one of claims 1-4 is completed.

7. A computer readable storage medium characterized in that, A computer program product for storing computer instructions, which are executed by a processor to complete the method of any one of claims 1-4.

8. A computer program product, characterised in that, The computer program product comprises a computer program, which is executed by a processor to complete the method of any one of claims 1-4.

Images