Deep learning method for uniform background light and shadow based on Transform model

Abstract

The invention discloses a deep learning method for uniform background light and shadow based on a Transform model. The method comprises the steps that an input image and a corresponding portrait main body mask are acquired, and the input image comprises an area with uneven background light and shadow; preprocessing the input image and the portrait main body mask, including image standardization and feature fusion, to obtain a fused feature map; based on a patch segmentation method, converting the fusion feature map into sequence features; encoding the sequence features to output enhanced sequence features, modeling a light and shadow distribution dependency relationship of full image pixels through a global attention mechanism, and distinguishing portrait main body features and background features based on the portrait main body mask; and based on the inverse logic of patch segmentation, recovering the enhanced sequence features into a spatial feature graph and the like. According to the method, semantic mask prior, global context modeling and adaptive residual correction are organically integrated, and a solution is provided for solving the core problem in background light and shadow homogenization.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a deep learning method for achieving uniform background lighting and shadow based on the Transformer model. Background Technology

[0002] Uneven background lighting is a typical problem in digital image acquisition and processing, primarily caused by both hardware and environmental factors. On the hardware side, uneven distribution of image sensor sensitivity, lens optical distortion, and deviations in auxiliary lighting angles can easily lead to localized overexposure, underexposure, or brightness patches in the image background. On the environmental side, backlighting, sidelighting, and shadows under natural light conditions, as well as multiple light sources indoors, further exacerbate the inconsistency in background brightness distribution. This problem directly affects the subsequent image analysis and application: in consumer scenarios, differences in background brightness weaken the visual representation of the subject, affecting the viewing experience and user experience; in professional fields such as security monitoring, medical image analysis, or industrial visual inspection, uneven background lighting can interfere with the extraction and recognition of target features, reducing algorithm accuracy and system reliability. Therefore, research on background lighting uniformity algorithms aims to suppress or eliminate non-uniform illumination in the background area through technical means, providing stable and consistent background conditions for subsequent image processing, thus simultaneously improving visual quality and ensuring analytical performance.

[0003] Existing methods for achieving uniform background lighting can be categorized into two types: traditional image processing and deep learning. Each type has its limitations. Traditional methods primarily rely on statistical models of image brightness or local filtering to achieve global or regional brightness adjustments. For example, global methods such as histogram equalization and gamma correction can improve the overall brightness distribution, but they cannot distinguish between foreground and background, easily leading to distortion of the subject area. Local adaptive methods (such as CLAHE and bilateral filtering) can alleviate local brightness unevenness, but may cause edge blurring, high computational costs, and insufficient robustness to complex and changing lighting conditions. Deep learning methods, leveraging data-driven strategies, have developed solutions such as two-stage networks based on image segmentation and end-to-end lighting correction networks. Two-stage methods depend on high-precision segmentation results and are prone to correction deviations due to segmentation errors. End-to-end methods, while directly learning correction mappings, are limited by the scale and diversity of training data, suffer from subject style distortion, high computational resource requirements, and are difficult to deploy in lightweight or real-time scenarios. Overall, existing methods still face common challenges in terms of synergy between subject protection and background correction, generalization ability in complex scenes, and balancing processing efficiency and effectiveness.

[0004] In summary, while various methods for background lighting uniformity have emerged and made some progress, the core challenges in practical applications remain unresolved. Traditional methods offer advantages in efficiency and interpretability, but their correction capabilities are limited. Deep learning methods have improved correction performance but are constrained by data availability, computational power, and generalization ability. Current technologies generally suffer from insufficient foreground and background co-processing, weak adaptability to complex lighting conditions, and a difficulty in balancing real-time processing with high-quality output. Therefore, future research should focus on designing more robust and adaptive algorithm models that balance computational efficiency and applicability to meet the growing technical demands for background lighting uniformity across different scenarios. Summary of the Invention

[0005] In view of this, the purpose of this invention is to propose a deep learning method for uniform background lighting based on the Transformer model, which can at least solve one of the shortcomings mentioned in the background art.

[0006] According to one aspect of the present invention, a deep learning method for achieving uniform background lighting and shadow based on a Transformer model is provided, the method comprising:

[0007] Obtain the input image and the corresponding human subject mask, wherein the input image contains areas with uneven background lighting;

[0008] The input image and the subject mask are preprocessed, including image standardization and feature fusion, to obtain a fused feature map;

[0009] Based on the patch segmentation method, the fused feature map is converted into sequence features;

[0010] The sequence features are encoded to output enhanced sequence features. The light and shadow distribution dependency of all pixels in the image is modeled through a global attention mechanism. The subject features of the portrait are distinguished from the background features based on the subject mask.

[0011] Based on the inverse logic of patch segmentation, the enhanced sequence features are restored into a spatial feature map;

[0012] The spatial feature map is synchronously input into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0013] The resolution of the residual image and the intensity field image is magnified to the resolution of the input image. The influence weight of the residual image in the input image is controlled by the intensity field. The fusion of the input image, the magnified residual image and the intensity field image is completed, and the result image is output.

[0014] Among the aforementioned technical solutions, a deep learning method based on the Transformer architecture is proposed to achieve intelligent and uniform processing of background lighting and shadows in portrait images. Its core innovation lies in constructing a mask-guided, globally aware parallel residual correction framework. This method explicitly distinguishes the foreground and background through a portrait subject mask and utilizes the global attention mechanism of the Transformer to model the lighting and shadow dependencies across the entire image. Crucially, a parallel dual-output head structure is designed to generate a background residual map and a spatially adaptive intensity field, respectively, achieving refined and controllable local illumination correction for the background region, rather than uniform processing of the entire image. Specifically,

[0015] First, in the feature modeling stage, the scheme fuses and serializes the input image with a portrait mask, and uses a Transformer encoder to capture long-distance pixel correlations. This enables a global understanding of the background lighting distribution patterns, overcoming the limitations of traditional local methods (such as CLAHE) in terms of limited field of view and poor adaptability to complex lighting patterns. Second, in the design of the correction mechanism, its parallel residual map and intensity field map outputs constitute the core innovation. The residual map focuses on estimating the brightness adjustment required for the background, while the intensity field map acts as a spatial weight controller, dynamically determining the intensity of this adjustment in different background regions. This design allows the algorithm to perform non-uniform, adaptive compensation for different degrees of unevenness in the background, such as applying stronger correction to dark areas while maintaining small changes to already uniform areas, thereby achieving background homogenization while minimizing distortion or interference with the subject area caused by "overprocessing". Finally, in the result generation stage, a strategy of first estimating at low resolution and then upsampling is adopted, balancing global consistency optimization and computational efficiency, providing convenience for potential practical deployment.

[0016] In summary, this solution organically integrates semantic mask priors, global context modeling, and adaptive residual correction to address the core issue of background lighting uniformity. Its innovations are mainly reflected in: 1) Synergy between subject protection and background processing: By clearly isolating processing areas through masking, the risk of erosion of the subject during the correction process is fundamentally avoided; 2) Refinement and adaptability of the correction strategy: The introduction of an intensity field-controlled residual fusion mechanism achieves an improvement from "global / local equalization" to "pixel-level adaptive modulation," more accurately handling complex unevenness patterns; 3) Enhanced global optimization capabilities: Leveraging the global attention of the Transformer, the algorithm's understanding and coordination of the overall lighting structure of the image is improved. This method can significantly improve the background lighting correction effect in complex scenes while maintaining the naturalness of the subject, achieving a better balance between effectiveness and efficiency.

[0017] In some embodiments, the fused feature map is converted into sequence features based on the patch segmentation method, specifically:

[0018] The fused feature map of H×W×X is divided into N non-overlapping patches according to a fixed size P×P, where X is the number of channels in the fused feature map; N is the number of patches, N=(H / P)×(W / P), and H and W are both integer multiples of P. If they are not satisfied, they are adjusted by edge padding; H and W are the height and width of the original image, respectively.

[0019] Each P×P×X patch is mapped to a D-dimensional feature vector through a linear transformation layer, and the output sequence features have an N×D dimension.

[0020] In the above technical solution, a method combining non-overlapping regular patch partitioning and linear projection is used to convert spatial image features into a serialized, dimension-aligned token sequence. This step not only achieves data format adaptation but also provides a regular input foundation rich in local information for subsequent global relational modeling by the Transformer encoder through structured dimensionality reduction and recombination. Specifically,

[0021] First, in the spatial structure reorganization stage, the scheme divides the fused feature map of H×W×X into N non-overlapping patches of a fixed size P×P. This operation spatially discretizes the image into a series of local visual units (patches), each patch retaining multi-channel features within a small region of the original image. The requirement that H and W are both integer multiples of P, or satisfied through edge padding, ensures the regularity of the partitioning and the integrity of the data, avoiding information loss or introducing complex boundary processing problems. This structured partitioning significantly reduces the data volume of subsequent processing (from H×W pixels to N patches) while preserving local spatial adjacency information.

[0022] Secondly, in the feature dimension mapping stage, each patch P×P×X is mapped to a D-dimensional feature vector through a learnable linear transformation layer. This operation achieves two key functions: first, channel fusion and dimensionality reduction, which compresses and integrates information from all spatial locations (P×P) and all original channels (X) within the patch into a unified D-dimensional representation; second, dimension alignment, ensuring that all patches are converted into feature vectors of the same length, thus forming a standard sequence of dimension N×D. The N×D sequence is the standard input to the Transformer architecture, where each patch vector acts as a "token," containing the comprehensive features of the corresponding image region. Compared to the earlier ViT model's method of directly flattening pixels, this two-dimensional to one-dimensional transformation better preserves local spatial structure information; simultaneously, the learnability of its linear projection allows the model to adaptively optimize the feature representation, providing a more expressive foundation for subsequent attention calculations.

[0023] In summary, the patch segmentation and linear projection serialization method described in this scheme is innovative in the following ways: 1) It achieves efficient and structured transformation from image data to sequence data, meeting the requirements of Transformer for visual tasks; 2) It strikes a balance between dimensionality reduction and information preservation, with patch segmentation reducing sequence length to control computational overhead while preserving key spatial context at the local region level; 3) It provides an optimizable feature interface, with the linear projection layer serving as a trainable module, enabling end-to-end collaborative optimization between the feature transformation process and the downstream light and shadow uniformity task. This step lays the data foundation for the entire model to subsequently utilize a global attention mechanism to deeply explore and model the complex dependencies of light and shadow distribution across the entire image.

[0024] In some embodiments, the sequence features are encoded to output enhanced sequence features, the light and shadow distribution dependencies of all pixels in the image are modeled through a global attention mechanism, and the subject features of the portrait are distinguished from the background features based on the subject mask. Specifically:

[0025] The sequence features are input into the backbone network of the Transformer model, which is composed of L stacked Transformer encoders. Each Transformer encoder layer shares the same structure, but the parameters are independent.

[0026] After performing layer normalization on the multi-head self-attention module of each Transformer encoder layer, the input N×D dimensional sequence features are split into M N×(D / ,) dimensional sub-features according to the preset number of heads. The attention weights between different patches are calculated using the following formula:

[0027]

[0028] Where Q, K, and V are the query, key, and value matrices, respectively; d k =D / M is the dimension of a single-head feature; N is the sequence length, and D is the dimension of each feature vector;

[0029] After layer normalization of the feedforward network of each Transformer encoder layer, a nonlinear transformation is performed on the feature input feedforward network after processing by the multi-head self-attention module. The network structure consists of linear layers, GELU activation function, and linear layers, with the hidden layer dimension expanded to 4D.

[0030] After processing by the feedforward network, the output is an N×D dimensional enhanced sequence feature.

[0031] The above technical solution provides a specific process for deep encoding and enhancement of sequence features using a Transformer backbone network. Its core lies in constructing a mask-aware, multi-layered stacked Transformer encoder architecture. Through iterative global attention calculation and feedforward transformation, the input patch sequence features are progressively refined into enhanced features containing the dependence of the entire image's light and shadow distribution. This process not only models the long-distance contextual relationships of the image, but more importantly, by implicitly or explicitly integrating the subject mask information into the attention mechanism, it achieves differentiated processing of foreground (subject) and background features, laying a feature foundation for subsequent targeted background correction. Specifically, this encoding enhancement process works synergistically through the following mechanisms:

[0032] Deeply stacked encoder structure: This structure uses stacked Transformer encoders with identical L-layer structures, each with independent parameters. This deep structure allows for layer-by-layer abstraction and integration of features. Lower layers capture local lighting patterns, while higher layers model more complex global lighting distributions and subject-background semantic relationships, achieving feature representation learning from local to global perspectives.

[0033] Multi-head self-attention module (global dependency modeling): In each encoder layer, the input features, after layer normalization, are split into M sub-features (heads), and attention is calculated in parallel. The formula essentially calculates the association weight (attention score) between any two patch feature vectors in the sequence. Through this mechanism, the lighting features of each background patch can interact with all patch features in the entire image (including other background areas and the subject area), enabling it to perceive the overall lighting structure and its relative relationship to the subject area. The scaling factor ensures gradient stability.

[0034] Feedforward Network (FFN, Feature Nonlinear Deepening): The attention-aggregated features are then passed through a feedforward network containing two linear layers with a GELU activation function (hidden layers extended to 4D). This step provides nonlinear transformation capabilities for the features of each patch, enabling the fusion of global information aggregated by the attention mechanism, and further extraction and encoding of higher-order features relevant to the lighting correction task.

[0035] Mask-guided feature differentiation mechanism: Based on the portrait subject mask, the network distinguishes between portrait subject features and background features. Mask guidance is introduced into the attention weight calculation (combining mask information to strengthen feature protection of the subject region, and suppressing excessive association between the background patch and the portrait subject patch when calculating the attention of the background patch). This allows the network to consciously learn and strengthen the light and shadow relationships within the background region in a global context, while weakening the background's erroneous borrowing or interference with the portrait subject features, ensuring that the enhanced background features more purely focus on their own light and shadow correction needs.

[0036] In summary, the innovation of the Transformer encoding module designed in this scheme lies in: 1) achieving long-distance global context modeling: overcoming the limitation of the receptive field of convolutional neural networks, enabling any background region to directly refer to the information of the whole image for feature correction; 2) embedding semantic guidance in global modeling: integrating the distinction between the subject and the background into the attention mechanism through mask information, realizing task-adaptive feature learning, and avoiding feature confusion that may be caused by indiscriminate global attention; 3) forming deep feature extraction: through multiple stacking of the "MSA-FFN" module, the features are deepened and enhanced layer by layer, and the final output of N×D enhanced sequence features provides high-dimensional, global feature support for the final generation of spatially adaptive residual maps and intensity fields.

[0037] In some embodiments, based on the inverse logic of patch segmentation, the enhanced sequence features are restored to a spatial feature map, specifically:

[0038] Receive N×D dimensional enhanced sequence features, where N is the sequence length and D is the dimension of each feature vector;

[0039] Based on the original image height H, width W, and patch size P, the target spatial dimensions of the spatial feature map are calculated as height H / P and width W / P; where the sequence length N must satisfy N=(H / P)×(W / P);

[0040] Based on the spatial correspondence during patch segmentation, the N×D-dimensional enhanced sequence features are rearranged and reshaped into a (H / P)×(W / P)×D spatial feature map.

[0041] The above technical solution provides specific decoding steps for converting the serialized features output by the Transformer encoder back into a two-dimensional spatial structure. Its core lies in performing a reversible reshaping operation from the patch sequence to the spatial feature map, aiming to losslessly restore the spatial topology of the features and lay the foundation for subsequent generation of spatially aligned lighting correction parameters (residual map and intensity field map). Specifically,

[0042] First, in the dimension and layout determination stage, based on the known height (H), width (W), and patch size (P) of the original input image, the size of the target spatial feature map is calculated to be (H / P) × (W / P). This ensures that the mapping from a sequence of length N to a spatial grid with (H / P) × (W / P) locations is one-to-one and completely reversible, thus avoiding information misalignment or loss. Second, in the data structure reshaping stage, the received N×D enhanced sequence features are rearranged and dimensionally transformed according to the spatial position order of each patch in the original image during the encoding stage, ultimately reshaping into a (H / P) × (W / P) tensor. This operation is the reverse process of the spatial-to-sequence transformation in the encoding stage, accurately transforming the D-dimensional feature vector of each patch, which has been fully refined and enhanced by global attention, back to its original two-dimensional spatial grid position.

[0043] In summary, the innovation of this feature map recovery step lies in: 1) ensuring the alignment of feature spatial locations: through defined computational rules and reversible mapping, it ensures that the spatial context of features after complex global transformations is preserved, which is a prerequisite for generating spatially adaptive residual maps and intensity field maps; 2) realizing the transformation of feature representation: it completes the transformation from sequence data suitable for global relation modeling to regular feature maps suitable for spatial convolution or upsampling operations; 3) maintaining the fidelity of information transmission: this process is a deterministic, parameter-free reshaping operation, avoiding the introduction of additional noise or bias, so that the rich global semantics and lighting dependencies learned by the Transformer encoder can be completely and losslessly transmitted to downstream modules. Therefore, this design ensures that the results of the initial global analysis can be effectively applied to subsequent localized, pixel-level background lighting correction.

[0044] In some embodiments, the spatial feature map is synchronously input into a first output head and a second output head in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations. Specifically:

[0045] The first and second output heads synchronously receive spatial feature maps as input;

[0046] For the first output head, spatial features of the spatial feature map are extracted, and the distribution is stabilized through a batch normalization layer. Nonlinearity is introduced using the GELU activation function. The processed spatial features are then passed through a convolutional layer to map their channel number to be consistent with the channel number of the output image, and finally, a residual map is output.

[0047] For the second output head, spatial features of the spatial feature map are extracted, and the distribution is stabilized through a batch normalization layer. Nonlinearity is introduced using the GELU activation function. The processed spatial features are passed through a convolutional layer, and their channel number is mapped to 1. After that, the Sigmoid activation function is applied to constrain the output value to the range of [0,1], and finally the intensity field map is output.

[0048] The above technical solution provides a step for generating specific lighting correction parameters from the encoded spatial feature map. Its innovation lies in constructing a parallel, dual-branch, task-decoupled parameter generation head, capable of simultaneously deriving the residual correction amount and spatially adaptive modulation intensity for the background region from a shared feature representation. This design deconstructs the background lighting homogenization task into two sub-problems: "what to correct" (residual map) and "how to correct" (intensity field map), solving them separately through parallel processing paths. Ultimately, the synergistic effect of both sub-problems achieves refined, adaptive illumination adjustment of the background region. Specifically, the technical implementation of this parallel dual-output head is embodied in the following closely coordinated mechanism:

[0049] Shared features and branch specialization: Both output heads synchronously receive the same spatial feature map as input, ensuring that the residual map and intensity field map originate from the same set of globally modeled enhanced features in both semantics and spatial location, possessing inherent consistency. Based on this, each output head undergoes feature specialization processing through independent lightweight network structures (both containing batch normalization, GELU activation, and convolutional layers), focusing on learning output mappings of different properties.

[0050] First output head: Residual map generation. The goal of this branch is to predict a low-resolution background brightness adjustment map. The technical approach is as follows: batch normalization and GELU non-linear activation are applied to the input features to stabilize and enrich the feature representation. Then, a convolutional layer maps the number of channels to match the number of channels in the output image (e.g., 3 channels in an RGB image). This convolutional layer learns to convert high-dimensional features into specific pixel-level adjustment values, and the output residual map directly represents the amount of brightness increase or decrease required to achieve uniformity in each background region at a low-resolution scale.

[0051] The second output head: intensity field map generation. This branch aims to predict a modulation map with the same resolution and spatial weights in the range [0,1]. Its initial processing flow is similar to the first head (batch normalization, GELU), but the final convolutional layer maps the number of channels to 1, generating single-channel features. The key innovation lies in the applied Sigmoid activation function, which constrains the output value to the [0,1] interval. This design gives the intensity field map a clear physical meaning: its value represents the proportion of residual correction applied at the corresponding spatial location (0 for no correction, 1 for full correction). This allows the algorithm to dynamically determine the correction strength based on the characteristics of local background regions (such as texture complexity and initial non-uniformity). For example, weaker correction can be applied to edge regions or relatively uniform regions to avoid artifacts, while stronger correction is allowed in large, uniform dark areas.

[0052] In summary, the parallel dual-output head structure designed in this scheme is key to achieving adaptive background lighting uniformity. Its innovations lie in: 1) Decoupling and specialization of the correction logic: decomposing the complex correction task into two more learnable and interpretable sub-tasks—residual estimation and intensity modulation—improving the model's learning efficiency and generalization ability; 2) Introducing an interpretable spatial adaptive mechanism: through the intensity field map, the model is endowed with the ability to flexibly adjust the correction intensity according to the local context of the image, which helps to achieve refined processing and balance the correction effect with naturalness; 3) Ensuring the inherent consistency between parameters: based on parallel generation of shared features, it ensures that the residual map and the intensity field map are aligned in spatial structure and consistent in semantic understanding. This design enables the subsequent fusion step (residual application under intensity field control) to be realized, ultimately outputting a result image that significantly improves background uniformity while maximizing visual naturalness.

[0053] According to another aspect of the present invention, a training method for a Transformer model is provided, based on the above method; wherein the Transformer model includes an input layer, a preprocessing layer, a patch embedding layer, a Transformer backbone, a feature reshaping layer, a dual-output head layer, and a prediction layer; the method includes:

[0054] The training image, its corresponding ground truth image, and the subject mask are input into the layer; the ground truth image is a uniformly lit and shadowed image corresponding to the training image.

[0055] The training image and the subject mask are input into the preprocessing layer via the input layer for preprocessing, including image standardization and feature fusion, to obtain a fused feature map.

[0056] The fused feature map is input into the patch embedding layer, and based on the patch segmentation method, the fused feature map is converted into sequence features;

[0057] The sequence features are input into the Transformer backbone, and the sequence features are encoded based on the Transformer backbone to output enhanced sequence features. The light and shadow distribution dependency of all pixels in the image is modeled through a global attention mechanism, and the subject features of the portrait are distinguished from the background features based on the subject mask.

[0058] The enhanced sequence features are input into the feature reshaping layer, and the enhanced sequence features are restored into a spatial feature map based on the inverse logic of patch segmentation.

[0059] The spatial feature map is input into the dual-output head layer. The spatial feature map is synchronously input into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0060] The low-resolution residual map and low-resolution intensity field map are input into the prediction layer. The training image and ground truth map are downsampled to the same dimension as the residual map and intensity field map. The influence weight of the residual map in the input image is controlled by the intensity field. The downsampled training image is fused with the low-resolution residual map and low-resolution intensity field map to output a low-resolution prediction map. The total loss is calculated based on the low-resolution prediction map and the downsampled ground truth map. The gradient of the total loss is backpropagated to the Transformer backbone and the dual-output head layer to train the Transformer model.

[0061] In the above technical solution, a complete end-to-end training process for the aforementioned uniform background lighting model is constructed. Its innovation lies in designing a collaborative training method that performs efficient supervision and gradient backpropagation in a low-resolution space. The core of this method is to achieve direct and efficient supervision of the complex parameter generation process (especially the residual map and intensity field map) by directly calculating the prediction loss at a low-resolution scale aligned with the model's internal representation. Specifically, the prediction layer is the key to this training method. Instead of directly upsampling the low-resolution residual map and intensity field map and comparing it with the full-resolution ground truth map, it first downsamples the training image and its corresponding ground truth map (the ideal image with uniform lighting) to the same size as the low-resolution parameter map generated by the dual output heads. Within this low-dimensional space, the images are fused according to the same logic of "intensity field-controlled residual weighting" to generate a low-resolution prediction map, and the loss is calculated with the downsampled ground truth map. This design has two advantages: first, it significantly reduces the huge computational and memory overhead caused by pixel-level comparison of high-resolution images; second, it enables the loss function to directly supervise the generation quality of the two core output parameters, the residual map and the intensity field map, avoiding interference or blurring of gradient signals that may be introduced during the upsampling process.

[0062] The gradient of the calculated total loss is used to update the parameters of the dual-output head layer, the Transformer backbone, and even shallower layers of the network simultaneously through backpropagation. That is, the Transformer backbone, responsible for global understanding, and the dual-output head, responsible for specific parameter generation, are jointly trained under a unified optimization objective. The intensity field generation head learns how to allocate correction weights, the residual map generation head learns how to estimate correction amounts, and the Transformer backbone learns how to provide high-quality feature representations that simultaneously support both tasks. These three layers co-evolve through gradient flow.

[0063] In summary, this training scheme is a highly efficient optimization strategy that is highly adapted to the innovative model architecture. Its innovation lies in: 1) Direct supervision of complex correction mechanisms: By constructing supervision signals at the native resolution of the parametric map, the model can accurately learn the generation of "residuals" and "modulation intensity," which is the foundation for adaptive correction; 2) Achieving a balance between computational efficiency and supervision effectiveness: Low-resolution loss calculation significantly improves training speed and reduces resource consumption. Simultaneously, since the supervision signals and the model's core output are at the same scale, the clarity and effectiveness of the optimization objective are guaranteed; 3) Ensuring collaborative learning between global and local components: End-to-end backpropagation ensures that the feature extraction backbone network and the task-specific output head can optimize towards the common goal of final image quality, promoting a deep integration of global context modeling capabilities and local parameter generation capabilities.

[0064] In some embodiments, the training images are obtained by at least one of the following methods: a. using an image editing tool to automatically generate training image-ground image pairs in batches through script execution; b. using 3D scene and 3D character models to generate training image-ground image pairs in batches covering various lighting conditions; c. using a generative adversarial network to generate training image-ground image pairs.

[0065] Calculate the standard deviation σ of the background region in the training image and the standard deviation σ of the brightness in the background region of the ground truth image. Compare the two standard deviations with preset thresholds to determine whether they are qualified. Use a feature matching algorithm to calculate the feature matching rate of the subject region of the human figure in the training image-ground truth image. Compare the feature matching rate with a preset threshold to determine whether they are qualified. Use a scene classification model to determine whether the lighting effect of the training image matches its scene type. Compare the classification confidence with a preset threshold to determine whether it is qualified. Use the qualified image pairs as training image pairs. Mark the reasons for the unqualified image pairs and regenerate them.

[0066] The above technical solution provides a process for preparing training data (image pairs) for training the aforementioned Transformer model. Its core lies in constructing a data preparation method that combines "diversified generation" and "multi-dimensional filtering." Specifically,

[0067] At the data generation level, a multi-faceted and complementary technical approach is adopted to ensure the diversity of the dataset. The solution provides three optional generation methods: a) script-based generation based on image editing tools, which simulates the addition or removal of uneven lighting and shadows on real images in a procedural manner. The advantage is that it is relatively low cost and can generate a variety of variations based on real images; b) rendering generation based on 3D scenes and models, which can obtain precisely paired and ground truth "perfect" image pairs by setting different lighting conditions and rendering in a fully controllable virtual environment, and is particularly good at constructing extreme or complex lighting scenes; c) synthetic generation based on generative adversarial networks, which uses the powerful generative capabilities of GANs to create new uneven lighting and shadow images that may exceed the existing distribution and their corrected versions, which helps to improve the generalization and creative capabilities of the model.

[0068] At the quality screening level, a progressive and quantifiable triple evaluation standard was established. First, based on the comparison of the standard deviation of background region brightness, the standard deviation is used to quantitatively verify whether the ground truth image has achieved a substantial improvement in background uniformity compared to the training image, which is a fundamental prerequisite for data validity. Second, based on the feature matching rate calculation of the subject region, the aim is to ensure that the identity and detailed features of the subject are preserved during the conversion from the training image to the ground truth image, preventing the introduction of "harmful" samples with distorted subjects and ensuring that the model achieves the goal of subject fidelity. Finally, a scene classification model is introduced to judge the reasonableness of the generated image, evaluating whether the lighting and shadow effects conform to the common sense of its scene (such as indoor, outdoor, backlight, etc.), filtering out outlier samples that, although uniform, have lighting and shadows that violate physical laws or visual habits, thus ensuring the naturalness and credibility of the correction results.

[0069] In summary, this data preparation scheme is a preliminary step in the entire model training process. Its innovations lie in: 1) Constructing a task-oriented data generation paradigm: By combining multiple technologies, it systematically solves the problem of obtaining "perfect ground truth images" on a large scale in the background lighting homogenization task; 2) Implementing goal-oriented quality control: Transforming the three goals expected by the model—"uniform background," "fidelity of the subject," and "naturalness of the result"—into calculable and comparable objective indicators (standard deviation, matching rate, classification confidence), achieving automated and standardized filtering of data quality, and improving the purity of the dataset and training efficiency; 3) Forming a closed-loop iterative optimization process: Unqualified image pairs are labeled with reasons and regenerated, allowing the data preparation process itself to continuously optimize and improve. This entire scheme ensures that every sample used to train the model is validated, thereby reducing the interference of noisy data on model learning.

[0070] According to another aspect of the present invention, a deep learning device for achieving uniform background lighting and shadow based on a Transformer model is provided. Based on the above-described method, the device comprises:

[0071] The input module is used to acquire an input image and the corresponding human subject mask, wherein the input image contains areas with uneven background lighting;

[0072] The preprocessing module is used to preprocess the input image and the human subject mask, including image standardization and feature fusion, to obtain a fused feature map;

[0073] The patch embedding module is used to convert the fused feature map into sequence features based on the patch segmentation method;

[0074] The Transformer backbone module is used to encode the sequence features and output enhanced sequence features. It models the light and shadow distribution dependency of all pixels in the image through a global attention mechanism and distinguishes the subject features of the portrait from the background features based on the subject mask.

[0075] The feature reshaping module is used to restore the enhanced sequence features into a spatial feature map based on the inverse logic of patch segmentation;

[0076] The dual-output head module is used to synchronously input the spatial feature map into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0077] The prediction module is used to upscale the resolution of the residual image and the intensity field image to the resolution of the input image, control the weight of the residual image in the input image through the intensity field, and complete the fusion of the input image, the upscaled residual image, and the intensity field image, outputting a result image; and,

[0078] The training image and ground truth image are downsampled to the same dimension as the residual image and intensity field image. The influence weight of the residual image in the input image is controlled by the intensity field. The downsampled training image is fused with the low-resolution residual image and low-resolution intensity field image to output a low-resolution prediction image. The total loss is calculated based on the low-resolution prediction image and the downsampled ground truth image. The gradient of the total loss is backpropagated to the Transformer backbone and the dual-output head layer to train the Transformer model.

[0079] In order to better utilize the above method, this application proposes a deep learning device for uniform background lighting and shadow based on the Transformer model. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0080] According to another aspect of the present invention, a deep learning device for uniform background lighting based on a Transformer model is provided, comprising:

[0081] At least one processor and a memory communicatively connected to said at least one processor;

[0082] The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the method described above.

[0083] In the above technical solution, to better operate and process the method, the method is stored in memory, and the processor executes the stored method. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here.

[0084] According to another aspect of the present invention, a computer-readable storage medium is provided storing a computer program that, when executed by a processor, implements the above-described method.

[0085] In the above technical solution, to better operate and use the method, the method is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0086] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0087] Figure 1 This is a schematic flowchart of an embodiment of the reasoning process of the present invention;

[0088] Figure 2 This is a flowchart illustrating an embodiment of the training process of the present invention;

[0089] Figure 3 This is a schematic diagram of the model architecture of an embodiment of the training process of the present invention;

[0090] Figure 4 This is a data pair manufacturing process flowchart of an embodiment of the training process of the present invention;

[0091] Figure 5 This is a schematic diagram of an embodiment of the device of the present invention. Detailed Implementation

[0092] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] Example 1

[0094] Please see Figure 1 A deep learning method for uniform background lighting and shadow based on the Transformer model, the method comprising:

[0095] S1. Obtain the input image and the corresponding human subject mask, wherein the input image contains areas with uneven background lighting;

[0096] S2. Preprocess the input image and the human subject mask, including image standardization and feature fusion, to obtain a fused feature map;

[0097] S3. Based on the patch segmentation method, the fused feature map is converted into sequence features;

[0098] S4. Encode the sequence features to output enhanced sequence features, model the light and shadow distribution dependency of all pixels in the image through a global attention mechanism, and distinguish the subject features of the portrait from the background features based on the subject mask.

[0099] S5. Based on the inverse logic of patch segmentation, the enhanced sequence features are restored into a spatial feature map;

[0100] S6. The spatial feature map is synchronously input into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0101] S7. Magnify the resolution of the residual map and the intensity field map to the resolution of the input image, control the weight of the residual map in the input image through the intensity field, and complete the fusion of the input image, the magnified residual map and the intensity field map, and output the result image.

[0102] In this embodiment, S3, based on the patch segmentation method, the fused feature map is converted into sequence features, specifically:

[0103] The fused feature map of H×W×X is divided into N non-overlapping patches according to a fixed size P×P, where X is the number of channels in the fused feature map; N is the number of patches, N=(H / P)×(W / P), and H and W are both integer multiples of P. If they are not satisfied, they are adjusted by edge padding; H and W are the height and width of the original image, respectively.

[0104] Each P×P×X patch is mapped to a D-dimensional feature vector through a linear transformation layer, and the output sequence features have an N×D dimension.

[0105] In this embodiment, S4, the sequence features are encoded to output enhanced sequence features. A global attention mechanism is used to model the light and shadow distribution dependency of all pixels in the image. The subject features are then distinguished from background features based on the subject mask. Specifically:

[0106] The sequence features are input into the backbone network of the Transformer model, which is composed of L stacked Transformer encoders. Each Transformer encoder layer shares the same structure, but the parameters are independent.

[0107] After performing layer normalization on the multi-head self-attention module of each Transformer encoder layer, the input N×D dimensional sequence features are split into M N×(D / M) dimensional sub-features according to the preset number of heads M, and the attention weights between different patches are calculated as follows:

[0108]

[0109] Where Q, K, and V are the query, key, and value matrices, respectively; d k=D / M is the dimension of a single-head feature; N is the sequence length, and D is the dimension of each feature vector;

[0110] After layer normalization of the feedforward network of each Transformer encoder layer, a nonlinear transformation is performed on the feature input feedforward network after processing by the multi-head self-attention module. The network structure consists of linear layers, GELU activation function, and linear layers, with the hidden layer dimension expanded to 4D.

[0111] After processing by the feedforward network, the output is an N×D dimensional enhanced sequence feature.

[0112] In this embodiment, S5, based on the inverse logic of patch segmentation, restores the enhanced sequence features into a spatial feature map, specifically:

[0113] Receive N×D dimensional enhanced sequence features, where N is the sequence length and D is the dimension of each feature vector;

[0114] Based on the original image height H, width W, and patch size P, the target spatial dimensions of the spatial feature map are calculated as height H / P and width W / P; where the sequence length N must satisfy N=(H / P)×(W / P);

[0115] Based on the spatial correspondence during patch segmentation, the N×D-dimensional enhanced sequence features are rearranged and reshaped into a (H / P)×(W / P)×D spatial feature map.

[0116] In this embodiment, S6, the spatial feature map is synchronously input into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations. Specifically:

[0117] The first and second output heads synchronously receive spatial feature maps as input;

[0118] For the first output head, spatial features of the spatial feature map are extracted, and the distribution is stabilized through a batch normalization layer. Nonlinearity is introduced using the GELU activation function. The processed spatial features are then passed through a convolutional layer to map their channel number to be consistent with the channel number of the output image, and finally, a residual map is output.

[0119] For the second output head, spatial features of the spatial feature map are extracted, and the distribution is stabilized through a batch normalization layer. Nonlinearity is introduced using the GELU activation function. The processed spatial features are passed through a convolutional layer, and their channel number is mapped to 1. After that, the Sigmoid activation function is applied to constrain the output value to the range of [0,1], and finally the intensity field map is output.

[0120] Example 2

[0121] The second embodiment is a training step built on the basis of the first embodiment. The implementation methods of the first embodiment and the second embodiment overlap, but the training method can more comprehensively and profoundly demonstrate the completeness of the model or method process. Therefore, the parts of each step in the first embodiment that have not been described in detail will be described in detail in this embodiment.

[0122] Please see Figure 2 , Figure 3 A training method for a Transformer model, based on the method described in one embodiment; wherein the Transformer model includes an input layer, a preprocessing layer, a patch embedding layer, a Transformer backbone, a feature reshaping layer, a dual-output head layer, and a prediction layer; the method includes:

[0123] S1. Input the training image, the corresponding ground truth image, and the subject mask into the layer; the ground truth image is a uniformly lit and shadowed image corresponding to the training image.

[0124] For example, the input layer provides raw data input for model training, containing two types of core data for subsequent feature fusion and lighting correction guidance.

[0125] Input data details:

[0126] Image of a person with uneven lighting (denoted as img): The dimensions are H×W×3 (H is the image height, W is the image width, and 3 is the RGB three channels). The image contains the background lighting unevenness area to be corrected (such as local bright spots, dark areas, and areas with multiple light sources superimposed) and the complete portrait subject.

[0127] The subject mask (denoted as mask) has dimensions of H×W×1 (single channel) and pixel values ​​of only 0 or 1. "1" corresponds to the subject area of ​​the portrait (such as face, hair, and clothing texture), and "0" corresponds to the background area. It is used to guide subsequent modules to accurately distinguish between the subject and the background and avoid the subject details being interfered with by light and shadow correction.

[0128] Data flow: The img and mask output from the input layer are directly passed to the preprocessing layer without any data dimension transformation.

[0129] In this embodiment, the training images are obtained through at least one of the following methods: a. using an image editing tool to automatically generate training image-ground image pairs in batches through script execution; b. using 3D scene and 3D character models to generate training image-ground image pairs in batches covering various lighting conditions; c. using a generative adversarial network to generate training image-ground image pairs.

[0130] Calculate the standard deviation σ of the background region in the training image and the standard deviation σ of the brightness in the background region of the ground truth image. Compare the two standard deviations with preset thresholds to determine whether they are qualified. Use a feature matching algorithm to calculate the feature matching rate of the subject region of the human figure in the training image-ground truth image. Compare the feature matching rate with a preset threshold to determine whether they are qualified. Use a scene classification model to determine whether the lighting effect of the training image matches its scene type. Compare the classification confidence with a preset threshold to determine whether it is qualified. Use the qualified image pairs as training image pairs. Mark the reasons for the unqualified image pairs and regenerate them.

[0131] For example, in the training process of a portrait background uniform lighting model, high-quality "input-label" data pairs are the core element determining the model's performance. The input samples need to cover different background lighting unevenness patterns in various scenes (such as local bright spots, dark areas, light-dark boundaries, and multiple light sources), while the label samples need to correspond to the "ideal uniform lighting state" of the input samples (i.e., consistent background brightness, no lighting interference, and intact details of the portrait subject). However, existing data acquisition methods have significant drawbacks:

[0132] Real data collection is difficult: It requires the construction of a "light and shadow unevenness-uniformity" comparison scene in a controlled environment (such as a professional photography studio), which is costly, time-consuming, and difficult to cover the complex light and shadow patterns in natural scenes (such as outdoor backlighting and indoor mixed light sources).

[0133] Manual labeling has low accuracy: If a real image with uneven lighting is corrected into a uniform image by manual editing (such as Photoshop) and used as a label, the label accuracy is easily inconsistent due to differences in subjective judgment, and batch generation is not possible;

[0134] Existing forgery methods are limited: traditional data augmentation (such as random brightness adjustment) can only generate simple light and shadow changes and cannot simulate local uneven lighting in real scenes (such as light spots projected by windows and shadows formed by people blocking the light), resulting in weak model generalization ability.

[0135] Therefore, there is an urgent need for a technical solution that can generate "light and shadow unevenness-uniformity" comparison data pairs in batches, in a diverse and high-fidelity manner, in order to support the efficient training of background light and shadow uniformity models.

[0136] This invention employs a "three-path fusion" forgery strategy, generating data pairs through rule-based forgery, scenario-based forgery, and intelligent forgery respectively, covering different accuracy and scenario requirements. Please refer to [link / reference]. Figure 4 The specific steps are as follows:

[0137] Path 1: Rule-based forgery using image editing tools (suitable for quickly generating basic data pairs)

[0138] Using professional image editing tools such as Adobe Photoshop and GIMP, combined with automated script execution, batch generation of "uneven lighting input - uniform labels" is achieved. The steps include:

[0139] Basic material preparation: Collect or acquire a "clean portrait image" B (with a solid color or simple texture background and a complete portrait subject) without background light and shadow interference, as the basic template for data pair generation;

[0140] Input sample S in Generation:

[0141] a. Construct an uneven lighting layer using the Gradient Tool and Brush Tool: ① Create a transparent layer L1 with the same size as B; ② Draw different types of lighting interference on L1 (e.g., use Radial Gradient to simulate point light spot, Linear Gradient to simulate backlight gradient, and Soft Brush to draw irregular shadows); ③ Set the blending mode of L1 to Overlay, Soft Light, or Multiply, and adjust the opacity to 10%-40%, blending it with B to obtain S. in (The background contains controllable uneven lighting);

[0142] b. Use the "Liquify Tool" to fine-tune the background area (such as slightly stretching the background texture and adjusting the background edge outline) to simulate background deformation in real shooting and improve S. in The authenticity of the scene;

[0143] Label Sample S out Generation: Directly use the original "clean portrait image" B as S out Ensure the background is free of light and shadow interference, and that it is consistent with S in The subject of the portrait is completely identical;

[0144] Automated batch processing: Record the above steps using Photoshop Action scripts to batch import multiple sets of "clean portrait images" and automatically generate corresponding data pairs (S in ,S out It supports generating 1,000-10,000 sets of basic data in a single session.

[0145] Path 2: Contextualized forgery based on parametric models (suitable for generating complex scene data pairs)

[0146] Using 3D modeling tools such as Blender and Cinema 4D, parametric lighting and shadow scenes are constructed, generating input samples with realistic physically based lighting and shadows and corresponding labels. The steps include:

[0147] 3D scene construction:

[0148] a. Import a 3D human portrait model M (a high-precision model can be obtained by scanning a real person, or an open-source 3D human portrait model library can be used), adjust the model's posture and expression, and simulate the appearance of a real human portrait;

[0149] b. Build background scenes (such as indoor living rooms, outdoor streets, photography studios, etc.), import background object models (such as furniture, trees, lamps), and set the reflectivity and color of background materials (such as fabric, walls, metal).

[0150] Parametric control of lighting and shadow:

[0151] a. Add light sources to the scene (such as parallel light to simulate sunlight, point light to simulate a desk lamp, and area light to simulate a fill light). Adjust the position (x / y / z coordinates), intensity (100-10000cd), color temperature (2700K-6500K), and illumination angle (0°-90°) of the light source by adjusting the parameters to generate different types of uneven background lighting (such as shadows formed by light source occlusion and light and dark interlacing formed by multiple light sources).

[0152] b. Rendering and generating input sample S in Set the rendering resolution (e.g., 1024×1024, 2048×2048) to render an image S containing a 3D human figure and a parametrically uneven lighting background. in ;

[0153] Label Sample S out Generation: Keeping the 3D human portrait model M, background scene, and rendering parameters unchanged, remove all scene light sources, retain only the global ambient light (intensity set to the scene average brightness), and re-render to obtain an image with no background light and shadow interference and uniform brightness, which will be used as the label sample S. out Scene diversification expansion: By adjusting the light source parameters, background scene type, and human figure model pose, batch data pairs covering different scenes such as "indoor / outdoor", "day / night", and "single light source / multiple light sources" are generated. Each set of data is accompanied by light and shadow parameter labels (such as the number of light sources, intensity, and angle) for scene adaptive training of the model.

[0154] Path 3: Intelligent forgery based on generative adversarial networks (suitable for generating high-fidelity natural scene data pairs)

[0155] Based on generative adversarial networks such as StyleGAN3 and ProGAN, and combined with light and shadow transfer technology, data pairs of "uneven light and shadow" conforming to the laws of natural light and shadow are generated. The steps include:

[0156] Pre-trained model preparation:

[0157] a. Training the StyleGAN3 portrait generation model: Using large-scale real portrait datasets (such as CelebA-HQ, FFHQ) as training data, train the StyleGAN3 model so that it can generate high-resolution (1024×1024), high-fidelity natural portrait images, and support fine-tuning of portrait pose, expression, and background style.

[0158] b. Training the light and shadow transfer model: Using real comparison data of "uneven light and shadow image - uniform light and shadow image" (a small amount of manually labeled data) as training data, train the U-Net-based light and shadow transfer network T so that it can transfer the light and shadow style in the image from "uneven" to "uniform" while keeping the image content (such as the subject of the portrait and the background texture) unchanged.

[0159] Input sample S in generate:

[0160] a. Generate a "basic natural portrait image" C using StyleGAN3 (the background contains natural textures and has no obvious uneven lighting).

[0161] b. Collect a library of light and shadow interference images G from real scenes (such as background images containing window light spots, tree shadows, and light spot bright spots), randomly select light and shadow areas in G, and superimpose the light and shadow interference onto the background area of ​​C using image fusion technology (such as Poisson fusion) to obtain a preliminary uneven light and shadow image C1;

[0162] c. Input C1 into the "fine-tuning module" of the pre-trained StyleGAN3 model to adjust the model's style vector, making the lighting transitions in C1 more natural (e.g., avoiding harsh edges in lighting), and finally generate the input sample S. in ;

[0163] Label Sample S out generate:

[0164] a. S in Input the light and shadow transfer network T, and network T will apply the learned light and shadow homogenization rules to S. in The background area is adjusted for lighting and shadows while maintaining the details of the main subject of the portrait, resulting in an image with uniform background lighting and shadows.

[0165] b. Introduce "Perceptual Loss" and "Structural Similarity Loss" (SSIMLoss) to optimize the transferred image, ensuring S... out Background brightness deviation ≤3%, and S in The subject structure similarity of the portrait is ≥95%, and the final labeled sample S is obtained. out Data augmentation and filtering: For the generated (S) in ,Sout Data pairs are augmented by random flipping, rotation (±15°), scaling (0.8-1.2 times), etc., to expand the data scale; data pairs with a quality score ≥0.85 are selected by pre-trained image quality assessment models (such as NIQE, PIQE) to remove blurry and distorted low-quality data and ensure the effectiveness of training data.

[0166] To ensure that the generated data pairs meet the model training requirements, this invention designs a data quality verification module to perform multi-dimensional verification on the data pairs generated through the three paths mentioned above. The steps include:

[0167] Light and shadow uniformity verification: Calculate the label sample S out If the standard deviation of the brightness of the background area σ is ≤ 5 (grayscale range 0-255), then the background lighting is considered uniform; calculate the input sample S. in If the standard deviation of brightness σ' in the background area is ≥20, then the background is determined to have effective uneven lighting.

[0168] Human subject consistency verification: The SIFT feature matching algorithm is used to calculate S. in With S out The feature matching rate of the main body area (face and upper body) of the person is determined. If the matching rate is ≥90%, the main body of the portrait is judged to be without distortion or offset.

[0169] Scene authenticity verification: Using a pre-trained scene classification model (such as ResNet50), determine S in Whether the background lighting and shadow patterns match the corresponding scene (e.g., the lighting and shadows in an indoor scene should conform to the indoor light source rules). If the scene matching degree is ≥85%, the data scene authenticity is deemed qualified.

[0170] Non-compliant data processing: For data pairs that fail validation, mark the reasons for non-compliance (such as substandard light and shadow uniformity or poor subject consistency), and feed them back to the corresponding generation path for parameter adjustment (such as adjusting the light and shadow transparency of image editing tools, the light source intensity of 3D models, and the style vector of GAN models). Regenerate the data pairs and validate them again until they meet the quality standards.

[0171] S2. The training image and the subject mask are input into the preprocessing layer via the input layer for preprocessing, including image standardization and feature fusion, to obtain a fused feature map.

[0172] For example, the preprocessing layer is used to eliminate the magnitude difference in the input data, fuse the subject location information with the image content, and provide standardized input for Transformer feature extraction.

[0173] Core Operations

[0174] Image normalization: The image size is downsampled to the network input size, and the pixel values ​​are linearly mapped from the original [0,255] interval to the [-1,1] interval, as shown in the following formula:

[0175]

[0176] Where (x,y) are pixel coordinates, and c∈{R,G,B} are color channels. Standardization is used to eliminate the interference of differences in pixel values ​​between different images on model training.

[0177] Feature fusion: The standardized image norm The mask is concatenated with the image in the channel dimension to generate a fused feature map with dimensions H×W×4 (3 RGB channels + 1 mask channel), which realizes the early association between the position information of the human subject and the image content, and provides prior information for subsequent global modeling of Transformer.

[0178] Data augmentation: Randomly enhance color, contrast, and saturation; randomly rotate the image.

[0179] Data flow: The H×W×4 fused feature map output from the preprocessing layer is passed to the patch embedding layer.

[0180] S3. Input the fused feature map into the patch embedding layer, and convert the fused feature map into sequence features based on the patch segmentation method;

[0181] For example, the patch embedding layer is used to convert 2D fused feature maps into sequence features that can be processed by Transformer while preserving spatial location information.

[0182] Core Operations

[0183] Patch segmentation: Divide the fused feature map of H×W×4 into N non-overlapping patches according to a fixed size P×P (e.g., 16×16), where the number of patches N = (H / P)×(W / P) (H and W must be integer multiples of P; if not, adjust by edge filling).

[0184] Linear projection: Each P×P×4 patch is mapped to a D-dimensional feature vector (e.g., D=768) through a linear transformation layer, and the final output is a sequence feature with an N×D dimension, which adapts to the Transformer's requirements for sequence input, while preserving the spatial semantic information of the patch through projection.

[0185] Data flow: The N×D sequence features output from the patch embedding layer are passed to the Transformer Backbone.

[0186] S4. Input the sequence features into the Transformer backbone, encode the sequence features based on the Transformer backbone to output enhanced sequence features, model the light and shadow distribution dependency of all pixels in the image through a global attention mechanism, and distinguish the subject features of the portrait from the background features based on the subject mask.

[0187] For example, the Transformer Backbone is the core feature extraction module of the model. It models the global distribution pattern of background light and shadow through a global attention mechanism, distinguishes the features of the subject from the background, and provides high-quality feature support for the subsequent dual-branch output.

[0188] Structure and Operation

[0189] Overall structure: It consists of L layers of Transformer Encoders stacked together (e.g., L=12), with each Encoder sharing the same structure and only having independent parameters;

[0190] Single-layer encoder operations:

[0191] Multi-head self-attention module: The N×D sequence features are split into M N×(D / M) sub-features according to the number of heads M (e.g., M=12), and the attention weights between different patches are calculated using the following formula:

[0192]

[0193] Where Q, K, and V are the query, key, and value matrices, respectively, and d k =D / M is a single-head feature dimension. It captures the distribution and correlation of background light and shadow across the entire image through global attention, and strengthens the feature protection of the main body area by combining mask information.

[0194] Feedforward Network (FFN): Performs a non-linear transformation on the attention output features, with a structure of "Linear→GELU→Linear", where the hidden layer dimension is 4D, as shown in the following formula:

[0195] FFN(x)=Linear2(GELU(Linear1(x)))

[0196] Used to enhance the expressive power of features and adapt to the feature learning needs of complex lighting and shadow scenes;

[0197] Layer normalization operation: Layer normalization is performed at the inputs of the multi-head self-attention module and the feedforward network, respectively, as shown in the following formula:

[0198]

[0199] Where E[x] and Var[x] are the mean and variance of the features, respectively, γ and β are learnable parameters, and ∈ is a local minimum (e.g., 1e-5) to avoid division by zero error and stabilize the model training process.

[0200] Data flow: The N×D enhanced sequence features output by the Transformer Backbone are passed to the feature reshaping layer.

[0201] S5. Input the enhanced sequence features into the feature reshaping layer, and restore the enhanced sequence features into a spatial feature map based on the inverse logic of patch segmentation.

[0202] For example, the feature reshaping layer is used to restore the sequence features output by the Transformer into a 2D spatial feature map, retaining the spatial location information of the light and shadow features, and providing structured feature input for the dual-output Head.

[0203] Core Operations

[0204] Based on the inverse logic of patch segmentation, the N×D sequence features are reconstructed into a 2D feature map of (H / P)×(W / P)×D: where N=(H / P)×(W / P). By reshaping the dimensions, the "patch sequence" is restored to a "spatial feature map", ensuring that the features at each position correspond one-to-one with the spatial regions in the original image, laying the foundation for the subsequent generation of spatialized residuals and intensity fields.

[0205] Data flow: The (H / P)×(W / P)×D 2D feature map output by the feature reshaping layer is synchronously transmitted to the residual head and the alpha head.

[0206] S6. Input the spatial feature map into the dual-output head layer, and simultaneously input the spatial feature map into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0207] For example, the dual-output Head layer is used to generate "residuals" and "intensity fields" for background lighting correction based on 2D feature maps, and is the core execution module for the model to achieve uniform lighting.

[0208] Details of each Head are as follows:

[0209] (1) Residual Head:

[0210] Structure: Composed of “3×3 convolutional layer → batch normalization layer → GELU activation function → 1×1 convolutional layer”;

[0211] Function: Learns the residual compensation amount of background lighting and shadow, that is, outputs the correction value corresponding to the unevenness of background lighting and shadow through feature mapping;

[0212] Output: Low-resolution residual C, with dimensions (H / P)×(W / P)× (corresponding to the three RGB channels, ensuring that the lighting and shadows of each channel can be corrected independently).

[0213] (2) Alpha Head:

[0214] Structure: Consistent with the basic structure of the residual head, except for the addition of a Sigmoid activation function at the end;

[0215] Function: Learn the weight of residual effects to control the intensity of lighting and shadow correction in different areas;

[0216] Output: Low-resolution intensity field Alpha, with dimensions (H / P)×(W / P)×1, and pixel values ​​ranging from [0,1] (the closer the value is to 1, the stronger the correction effect of the residual in the corresponding region; the closer the value is to 0, the closer the corresponding region is to the original image, avoiding over-correction of the main region).

[0217] Data flow: The C output from the residual head and the Alpha output from the Alpha head are jointly transmitted to the training loss calculation module.

[0218] S7. Input the low-resolution residual map and low-resolution intensity field map into the prediction layer, and downsample the training image and ground truth map to the same dimension as the residual map and intensity field map; control the weight of the residual map in the input image through the intensity field to complete the fusion of the downsampled training image with the low-resolution residual map and low-resolution intensity field map, and output the low-resolution prediction map; calculate the total loss based on the low-resolution prediction map and the downsampled ground truth map; backpropagate the gradient of the total loss to the Transformer backbone and the dual-output head layer to train the Transformer model.

[0219] For example, the prediction layer (used for calculating the training loss during training and for outputting the inference result, i.e., a portrait image with uniform background lighting, differs from the inference process here. In the prediction stage, the image needs to be downsampled to calculate the loss value, while in the inference stage, the image needs to be upsampled to output the final result image) provides gradient signals for model parameter optimization through loss calculation at low resolution. It is the core of the training loop and the key difference between this training architecture and the inference architecture.

[0220] Ground truth downsampling: The training uniform ground truth map GT (original dimension H×W×3) is downsampled to (H / P)×(W / P)×3 through bilinear interpolation to ensure that it is consistent with the dimensions of C and Alpha, and to avoid loss calculation errors caused by resolution mismatch.

[0221] Low-resolution prediction map generation: The low-resolution prediction map R' is calculated based on C and Alpha, using the following formula:

[0222] R′(x,y,c)=img down (x,y,c)+Alpha(x,y)×C(x,y,c)

[0223] Among them, img down The image is a low-resolution image after downsampling (dimensions (H / P) × (W / P) × 3), where (x, y) are the pixel coordinates of the low-resolution image and c is the color channel;

[0224] Multiple loss function calculation: Calculate the total loss L based on R' and the downsampled ground truth. total The formula is as follows:

[0225] L total =L vgg (GT,R′)+L gan (GT,R′)

[0226] Gradient backpropagation and parameter optimization: L total The gradients are backpropagated to the Transformer Backbone and the dual-output Head, and all model parameters are updated by the Adam optimizer (learning rate such as 1e-4), completing one training iteration.

[0227] Based on Embodiments 1 and 2, this invention aims to provide a background lighting uniformity model based on Transformer as the backbone. By leveraging rich data on manufacturing methods and simulating real-world scenes, it addresses the shortcomings of existing background lighting uniformity technologies in subject protection, complex lighting generalization, and detail fitting accuracy. It achieves synergistic optimization of "precise background lighting uniformity" and "lossless preservation of the subject," while simultaneously improving the model's adaptability to various lighting unevenness scenarios (such as multiple light sources, gradient lighting, and strong local interference), meeting the high-fidelity lighting correction needs of both consumer and professional portrait processing scenarios. The advantages of this invention are as follows:

[0228] 1. Address the problem of insufficient collaborative processing between the subject and background in existing technologies.

[0229] Existing technologies (such as traditional CLAHE filtering and CNN-based GAN models) have difficulty accurately distinguishing between the subject and background areas in a portrait: traditional methods tend to blur the edges of the subject, and CNN-based GANs, due to the limitation of local receptive fields, cannot effectively capture the global correlation between the subject and the background, often resulting in "over-correction of the background" or "destruction of subject details" (such as distortion of hair strands and clothing textures).

[0230] This invention uses Transformer as the backbone, and its self-attention mechanism can model the global dependency relationship of all pixels in the image. Combined with the human subject mask 〖"mask"〗_a, the subject region is accurately located. In the operation of generator G(A,〖"mask"〗_a), attention weight allocation prioritizes the protection of subject pixels (such as face and hair), and only outputs a targeted intensity field "Alpha" and residual C to the background region, so as to avoid the subject details being interfered with by the light and shadow correction.

[0231] 2. Address the issue of "weak generalization ability for complex lighting and shadow scenes" in existing technologies.

[0232] Existing GAN models (such as CNN-based generators) are not good at fitting unstructured light and shadow (such as dappled tree shadows or multiple light sources in an indoor environment): the local receptive field of CNN cannot capture the global distribution pattern of light and shadow, which makes the generated image R prone to "incomplete light and shadow correction" (such as residual local dark areas) or "over-correction" (such as uniform background brightness but no texture layer).

[0233] The global attention mechanism of the Transformer backbone in this invention can efficiently learn the global distribution features of complex light and shadow. In the generator training, by modeling the correlation between light and shadow pixels in the whole image, the model can adapt to diverse scenes such as "single light source / multiple light source", "uniform light and shadow / gradual light and shadow", and "strong interference light and shadow / weak interference light and shadow", ensuring that the generated R in different scenes conforms to the uniformity of real light and shadow.

[0234] Example 3

[0235] Please see Figure 5 A deep learning device for achieving uniform background lighting and shadow based on the Transformer model, based on the methods described in Embodiment 1 and Embodiment 2, the device comprising:

[0236] The input module is used to acquire an input image and the corresponding human subject mask, wherein the input image contains areas with uneven background lighting;

[0237] The preprocessing module is used to preprocess the input image and the human subject mask, including image standardization and feature fusion, to obtain a fused feature map;

[0238] The patch embedding module is used to convert the fused feature map into sequence features based on the patch segmentation method;

[0239] The Transformer backbone module is used to encode the sequence features and output enhanced sequence features. It models the light and shadow distribution dependency of all pixels in the image through a global attention mechanism and distinguishes the subject features of the portrait from the background features based on the subject mask.

[0240] The feature reshaping module is used to restore the enhanced sequence features into a spatial feature map based on the inverse logic of patch segmentation;

[0241] The dual-output head module is used to synchronously input the spatial feature map into the first and second output heads in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background region, respectively. The intensity field map represents the weights applied to the residual map at different spatial locations.

[0242] The prediction module is used to upscale the resolution of the residual image and the intensity field image to the resolution of the input image, control the weight of the residual image in the input image through the intensity field, and complete the fusion of the input image, the upscaled residual image, and the intensity field image, outputting a result image; and,

[0243] The training image and ground truth image are downsampled to the same dimension as the residual image and intensity field image. The influence weight of the residual image in the input image is controlled by the intensity field. The downsampled training image is fused with the low-resolution residual image and low-resolution intensity field image to output a low-resolution prediction image. The total loss is calculated based on the low-resolution prediction image and the downsampled ground truth image. The gradient of the total loss is backpropagated to the Transformer backbone and the dual-output head layer to train the Transformer model.

[0244] In this embodiment, in order to better utilize the methods described in Embodiment 1 and Embodiment 2, this application proposes a deep learning device for uniform background lighting and shadow based on the Transformer model. Each module corresponds to each step of the above method, and its specific principle has been described above and will not be repeated here.

[0245] Example 4

[0246] A deep learning device based on the Transformer model with uniform background lighting and shadows, comprising:

[0247] At least one processor and a memory communicatively connected to said at least one processor;

[0248] The memory stores instructions that can be executed by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the methods described in Embodiment 1 and Embodiment 2.

[0249] In this embodiment, to better run and process the methods described in Embodiment 1 and Embodiment 2, the above methods are stored in a memory, and the stored methods are executed using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0250] Example 5

[0251] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described in one and two embodiments.

[0252] In this embodiment, to better operate and use the methods described in Embodiment 1 and Embodiment 2, the above methods are stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principles and effects of each step have been described above and will not be elaborated upon here.

[0253] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A deep learning method for uniform background light and shadow based on a Transformer model, characterized in that, The method comprises: acquiring an input image and a corresponding portrait subject mask, the input image containing a background light and shadow uneven area; preprocessing the input image and the portrait subject mask, including image standardization and feature fusion, to obtain a fused feature map; converting the fused feature map into a sequence feature based on a patch segmentation method; encoding the sequence feature to output an enhanced sequence feature, modeling the light and shadow distribution dependency of all image pixels through a global attention mechanism, and distinguishing portrait subject features from background features based on the portrait subject mask; restoring the enhanced sequence feature into a spatial feature map based on the inverse logic of patch segmentation; synchronously inputting the spatial feature map into a first output head and a second output head in parallel to generate a low-resolution residual map and a low-resolution intensity field map for the background area, the intensity field map representing the application weight of the residual map at different spatial positions; upscaling the resolution of the residual map and the intensity field map to the resolution of the input image, controlling the weight of the residual map in the input image through the intensity field, and fusing the input image, the upscaled residual map, and the intensity field map to output a result image.

2. The deep learning method for background light and shadow uniformity based on the Transformer model according to claim 1, wherein the fused feature map is converted into a sequence feature based on a patch segmentation method, specifically: the fused feature map of HxWxX is segmented into N non-overlapping patches with a fixed size PxP, wherein X is the number of channels of the fused feature map, N is the number of patches, N=(H / P)×(W / P), and H and W are both integer multiples of P, if not, edge padding is used for adjustment; H and W are the height and width of the original image, respectively; each P×P×X patch is mapped to a D-dimensional feature vector through a linear transformation layer, and an N×D sequence feature is output.

3. The deep learning method for background light and shadow uniformity based on the Transformer model according to claim 1, wherein the sequence feature is encoded to output an enhanced sequence feature, the light and shadow distribution dependency of all image pixels is modeled through a global attention mechanism, and portrait subject features are distinguished from background features based on the portrait subject mask, specifically: the sequence feature is input into the backbone network of the Transformer model, which is stacked by L layers of Transformer encoders, each layer of Transformer encoder shares the same structure and only the parameters are independent; after layer normalization of the multi-head self-attention module of each layer of Transformer encoder, the input N×D sequence feature is split into M N×(D / M) sub-features according to the preset number of heads M, and the attention weight between different patches is calculated, the formula is as follows: after layer normalization of the feedforward network of each layer of Transformer encoder, the feature processed by the multi-head self-attention module is input into the feedforward network for nonlinear transformation; the network structure is a linear layer, a GELU activation function, and a linear layer, and the dimension of the hidden layer is expanded to 4D. ​ where Q, K, V are query, key, value matrices respectively; d k = D / M is the single-head feature dimension; N is the sequence length, and D is the dimension of each feature vector. ​ After processing by the feedforward network, an enhanced sequence feature of NxD dimension is output.

4. The deep learning method for background light and shadow uniformity based on a Transformer model according to claim 1, wherein, based on patch division inverse logic, the enhanced sequence feature is restored to a spatial feature map, specifically: receiving an enhanced sequence feature of NxD dimension, wherein N is a sequence length and D is a dimension of each feature vector; calculating a target spatial size of the spatial feature map as H / P in height and W / P in width according to an original image height H, a width W and a patch size P; wherein the sequence length N must satisfy N=(H / P)×(W / P); and rearranging and reshaping the enhanced sequence feature of NxD dimension into a spatial feature map of (H / P)×(W / P)×D according to a spatial correspondence relationship in patch division.

5. The deep learning method for background light and shadow uniformity based on a Transformer model according to claim 1, wherein, the spatial feature map is synchronously input into a first output head and a second output head in parallel to respectively generate a low-resolution residual map and a low-resolution intensity field map for a background region, and the intensity field map represents an application weight of the residual map at different spatial positions, specifically: the first output head and the second output head synchronously receive the spatial feature map as input; for the first output head, spatial features of the spatial feature map are extracted, a batch normalization layer is used to stabilize the distribution, and a GELU activation function is used to introduce nonlinearity; the processed spatial features are input into a convolution layer to map the number of channels to be consistent with the number of channels of the output image, and finally a residual map is output; for the second output head, spatial features of the spatial feature map are extracted, a batch normalization layer is used to stabilize the distribution, and a GELU activation function is used to introduce nonlinearity; the processed spatial features are input into a convolution layer to map the number of channels to 1; and then a Sigmoid activation function is applied to constrain the output value in the range of [0, 1], and finally an intensity field map is output. The method according to any one of claims 1-5, wherein the Transformer model comprises an input layer, a preprocessing layer, a patch embedding layer, a Transformer backbone, a feature reshaping layer, a double output head layer, and a prediction layer; and the method comprises: inputting a training image, a ground truth image corresponding to the training image, and a portrait subject mask into the input layer; the ground truth image is an image with uniform light and shadow corresponding to the training image; inputting the training image and the portrait subject mask into the preprocessing layer via the input layer for preprocessing, including image standardization and feature fusion, to obtain a fused feature map; inputting the fused feature map into the patch embedding layer to convert the fused feature map into a sequence feature based on a patch division method; inputting the sequence feature into the Transformer backbone to encode the sequence feature and output an enhanced sequence feature, model the light and shadow distribution dependency relationship of all image pixels through a global attention mechanism, and distinguish portrait subject features and background features based on the portrait subject mask; and inputting the enhanced sequence feature into the feature reshaping layer to restore the enhanced sequence feature to a spatial feature map. ​ ​ ​ ​ ​ ​ ​ ​ ​ 6. A method for training a Transformer model, comprising: ​ ​ ​ ​ ​ The enhanced sequence features are input into a feature remodeling layer to restore the enhanced sequence features to spatial feature maps based on inverse logic of patch segmentation; The spatial feature maps are input into a double-output head layer, and the spatial feature maps are synchronously input into a first output head and a second output head in parallel to generate a low-resolution residual map and a low-resolution intensity field map for a background area, respectively, and the intensity field map represents an application weight of the residual map at different spatial positions; The low-resolution residual map and the low-resolution intensity field map are input into a prediction layer, and the training image and the ground truth map are down-sampled to be consistent with the dimensions of the residual map and the intensity field map; the application weight of the residual map in the input image is controlled by the intensity field, the fusion of the down-sampled training image and the low-resolution residual map and the low-resolution intensity field map is completed, and a low-resolution prediction map is output; the total loss is calculated based on the low-resolution prediction map and the down-sampled ground truth map; the gradient of the total loss is reversely transmitted to the Transformer backbone and the double-output head layer, and the Transformer model is trained.

7. The training method of claim 6, wherein the training image is obtained by at least one of the following: a. using an image editing tool to automatically execute a script of the tool to batch generate image pairs of the training image and the ground truth map; b. using a 3D scene and a 3D character model to batch generate image pairs of the training image and the ground truth map covering multiple lighting conditions; c. using a generative adversarial network to generate image pairs of the training image and the ground truth map. The standard deviation σ of the background area of the training image and the brightness standard deviation σ of the background area of the ground truth map are calculated, and the two standard deviations are compared with a preset threshold to determine whether they are qualified; a feature matching algorithm is used to calculate a feature matching rate of a portrait subject area in the training image and the ground truth map, and the feature matching rate is compared with a preset threshold to determine whether it is qualified; a scene classification model is used to determine whether the light and shadow effect of the training image conforms to the scene type, and the classification confidence is compared with a preset threshold to determine whether it is qualified; the above qualified image pairs are used as training image pairs, and the unqualified image pairs are marked and regenerated. The device comprises: an input module configured to obtain an input image and a corresponding portrait subject mask, the input image containing a background area with uneven light and shadow; 8. A deep learning device for background light and shadow uniformity based on a Transformer model, characterized in that, a preprocessing module configured to preprocess the input image and the portrait subject mask, including image standardization and feature fusion, to obtain a fused feature map; a patch embedding module configured to convert the fused feature map into sequence features based on patch segmentation; a Transformer backbone module configured to encode the sequence features to output enhanced sequence features, model the light and shadow distribution dependency of all image pixels through a global attention mechanism, and distinguish portrait subject features and background features based on the portrait subject mask; a feature remodeling module configured to restore the enhanced sequence features to spatial feature maps based on inverse logic of patch segmentation; and a double-output head layer configured to synchronously input the spatial feature maps into a first output head and a second output head in parallel to generate a low-resolution residual map and a low-resolution intensity field map for a background area, respectively, and the intensity field map represents an application weight of the residual map at different spatial positions. ​ The dual output head module is used for generating a low resolution residual map and a low resolution intensity field map for a background region respectively by synchronously inputting a spatial feature map into a parallel first output head and a second output head, and the intensity field map represents an application weight of the residual map at different spatial positions; The prediction module is used for upsampling resolutions of the residual map and the intensity field map to a resolution of an input image, controlling an action weight of the residual map in the input image by the intensity field map, completing fusion of the input image, the upsampled residual map and the intensity field map, and outputting a result map; and The training image and the ground truth map are downsampled to be consistent with dimensions of the residual map and the intensity field map, fusion of the downsampled training image, the low resolution residual map and the low resolution intensity field map is completed by controlling an action weight of the residual map in the input image by the intensity field map, a low resolution prediction map is outputted, a total loss is calculated based on the low resolution prediction map and the downsampled ground truth map, and the total loss is reversely transmitted to the Transformer main body and the dual output head layer to train the Transformer model.

9. A deep learning device for background light and shadow uniformity based on a Transformer model, characterized in that, The method comprises: at least one processor and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 7.

10. A computer readable storage medium storing a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images