Light source detection model processing method and device, readable storage medium and program product

Abstract

The invention relates to a light source detection model processing method and device, a readable storage medium and a computer program product. The method comprises the following steps: extracting a sample face feature map from a sample face image, wherein the sample face image is obtained by rendering a sample illumination spherical harmonic coefficient; performing illumination prediction based on the sample face feature map to obtain a first illumination spherical harmonic coefficient; obtaining a reference illumination spherical harmonic coefficient, and performing image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficient to obtain a reconstructed face image; performing illumination prediction on the reconstructed face image to obtain a second illumination spherical harmonic coefficient; determining spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determining spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient; and performing model training based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model. The method can improve the accuracy of light source detection of the image.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a method, apparatus, computer device, computer-readable storage medium, and computer program product for processing a light source detection model. Background Technology

[0002] With the development of computer technology, skin analysis technology based on mobile phone photography has emerged. Mobile phone skin analysis involves very complex lighting environments that can affect subsequent skin analysis results. Therefore, facial light source detection is a crucial step. Accurately identifying light sources (such as direction and intensity) improves and optimizes skin analysis algorithms, reducing the interference of ambient light on skin tone, blemishes, pores, etc.

[0003] Traditional face light source detection methods typically involve classifying lighting conditions into several preset categories (such as "cool / warm light," "natural light," "backlight," etc.) to label the data and train a simple multi-class CNN network. This approach can adapt different skin texture algorithms to different lighting conditions, making it highly practical. However, it can only perform coarse-grained judgments, and the output lighting data is not accurate. Summary of the Invention

[0004] Based on this, this application provides a light source detection model processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product, which can improve the accuracy of light source detection in images.

[0005] On the one hand, this application provides a method for processing a light source detection model, including:

[0006] Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0007] Illumination prediction is performed based on the sample face feature map to obtain the first illumination spherical harmonic coefficient;

[0008] Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image;

[0009] Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient;

[0010] Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0011] The model is trained based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0012] On the one hand, this application also provides a light source detection model processing device, including:

[0013] The extraction module is used to extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0014] The first prediction module is used to predict illumination based on the sample face feature map to obtain the first illumination spherical harmonic coefficient.

[0015] The reconstruction module is used to obtain the reference illumination spherical harmonic coefficients, and to perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image;

[0016] The second prediction module is used to predict the illumination of the reconstructed face image and obtain the second illumination spherical harmonic coefficient.

[0017] The loss determination module is used to determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and to determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0018] The training module is used to train the model based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0019] On the one hand, this application also provides a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to perform the following steps:

[0020] Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0021] Illumination prediction is performed based on the sample face feature map to obtain the first illumination spherical harmonic coefficient;

[0022] Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image;

[0023] Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient;

[0024] Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0025] The model is trained based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0026] On the one hand, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, performs the following steps:

[0027] Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0028] Illumination prediction is performed based on the sample face feature map to obtain the first illumination spherical harmonic coefficient;

[0029] Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image;

[0030] Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient;

[0031] Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0032] The model is trained based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0033] On the one hand, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps:

[0034] Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0035] Illumination prediction is performed based on the sample face feature map to obtain the first illumination spherical harmonic coefficient;

[0036] Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image;

[0037] Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient;

[0038] Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0039] The model is trained based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0040] The aforementioned light source detection model processing method, apparatus, computer equipment, computer-readable storage medium, and computer program product use the sample illumination spherical harmonic coefficient rendering and the sample face image obtained from the sample illumination spherical harmonic coefficient rendering as training sample pairs. Sample face feature maps are extracted from the sample face images, and the first illumination spherical harmonic coefficient is directly predicted based on the sample face feature maps. The sample illumination spherical harmonic coefficient is introduced as a supervision signal (i.e., spherical harmonic coefficient prediction loss), enabling the model to learn illumination patterns and improving its ability to resolve complex illumination conditions. Reference illumination spherical harmonic coefficients are obtained, and image reconstruction is performed based on the sample face feature maps and reference illumination spherical harmonic coefficients to obtain a reconstructed face image. Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient. During the image reconstruction stage, the sample face feature maps are combined with the reference illumination spherical harmonic coefficients, allowing the model to adapt to new illumination conditions while maintaining identity features. The spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient is determined. The discrimination loss of the second illumination spherical harmonic coefficient is used for verification to ensure that the illumination attributes of the reconstructed image strictly match the reference coefficients, thereby separating illumination from identity features. Furthermore, the prediction error from the first stage is propagated to the second stage through the reconstructed image. The spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss form a joint optimization objective for model training, enhancing the model's robustness to illumination perturbations and making the trained light source detection model more accurate in detecting human portrait light sources in images. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an application environment diagram of the light source detection model processing method in one embodiment;

[0043] Figure 2 This is a flowchart illustrating the light source detection model processing method in one embodiment;

[0044] Figure 3 This is a schematic diagram of the architecture of a light source detection model in one embodiment;

[0045] Figure 4 This is a schematic diagram of the architecture of the illumination conversion module in one embodiment;

[0046] Figure 5 This is a flowchart illustrating a light source detection method in one embodiment;

[0047] Figure 6 This is a schematic diagram of the process for calculating portrait lighting parameters from the spherical harmonic coefficient in one embodiment;

[0048] Figure 7 This is a structural block diagram of a light source detection model processing device in one embodiment;

[0049] Figure 8 This is a structural block diagram of the light source detection device in one embodiment;

[0050] Figure 9 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0051] To make the objectives, technical solutions, and beneficial effects of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0052] The light source detection model processing method provided in this application embodiment can be applied to, for example... Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be integrated onto server 104 or placed on a cloud or other network server. Terminal 102 can obtain sample face images from server 104. Terminal 102 extracts sample face feature maps from the sample face images, which are rendered using sample illumination spherical harmonic coefficients. Terminal 102 performs illumination prediction based on the sample face feature maps to obtain first illumination spherical harmonic coefficients. Terminal 102 obtains reference illumination spherical harmonic coefficients and performs image reconstruction based on the sample face feature maps and reference illumination spherical harmonic coefficients to obtain a reconstructed face image. Terminal 102 performs illumination prediction on the reconstructed face image to obtain second illumination spherical harmonic coefficients. Terminal 102 determines the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficients and the sample illumination spherical harmonic coefficients, and determines the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficients and the reference illumination spherical harmonic coefficients. Terminal 102 trains the model based on spherical harmonic coefficient prediction loss and spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0053] The terminal 102 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, IoT devices, and portable wearable devices. IoT devices can include smart speakers, smart TVs, smart air conditioners, smart in-vehicle systems, and projection devices. Portable wearable devices can include smartwatches, smart bracelets, and head-mounted displays. Head-mounted displays can be virtual reality (VR) devices, augmented reality (AR) devices, and smart glasses. The server 104 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0054] In one exemplary embodiment, such as Figure 2 As shown, a light source detection model processing method is provided, which can be applied to computer devices (such as...) Figure 1 Taking the terminal or server in the middle as an example, the explanation includes:

[0055] Step 202: Extract sample face feature maps from the sample face images. The sample face images are obtained by rendering with the spherical harmonic coefficients of the sample illumination.

[0056] Specifically, when constructing the training dataset, physically based rendering simulation can be used instead of directly capturing real images. The computer can first generate a high-quality 3D face model by scanning real faces to obtain mesh geometry, high-resolution texture, and normal maps. The same 3D portrait is then rendered under different lighting conditions using the PBRT rendering tool. Each 3D model is rendered under approximately 900 different ambient lights. Each ambient light is represented by a spherical harmonic coefficient vector. Through rendering, the spherical harmonic coefficients of each image are fully known, allowing for the formation of one-to-one training data pairs. That is, given a set of sample lighting spherical harmonic coefficients L, the rendering engine generates a sample face image Img_L, so the data pair is in the form of (Img_L, L).

[0057] In one embodiment, the rendering engine outputs an RGB image with a resolution of 1024x1024 after rendering.

[0058] Step 204: Based on the sample face feature map, perform illumination prediction to obtain the first illumination spherical harmonic coefficient.

[0059] Spherical harmonic functions (i.e., spherical harmonic coefficients of illumination) are a method of expressing functions on a sphere. They are a family of orthogonal functions in spherical coordinates, much like the expansion of sine and cosine functions in Fourier transform onto a plane. Spherical harmonic functions are used to represent brightness variations under diffuse illumination. They can represent complex ambient light at relatively low orders, and their fixed-dimensional vector characteristics make them easy to integrate with neural networks.

[0060] The first illumination spherical harmonic coefficient represents the illumination information in the sample face image predicted by the model. This illumination information includes the brightness information of the sample face image.

[0061] The first illumination spherical harmonic coefficient may include 9 spherical harmonic coefficients, i.e., 9-dimensional spherical harmonic coefficients. The sample illumination spherical harmonic coefficient may include 9 spherical harmonic coefficients, i.e., 9-dimensional spherical harmonic coefficients.

[0062] Specifically, the computer equipment performs illumination prediction based on the sample face feature map to obtain the first illumination spherical harmonic coefficient.

[0063] Step 206: Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain the reconstructed face image.

[0064] Specifically, the computer equipment acquires the spherical harmonic coefficients of the reference illumination, and performs image reconstruction on the sample face feature map based on the reference illumination spherical harmonic coefficients to obtain the reconstructed face image.

[0065] In this embodiment, the reference illumination spherical harmonic coefficient can be a spherical harmonic coefficient that is different from the sample illumination spherical harmonic coefficient.

[0066] Step 208: Perform illumination prediction on the reconstructed face image to obtain the second illumination spherical harmonic coefficient.

[0067] The second illumination spherical harmonic coefficient represents the illumination information in the reconstructed face image predicted by the model. This illumination information in the reconstructed face image includes the brightness information of the reconstructed face image.

[0068] Specifically, the computer equipment can predict the illumination of the reconstructed face image to obtain the second illumination spherical harmonic coefficient.

[0069] Step 210: Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient.

[0070] Specifically, the computer device can calculate the difference between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and use this difference as the spherical harmonic coefficient prediction loss.

[0071] For example, the prediction loss for spherical harmonics is MSE_loss, and a total of 9 spherical harmonics are predicted.

[0072] The prediction loss MSE_loss for spherical harmonic coefficients can be calculated using the following formula:

[0073]

[0074] Where G represents the encoder of the model, This represents the nine spherical harmonic coefficients of the model's predicted output, namely the first illumination spherical harmonic coefficient. This represents the spherical harmonic coefficients given in the data preprocessing, i.e., the spherical harmonic coefficients of the sample illumination.

[0075] Specifically, the computer device can calculate the difference between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient, and use this difference as the spherical harmonic coefficient discrimination loss.

[0076] For example, the spherical harmonic coefficient discrimination loss is called Consistency_loss, which can be calculated using the following formula:

[0077]

[0078] in, This indicates that the new spherical harmonic coefficients, namely the second illumination spherical harmonic coefficients, are estimated from the output reconstructed face image using a spherical harmonic coefficient discrimination network with a frozen parameter. This represents the spherical harmonic coefficients of the input reference illumination. Spherical harmonic coefficient discrimination loss is a type of closed-loop loss. lose This encourages the network to output reconstructed images that are physically plausible, while also supervising the encoder to extract features more meticulously.

[0079] Step 212: Train the model based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain the light source detection model.

[0080] Specifically, the computer equipment trains the model based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss, and stops training when the training stops, thus obtaining the light source detection model.

[0081] The training stopping condition can be that the number of training iterations reaches a preset number, or that the loss is less than or equal to a loss threshold.

[0082] In one embodiment, the target loss can be calculated based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss. The process stops when the target loss is less than or equal to the loss threshold, thus obtaining the light source detection model.

[0083] In one embodiment, such as Figure 3 As shown in the figure, this embodiment uses a deep learning method to design a real-time light source detection network, i.e., a light source detection model. The overall structure of the network is as follows: Figure 3 As shown. During the training phase, the model consists of an encoder-decoder structure and a pre-trained spherical harmonic coefficient discriminant network. The encoder and decoder are connected by several skip connections. The model input image is a 512x512 rendered face image. The network backbone uses residual modules based on depthwise separable convolutions. Depthwise separable convolutions are used to reduce parameters and improve model speed. At the network "bottleneck," an illumination Transformer module, a convolutional output layer, and a convolutional input layer are concatenated. The convolutional output layer is responsible for outputting 9-dimensional illumination spherical harmonic coefficients, i.e., outputting the first illumination spherical harmonic coefficients. The convolutional input layer is only used during the training phase. Its input is also 9-dimensional illumination spherical harmonic coefficients, i.e., inputting reference illumination spherical harmonic coefficients. This guides the network to reconstruct and generate reconstructed face images with different illuminations based on different illumination spherical harmonic coefficients, thereby indirectly improving the encoder's feature extraction capability by calculating the spherical harmonic coefficient discriminant loss. The training phase involves a large number of spherical harmonic coefficient discrimination network parameters, which are used to calculate the spherical harmonic coefficient discrimination loss along with the input reference illumination spherical harmonic coefficients, thereby improving the robustness of the encoder.

[0084] During the model inference phase, only retain Figure 3 The encoder, illumination Transformer module, and convolutional output layer in the structure have parameters that, while ensuring accuracy, can meet real-time performance requirements.

[0085] In the aforementioned light source detection model processing method, the sample illumination spherical harmonic coefficient rendering and the sample face image obtained from the sample illumination spherical harmonic coefficient rendering are used as training sample pairs. Sample face feature maps are extracted from the sample face images, and the first illumination spherical harmonic coefficient is directly predicted based on the sample face feature maps. The sample illumination spherical harmonic coefficient is introduced as a supervision signal (i.e., spherical harmonic coefficient prediction loss), enabling the model to learn illumination rules and improving the model's ability to resolve complex illumination conditions. Reference illumination spherical harmonic coefficients are obtained, and image reconstruction is performed based on the sample face feature maps and reference illumination spherical harmonic coefficients to obtain a reconstructed face image. Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient. During the image reconstruction stage, the sample face feature maps are combined with the reference illumination spherical harmonic coefficients, allowing the model to adapt to new illumination conditions while maintaining identity features. The spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient is determined. The discrimination loss of the second illumination spherical harmonic coefficient is used for verification to ensure that the illumination attributes of the reconstructed image strictly match the reference coefficients, thereby separating illumination from identity features. Furthermore, the prediction error in the first stage is passed to the second stage through the reconstructed image. The spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss form a joint optimization objective for model training, which enhances the robustness of the model to illumination disturbances and makes the trained light source detection model more accurate in detecting human portrait light sources in the image.

[0086] In an exemplary embodiment, illumination prediction based on a sample face feature map to obtain a first illumination spherical harmonic coefficient includes: performing attention allocation processing on the sample face feature map to obtain an attention allocation feature map; and performing illumination prediction on the attention allocation feature map to obtain the first illumination spherical harmonic coefficient.

[0087] Image reconstruction based on sample face feature maps and reference illumination spherical harmonic coefficients to obtain reconstructed face images includes: image reconstruction based on sample face feature maps, attention allocation feature maps, and reference illumination spherical harmonic coefficients to obtain reconstructed face images.

[0088] Specifically, the computer equipment performs attention allocation processing on the sample face feature map to obtain an attention allocation feature map. Illumination prediction is then performed on the attention allocation feature map to obtain the first illumination spherical harmonic coefficients.

[0089] Computer equipment reconstructs images based on sample face feature maps, attention allocation feature maps, and reference illumination spherical harmonic coefficients to obtain reconstructed face images.

[0090] In one embodiment, attention allocation processing can be performed on the sample facial feature map using the illumination Transformer module. For example... Figure 4 As shown, the lighting Transformer module consists of two residual blocks and a tail channel attention module, as follows: Figure 4 As shown. Assume the input feature tensor is... At the beginning of each residual block, a LayerNorm (LN) layer is set to normalize the input features, stabilize the entire network training process, and accelerate convergence. .

[0091] Then, a W-MSA module is connected, which divides the feature map passed in from the upper layer into multiple non-overlapping windows, each window being of size [size missing]. This saves significant computational costs and accelerates network inference. Self-attention is then calculated within each window. This module utilizes a multi-head attention mechanism, which can focus on different regions from different perspectives, learning multi-scale, multi-location, and multi-semantic relationships. For features within each window, the following operations are performed:

[0092] 1) Obtain QKV through linear layer mapping:

[0093]

[0094] 2) Implement the following attention formula using a series of matrix multiplication layers, scale layers, and softmax layers:

[0095]

[0096] in, This represents a learnable relative positional encoding. This represents the dimension of each head in the multi-head attention process. The dot-product layer is used to capture feature similarity, and then Softmax is used to assign the attention weights, divided by... This is to stabilize the gradient.

[0097] 3) Perform multi-head concatenation and mapping back to the original dimension output:

[0098]

[0099] An MLP was added to the second residual block to increase the module's nonlinear expressive power. It consists of two fully connected layers with GELU activation function.

[0100]

[0101] Finally, the module passes through a feature map channel weighting (CA) module, which performs a weighted processing on the merged feature maps, assigning larger weights to more important feature maps to improve the module's feature discrimination ability. The specific implementation is as follows: The feature map input size of this module is... Perform a global average pooling operation along the channel dimension on the feature map to obtain a channel-based feature map. , and then The input is fed into a two-layer structure consisting of two 1x1 convolutions, forming a simple perceptron. To reduce network parameter overhead, the hidden layer activation size of the perceptron is set to... Where r is the percentage reduction in the number of channels, which can be set. The output channel count is 4. Therefore, the first convolutional layer has 4 output channels. The activation function is PReLU, and the output channels of the second convolutional layer are restored to channels. Finally, the channel-based weight coefficients are obtained through the sigmoid function. Finally, the module undergoes a 1x1 convolution followed by a PReLU operation to restore the channels of the feature map to the same level as before the input module, resulting in the attention allocation feature map. This ensures that the stacking of modules can continue without increasing computational overhead.

[0102] In this embodiment, an attention mechanism is introduced to dynamically assign weights to the sample face feature map, focusing on the light-sensitive area (such as the highlight / shadow area) to generate an attention-assigned feature map. This not only improves the prediction accuracy of the first illumination spherical harmonic coefficient, but also combines the original feature map with the attention feature in the image reconstruction stage to perform multi-scale illumination fusion. This allows the reconstructed face image to maintain the integrity of the identity features and accurately reflect the physical characteristics of the reference illumination spherical harmonic coefficient. Finally, a balance optimization of illumination decoupling and feature preservation is achieved through dual supervision (prediction + discrimination).

[0103] In one exemplary embodiment, the method further includes:

[0104] Determine the illumination reconstruction loss between the reconstructed face image and the reference face image;

[0105] The model is trained based on spherical harmonic coefficient prediction loss and spherical harmonic coefficient discrimination loss to obtain a light source detection model, including:

[0106] The target loss is determined based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss; the model is then trained based on the target loss to obtain the light source detection model.

[0107] Specifically, the computer device can calculate the illumination reconstruction loss at each pixel of the reconstructed face image and the reference face image. The target loss is obtained by summing or weighted summing the spherical harmonic coefficient prediction loss, spherical harmonic coefficient discrimination loss, and illumination reconstruction loss. The computer device then trains the model based on the target loss until a training stopping condition is met, thus obtaining the light source detection model.

[0108] In one embodiment, the illumination reconstruction loss is denoted as Reconstruction_Loss, which can be calculated using the following formula:

[0109]

[0110] in, These are the pixels in the reconstructed face image output by the model. These are the pixels in the reference face image. The illumination reconstruction loss represents the convergence of the contrast between the reconstructed face image and the reference face image at the pixel level. The reference face image is the target rendered image, which can be a face image rendered using the PBRT rendering tool.

[0111] In this embodiment, by introducing illumination reconstruction loss, a triple supervision mechanism is constructed on the basis of the original spherical harmonic coefficient prediction loss and discrimination loss. This not only ensures the accuracy of the model's illumination prediction in the spherical harmonic coefficient space (physical parameter level), but also directly constrains the illumination consistency between the generated image and the real reference image through pixel-level illumination reconstruction loss (visual perception level). This forms a multi-granularity optimization objective from parameters to image, significantly improving the illumination decomposition accuracy and image reconstruction realism of the light source detection model, and ultimately achieving more robust illumination estimation and more natural rendering effects.

[0112] In an exemplary embodiment, the target loss is determined based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss, including:

[0113] The local shadow contrast loss between the region of interest in the reconstructed face image and the region of interest in the reference face image is determined; the target loss is determined based on the spherical harmonic coefficient prediction loss, spherical harmonic coefficient discrimination loss, illumination reconstruction loss, and local shadow contrast loss.

[0114] Among them, local shadow contrast loss is a shadow contrast loss of a local area, used to represent the contrast of areas with changes in illumination, and can reflect the reconstruction of the image.

[0115] Specifically, the computer device can calculate the local shadow contrast loss between the region of interest (ROI) of the reconstructed face image and the ROI of the reference face image. The target loss is obtained by summing or weighted summing the spherical harmonic coefficient prediction loss, spherical harmonic coefficient discrimination loss, illumination reconstruction loss, and local shadow contrast loss. The computer device trains the model based on the target loss until the training stops when the stopping condition is met, thus obtaining the light source detection model.

[0116] In one embodiment, the local shadow contrast loss is called Shadow_loss, which can be calculated using the following formula:

[0117]

[0118] in, Represents the gradient. This represents a user-defined region of interest. Here, the Sobel operator is used to extract the edges between the reconstructed face image and the reference face image, calculating differences in regions of interest where facial features are prone to indentation, such as the nose and eye sockets. Local shadow contrast loss enhances contrast control in areas of varying illumination, resulting in more natural shadows. The more realistic the reconstructed face image generated by the decoder, the better it guides the encoder in extracting more discriminative features.

[0119] Therefore, the target loss for training this model can be expressed as:

[0120]

[0121] in, This represents the corresponding weight of the loss function.

[0122] In one embodiment, the AdamW optimizer is used. The network input consists of a 3-channel RGB image and 9-dimensional output image illumination spherical harmonic coefficients. The encoder and convolutional output layer output the predicted input image illumination spherical harmonic coefficients, and the decoder outputs a reconstructed face image with a different illumination. The initial training rate is 0.001, the weight decay is 0.05, and the number of iterations is 1000K. The parameters are adjusted during actual training. .

[0123] In one embodiment, 100 3D portrait renderings with 900 different lighting conditions are collected as training data, totaling 90,000 images. The test set contains 10 3D portrait renderings with 900 different lighting conditions from the training set, totaling 9,000 images.

[0124] In this embodiment, by introducing local shadow contrast loss, the model further focuses on the illumination consistency of key facial regions (such as the eyes, bridge of the nose, and other highlight-sensitive areas) on the basis of the original triple loss (spherical harmonic coefficient prediction, discrimination, and illumination reconstruction). Contrastive learning is used to enhance the model's ability to perceive local illumination differences, thereby forming a synergistic optimization between global parameter constraints and local detail restoration. Ultimately, the light source detection model can not only accurately predict the overall illumination parameters, but also finely reconstruct and restore the illumination reflection characteristics of each area of ​​the face, significantly improving detail preservation and rendering realism under complex lighting conditions.

[0125] In one exemplary embodiment, the method further includes:

[0126] The first illumination spherical harmonic coefficient is converted into portrait illumination parameters, which include the angle and intensity of the light source in the sample face image.

[0127] The angle of the light source includes the pitch angle and the azimuth angle.

[0128] Specifically, the normal vector of the normal to a preset unit sphere is obtained. The illumination spherical harmonic coefficient and the normal vector are input into a preset spherical harmonic function to obtain a brightness map of the unit circle. Illuminated regions that meet the brightness conditions are selected from the brightness map of the unit circle. Based on the illumination intensity of each pixel in the illuminated region, the illumination intensity of the light source in the sample face image is determined. Based on the coordinates of each pixel in the illuminated region, the average coordinates of the illuminated region are determined, and the average coordinates are converted into polar coordinates to obtain the pitch and azimuth angles of the light source in the sample face image.

[0129] In one embodiment, the computer device can acquire reference portrait illumination parameters, calculate the illumination parameter loss between the reference portrait illumination parameters and the portrait illumination parameters, and determine the target loss based on the illumination parameter loss, the spherical harmonic coefficient prediction loss, and the spherical harmonic coefficient discrimination loss.

[0130] Alternatively, the target loss can be determined based on the illumination parameter loss, the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss.

[0131] Alternatively, the target loss can be determined based on the illumination parameter loss, the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, the illumination reconstruction loss, and the local shadow contrast loss.

[0132] In this embodiment, the model training can be further combined with illumination parameter loss to make the conversion process of illumination spherical harmonic coefficients into portrait illumination parameters more accurate.

[0133] In one exemplary embodiment, such as Figure 5 As shown, a light source detection method is provided, which can be applied to computer devices (such as...) Figure 1 Taking the terminal or server in the middle as an example, the explanation includes:

[0134] Step 502: Obtain a face image and extract facial feature maps from the face image.

[0135] Among them, facial images are images that require light source detection.

[0136] Specifically, the computing device can extract features from a face image to obtain a face feature map.

[0137] Step 504: Based on the face feature map, perform illumination prediction to obtain the illumination spherical harmonic coefficients corresponding to the light source in the face image.

[0138] Specifically, computer equipment can predict illumination based on facial feature maps to obtain the illumination spherical harmonic coefficients corresponding to the light source in the facial image.

[0139] Furthermore, the computer device can perform attention allocation on the facial feature map to obtain an attention allocation feature map. Based on the attention allocation feature map, illumination prediction is performed to obtain the illumination spherical harmonic coefficients corresponding to the light source in the facial image.

[0140] In this embodiment, the light source detection method is executed through a light source detection model, which includes an encoder, an illumination Transformer module, and a convolutional output layer. The encoder extracts a facial feature map, and the illumination Transformer module performs attention allocation on the facial feature map to obtain an attention allocation feature map. The convolutional output layer then performs illumination prediction on the attention allocation feature map to obtain the illumination spherical harmonic coefficients corresponding to the facial image.

[0141] Step 506: Convert the spherical harmonic coefficients of illumination into portrait illumination parameters, which include the angle and intensity of the light source in the face image.

[0142] Specifically, the computer equipment converts the spherical harmonic coefficients of illumination into portrait illumination parameters.

[0143] In one embodiment, the portrait lighting parameters include pitch angle and azimuth angle.

[0144] In one embodiment, the light source detection model further includes a conversion layer for converting illumination spherical harmonics into portrait illumination parameters.

[0145] In this embodiment, by extracting feature maps from face images and predicting illumination spherical harmonic coefficients, real-time prediction of single-image face light source detection can be achieved. Ambient light parameters (angle, intensity) can be regressed from a single face image, realizing efficient and accurate light source environment analysis. It can quickly locate the illumination direction and intensity in the face scene, providing reliable illumination information for applications such as image processing, face recognition, and virtual reality. At the same time, the physical representation of spherical harmonic coefficients ensures the robustness and interpretability of illumination estimation.

[0146] Furthermore, mobile devices can obtain light source detection results using only the front-facing camera without relying on other sensors (such as infrared or depth maps), making it compatible with various photo-based skin measurement solutions. Compared to traditional methods, this embodiment is highly lightweight, highly accurate, and easy to deploy, and can be adapted to various devices.

[0147] In one embodiment, the angle of the light source includes a pitch angle and a azimuth angle; converting the illumination spherical harmonic coefficient into portrait illumination parameters includes:

[0148] Obtain the normal vector of the normal to the preset unit sphere, input the illumination spherical harmonic coefficient and the normal vector into the preset spherical harmonic function to obtain the brightness map of the unit circle; filter the illumination regions that meet the brightness conditions from the brightness map of the unit circle; determine the illumination intensity of the light source in the face image based on the illumination intensity of each pixel in the illumination region; determine the average coordinates of the illumination region based on the coordinates of each pixel in the illumination region, and convert the average coordinates into polar coordinates to obtain the pitch angle and azimuth angle of the light source.

[0149] The brightness map represents the brightness value of each pixel (x, y) in the face image.

[0150] Specifically, the computer device obtains the normal vector of the normal to a preset unit sphere, inputs the illumination spherical harmonic coefficients and the normal vector into a preset spherical harmonic function, and obtains a brightness map of the unit circle. This brightness map is used to represent the direction and intensity of illumination.

[0151] Computer equipment can determine the brightness value of each pixel in the brightness map of a unit circle, filter out the pixels whose brightness values ​​meet the brightness conditions, and the area where these filtered pixels are located is the illuminated area.

[0152] The brightness condition can be that the brightness value reaches a brightness threshold. For example, pixels in the brightness map whose brightness values ​​meet the brightness threshold can be selected.

[0153] Furthermore, the computer device can perform histogram filtering based on the brightness map of the unit circle to obtain the brightest area in the brightness map as the illumination area.

[0154] The computer device can determine the average intensity of the illuminated area based on the illumination intensity of each pixel in the illuminated area, and use this average intensity as the illumination intensity of the light source in the face image. Based on the coordinates of each pixel in the illuminated area, the average coordinates of the illuminated area are determined, and the average coordinates are converted into polar coordinates to obtain the pitch angle and azimuth angle of the light source.

[0155] In one embodiment, such as Figure 6 As shown, after the light source detection model outputs the single-image predicted illumination spherical harmonic coefficients, post-processing is required to obtain the corresponding portrait illumination parameters.

[0156] The prior knowledge is that in a spherical coordinate system, a point is represented by a triplet. Indicated, where: r represents radial distance, The θ represents the pitch angle in a spherical coordinate system, ranging from 0 to 1. ), This represents the azimuth angle (Phi) in a spherical coordinate system, ranging from 0 to... ).

[0157] Portrait lighting parameters can be expressed as pitch angle Theta, azimuth angle Phi, and illumination intensity I. Now let's consider the lighting direction... Defined as a function on the unit sphere, it can be expanded using spherical harmonic coefficients:

[0158]

[0159] is a spherical harmonic function, which is a set of orthogonal basis functions on the unit sphere.

[0160] These are the spherical harmonic coefficients output by the model, representing the projection of the function at the 1st and mth orders. In this embodiment, a shade map under illumination can be synthesized using a preset spherical harmonic function, converting polar coordinates to rectangular coordinates, and combining the normal map N(x, y) of the unit sphere with the illumination spherical harmonic coefficients. Combined, calculate the shade value of each pixel within a unit sphere:

[0161]

[0162] The formula expresses illumination as spherical harmonic coefficients. By inputting the illumination spherical harmonic coefficients and the normal vector of the normal into a preset spherical harmonic function (i.e., "which direction to look at"), the result can be obtained. Figure 6 The unit circle brightness diagram in the lower left corner represents the direction and intensity of the illumination. Here, a third-order spherical harmonic function is sufficient to clearly represent the brightness distribution of the illumination; therefore, both the illumination spherical harmonic coefficients and the preset spherical harmonic functions are nine, i.e.:

[0163]

[0164] Since specific values ​​for the Theta and Phi angles are required, filtering is needed after obtaining the brightness map of the unit circle. The brightest illuminated areas are selected using a histogram. Next, the average intensity is calculated using the illumination intensity of each pixel in the illuminated area; this average intensity is the illumination intensity. The average coordinates are calculated using the coordinates of each pixel in the illuminated area. These average coordinates are rectangular coordinates, which are then converted to polar coordinates using inverse trigonometric functions, yielding the pitch angle Theta and azimuth angle Phi of the light source.

[0165] In this embodiment, by combining the spherical harmonic coefficients of illumination with the preset unit sphere normal vector, a unit circle brightness map is generated using the spherical harmonic function. High-brightness areas are then selected from this map, and the pixel intensity and coordinate distribution are analyzed, enabling accurate estimation of light source parameters in face images. Furthermore, not only is the illumination intensity of the light source accurately calculated, but the pitch and azimuth angles of the light source are also obtained through polar coordinate transformation, thereby comprehensively reconstructing the three-dimensional lighting environment and providing high-precision illumination direction and intensity data support for applications such as virtual lighting synthesis and face reconstruction.

[0166] In one embodiment, a light source detection model processing method and a light source detection method are provided, applied to a computer device, including:

[0167] Light source detection model processing method:

[0168] Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients;

[0169] Attention allocation processing is performed on the sample face feature map to obtain the attention allocation feature map;

[0170] Illumination prediction is performed on the attention allocation feature map to obtain the first illumination spherical harmonic coefficient;

[0171] Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map, attention allocation feature map, and reference illumination spherical harmonic coefficients to obtain the reconstructed face image;

[0172] Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient;

[0173] Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient;

[0174] Determine the illumination reconstruction loss between the reconstructed face image and the reference face image, and determine the local shadow contrast loss between the region of interest in the reconstructed face image and the region of interest in the reference face image;

[0175] The target loss is determined based on the spherical harmonic coefficient prediction loss, spherical harmonic coefficient discrimination loss, illumination reconstruction loss, and local shadow contrast loss.

[0176] The model is trained based on the target loss to obtain a light source detection model.

[0177] Light source detection method:

[0178] Acquire a face image, and use a light source detection model to perform attention allocation processing on the face feature map to obtain an attention allocation feature map;

[0179] By using a light source detection model and performing illumination prediction based on attention allocation feature maps, the illumination spherical harmonic coefficients corresponding to the light source in the face image are obtained.

[0180] By using the light source detection model, the normal vector of the normal of the preset unit sphere is obtained. The illumination spherical harmonic coefficient and the normal vector are input into the preset spherical harmonic function to obtain the brightness map of the unit circle.

[0181] Using a light source detection model, illuminated areas that meet the brightness conditions are selected from the brightness map of the unit circle.

[0182] The illumination intensity of the light source in the face image is determined based on the illumination intensity of each pixel in the illuminated area using a light source detection model.

[0183] By using a light source detection model, the average coordinates of the illuminated area are determined based on the coordinates of each pixel in the illuminated area. The average coordinates are then converted into polar coordinates to obtain the pitch and azimuth angles of the light source in the face image.

[0184] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0185] Based on the same inventive concept, this application also provides a light source detection model processing apparatus for implementing the light source detection model processing method described above. The solution provided by this apparatus is similar to the implementation scheme described in the above method; therefore, the specific limitations in one or more embodiments of the light source detection model processing apparatus provided below can be found in the limitations of the light source detection model processing method described above, and will not be repeated here.

[0186] In one exemplary embodiment, such as Figure 7 As shown, a light source detection model processing device 700 is provided, comprising:

[0187] The extraction module 702 is used to extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients.

[0188] The first prediction module 704 is used to predict illumination based on the sample face feature map and obtain the first illumination spherical harmonic coefficient.

[0189] The reconstruction module 706 is used to obtain the reference illumination spherical harmonic coefficients, and to perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain the reconstructed face image.

[0190] The second prediction module 708 is used to predict the illumination of the reconstructed face image and obtain the second illumination spherical harmonic coefficient.

[0191] The loss determination module 710 is used to determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and to determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient.

[0192] Training module 712 is used to train the model based on spherical harmonic coefficient prediction loss and spherical harmonic coefficient discrimination loss to obtain a light source detection model.

[0193] In an exemplary embodiment, the first prediction module is further configured to perform attention allocation processing on the sample face feature map to obtain an attention allocation feature map; and perform illumination prediction on the attention allocation feature map to obtain a first illumination spherical harmonic coefficient.

[0194] The reconstruction module is also used to reconstruct images based on sample face feature maps, attention allocation feature maps, and reference illumination spherical harmonic coefficients to obtain reconstructed face images.

[0195] In one exemplary embodiment, the loss determination module is further configured to determine the illumination reconstruction loss between the reconstructed face image and the reference face image;

[0196] The training module is also used to determine the target loss based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss; and to train the model based on the target loss to obtain the light source detection model.

[0197] In an exemplary embodiment, the loss determination module is further configured to determine the local shadow contrast loss between the region of interest of the reconstructed face image and the region of interest of the reference face image.

[0198] The training module is also used to determine the target loss based on spherical harmonic coefficient prediction loss, spherical harmonic coefficient discrimination loss, illumination reconstruction loss, and local shadow contrast loss.

[0199] In one exemplary embodiment, the device further includes:

[0200] The conversion module is used to convert the first illumination spherical harmonic coefficients into portrait illumination parameters, which include the angle and intensity of the light source in the sample face image.

[0201] Based on the same inventive concept, this application also provides a light source detection device for implementing the light source detection method described above. The solution provided by this device is similar to the solution described in the above method; therefore, the specific limitations in one or more light source detection device embodiments provided below can be found in the limitations of the light source detection method described above, and will not be repeated here.

[0202] In one exemplary embodiment, such as Figure 8 As shown, a light source detection device 800 is provided, comprising:

[0203] The acquisition module 802 is used to acquire face images and extract face feature maps from the face images.

[0204] The prediction module 804 is used to predict illumination based on the face feature map and obtain the illumination spherical harmonic coefficients corresponding to the light source in the face image.

[0205] The conversion module 806 is used to convert the spherical harmonic coefficients of illumination into portrait illumination parameters, which include the angle and intensity of the light source in the face image.

[0206] In one embodiment, the angle of the light source includes the pitch angle and the azimuth angle; the conversion module 806 is further configured to obtain the normal vector of the normal of a preset unit sphere, input the illumination spherical harmonic coefficient and the normal vector into the preset spherical harmonic function to obtain the brightness map of the unit circle; filter out the illumination areas that meet the brightness conditions from the brightness map of the unit circle; determine the illumination intensity of the light source in the face image based on the illumination intensity of each pixel in the illumination area; determine the average coordinates of the illumination area based on the coordinates of each pixel in the illumination area, and convert the average coordinates into polar coordinates to obtain the pitch angle and azimuth angle of the light source.

[0207] The aforementioned light source detection model processing device and light source detection device, each module of which can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device in hardware form, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0208] In one exemplary embodiment, a computer device is provided, which may be a terminal or a server. Taking a terminal as an example, its internal structure diagram may be as follows: Figure 9As shown, the computer device includes a processor, memory, input / output interface, communication interface, display unit, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface, display unit, and input device are also connected to the system bus via the input / output interface. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The input / output interface is used for exchanging information between the processor and external devices. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When executed by the processor, the computer program implements a light source detection model processing method and a light source detection method. The display unit is used to form a visually visible image and can be a display screen, projection device, or virtual reality imaging device. The display screen can be an LCD screen or an e-ink screen. The input device of the computer device can be a touch layer covering the display screen, or buttons, trackballs, or touchpads set on the casing of the computer device, or external keyboards, touchpads, or mice, etc.

[0209] Those skilled in the art will understand that Figure 9 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0210] In one exemplary embodiment, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the embodiments described above.

[0211] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the above embodiments.

[0212] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps of the above embodiments.

[0213] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0214] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0215] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0216] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for processing a light source detection model, characterized in that, The method includes: Extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients; Illumination prediction is performed based on the sample face feature map to obtain the first illumination spherical harmonic coefficient; Obtain the reference illumination spherical harmonic coefficients, and perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image; Illumination prediction is performed on the reconstructed face image to obtain the second illumination spherical harmonic coefficient; Determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient; The model is trained based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

2. The method according to claim 1, characterized in that, The step of predicting illumination based on the sample face feature map to obtain the first illumination spherical harmonic coefficient includes: Attention allocation processing is performed on the sample face feature map to obtain attention allocation feature map; Illumination prediction is performed on the attention allocation feature map to obtain the first illumination spherical harmonic coefficient; The step of reconstructing the image based on the sample face feature map and the reference illumination spherical harmonic coefficient to obtain the reconstructed face image includes: Image reconstruction is performed based on the sample face feature map, the attention allocation feature map, and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image.

3. The method according to claim 1, characterized in that, The method further includes: Determine the illumination reconstruction loss between the reconstructed face image and the reference face image; The process of training the model based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model includes: Based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss, the target loss is determined; The model is trained based on the target loss to obtain a light source detection model.

4. The method according to claim 3, characterized in that, The determination of the target loss based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, and the illumination reconstruction loss includes: Determine the local shadow contrast loss between the region of interest in the reconstructed face image and the region of interest in the reference face image; The target loss is determined based on the spherical harmonic coefficient prediction loss, the spherical harmonic coefficient discrimination loss, the illumination reconstruction loss, and the local shadow contrast loss.

5. The method according to any one of claims 1 to 4, characterized in that, The method further includes: The first illumination spherical harmonic coefficient is converted into portrait illumination parameters, which include the angle and intensity of the light source in the sample face image.

6. A method for detecting a light source, characterized in that, The method further includes: Acquire a face image, and extract a face feature map from the face image; Illumination prediction is performed based on the facial feature map to obtain the spherical harmonic coefficients of the light source in the facial image; The illumination spherical harmonic coefficients are converted into portrait illumination parameters, which include the angle and intensity of the light source in the face image.

7. The method according to claim 6, characterized in that, The angle of the light source includes pitch angle and azimuth angle; the conversion of the illumination spherical harmonic coefficient into portrait lighting parameters includes: Obtain the normal vector of the normal of a preset unit sphere, input the illumination spherical harmonic coefficient and the normal vector into the preset spherical harmonic function to obtain the brightness map of the unit circle; Illuminated areas that meet the brightness conditions are selected from the brightness diagram of the unit circle; The illumination intensity of the light source in the face image is determined based on the illumination intensity of each pixel in the illumination area. Based on the coordinates of each pixel in the illuminated area, the average coordinates of the illuminated area are determined, and the average coordinates are converted into polar coordinates to obtain the pitch angle and azimuth angle of the light source.

8. A light source detection model processing device, characterized in that, The device includes: The extraction module is used to extract sample face feature maps from sample face images, which are obtained by rendering with sample illumination spherical harmonic coefficients; The first prediction module is used to predict illumination based on the sample face feature map to obtain the first illumination spherical harmonic coefficient. The reconstruction module is used to obtain the reference illumination spherical harmonic coefficients, and to perform image reconstruction based on the sample face feature map and the reference illumination spherical harmonic coefficients to obtain a reconstructed face image; The second prediction module is used to predict the illumination of the reconstructed face image and obtain the second illumination spherical harmonic coefficient. The loss determination module is used to determine the spherical harmonic coefficient prediction loss between the first illumination spherical harmonic coefficient and the sample illumination spherical harmonic coefficient, and to determine the spherical harmonic coefficient discrimination loss between the second illumination spherical harmonic coefficient and the reference illumination spherical harmonic coefficient; The training module is used to train the model based on the spherical harmonic coefficient prediction loss and the spherical harmonic coefficient discrimination loss to obtain a light source detection model.

9. A light source detection device, characterized in that, The device includes: The acquisition module is used to acquire a face image and extract a face feature map from the face image; The prediction module is used to predict illumination based on the face feature map and obtain the illumination spherical harmonic coefficients corresponding to the light source in the face image. The conversion module is used to convert the illumination spherical harmonic coefficients into portrait illumination parameters, which include the angle and intensity of the light source in the face image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 7.

Images