Method and system for face recognition image color correction with multispectral ambient light detection

Abstract

This invention provides a method and system for color correction of facial recognition images based on multispectral ambient light detection, belonging to the fields of image processing and computer vision technology. The method simultaneously acquires an initial facial image and an ambient light intensity sequence. It utilizes the spectral scattering difference between the epidermis and dermis to perform multidimensional decomposition of the image, separating the facial background image from the illumination interference image. Based on luminous flux characteristics, the interference image is decoupled into surface reflection and subcutaneous scattering images, and the incident light path is extracted. A facial undulation gradient is constructed from the background image, and a light decay mapping table is generated by combining it with the ambient light phase delay, thereby calculating the color difference compensation matrix. Subsequently, phase synchronization is performed using the spectral phase difference to obtain the target correction matrix. Finally, pixel spectral reconstruction is performed based on this matrix, and standard light source rendering is applied. This invention effectively eliminates the influence of complex illumination and significantly improves the color reproduction accuracy of facial recognition.

Description

Technical Field

[0001] This invention relates to the fields of image processing and computer vision technology, and in particular to a method and system for color correction of facial recognition images based on multispectral ambient light detection. Background Technology

[0002] Facial recognition technology has been widely applied in security monitoring, mobile payment, and human-computer interaction, its core being the extraction of stable and reliable identity features from captured facial images. However, in real-world deployment environments, facial image acquisition is often affected by complex and variable lighting conditions, especially in multispectral environments where multiple light sources, such as fluorescent lights, LED white light, natural light, and various decorative lighting, coexist. These different light sources have different spectral energy distributions, flicker frequencies, and spatial projection directions, creating complex lighting superposition effects when applied to the face. This superposition effect leads to severe color shifts, localized overexposure of highlights, loss of detail in shadow areas, and skin color distortion in facial images, severely limiting the accuracy and robustness of facial recognition systems in real-world environments.

[0003] Existing facial image color correction methods mainly fall into three categories. The first category is white balance methods based on the gray-world assumption. This method assumes that the average value of all colors in an image tends towards gray and adjusts the color channels of the entire image using global statistics. However, since skin color occupies a large proportion of facial images, the global color mean is dominated by skin color and deviates from the gray assumption, easily leading to correction bias. Furthermore, this method cannot perform local adaptive correction for illumination differences in different areas of the face. The second category is illumination compensation methods based on Retinex theory. This method decomposes the image into reflection and illumination components to achieve illumination compensation. However, traditional Retinex methods struggle to accurately separate complex incident light paths in multi-light source environments, especially for uneven illumination areas caused by facial contours. Insufficient separation accuracy results in unsatisfactory correction effects. The third category is end-to-end correction methods based on deep learning. While they perform well on specific datasets, they rely on a large amount of labeled data for training. Their generalization ability is limited by the illumination scene coverage of the training data, and their correction accuracy drops significantly when facing unseen multispectral combinations.

[0004] In summary, existing technologies generally suffer from the following shortcomings: they do not fully consider the multi-layered optical characteristics of skin tissue, namely the differences in scattering and transmission of different wavelengths of light between the epidermis and dermis; they do not incorporate the spatial modulation effect of light attenuation caused by the undulations of the three-dimensional facial structure; and they do not perform synchronous correction for the temporal phase characteristics of multispectral ambient light. These shortcomings make it difficult for existing methods to achieve high-precision facial color reproduction in complex multispectral lighting environments, thus affecting the accuracy and reliability of subsequent facial recognition. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of existing facial image color correction methods that fail to fully utilize the multi-layer optical properties of the skin, the three-dimensional undulation structure of the face, and the temporal phase characteristics of multispectral ambient light for comprehensive correction. This invention provides a facial recognition image color correction method and system based on multispectral ambient light detection, which can accurately restore the true color of the face under complex multispectral lighting conditions, thereby improving the accuracy and robustness of facial recognition.

[0006] The objective of this invention is achieved through the following measures: The first aspect of this invention provides a color correction method for facial recognition images based on multispectral ambient light detection, comprising: Simultaneously acquire initial facial images and ambient light intensity sequences under multispectral conditions; Based on the spectral scattering difference between the epidermis and dermis, the initial facial image is decomposed in multiple dimensions to separate the facial background image and the illumination interference image of the initial facial image. Based on the luminous flux characteristics of the ambient light intensity sequence, the illumination interference map is analyzed, the incident light path of the illumination interference map is extracted, and the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map based on the incident light path. Extract the high-convex regions of the face from the background image, calculate the chromaticity gradient rate of each pixel in the high-convex regions, and construct the facial undulation gradient of the high-convex regions based on the chromaticity gradient rate. The ambient light intensity sequence is analyzed to obtain the phase delay of the ambient light. The phase delay is fused with the facial undulation gradient to generate a light decay mapping table. The color difference compensation matrix of the initial facial image is calculated based on the light decay mapping table. Calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map, and perform optical phase synchronization on the color difference compensation matrix based on the spectral phase difference to obtain the target correction matrix of the initial facial image; Using the target correction matrix as the reflectance benchmark for the facial background image, pixel spectral reconstruction is performed on the initial facial image, and standard light source rendering is applied to output the color-corrected target facial image.

[0007] Optionally, based on the spectral scattering difference between the epidermis and dermis, the initial facial image is decomposed into multiple dimensions to separate the facial background image and the illumination interference image of the initial facial image, including: Obtain the pixel grayscale values ​​corresponding to different spectral bands in the initial facial image; The set of pixel gray values ​​corresponding to spectral bands with wavelengths greater than a set wavelength threshold is defined as the epidermal scattering and transmission set, and the set of pixel gray values ​​corresponding to spectral bands with wavelengths less than or equal to a set wavelength threshold is defined as the dermal diffuse reflection set. Calculate the gray-level ratio of the epidermal scattering and transmission set and the dermal diffuse reflection set at the same spatial coordinates for each pixel, and use the gray-level ratio of each pixel as the spectral scattering difference. Traverse the initial facial image according to spatial coordinates, extract and summarize the pixels with corresponding spectral scattering differences less than the set stripping threshold into a facial background image, and summarize the remaining pixels into an illumination interference image.

[0008] Optionally, based on the luminous flux characteristics of the ambient light intensity sequence, the illumination interference map is analyzed, and the incident light path of the illumination interference map is extracted, including: Find the local maximum point corresponding to the light intensity value in the ambient light intensity sequence, and locate the spatial coordinates of the local maximum point as the emission endpoint of the ambient light. Extract continuous pixel regions in the illumination interference map whose brightness values ​​exceed a set brightness threshold, calculate the geometric center coordinates of the continuous pixel regions, and use them as the endpoints of the light-receiving projection. Obtain the normal perpendicular vector of the facial region where the light-receiving projection endpoint is located; The initial ray direction is obtained by connecting the transmitting end and the receiving projection end, and the initial ray direction is vector-projected along the direction of the normal perpendicular vector to obtain the incident light path of the illumination interference map.

[0009] Optionally, based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map, including: Calculate the angle between the incident light path and the normal perpendicular vector at the endpoint of the light-receiving projection, and use it as the incident deflection angle; Based on the physical law of light refraction on different media surfaces, calculate the surface reflection ratio and light transmission ratio of the light corresponding to the incident deflection angle. The sum of the surface reflection ratio and the light transmission ratio is one. Multiply the brightness value of each pixel in the illumination interference map by the surface reflection ratio of the light to obtain the surface reflection map; Multiply the brightness value of each pixel in the illumination interference map by the light transmission ratio to obtain the subcutaneous scattering map.

[0010] Optionally, the high-convex region of the face is extracted from the background image of the face, and the chromaticity gradient rate of each pixel within the high-convex region of the face is calculated, including: In the background image of the face, the pixel brightness is compared line by line to find the local brightness extreme point. With the local brightness extreme point as the center, the pixels are expanded outward layer by layer until the brightness difference between two adjacent layers of pixels reaches the decay inflection point from positive to negative. The expanded and covered pixel set is then enveloped as the high convex area of ​​the face. Within the high-convex area of ​​the face, obtain the color channel values ​​of the target pixel and its adjacent pixels; Calculate the difference in color channel values ​​between the target pixel and its neighboring pixels in the corresponding color channel. Use the ratio of this difference to the linear distance between the target pixel and its neighboring pixels as the chromaticity gradient rate of the target pixel.

[0011] Optionally, a facial undulation gradient is constructed based on the chromaticity gradient rate, including: Establish a direct proportional mapping function between the chromaticity gradient rate and the change in spatial height; By substituting the chromaticity gradient rate of each pixel in the high convex area of ​​the face into the proportional mapping function, the height drop of each pixel relative to the local brightness extreme point is calculated. By binding and associating the planar position of all pixels in the high convex area of ​​the face with the corresponding height drop in the two-dimensional coordinate system, the facial undulation gradient of the high convex area is obtained.

[0012] Optionally, the ambient light intensity sequence is analyzed to obtain the phase lag of the ambient light. The phase lag is then fused with the facial undulation gradient to generate a light decay mapping table. Based on the light decay mapping table, the color difference compensation matrix of the initial facial image is calculated, including: The time span between two adjacent light intensity troughs in the ambient light intensity sequence is extracted as the emission period of the ambient light. The phase delay of the ambient light reaching the face is derived based on the emission period and the light wave propagation speed. Multiply the height drop of each pixel in the facial undulation gradient by the phase delay to obtain the light wave propagation loss value at each pixel, and record the correspondence between the light wave propagation loss value and the two-dimensional plane coordinates of the pixel as a light decay mapping table. Find the light wave propagation loss value corresponding to each pixel position in the initial facial image in the light decay mapping table, calculate the reciprocal of the light wave propagation loss value, arrange the reciprocals corresponding to all pixel positions according to the pixel array of the initial facial image, and obtain the color difference compensation matrix of the initial facial image.

[0013] Optionally, the spectral phase difference between the surface reflection map and the subcutaneous scattering map is calculated, and the color difference compensation matrix is ​​synchronized with the light phase based on the spectral phase difference to obtain the target correction matrix of the initial facial image, including: The times when the light intensity peaks in the same spectral band are counted in the surface reflection map and the subcutaneous scattering map respectively. The peak time of the surface reflection map is subtracted from the peak time of the subcutaneous scattering map, and the absolute value of the difference is taken as the spectral phase difference. The numerical proportion of the spectral phase difference to the entire ambient light emission period is calculated and used as the phase lag coefficient. Multiply the values ​​of each element in the color difference compensation matrix by the phase lag coefficient, and replace the original matrix with the matrix corresponding to the product result to obtain the target correction matrix for the initial facial image.

[0014] Optionally, using the target correction matrix as the reflectance reference for the facial background image, pixel spectral reconstruction is performed on the initial facial image, and standard light source rendering is applied to output a color-corrected target facial image, including: The original color channel values ​​of each pixel in the initial facial image are multiplied with the corresponding element values ​​in the target correction matrix. The product is then used to overwrite the original color channel values ​​to complete pixel spectral reconstruction and obtain the transitional facial image. Obtain the single constant bias term corresponding to the preset uniform white light source; A single constant bias term is added to the light intensity value of each pixel in the transition face image, thereby increasing the overall light intensity value while keeping the color channel ratio unchanged, and outputting a color-corrected target face image.

[0015] A second aspect of the present invention provides a face recognition image color correction system based on multispectral ambient light detection, comprising: The acquisition module is used to simultaneously acquire initial facial images and ambient light intensity sequences under multispectral environments; The decomposition module is used to perform multidimensional decomposition on the initial facial image based on the spectral scattering difference between the epidermis and dermis, separating the facial background image and the illumination interference image of the initial facial image. The analysis module is used to analyze the illumination interference map based on the luminous flux characteristics of the ambient light intensity sequence and extract the incident light path of the illumination interference map; based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map; The module is used to extract the high-convex regions of the face from the background image and calculate the chromaticity gradient rate of each pixel in the high-convex regions; based on the chromaticity gradient rate, the facial undulation gradient of the high-convex regions is constructed. The calculation module is used to parse the ambient light intensity sequence and obtain the phase delay of the ambient light; it fuses the phase delay with the facial undulation gradient to generate a light decay mapping table, and calculates the color difference compensation matrix of the initial facial image based on the light decay mapping table. The synchronization module is used to calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map; based on the spectral phase difference, the color difference compensation matrix is ​​optically phase synchronized to obtain the target correction matrix of the initial facial image; The reconstruction module is used to perform pixel spectral reconstruction on the initial facial image using the target correction matrix as the reflectance reference of the facial background image, and to apply standard light source rendering to output the color-corrected target facial image.

[0016] As can be seen from the above technical solutions, the present invention has the following advantages: This invention performs multidimensional decomposition of facial images based on the spectral scattering differences between the epidermis and dermis, accurately separating the facial background color information from illumination interference information, avoiding correction biases caused by the mixing of skin color and illumination in traditional methods. By analyzing the luminous flux characteristics of the ambient light intensity sequence to extract the incident light path and decoupling the illumination interference map into a surface reflection map and a subcutaneous scattering map, it achieves refined decomposition and analysis of complex multispectral light sources. By extracting high-convex regions from the facial background map and calculating the chromaticity gradient rate to construct a facial undulation gradient, the undulation features of the three-dimensional facial structure are incorporated into the spatial modulation calculation of illumination attenuation, overcoming the shortcomings of existing methods that ignore the influence of facial three-dimensional morphology on illumination. By fusing the phase delay of ambient light with the facial undulation gradient to generate an illumination attenuation mapping table and calculating the color difference compensation matrix, pixel-level accurate compensation for illumination attenuation is achieved. By calculating the spectral phase difference between the surface reflection map and the subcutaneous scattering map to synchronize the light phase of the color difference compensation matrix, the temporal consistency between the correction process and actual illumination is ensured. Finally, the initial facial image is reconstructed using the target correction matrix and rendered with a standard light source to output a high-precision color-corrected target facial image, which significantly improves the accuracy and robustness of facial recognition under complex multispectral lighting conditions. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the overall steps of a face recognition image color correction method based on multispectral ambient light detection, as provided in an embodiment of the present invention. Figure 2 This is a schematic diagram of a multilayer optical scattering structure of skin provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the multidimensional decomposition process of facial images provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of the incident light path extraction and illumination interference diagram decoupling process provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of the process for generating the optical attenuation mapping table and calculating the color difference compensation matrix provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the multidimensional decomposition result of a facial image in an underground parking lot entrance scene provided in an embodiment of the present invention; Figure 7This is a schematic diagram illustrating the before-and-after facial color comparison and CIEDE2000 color difference evaluation provided in an embodiment of the present invention. Figure 8 This is a structural block diagram of a multispectral ambient light detection facial recognition image color correction system provided in an embodiment of the present invention. Detailed Implementation

[0019] This invention provides a method and system for color correction of facial recognition images based on multispectral ambient light detection. It solves the technical problem in existing facial image color correction methods that fail to fully utilize the multi-layered optical properties of the skin, the three-dimensional undulation structure of the face, and the temporal phase characteristics of multispectral ambient light for comprehensive correction. This invention is applicable to the image preprocessing stage of facial recognition systems and can also be used in applications requiring high-fidelity facial color reproduction, such as video conferencing and telemedicine.

[0020] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] Please see Figure 1 , Figure 1 This is a flowchart illustrating the overall steps of a face recognition image color correction method based on multispectral ambient light detection, provided in an embodiment of the present invention. The method provided by the present invention includes the following steps: S101. Simultaneously acquire initial facial images and ambient light intensity sequences under multispectral conditions.

[0022] A multispectral environment refers to a scene where two or more light sources with different spectral characteristics are present simultaneously. For example, in an office setting, fluorescent lights on the ceiling produce periodic flickering light at a frequency of approximately 50Hz, natural light from windows provides continuous broadband illumination, and desk lamps produce localized warm-toned light. These different light sources have varying spectral energy distributions, flicker frequencies, and spatial projection directions, collectively creating a complex multispectral illumination environment.

[0023] The initial facial image refers to the raw facial image captured directly by a multispectral camera or an image sensor supporting multi-band acquisition under the aforementioned multispectral environment, without any color correction processing. This image contains pixel information in the visible light band and the near-infrared band. The color channel values ​​of each pixel are affected by the combined influence of multiple light sources in the environment, resulting in color characteristics that deviate from the true skin tone.

[0024] Ambient light intensity sequence refers to time-series data obtained by continuously sampling ambient light intensity using a spectral sensor array triggered synchronously with the camera while acquiring the initial facial image. This sequence records the changes in ambient light intensity at different spatial locations and wavelengths over a period of time near the acquisition moment. The sampling frequency of the ambient light intensity sequence must meet the requirements of the Nyquist sampling theorem, i.e., it must be no less than twice the flicker frequency of the ambient light source to ensure complete capture of the periodic changes in the light source. For example, when there is a 50Hz AC-driven fluorescent lamp in the environment, the sampling frequency should be no less than 100Hz, and the specific frequency can be set according to actual needs.

[0025] Synchronous acquisition refers to the strict alignment of the exposure time window of the initial facial image with the sampling time window of the ambient light intensity sequence on the time axis, ensuring that both types of data reflect the illumination state at the same moment. At the hardware level, this can be achieved by simultaneously activating camera exposure and spectral sensor sampling via an external trigger signal. The significance of synchronous acquisition lies in providing a time-consistent data foundation for subsequent correction calculations based on ambient light phase characteristics, avoiding illumination state misalignment caused by acquisition time deviations.

[0026] This step acquires time-aligned facial image data and ambient lighting data through a synchronous acquisition mechanism, providing complete and consistent raw data support for subsequent multidimensional decomposition, incident light path analysis, and phase delay calculation, thus ensuring the reliability of the data foundation for the entire color correction process.

[0027] After simultaneously acquiring the initial facial image and ambient light intensity sequence under a multispectral environment, the resulting initial facial image contains both the inherent color information of the facial skin and illumination interference information from external multispectral light sources. Directly performing color correction on the original image, which contains both types of information, would fail to distinguish which color deviations originate from external illumination and which are due to the skin's inherent color characteristics, leading to insufficient correction accuracy. Therefore, it is necessary to separate the background color information from the illumination interference information in the facial image to allow for subsequent refined analysis and compensation of the illumination interference components.

[0028] S102. Based on the spectral scattering difference between the epidermis and dermis, the initial facial image is decomposed in multiple dimensions to separate the facial background image and the illumination interference image of the initial facial image.

[0029] Please see Figure 2 , Figure 2 This is a schematic diagram of a multilayer optical scattering structure of skin provided in an embodiment of the present invention.

[0030] like Figure 2As shown, human skin tissue has a typical layered optical structure. The epidermis, the outermost layer of the skin, is approximately 0.05 to 0.15 mm thick and mainly contains melanin granules, exhibiting strong absorption and scattering of short-wavelength visible light. The dermis, located beneath the epidermis, is approximately 1 to 4 mm thick and contains components such as hemoglobin and collagen fibers, exhibiting strong diffuse reflection characteristics of long-wavelength and near-infrared visible light. This layered optical difference means that light of different wavelengths undergoes different scattering and transmission paths within the skin: short-wavelength light is mainly scattered and reflected back within the epidermis, while long-wavelength light can penetrate the epidermis and reach the dermis before being diffusely reflected back. This invention utilizes this difference in optical properties to distinguish between facial background information and external light interference information by comparing the response intensity of different wavelengths.

[0031] Spectral scattering difference refers to the ratio of the grayscale response of the long-wavelength band to that of the short-wavelength band at the same pixel spatial coordinates. When a pixel location is only affected by the optical properties of the skin itself and there is no significant external light interference, this ratio tends to be stable and within a small range. However, when there is strong external light directly illuminating the area, the reflection and scattering of light on the skin surface will cause abnormal shifts in the grayscale values ​​of specific bands, resulting in a significant deviation of this ratio from the normal range.

[0032] The facial background image refers to the image component separated from the initial facial image, which mainly reflects the inherent color and texture information of the facial skin. The illumination interference image refers to the image component separated from the initial facial image, which mainly contains information about the effects of ambient light on the face, such as highlights, shadows, and color casts.

[0033] Please see Figure 3 , Figure 3 This is a schematic diagram of the multidimensional decomposition process of facial images provided in an embodiment of the present invention; as shown below. Figure 3 As shown, S102 specifically includes: Obtain the pixel grayscale values ​​corresponding to different spectral bands in the initial facial image; determine the set of pixel grayscale values ​​corresponding to spectral bands with wavelengths greater than a set wavelength threshold as the epidermal scattering and transmission set, and determine the set of pixel grayscale values ​​corresponding to spectral bands with wavelengths less than or equal to the set wavelength threshold as the dermal diffuse reflection set; calculate the grayscale ratio of the pixels in the epidermal scattering and transmission set and the dermal diffuse reflection set at the same spatial coordinates, and use the grayscale ratio of each pixel as the spectral scattering difference; traverse the initial facial image according to the spatial coordinates, extract and summarize the pixels with corresponding spectral scattering differences less than the set stripping threshold as the facial background map, and summarize the remaining pixels as the illumination interference map.

[0034] The specific implementation process is as follows: First, obtain the pixel grayscale values ​​corresponding to different spectral bands in the initial facial image. In a multispectral image, each pixel has grayscale values ​​for multiple spectral channels, each corresponding to the spectral response intensity of a different wavelength range.

[0035] Next, a wavelength threshold is set to divide the multispectral grayscale data of the pixels into two groups. The specific value of the wavelength threshold can be determined according to the band configuration of the multispectral camera used and the actual scene requirements; for example, it can be set to 650nm, but the specific value can be set according to actual needs. The set of pixel grayscale values ​​corresponding to spectral bands with wavelengths greater than this threshold is defined as the epidermal scattering and transmission set. This name is used because long-wavelength light can penetrate the epidermis to reach the dermis and return after diffuse reflection in the dermis. In this propagation process, the epidermis mainly acts as a medium for scattering and transmission. The set of pixel grayscale values ​​corresponding to spectral bands with wavelengths less than or equal to this threshold is defined as the dermal diffuse reflection set. This is because short-wavelength light is strongly scattered in the epidermis, and the energy reaching the dermis is limited. The contribution of dermal diffuse reflection in the returned grayscale signal is dominant.

[0036] Next, the spectral scattering difference is calculated pixel by pixel. For a pixel with a certain spatial coordinate, the average gray value of each channel in the epidermal scattering transmission set and the average gray value of each channel in the dermal diffuse reflection set are taken respectively. The ratio of the two is the spectral scattering difference of that pixel location.

[0037] Finally, the entire initial facial image is traversed pixel by pixel according to spatial coordinates, and the spectral scattering difference of each pixel is compared with a set stripping threshold. The stripping threshold can be calibrated and determined according to the actual acquisition environment; for example, it can be set to 1.2, and the specific value can be set according to actual needs. When the spectral scattering difference of a pixel is less than the stripping threshold, it indicates that the grayscale response ratio of the two bands at that pixel is within the normal range, and its grayscale value is mainly determined by the inherent optical characteristics of the skin, with less interference from external lighting. This value is extracted and summarized as part of the facial background image. When the spectral scattering difference of a pixel is greater than or equal to the stripping threshold, it indicates that the grayscale response at that pixel is significantly affected by external lighting, and this value is classified into the lighting interference map.

[0038] This step achieves precise separation of facial images by utilizing the scattering differences of the multi-layered optical structure of skin tissue, effectively decoupling the background color information of the face from the illumination interference information. Compared with traditional methods such as Retinex, which only separate based on brightness or frequency, this step performs layered decomposition based on the optical physics mechanism of the skin, resulting in higher separation accuracy and laying a reliable foundation for subsequent refined analysis and color compensation of illumination interference images.

[0039] After the multidimensional decomposition in S102, the illumination interference map has concentrated information on the various illumination effects of ambient light on the face. However, the brightness distribution in the illumination interference map is actually a superposition of optical components with two different physical mechanisms: one part is the surface highlight component formed by direct specular reflection of light on the skin surface, whose color characteristics are mainly determined by the spectrum of the light source itself; the other part is the diffuse component formed by light penetrating the skin surface and undergoing multiple scatterings in the subcutaneous tissue before returning, whose color characteristics are modulated by both the spectrum of the light source and the optical properties of the subcutaneous tissue. These two components have different mechanisms of influence on facial color and need to be treated differently in subsequent correction. Therefore, the next step is to extract the incident light path and, based on this, further decouple the illumination interference map into a surface reflection map and a subcutaneous scattering map.

[0040] S103. Based on the luminous flux characteristics of the ambient light intensity sequence, analyze the illumination interference map and extract the incident light path of the illumination interference map; based on the incident light path, decouple the illumination interference map into a surface reflection map and a subcutaneous scattering map.

[0041] The core task of this step is to determine the spatial propagation path of light from the light source to the face, and based on the relationship between the incident angle of the light and the facial surface along this path, to quantitatively separate the surface reflection component and the subcutaneous scattering component mixed in the illumination interference diagram.

[0042] The incident light path refers to the spatial propagation path of ambient light from the emitting endpoint of the light source to the illuminated area of ​​the face. It reflects the actual incident direction of the light relative to the local surface of the face. The emitting endpoint refers to the spatial coordinate position corresponding to the local maximum value of the illumination radiation intensity in the spatial sampling position of the ambient light intensity sequence, representing the equivalent spatial position of the light source. The illuminated projection endpoint refers to the geometric center position of the bright area directly illuminated in the illumination interference diagram, representing the center position of the light spot formed after the light is projected onto the face. The normal perpendicular vector is the unit direction vector perpendicular to the facial surface, reflecting the orientation of the local surface of the face.

[0043] Furthermore, the specific implementation process of S103 is divided into two processes: incident light path extraction and illumination interference map decoupling, such as... Figure 4 As shown, it specifically includes: In the ambient light intensity sequence, find the local maximum point corresponding to the light intensity value, and locate the spatial coordinates of the local maximum point as the emission endpoint of the ambient light; extract the continuous pixel region in the illumination interference map where the brightness value exceeds the set brightness threshold, calculate the geometric center coordinates of the continuous pixel region, and use it as the light receiving projection endpoint; obtain the normal perpendicular vector of the facial region where the light receiving projection endpoint is located; connect the emission endpoint and the light receiving projection endpoint to obtain the initial light direction, and vector project the initial light direction along the direction of the normal perpendicular vector to obtain the incident light path of the illumination interference map.

[0044] Calculate the angle between the incident light path and the normal perpendicular vector at the endpoint of the projected light, and use it as the incident deflection angle. Based on the physical laws of light refraction on different media surfaces, calculate the surface reflection ratio and light transmission ratio corresponding to the incident deflection angle. The surface reflection ratio and light transmission ratio are summed to one. Multiply the brightness value of each pixel in the illumination interference map by the surface reflection ratio to obtain the surface reflection map. Multiply the brightness value of each pixel in the illumination interference map by the light transmission ratio to obtain the subcutaneous scattering map.

[0045] During the incident light path extraction process, local extremum analysis is first performed on the light intensity values ​​at each spatial sampling location in the ambient light intensity sequence. A sliding window extremum detection method is employed, comparing the light intensity value of each spatial sampling point within its preset neighborhood to determine if it is the maximum value. When the light intensity value of a sampling point is greater than the light intensity values ​​of all other sampling points in its neighborhood, that point is determined to be a local maximum, and its spatial coordinates are located as the emission endpoint of the ambient light.

[0046] Next, highlight regions are extracted from the illumination interference map. The average brightness value of all pixels in the illumination interference map is multiplied by a preset scaling factor to obtain the brightness threshold, for example, this scaling factor can be set to 1.5, and the specific value can be set according to actual needs. A connected component labeling algorithm is used to perform connectivity analysis on pixels in the illumination interference map whose brightness values ​​exceed the brightness threshold, and pixels exceeding the threshold with spatial adjacency are grouped into the same continuous pixel region. For each continuous pixel region, the arithmetic mean of the spatial coordinates of all its pixels is calculated, which is the geometric center coordinate of the region and is used as the endpoint of the illuminated projection.

[0047] Then, the normal perpendicular vector of the facial region where the illuminated projection endpoint is located is obtained. In this embodiment, an approximate estimation method based on image gray-level gradient is adopted: the gray-level gradient of the facial background image is calculated in the neighborhood of the illuminated projection endpoint along the horizontal and vertical directions, respectively. The horizontal gradient component and the vertical gradient component are extracted using the Sobel operator, and then the normal direction of the facial surface at that point is deduced through the cross product operation of the gradient directions.

[0048] By spatially connecting the emitting endpoint and the receiving projection endpoint, the initial ray direction is obtained. Since the facial surface has three-dimensional curved features rather than being a plane, the actual incident direction of the light reaching the face needs to consider the normal direction of the local facial surface. Therefore, a vector projection operation is performed on the initial ray direction along the direction perpendicular to the normal vector, that is, the projection component of the direction vector of the initial ray direction onto the perpendicular to the normal vector is calculated, resulting in the corrected incident light path. This incident light path accurately reflects the actual incident direction of the light relative to the local facial surface.

[0049] During the decoupling process of the illumination interference diagram, the spatial angle between the incident light path and the normal perpendicular vector at the endpoint of the illuminated projection is calculated, and this angle is defined as the incident deflection angle. The incident deflection angle reflects the degree to which the light deviates from the normal direction of the facial surface: when the incident deflection angle is small, that is, the light is close to perpendicular to the facial surface, the proportion of light energy penetrating the skin and entering the subcutaneous tissue is high, and the proportion of light energy reflected on the surface is low; when the incident deflection angle is large, that is, the light is incident on the facial surface at a large angle, the proportion of light energy reflected on the surface is high, and the proportion of light energy penetrating into the subcutaneous tissue is low.

[0050] The surface reflection and transmission ratios of light rays corresponding to the incident deflection angle are calculated using Fresnel's law of reflection. Fresnel's law of reflection describes the energy distribution relationship between reflection and transmission at the interface when light travels from one medium to another. For unpolarized light entering the skin surface from air, the refractive index of air is taken as 1.0, and the refractive index of skin as 1.45. According to Fresnel's equations, the refraction angle is first calculated using Snell's law of refraction. Then, the reflectance of the s-polarized component and the p-polarized component are calculated separately, and the arithmetic mean of the two is taken as the total surface reflection ratio R. The transmission ratio T is equal to 1 minus R, that is, the sum of R and T is always 1.

[0051] Multiplying the brightness value of each pixel in the illumination interference map by the surface reflection ratio R yields the surface reflection map. The value of each pixel in the surface reflection map represents the illumination intensity at that location caused by specular reflection from the surface. Multiplying the brightness value of each pixel in the illumination interference map by the light transmission ratio T yields the subcutaneous scattering map. The value of each pixel in the subcutaneous scattering map represents the illumination intensity at that location caused by light penetrating the skin, being scattered multiple times by the subcutaneous tissue, and then returning.

[0052] This step, by accurately extracting the incident light path and combining it with Fresnel's law of reflection for quantitative decoupling, successfully separates two types of optical components with different physical mechanisms in the illumination interference diagram. This provides a clear data foundation for subsequent differentiated processing of surface reflection and subcutaneous scattering, avoiding insufficient correction accuracy caused by the mixed processing of the two types of components.

[0053] Having completed the aforementioned steps, we now have a facial background map, as well as surface reflection and subcutaneous scattering maps decoupled from the illumination interference map. However, the face is not a planar structure; areas such as the bridge of the nose, cheekbones, and forehead are relatively convex, while areas such as the jawline and eye sockets are relatively concave. This three-dimensional undulating structure means that the distance from different locations on the face to the light source varies. The tops of convex areas closer to the light source receive greater illumination intensity, while the sides of convex areas and concave areas farther away receive relatively weaker illumination intensity. If we do not consider the spatial modulation effect of the three-dimensional undulating structure of the face on illumination attenuation, simply using a uniform compensation coefficient to color correct the entire image will not eliminate the spatially uneven color cast caused by facial undulations. Therefore, it is necessary to extract the features of highly convex areas of the face from the facial background map and construct the facial undulation gradient through chromaticity gradient analysis to provide three-dimensional structural information for subsequent pixel-level accurate compensation.

[0054] S104. Extract the high-convex region of the face from the background image of the face, and calculate the chromaticity gradient rate of each pixel in the high-convex region of the face; construct the facial undulation gradient of the high-convex region of the face based on the chromaticity gradient rate.

[0055] The task of this step is to locate the protruding structural regions of the face from the background image, quantify the rate of color gradation along the spatial direction, and convert the color gradation information on the two-dimensional image into height field data that reflects the three-dimensional undulation of the face.

[0056] A facial convexity region refers to a localized area in the base image of the face where brightness decreases from the center outwards due to the protrusion of the three-dimensional facial structure, corresponding to the actual protruding structural parts of the face. Chromaticity gradient rate refers to the rate of change of color channel values ​​between adjacent pixels along the spatial direction within the facial convexity region, reflecting the speed of color gradation in that direction. Facial undulation gradient is a data structure describing the three-dimensional undulation morphology of the facial convexity region. It maps color gradation information on a two-dimensional plane to height change information in three-dimensional space, forming an approximate three-dimensional height field for the facial convexity region.

[0057] Furthermore, the specific implementation process of S104 includes three steps: extraction of high-convex facial regions, calculation of chromaticity gradient rate, and construction of facial undulation gradient. Specifically, it includes: In the background image of the face, the brightness of pixels is compared line by line to find the local brightness extreme point. With the local brightness extreme point as the center, the pixels are expanded outward layer by layer until the brightness difference between two adjacent layers of pixels reaches the attenuation inflection point from positive to negative. The expanded and covered pixel set is then enveloped as the high convex area of ​​the face.

[0058] Within the high-convex area of ​​the face, the color channel values ​​of the target pixel and its neighboring pixels are obtained; the difference between the color channel values ​​of the target pixel and its neighboring pixels in the corresponding color channel is calculated, and the ratio of the difference to the linear distance between the target pixel and its neighboring pixels is used as the chromaticity gradient rate of the corresponding target pixel.

[0059] Establish a direct proportional mapping function between the chromaticity gradient rate and the spatial height change; substitute the chromaticity gradient rate of each pixel in the high convex area of ​​the face into the direct proportional mapping function to calculate the height drop of each pixel relative to the local brightness extreme point; bind and associate the planar position of all pixels in the high convex area of ​​the face with the corresponding height drop one by one to obtain the facial undulation gradient of the high convex area of ​​the face.

[0060] During the extraction of high-protrusion areas on the face, the pixel brightness values ​​are first scanned line by line in the background image of the face, and the local extremum detection method is used to find the local brightness extremum points.

[0061] Specifically, for each pixel in the facial background image, the brightness value of that pixel is compared with the brightness values ​​of its four adjacent pixels (above, below, left, and right). If the brightness value of that pixel is greater than or equal to the brightness values ​​of all its adjacent pixels, it is marked as a candidate extremum point. Non-maximum suppression is applied to all candidate extremum points, and the pixel with the highest brightness in its neighborhood is retained as the final local brightness extremum point. The local brightness extremum point corresponds to the top position of the convex facial structure because, under roughly uniform lighting conditions, the top of the protrusion is closest to the light source and its surface normal direction is closest to the direction of light incidence, thus resulting in the highest brightness.

[0062] Then, taking each local brightness extreme point as the center, pixels are added to the periphery layer by layer using a layer-by-layer expansion strategy.

[0063] Specifically, the extreme point itself is first considered as the first layer, then all pixels directly adjacent to the first layer are considered as the second layer, and so on, expanding outward layer by layer. After each layer expansion is completed, the difference between the average brightness value of all pixels in the current layer and the average brightness value of all pixels in the previous layer is calculated. On the sloping surface of the facial protrusion, due to the decreasing light reception and color gradient leading to a decrease in brightness with increasing distance from the convex apex, this brightness difference shows a positive value and gradually decreases. When the difference reaches an inflection point from positive to negative, that is, when the average brightness of the current layer is no longer lower than the previous layer but begins to rise, it indicates that the expansion has reached the boundary between the protrusion and the surrounding flat area or other structures, and the expansion stops at this point. The set of all pixels covered from the center of the extreme point to this attenuation inflection point is encapsulated as a facial high-convex region. By performing the above expansion process on all local brightness extreme points in the facial background image, all high-convex regions in the face can be extracted.

[0064] During the chromaticity gradient rate calculation process, the chromaticity gradient rate is calculated for each pixel in the high convex area of ​​the face.

[0065] For any target pixel within the region, first obtain the RGB color channel values ​​of the target pixel, as well as the RGB color channel values ​​of its spatially adjacent pixels. Then, calculate the difference between the target pixel and its neighboring pixels for each color channel. For example, if the target pixel value is 180 and the neighboring pixel value is 172 in the R channel, the difference for that channel is 8. Divide this difference by the linear distance between the target pixel and its neighboring pixels to obtain the chromaticity gradient rate for that channel. When adjacent pixels are arranged horizontally or vertically, the linear distance is set to 1 in units of one pixel spacing. Perform a vector modulo operation on the chromaticity gradient rates of the three color channels, i.e., take the square root of the sum of the squares of the three channel chromaticity gradient rates, to obtain the overall chromaticity gradient rate of the target pixel. Perform the above calculation on all pixels within the high-convex area of ​​the face to obtain the chromaticity gradient rate distribution data for the entire region.

[0066] In the process of constructing facial contour gradients, a direct proportional mapping function is established between the chromaticity gradation rate and the change in spatial height. The physical basis of this direct proportional mapping function is that, under approximately uniform lighting conditions, the degree of color gradation from top to side on a facial protrusion is directly proportional to the rate of height decrease in that direction; that is, the more dramatic the chromaticity gradation, the faster the height decreases in three-dimensional space. The form of the mapping function is as follows: Where h is the height reduction, g is the chromaticity gradient rate, and k is the proportionality coefficient. The proportionality coefficient k can be obtained through prior facial size information, for example, by performing linear regression calibration based on the correspondence between the statistical prior value of the typical height of the bridge of the nose and the measured data of the chromaticity gradient rate of the bridge of the nose area. The specific value can be set according to actual needs.

[0067] By substituting the chromaticity gradient rate of each pixel within the high-convex area of ​​the face into a proportional mapping function, the height drop of each pixel relative to the local brightness extremum is calculated. At the extremum at the top of the convexity, the chromaticity gradient rate is approximately zero, and the height drop is also zero. Pixels farther from the top of the convexity and with a higher chromaticity gradient have a greater height drop. The planar positions of all pixels within the high-convex area of ​​the face in a two-dimensional coordinate system are then linked one-to-one with their corresponding height drops, forming a data structure containing three dimensions: horizontal coordinate, vertical coordinate, and height drop. This structure is then combined to obtain the facial undulation gradient of the high-convex area.

[0068] This step extracts convex structural features from the facial background image and constructs a three-dimensional height field, integrating the three-dimensional morphological information of the face into the subsequent illumination attenuation compensation calculation. This overcomes the shortcomings of existing methods that treat the face as a plane for uniform correction, and provides key three-dimensional structural data support for achieving pixel-level accurate color compensation.

[0069] After constructing the facial undulation gradient in S104, approximate three-dimensional height field data of the high-convex facial region has been obtained. Simultaneously, the ambient light intensity sequence acquired in S101 contains temporal variation information of the ambient light source. Ambient light, especially artificial light sources driven by alternating current, exhibits periodic flickering characteristics, with light intensity fluctuating periodically over time. As light waves propagate from the light source to different locations on the face, the differences in propagation distance caused by the three-dimensional undulations of the face introduce varying degrees of phase shift and light energy attenuation. Without combining the temporal phase characteristics of ambient light with the three-dimensional undulation structure information of the face for joint modeling, it is impossible to accurately quantify the actual degree of light attenuation at each pixel location. Therefore, it is necessary to extract the phase delay of ambient light from the ambient light intensity sequence and fuse it with the facial undulation gradient to generate a pixel-level light attenuation mapping table, thereby calculating the color difference compensation matrix.

[0070] S105. Analyze the ambient light intensity sequence to obtain the phase delay of the ambient light; fuse the phase delay with the facial undulation gradient to generate a light decay mapping table, and calculate the color difference compensation matrix of the initial facial image based on the light decay mapping table.

[0071] This step combines the temporal periodic characteristics of ambient light with the three-dimensional undulation structure characteristics of the face to accurately calculate the quantized value of illumination attenuation at each pixel location on the face, providing pixel-level precision for subsequent color compensation.

[0072] The luminous cycle of ambient light refers to the time required for an ambient light source to complete one full cycle of brightness change. For artificial light sources driven by alternating current, the luminous cycle is equal to the reciprocal of the driving frequency. For example, in a region where the alternating current frequency is 50Hz, the corresponding fluorescent lamp luminous cycle is 0.02 seconds, which can be set according to actual needs. The phase delay of ambient light refers to the phase shift of the light wave as it propagates from the light source to the reference plane on the face surface due to the propagation distance. Its magnitude is directly proportional to the propagation distance and inversely proportional to the wavelength of the light wave.

[0073] The light decay map is a two-dimensional data table aligned with the initial facial image space, where each element records the light wave propagation loss value at the corresponding pixel location. The chromatic aberration compensation matrix is ​​a two-dimensional matrix formed by arranging the propagation loss values ​​of each pixel in the light decay map after performing a reciprocal operation, where each element value represents the color compensation gain coefficient that needs to be applied at the corresponding pixel location.

[0074] Please see Figure 5 , Figure 5 This is a schematic diagram illustrating the process of generating the optical attenuation mapping table and calculating the chromatic aberration compensation matrix according to an embodiment of the present invention; as shown below. Figure 5 As shown, S105 specifically includes: The time span between two adjacent light intensity troughs in the ambient light intensity sequence is extracted as the emission period of the ambient light. Based on the emission period and the light wave propagation speed, the phase delay of the ambient light reaching the face is derived. The height drop of each pixel in the facial undulation gradient is multiplied by the phase delay to obtain the light wave propagation loss value at each pixel. The correspondence between the light wave propagation loss value and the two-dimensional plane coordinates of the pixel is recorded as a light decay mapping table. The light wave propagation loss value corresponding to each pixel position in the initial facial image is found in the light decay mapping table. The reciprocal of the light wave propagation loss value is calculated. The reciprocals corresponding to all pixel positions are arranged according to the pixel array of the initial facial image to obtain the color difference compensation matrix of the initial facial image.

[0075] The specific implementation process is as follows. First, the emission period is extracted from the ambient light intensity sequence. A light intensity trough refers to the moment when the light intensity in the ambient light intensity sequence reaches a local minimum. By performing temporal minimum detection on the ambient light intensity sequence, all moment points satisfying the local minimum condition are searched. Then, the time interval between two adjacent trough moment points is calculated; this interval is a complete emission period T. The average of multiple adjacent trough intervals is taken to improve the accuracy of the emission period estimation.

[0076] Then, based on the emission period T and the light propagation speed c, the phase delay of ambient light reaching the facial reference plane is derived. The light propagation speed c is taken as the speed of light, approximately 300 million meters per second. The propagation distance d from the light source to the face can be estimated by the spatial distance between the coordinates of the emission endpoint and the coordinates of the face center. The phase delay is equal to twice pi multiplied by the propagation distance d and then divided by the product of the light propagation speed c and the emission period T.

[0077] Multiplying the height drop of each pixel in the facial contour gradient by the phase delay yields the light wave propagation loss value at each pixel. The physical meaning of this calculation is: pixels at the top of the facial protrusions have zero height drop and minimal light wave propagation loss because they are closest to the light source; while pixels on the sides of the protrusions and in the recessed areas have larger height drops, requiring the light waves to travel a longer equivalent path to reach them, resulting in greater phase shift and energy attenuation during propagation, and thus greater light wave propagation loss values. The correspondence between the light wave propagation loss values ​​of each pixel and their two-dimensional planar coordinates is organized and recorded as a light attenuation mapping table.

[0078] The optical attenuation map table is used to find the light wave propagation loss value corresponding to each pixel position in the initial facial image, and the reciprocal of each loss value is taken. The purpose of the reciprocal operation is to convert the attenuation into a compensation value: the larger the attenuation of a pixel, the larger the reciprocal value, and the larger the corresponding compensation coefficient, so as to achieve accurate recovery of the attenuation in subsequent multiplication operations. The reciprocal values ​​corresponding to all pixel positions are arranged according to the pixel array of the initial facial image to obtain the color difference compensation matrix of the initial facial image.

[0079] This step generates a pixel-level precision light decay mapping table and color difference compensation matrix by jointly modeling the ambient light temporal features and facial 3D structural features. Compared with the traditional method of using global or regional uniform compensation coefficients, it can more accurately reflect the actual light attenuation differences at different locations on the face and significantly improve the spatial accuracy of color compensation.

[0080] The chromatic aberration compensation matrix obtained in S105 has already incorporated the spatial attenuation information of illumination caused by the three-dimensional undulations of the face. However, the surface reflection and subcutaneous scattering decoupled in S103 are formed after light propagates along different paths on the facial surface and in the subcutaneous tissue. There is a slight difference in the time it takes for the two to reach the image sensor, resulting in a phase difference in the spectral domain. If this phase difference is not corrected, directly using the chromatic aberration compensation matrix for color compensation may lead to a mismatch between the correction result and the temporal characteristics of the actual illumination, resulting in correction residuals. Therefore, it is necessary to calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map, and adjust the chromatic aberration compensation matrix accordingly to obtain the final target correction matrix.

[0081] S106. Calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map; based on the spectral phase difference, perform optical phase synchronization on the color difference compensation matrix to obtain the target correction matrix of the initial facial image.

[0082] The task of this step is to quantify the temporal offset difference between the two optical components, surface reflection and subcutaneous scattering, and to incorporate this difference information into the chromatic aberration compensation matrix for calibration.

[0083] Spectral phase difference refers to the absolute difference in the timing of the peak light intensity values ​​in the surface reflection map and the subcutaneous scattering map within the same spectral band, reflecting the degree of temporal shift between the two illumination components. The phase lag coefficient is the ratio of the spectral phase difference to the entire ambient light emission cycle, characterizing the proportion of time delay between subcutaneous scattered light and surface reflection within a complete emission cycle. The target correction matrix is ​​the final correction matrix after light phase synchronization adjustment, simultaneously considering illumination attenuation compensation in the spatial dimension and illumination phase synchronization in the temporal dimension; it is the core matrix used for final color correction.

[0084] Furthermore, S106 specifically includes: The times when the light intensity peaks in the same spectral band are counted in the surface reflection map and the subcutaneous scattering map respectively. The peak time of the surface reflection map is subtracted from the peak time of the subcutaneous scattering map, and the absolute value of the difference is taken as the spectral phase difference. The proportion of the spectral phase difference to the entire ambient light emission cycle is calculated as the phase lag coefficient. The element values ​​at each position in the color difference compensation matrix are multiplied by the phase lag coefficient, and the matrix corresponding to the product result replaces the original matrix as the target correction matrix of the initial facial image.

[0085] The specific implementation process is as follows: Temporal peak analysis is performed on the multispectral data of the surface reflectance map and the subcutaneous scattering map. For each spectral band, the moment when the light intensity value of that band reaches its maximum value in the surface reflectance map is recorded as the surface reflectance peak moment; similarly, the moment when the light intensity value of the same band reaches its maximum value in the subcutaneous scattering map is recorded as the subcutaneous scattering peak moment. For surface reflectance, since the path of light directly reflected from the skin surface is relatively short, the moment of its light intensity peak is usually very close to the moment of the light source radiation peak. For subcutaneous scattering, since light needs to penetrate the skin surface and undergo multiple scatterings in the dermis and other subcutaneous tissues before returning to the sensor, its propagation path is significantly longer than that of surface reflectance, therefore the peak moment is relatively delayed.

[0086] The peak time of surface reflection is subtracted from the peak time of subcutaneous scattering, and the absolute value of the difference is taken as the spectral phase difference. The spectral phase difference is calculated separately for multiple spectral bands and then averaged to improve the robustness of the estimation.

[0087] The phase lag coefficient is calculated by dividing the spectral phase difference by the emission period T of the entire ambient light. This coefficient reflects the degree of delay between subcutaneous scattered light and surface reflection within a complete illumination cycle.

[0088] Each element in the chromatic aberration compensation matrix is ​​multiplied by a phase lag coefficient. This multiplication scales and calibrates the compensation intensity of the chromatic aberration compensation matrix according to the actual illumination phase delay characteristics, ensuring the compensation amount matches the actual illumination temporal state. The matrix corresponding to the product is then used to replace the original chromatic aberration compensation matrix, serving as the target correction matrix for the initial facial image.

[0089] This step quantifies the temporal phase difference between surface reflection and subcutaneous scattering and incorporates it into the correction matrix, ensuring that the color compensation process remains consistent with the temporal characteristics of actual illumination. This avoids correction residuals caused by ignoring illumination phase differences and further improves correction accuracy.

[0090] S107. Using the target correction matrix as the reflectance reference of the facial background image, perform pixel spectral reconstruction on the initial facial image, apply standard light source rendering, and output the color-corrected target facial image.

[0091] This step is the final execution step in the entire color correction process. It applies the comprehensive compensation information contained in the target correction matrix to the initial facial image to complete the color correction and output the final result.

[0092] Reflectance benchmark refers to using the element values ​​of the target correction matrix as a reference standard for the reflectance of facial skin at each pixel location under ideal standard lighting conditions. Pixel spectral reconstruction refers to the process of correcting and adjusting the color channel values ​​of each pixel in the initial facial image using the target correction matrix. Transitional facial image refers to the intermediate result image after pixel spectral reconstruction but before standard light source rendering. Standard light source rendering refers to the process of adjusting the overall brightness of the transitional facial image to the desired level under standard light source illumination conditions.

[0093] Furthermore, S107 specifically includes: The original color channel values ​​of each pixel in the initial facial image are multiplied with the corresponding element values ​​in the target correction matrix. The product is then used to overwrite the original color channel values ​​to complete pixel spectral reconstruction and obtain the transitional facial image. A single constant bias term corresponding to a preset uniform white light source is obtained. The single constant bias term is added to the light intensity values ​​of each pixel in the transitional facial image to increase the overall light intensity value while keeping the color channel ratio unchanged. The color-corrected target facial image is then output.

[0094] The specific implementation process is as follows: The original color channel values ​​of each pixel in the initial facial image are multiplied pixel-by-pixel and channel-by-channel with the corresponding element values ​​in the target correction matrix. For the three RGB color channels, the original value of each channel is multiplied by the element value at the same position in the target correction matrix. The effect of this multiplication is: for pixels with severe illumination attenuation, the element value in the target correction matrix is ​​larger, and the color channel value of that pixel is amplified after the multiplication to compensate for the attenuation loss; for pixels with less illumination influence, the element value is smaller, and the adjustment range is correspondingly smaller. The product result is used to overwrite the original color channel values ​​to complete pixel spectral reconstruction, resulting in a transitional facial image. At this point, the color distribution of the transitional facial image has eliminated the effects of spatial non-uniform color shift and temporal phase deviation caused by multispectral ambient light, but the overall brightness level may deviate from the expected brightness under standard viewing conditions.

[0095] Obtain the single constant bias term corresponding to the preset uniform white light source. A uniform white light source refers to an ideal reference light source with a uniform spectral energy distribution across all color channels, such as the CIE standard light source D65. The single constant bias term is a scalar value, its magnitude determined by the difference between the brightness parameters of the standard light source and the overall average brightness of the transitional facial image. Specifically, the average brightness value of all pixels in the three color channels of the transitional facial image can be calculated. Subtracting this average brightness value from the target brightness value corresponding to the standard light source yields the single constant bias term. For example, this value can be 20, but the specific value can be set according to actual needs and the parameters of the standard light source.

[0096] Finally, a single constant bias term is added to the light intensity values ​​of each pixel in the transitional facial image. Since the same constant accumulation operation is applied to all color channels of all pixels, the proportional relationships between the color channels of each pixel remain unchanged; only the overall brightness level is uniformly increased. This ensures that color correction does not introduce new color casts, while simultaneously ensuring that the brightness level of the output image meets the requirements of standard viewing conditions. After the above processing, the color-corrected target facial image is output.

[0097] This step achieves accurate spectral reconstruction through pixel-by-pixel multiplication. Combined with brightness normalization processing using a constant bias term, it eliminates ambient light color shift while maintaining the relative proportions between color channels, ensuring the naturalness of the final output image's color and the appropriateness of its brightness.

[0098] For example, a facial recognition gate system at the entrance of an underground parking lot in a commercial complex is used as an application scenario to illustrate the complete implementation process of the method of the present invention in detail. In this scenario, three types of light sources exist simultaneously: warm white LED downlights with a frequency of 50Hz installed on the ceiling, emergency fluorescent lights above the entrance passage, and natural sunlight filtering in from the driveway. The spectral energy distribution, flicker frequency, and projection direction of the three types of light sources are different, creating a complex multispectral superposition illumination on the face of a person.

[0099] In S101, the multispectral camera built into the gate system and the four-channel spectral sensor array are simultaneously triggered by external signals to start data acquisition. The multispectral camera divides the visible light range of 450nm to 700nm into three channels: blue, green, and red, and also has a 750nm near-infrared channel, for a total of four channels to simultaneously expose and acquire the initial facial image. The spectral sensor array continuously samples the scene illumination at a sampling frequency of 200Hz, acquiring an ambient light intensity sequence of 0.5 seconds in length, for a total of 100 sampling points.

[0100] In S102, a wavelength threshold of 650nm is set. The gray values ​​of the red and near-infrared channels are assigned to the epidermal scattering and transmission set, while the gray values ​​of the blue and green channels are assigned to the dermal diffuse reflection set. Taking an initial facial image with a resolution of 640 x 480 as an example, the spectral scattering difference is calculated pixel by pixel. The spectral scattering difference in the central facial region, as well as the forehead and mid-cheek area, is concentrated between 0.8 and 1.1. The spectral scattering difference in the area on the left side of the face directly illuminated by LED downlights increases to 1.4 to 1.8, and the spectral scattering difference in the area on the right side of the face illuminated by emergency fluorescent lights increases to 1.3 to 1.6. With a stripping threshold of 1.2 set, after pixel-by-pixel comparison, the facial background image contains approximately 180,000 pixels, mainly distributed in the central and lower half of the face; the illumination interference image contains approximately 127,000 pixels, mainly concentrated in the left and upper right sides of the face, as well as the highlight area of ​​the forehead. Figure 6 The area marked in the middle is shown.

[0101] In S103, local extremum detection is performed on the ambient light intensity sequence to identify two emission endpoints: one corresponding to the direction of the ceiling LED downlight, located approximately 30 degrees to the left and directly above the face; the other corresponding to the direction of the emergency fluorescent light, located approximately 45 degrees to the upper right of the face. In the illumination interference map, using 1.5 times the average pixel brightness of 128 (i.e., 192) as the brightness threshold, a connected component labeling algorithm is used to extract two bright, continuous regions: the left region has an area of ​​approximately 8200 pixels, with its geometric center at column 185, row 210; the upper right region has an area of ​​approximately 5600 pixels, with its geometric center at column 420, row 155. The Sobel operator is used to calculate the normal perpendicular vectors for each of the two light-receiving projection endpoints, and vector projection is performed by connecting these vector vectors to the corresponding emission endpoints to obtain two incident light paths. The incident deflection angle of the light path on the left is calculated to be 28 degrees. The air refractive index is 1.0, and the skin refractive index is 1.45. Based on Fresnel's law of reflection, the surface reflection ratio R is calculated to be 0.04, and the light transmission ratio T is calculated to be 0.96. The incident deflection angle of the light path on the upper right is 52 degrees, corresponding to R of 0.09 and T of 0.91. Multiplying the brightness values ​​of each pixel in the illumination interference map by R and T respectively yields the surface reflection map and the subcutaneous scattering map.

[0102] In S104, three main facial protrusion regions are identified in the facial background image through local extremum detection and layer-by-layer expansion: the bridge of the nose region contains approximately 4200 pixels with a local brightness extremum of 168; the left cheekbone region contains approximately 3100 pixels with a brightness extremum of 155; and the right cheekbone region contains approximately 2900 pixels with a brightness extremum of 152. Taking the bridge of the nose region as an example, when expanding outward from the extremum to the 7th layer, the average brightness difference between adjacent layers changes from a positive value of 5.2 to a negative value of -1.3, indicating a decay inflection point and determining the boundary of the protrusion region of the bridge of the nose. Within this region, the RGB three-channel chromaticity gradient rate is calculated pixel by pixel and the vector magnitude is taken. The chromaticity gradient rate of the pixels at the top of the bridge of the nose is approximately 0.8, while the chromaticity gradient rate of the pixels on the sides of the nostrils increases to 6.5. Based on the calibration of a typical 8mm height of the human nose bridge and its corresponding maximum chroma gradient rate, a proportional mapping coefficient k is set to 0.3mm per unit gradient rate. Substituting the chroma gradient rate into the mapping function, the height reduction at the top of the nose bridge is 0.24mm, and the height reduction on the sides of the nose wings is 1.95mm. The planar coordinates of all pixels in the high-convex area are bound to the height reduction amount, and the facial undulation gradient is obtained by combining them.

[0103] In S105, the ambient light intensity sequence was subjected to temporal minimum detection, extracting 10 light intensity troughs with an average interval of 0.02 seconds between adjacent troughs, confirming the LED downlight's emission period T to be 0.02 seconds. The propagation distance d from the light source to the face was estimated to be 2.1 meters, and the phase delay was derived to be approximately 2.2 x 10^-6 radians. Multiplying the height drop of each pixel in the facial undulation gradient by the phase delay, the light wave propagation loss value of the pixel at the top of the bridge of the nose was 5.28 x 10^-7, and the loss value of the pixel on the side of the nose was 4.29 x 10^-6. The relationship between the pixel loss value and coordinates was recorded as a light decay mapping table, and the reciprocals of each loss value were arranged to form a color difference compensation matrix. The compensation coefficient corresponding to the top of the bridge of the nose was approximately 1.02, the compensation coefficient corresponding to the side of the nose was approximately 1.18, and the compensation coefficient for the recessed area at the edge of the face could reach as high as 1.35.

[0104] In S106, temporal peak analysis was performed on the surface reflectance map and subcutaneous scattering map in four spectral bands. Taking the red channel as an example, the peak time of the surface reflectance map was the 23rd sampling point of the sampling sequence, and the peak time of the subcutaneous scattering map was the 25th sampling point. The spectral phase difference was 0.01 seconds, corresponding to the time of two sampling points. The average spectral phase difference of the four bands was 0.0095 seconds. The phase lag coefficient was equal to 0.0095 divided by the emission period of 0.02, which was approximately 0.475. Multiplying each element in the color difference compensation matrix by 0.475 yielded the target correction matrix.

[0105] In step S107, the RGB channel values ​​of each pixel in the initial facial image are multiplied channel by channel with the corresponding element values ​​of the target correction matrix to complete pixel spectral reconstruction and obtain the transitional facial image. The average brightness value of all pixels in the transitional facial image is calculated to be 106, the target brightness value corresponding to the standard light source D65 is set to 128, and the single constant bias term is 22. This 22 is accumulated and added to the light intensity values ​​of each channel of all pixels to output the target facial image. Figure 7 As shown, the issues of a warm yellow tint on the left side of the face, a cool white tint on the upper right side, and overexposure and whitening of the bridge of the nose in the initial facial image before correction were effectively eliminated after correction, and the skin tone of each area of ​​the face became more consistent. Using the CIEDE2000 color difference formula, the color difference between the left side of the face and the standard skin tone before correction was 11.2, the upper right side was 9.7, and the bridge of the nose was 7.3. After correction, the color differences of these three areas decreased to 2.1, 1.9, and 1.5 respectively, and the overall average color difference decreased from 9.4 before correction to 1.8 after correction. When the corrected target facial image was sent to the facial recognition module of the gate system for identity verification, the recognition and matching accuracy improved from 89.6% before correction to 98.2% after correction.

[0106] Please see Figure 8 , Figure 8 This is a structural block diagram of a multispectral ambient light detection facial recognition image color correction system provided in an embodiment of the present invention.

[0107] This invention provides a multispectral ambient light detection-based facial recognition image color correction system, comprising: Acquisition module 81 is used to simultaneously acquire initial facial images and ambient light intensity sequences under multispectral conditions; The decomposition module 82 is used to perform multidimensional decomposition on the initial facial image based on the spectral scattering difference between the epidermis and dermis of the skin, and to separate the facial background image and the illumination interference image of the initial facial image. The analysis module 83 is used to analyze the illumination interference map based on the luminous flux characteristics of the ambient light intensity sequence and extract the incident light path of the illumination interference map; based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map; Module 84 is used to extract the high-convex region of the face from the background image of the face and calculate the chromaticity gradient rate of each pixel in the high-convex region of the face; based on the chromaticity gradient rate, the facial undulation gradient of the high-convex region of the face is constructed. The calculation module 85 is used to parse the ambient light intensity sequence and obtain the phase delay of the ambient light; it fuses the phase delay with the facial undulation gradient to generate a light decay mapping table, and calculates the color difference compensation matrix of the initial facial image based on the light decay mapping table. Synchronization module 86 is used to calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map; based on the spectral phase difference, the color difference compensation matrix is ​​optically phase synchronized to obtain the target correction matrix of the initial facial image; The reconstruction module 87 is used to perform pixel spectral reconstruction on the initial facial image using the target correction matrix as the reflectance reference of the facial background image, and to apply standard light source rendering to output the color-corrected target facial image.

[0108] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems and modules described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0109] The present invention also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the multispectral ambient light detection facial recognition image color correction method described above.

[0110] The present invention also provides a computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, it implements the steps of the multispectral ambient light detection facial recognition image color correction method described above.

[0111] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.

[0112] Embodiments of the present invention also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the face recognition image color correction method for multispectral ambient light detection.

[0113] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0114] The foregoing has provided a detailed description of a multispectral ambient light detection-based facial recognition image color correction method and system provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are merely for the purpose of helping to understand the method and core ideas of the present invention. It should be noted that those skilled in the art can make various improvements and modifications to the present invention without departing from its principles, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A method for color correction of facial recognition images based on multispectral ambient light detection, characterized in that, include: Simultaneously acquire initial facial images and ambient light intensity sequences under multispectral conditions; Based on the spectral scattering difference between the epidermis and dermis, the initial facial image is decomposed in multiple dimensions to separate the facial background image and the illumination interference image of the initial facial image. Based on the luminous flux characteristics of the ambient light intensity sequence, the illumination interference map is analyzed, and the incident light path of the illumination interference map is extracted; based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map; Extract the high-convex regions of the face from the background image and calculate the chromaticity gradient rate of each pixel within the high-convex regions; construct the facial undulation gradient of the high-convex regions based on the chromaticity gradient rate. The ambient light intensity sequence is analyzed to obtain the phase delay of the ambient light; the phase delay is fused with the facial undulation gradient to generate a light decay mapping table, and the color difference compensation matrix of the initial facial image is calculated based on the light decay mapping table. Calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map; Based on the spectral phase difference, the color difference compensation matrix is ​​optically phase synchronized to obtain the target correction matrix of the initial facial image; Using the target correction matrix as the reflectance reference of the facial background image, pixel spectral reconstruction is performed on the initial facial image, and standard light source rendering is applied to output the color-corrected target facial image.

2. The method according to claim 1, characterized in that, The illumination interference map is analyzed based on the luminous flux characteristics of the ambient light intensity sequence, and the incident light path of the illumination interference map is extracted. Based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map, including: Find the local maximum point corresponding to the light intensity value in the ambient light intensity sequence, and locate the spatial coordinates of the local maximum point as the emission endpoint of the ambient light. Extract the continuous pixel region in the illumination interference image where the brightness value exceeds a set brightness threshold, calculate the geometric center coordinates of the continuous pixel region, and use it as the endpoint of the light-receiving projection. Obtain the normal perpendicular vector of the facial region where the light-receiving projection endpoint is located; The initial light direction is obtained by connecting the transmitting endpoint and the receiving projection endpoint, and the initial light direction is vector-projected along the direction of the normal perpendicular vector to obtain the incident light path of the illumination interference map.

3. The method according to claim 1, characterized in that, The spectral phase difference between the surface reflection map and the subcutaneous scattering map is calculated. Based on the spectral phase difference, the color difference compensation matrix is ​​optically phase synchronized to obtain the target correction matrix of the initial facial image, including: The times when the light intensity peaks in the same spectral band are counted in the surface reflection map and the subcutaneous scattering map respectively. The peak time of the surface reflection map is subtracted from the peak time of the subcutaneous scattering map, and the absolute value of the difference is taken as the spectral phase difference. The proportion of the spectral phase difference to the entire ambient light emission period is calculated and used as the phase lag coefficient. Multiply the element values ​​at each position in the color difference compensation matrix by the phase lag coefficient, and replace the original matrix with the matrix corresponding to the product result to obtain the target correction matrix for the initial facial image.

4. The method according to claim 1, characterized in that, The process involves parsing the ambient light intensity sequence to obtain the phase lag of the ambient light; fusing the phase lag with the facial undulation gradient to generate a light decay mapping table; and calculating the color difference compensation matrix of the initial facial image based on the light decay mapping table, including: The time span between two adjacent light intensity troughs in the ambient light intensity sequence is extracted as the emission period of the ambient light. Based on the emission period and the light wave propagation speed, the phase delay of the ambient light reaching the face is derived. Multiply the height drop of each pixel in the facial undulation gradient by the phase delay to obtain the light wave propagation loss value at each pixel, and record the correspondence between the light wave propagation loss value and the two-dimensional plane coordinates of the pixel as a light decay mapping table. The light wave propagation loss value corresponding to each pixel position of the initial facial image is found in the light decay mapping table. The reciprocal of the light wave propagation loss value is calculated. The reciprocals corresponding to all pixel positions are arranged according to the pixel array of the initial facial image to obtain the color difference compensation matrix of the initial facial image.

5. The method according to claim 2, characterized in that, The step of decoupling the illumination interference map into a surface reflection map and a subcutaneous scattering map based on the incident light path includes: Calculate the angle between the incident light path and the normal perpendicular vector at the endpoint of the light-receiving projection, and use it as the incident deflection angle; Based on the physical law of light refraction on different media surfaces, the surface reflection ratio and light transmission ratio of the light corresponding to the incident deflection angle are calculated, and the sum of the surface reflection ratio and the light transmission ratio is one. Multiply the brightness value of each pixel in the illumination interference map by the surface reflection ratio of the light to obtain the surface reflection map; Multiply the brightness value of each pixel in the illumination interference map by the light transmission ratio to obtain the subcutaneous scattering map.

6. The method according to claim 1, characterized in that, The step of performing multidimensional decomposition on the initial facial image based on the spectral scattering difference between the epidermis and dermis to separate the facial background image and the illumination interference image of the initial facial image includes: Obtain the pixel grayscale values ​​corresponding to different spectral bands in the initial facial image; The set of pixel gray values ​​corresponding to spectral bands with wavelengths greater than a set wavelength threshold is determined as the epidermal scattering and transmission set, and the set of pixel gray values ​​corresponding to spectral bands with wavelengths less than or equal to the set wavelength threshold is determined as the dermal diffuse reflection set. Calculate the grayscale ratio of the epidermal layer scattering and transmission set and the dermal layer diffuse reflection set at the same spatial coordinates of the pixels, and use the grayscale ratio of each pixel as the spectral scattering difference; The initial facial image is traversed according to spatial coordinates. Pixels with spectral scattering differences less than a set stripping threshold are extracted and summarized into a facial background image, and the remaining pixels are summarized into an illumination interference image.

7. The method according to claim 1, characterized in that, The step of extracting the high-convex regions of the face from the background image and calculating the chromaticity gradient rate of each pixel within the high-convex regions of the face includes: In the background image of the face, the pixel brightness is compared row by row to find the local brightness extreme point. With the local brightness extreme point as the center, the pixels are expanded outward layer by layer until the brightness difference between two adjacent layers of pixels shows a decay inflection point from positive to negative. The expanded and covered pixel set is then enveloped as the high convex area of ​​the face. Within the high-convex area of ​​the face, the color channel values ​​of the target pixel and its adjacent pixels are obtained; Calculate the difference between the color channel values ​​of the target pixel and the adjacent pixel in the corresponding color channel, and use the ratio of the difference to the linear distance between the target pixel and the adjacent pixel as the chromaticity gradient rate of the corresponding target pixel.

8. The method according to claim 7, characterized in that, The step of constructing the facial undulation gradient of the high-convex area of ​​the face based on the chromaticity gradient rate includes: Establish a direct proportional mapping function between the chromaticity gradient rate and the change in spatial height; The chromaticity gradient rate of each pixel in the high convex area of ​​the face is successively substituted into the direct proportional mapping function to calculate the height drop of each pixel relative to the local brightness extreme point. By binding and associating the planar position of all pixels in the high convex area of ​​the face with the corresponding height drop in the two-dimensional coordinate system, the facial undulation gradient of the high convex area of ​​the face is obtained.

9. The method according to claim 1, characterized in that, The step of using the target correction matrix as the reflectance reference of the facial background image, performing pixel spectral reconstruction on the initial facial image, applying standard light source rendering, and outputting a color-corrected target facial image includes: The original color channel values ​​of each pixel in the initial facial image are multiplied by the corresponding element values ​​in the target correction matrix, and the product result is used to cover the original color channel values ​​to complete pixel spectral reconstruction and obtain the transitional facial image. Obtain the single constant bias term corresponding to the preset uniform white light source; The single constant bias term is accumulated to the light intensity value of each pixel in the transition face image, so that the light intensity value of the whole image is increased while the color channel ratio remains unchanged, and the color-corrected target face image is output.

10. A face recognition image color correction system based on multispectral ambient light detection, characterized in that, include: The acquisition module is used to simultaneously acquire initial facial images and ambient light intensity sequences under multispectral environments; The decomposition module is used to perform multidimensional decomposition on the initial facial image based on the spectral scattering difference between the epidermis and dermis of the skin, and to separate the facial background image and the illumination interference image of the initial facial image. The analysis module is used to analyze the illumination interference map based on the luminous flux characteristics of the ambient light intensity sequence and extract the incident light path of the illumination interference map. Based on the incident light path, the illumination interference map is decoupled into a surface reflection map and a subcutaneous scattering map; A construction module is used to extract the high-convex region of the face from the background image of the face and calculate the chromaticity gradient rate of each pixel in the high-convex region of the face; and construct the facial undulation gradient of the high-convex region of the face based on the chromaticity gradient rate. The calculation module is used to parse the ambient light intensity sequence and obtain the phase delay of the ambient light; fuse the phase delay with the facial undulation gradient to generate a light decay mapping table, and calculate the color difference compensation matrix of the initial facial image based on the light decay mapping table. The synchronization module is used to calculate the spectral phase difference between the surface reflection map and the subcutaneous scattering map; Based on the spectral phase difference, the color difference compensation matrix is ​​optically phase synchronized to obtain the target correction matrix of the initial facial image; The reconstruction module is used to perform pixel spectral reconstruction on the initial facial image using the target correction matrix as the reflectance reference of the facial background image, apply standard light source rendering, and output the color-corrected target facial image.

Images