A multi-color space feature fusion face living body anomaly detection method and system

Abstract

The application provides a multi-color space feature fusion face living body anomaly detection method and system, multi-color space transformation is performed on an input face image to be detected to obtain feature representation of the face image to be detected in multiple color spaces, and multi-scale feature representation is extracted from the feature representation; the multi-scale feature representation is subjected to sparse self-attention enhancement processing to obtain enhanced feature representation of each color space, and then weighted fusion is performed according to adaptive fusion weights to obtain fused multi-color space features; the fused multi-color space features are subjected to anomaly detection based on a pre-constructed memory bank, and whether the input face image to be detected is a living body face image is determined according to an anomaly score. The application provides a face living body detection technical solution that can balance detection accuracy, robustness and engineering feasibility in a complex application environment, accurately describes feature distribution of a real face sample, and identifies a fake attack sample as an abnormal sample.

Description

Technical Field

[0001] This application belongs to the field of image intelligent analysis technology, specifically relating to a method and system for detecting facial liveness anomalies by fusing multi-color space features. Background Technology

[0002] With the continuous promotion and large-scale deployment of facial recognition technology in scenarios such as financial payments, access control and security, identity authentication, and smart terminals, facial recognition systems are gradually becoming an important means of identity authentication in various information systems. In practical engineering applications, facial recognition systems typically face complex and ever-changing real-world environments. Their input data is affected by various factors such as differences in acquisition equipment, changes in lighting conditions, and user usage patterns. If the system lacks an effective security protection mechanism, it is highly susceptible to spoofing attacks, leading to security vulnerabilities. Therefore, in the overall architecture of a facial recognition system, a face liveness detection module is usually introduced at the front end to determine the authenticity of the input facial image or video before identity recognition, thereby preventing non-live data from entering the subsequent recognition process.

[0003] Existing face liveness detection technologies are typically used as a pre-security module in face recognition systems to determine the authenticity of acquired face images or video data. In practical engineering applications, related technical solutions mainly revolve around deep learning models to model the differences between real face samples and attack samples. One type of existing technology is based on discriminative deep learning models, modeling the face liveness detection problem as a binary classification task. This type of solution typically uses RGB or grayscale face images as input, extracts features through convolutional neural networks, and uses supervised learning to distinguish between real faces and attack samples. This solution has good detection performance under known attack types, but the model is highly dependent on the distribution of training data, and the detection performance is prone to decline when the actual attack method changes. Another type of existing technology attempts to improve detection robustness by introducing multiple feature representations, such as color space transformation or texture feature analysis of face images to separate brightness and chromaticity information, thereby enhancing the system's ability to perceive changes in lighting and differences in attack media. However, this type of solution usually uses a fixed feature fusion method in engineering implementation, making it difficult to adaptively adjust the importance of various features according to different application scenarios. In addition, some existing technical solutions guide the model to learn more discriminative feature representations by introducing auxiliary information or multi-task learning methods, such as jointly modeling geometric structures, reflection characteristics, or depth information. While these solutions improve detection accuracy to some extent, they often have high requirements for acquisition equipment or operating environment, resulting in significant system complexity. In recent years, anomaly detection has also been introduced into the field of face liveness detection. These solutions typically use only real face samples to model the feature distribution and identify samples that deviate from the normal distribution as attack samples, thus alleviating the dependence on attack sample annotation to some extent. However, existing anomaly detection solutions are mostly based on single feature representations and mainly rely on global feature distribution for judgment, and their ability to characterize local forged areas and fine-grained attack traces on faces remains limited.

[0004] While existing face liveness detection technologies have improved the security of face recognition systems to some extent, they still have many shortcomings in practical engineering applications and complex attack scenarios, mainly in the following aspects. First, many existing face liveness detection schemes are still based on discriminative deep learning models, simplifying the liveness detection problem into a binary classification problem between real faces and attack samples. These schemes typically rely on the types and distribution of attack samples collected during the training phase. When new attack methods or attack media change in practical applications, the model struggles to effectively identify unseen attack samples, leading to a significant drop in detection performance and potential security risks. This technical approach, which heavily relies on prior knowledge of attack samples, struggles to meet the security and stability requirements of open environments and long-term operating systems. Second, at the feature modeling level, most existing technologies still use RGB or grayscale images as the main input, modeling facial features only within a single color space. This feature representation is easily affected by environmental factors under complex lighting conditions, different imaging devices, or different attack media, making it difficult to fully reflect the differences between real faces and forged samples in terms of color response, reflectivity, and material structure. While some solutions attempt to incorporate features from multiple color spaces, they typically employ fixed rules or simple splicing methods for fusion, lacking an adaptive modeling mechanism capable of discriminating between different color space features. This results in the underutilization of complementary information between multiple color spaces. Furthermore, from a spatial modeling perspective, most existing face liveness detection methods focus on global feature modeling of the entire face region, neglecting targeted analysis of forgery traces in local areas. In real-world attack scenarios, attack samples often exhibit anomalous features only in local areas (such as the eyes, mouth, and skin texture areas). Global feature modeling methods easily mask these local anomalies, reducing the model's ability to perceive fine-grained attack features and affecting overall detection accuracy. Moreover, while face liveness detection solutions that have introduced anomaly detection in recent years have alleviated the dependence on attack sample annotation to some extent, existing solutions mostly model the distribution of real face features based on a single feature representation, with anomaly detection primarily relying on distribution biases in the global feature space. This modeling approach struggles to effectively characterize the multi-scale, multi-regional distribution of facial features and is insufficiently sensitive to anomalous responses in localized forged regions within complex attack samples, limiting the detection effectiveness and robustness of anomaly detection methods in real-world, complex scenarios. Finally, from an engineering application perspective, some existing methods rely on additional sensor information or stringent acquisition conditions, placing high demands on system hardware configuration and operating environment, increasing system deployment and maintenance costs, and hindering large-scale deployment in practical applications. Furthermore, the lack of a unified, collaborative feature modeling and decision-making mechanism among different modules also impacts the overall system performance stability to some extent.

[0005] In summary, existing technologies still have significant shortcomings in dealing with unknown attacks, adaptive modeling of multi-color space features, perception of local forgery regions, and refined modeling of anomaly detection. There is an urgent need for a face liveness detection technology solution that can balance detection accuracy, robustness, and engineering feasibility in complex application environments. Summary of the Invention

[0006] To address the aforementioned problems in the existing technology, this application provides a method and system for detecting facial liveness anomalies through multi-color space feature fusion. The technical problem to be solved by this application is achieved through the following technical solution: A method for detecting facial liveness anomalies through multi-color space feature fusion includes: S100, Perform multi-color space transformation on the input face image to be tested to obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; S200, the multi-scale feature representation is subjected to sparse self-attention enhancement processing to obtain the enhanced feature representations of each color space; S300, the enhanced color space feature representations are weighted and fused according to adaptive fusion weights to obtain the fused multi-color space features; S400, based on the pre-built memory bank, performs anomaly detection on the fused multi-color space features, and determines whether the input test face image is a live face image based on the detected anomaly score.

[0007] A face liveness detection system based on multi-color space feature fusion includes: The multi-color space parallel feature extraction module is used to perform multi-color space transformation on the input face image to be tested, obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; The sparse self-attention feature enhancement module is used to perform sparse self-attention enhancement processing on the multi-scale feature representation to obtain the enhanced feature representations of each color space. The multi-color space adaptive fusion module is used to perform weighted fusion of the enhanced color space feature representations according to adaptive fusion weights to obtain the fused multi-color space features. The anomaly detection module based on the memory bank is used to perform anomaly detection on the fused multi-color space features based on the pre-built memory bank, and to determine whether the input face image to be tested is a live face image based on the detected anomaly score.

[0008] Beneficial effects: 1. This invention employs a face liveness detection framework based on anomaly detection. It utilizes only real face samples for feature modeling and distribution learning, treating all types of attack samples as anomalies deviating from the true distribution for detection. This approach effectively reduces dependence on known attack types and the number of attack samples, avoiding the problem of traditional supervised learning methods requiring frequent data re-collection and labeling when facing new attack methods. It significantly improves the system's generalization ability and practicality for unknown attack types.

[0009] 2. This invention introduces a parallel modeling mechanism for facial features using multiple color spaces, including RGB, YUV, and YCbCr, to fully explore the complementary relationship between luminance and chromaticity information. This allows the model to perceive the texture, structure, and color variation features of the face from different color representation perspectives. This multi-color space modeling approach effectively alleviates the performance degradation problem of single RGB representation under complex lighting conditions and differences in camera equipment, improving the stability and robustness of face liveness detection in cross-scene and cross-device applications.

[0010] 3. This invention introduces a sparse self-attention mechanism during feature extraction. By focusing on modeling local facial regions, it enhances the model's ability to focus on local anomalies. This mechanism can effectively capture fine-grained forgery features such as printed textures, screen reflections, and edge discontinuities, improving the model's accuracy in perceiving local attack traces and thus enhancing the overall reliability of liveness detection.

[0011] 4. This invention, by setting up a multi-color space adaptive feature fusion module, dynamically adjusts the weight ratio of different color space features in the fusion process according to the feature distribution of the input face sample, avoiding the redundant information superposition problem caused by traditional fixed weight or simple splicing fusion methods. This adaptive fusion strategy can highlight the color space features that are more discriminative for the current sample, improve the distinguishability and expressive power of the fused features, and thus further improve the accuracy of liveness detection.

[0012] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0013] Figure 1 This is a flowchart illustrating a face liveness anomaly detection method based on multi-color space feature fusion provided in this application; Figure 2 This is a schematic diagram of the overall detection process of a face liveness anomaly detection system provided in this application, which integrates multi-color space feature fusion. Detailed Implementation

[0014] The present application will be described in further detail below with reference to specific embodiments, but the implementation of the present application is not limited thereto.

[0015] To address the issues of unstable face liveness detection performance and insufficient adaptability to unknown attacks in complex application environments, existing technologies aim to propose a face liveness detection method and system based on multi-color space feature fusion and anomaly detection. This aims to improve the security, robustness, and reliability of face liveness detection in practical engineering applications, specifically solving the following technical problems: (1) To address the problem that existing face liveness detection methods based on discriminative binary classification models are highly dependent on the type and distribution of attack samples and have significantly reduced detection performance when facing unseen attack methods, a detection mechanism is provided that does not require modeling of all attack samples and can effectively improve the ability to identify unknown attacks.

[0016] (2) To address the problem that existing methods mostly use a single color space for feature modeling, which makes it difficult to fully utilize the complementary characteristics between brightness and chromaticity information, resulting in insufficient robustness of detection under complex lighting conditions, different imaging devices, or cross-scene applications, a multi-color space parallel modeling facial feature representation method is provided.

[0017] (3) To address the problem that existing methods mainly rely on global facial features for discrimination, and do not pay enough attention to the fine-grained abnormal features generated by forgery attacks in local areas, making it difficult to effectively perceive local forgery traces, a feature extraction mechanism that can enhance the ability to model local abnormal areas of the face is provided.

[0018] (4) To address the problem that existing multi-color space feature fusion methods typically employ fixed weights or simple splicing structures, which cannot adaptively adjust the contribution of each color space feature according to different input samples, resulting in limited feature fusion efficiency and discrimination ability, a fusion mechanism that can adaptively adjust the fusion relationship of multi-color space features according to input features is provided.

[0019] Combination Figure 1 and Figure 2 This application provides a method for detecting facial liveness anomalies through multi-color space feature fusion, including: S100, Perform multi-color space transformation on the input face image to be tested to obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; In one specific embodiment of this application, S100 includes: S110, convert the input face image to be tested into feature representations in RGB color space, YUV color space and YCbCr color space respectively; S120 extracts feature maps from the feature representations in the RGB color space, YUV color space, and YCbCr color space through independent deep convolutional neural network branches; S130, feature maps are extracted from multiple intermediate layers of the deep convolutional neural network in each branch, and the extracted feature maps are spatially scale aligned and channel-stitched to form multi-scale feature representations corresponding to RGB, YUV and YCbCr color spaces.

[0020] refer to Figure 2 This invention performs multi-color space transformation processing on the face image under test to fully explore the complementary characteristics of different color spaces in expressing luminance and chrominance information. Specifically, the input face image under test is an RGB color image. Through a pre-defined color space transformation method, the RGB image is converted into multiple color space representations, including at least RGB, YUV, and YCbCr. The feature representations in the RGB, YUV, and YCbCr color spaces are simply referred to as RGB, YUV, and YCbCr representations. The YUV and YCbCr color spaces can effectively decouple the luminance and chrominance components, which helps reduce the impact of illumination changes on the feature modeling process and highlights the differences in color response and imaging characteristics of different attack media. For each color space, this invention sets up an independent feature extraction branch. Each branch includes a deep convolutional neural network backbone structure for extracting feature representations from the face image in the corresponding color space. In the specific implementation process, each color space branch extracts feature maps from multiple intermediate layers of the backbone network, and performs spatial scale alignment and channel concatenation on the feature maps of different levels, thereby obtaining multi-scale feature representations that simultaneously contain low-level texture information and high-level semantic information, in order to comprehensively characterize the facial appearance features under that color space. Through the above parallel feature extraction method, this invention can obtain multi-scale facial feature representations corresponding to multiple color spaces, providing basic input for subsequent feature enhancement and fusion processing.

[0021] S200, the multi-scale feature representation is subjected to sparse self-attention enhancement processing to obtain the enhanced feature representations of each color space; In one specific embodiment of this application, S200 specifically includes: S210, the multi-scale feature map of each color space branch is divided into multiple non-overlapping local regions in the spatial dimension; S220, Self-attention calculation is performed independently in each local region to enhance the correlation between features within the region, thereby obtaining enhanced features in the local region; S230 reorganizes all local region enhancement features to obtain enhanced color space feature representations.

[0022] To enhance the model's ability to perceive local anomalies in forgery attacks, this invention introduces sparse self-attention feature enhancement technology into each color space feature branch. In real-world face liveness detection scenarios, forgery attacks often generate anomalous features only in local areas, such as local texture discontinuities, abnormal edge transitions, or distorted reflection properties. If a global self-attention mechanism is used for feature modeling, it easily introduces a large amount of information from regions unrelated to forgery, thereby weakening the focus on key anomalies. Based on the above considerations, this invention divides the input feature map into multiple local regions in the spatial dimension and performs self-attention calculation independently within each local region. By limiting the spatial scope of attention calculation, the model focuses on the relationships between features within local regions, thereby enhancing its ability to model local anomaly patterns. After completing the self-attention calculation in each local region, the output features of each local region are recombined to form a feature map enhanced by sparse self-attention. This feature map significantly improves the ability to express local forgery traces while retaining overall structural information. The aforementioned sparse self-attention feature enhancement module operates on the feature branches of each color space, enabling face features in different color spaces to have stronger local discrimination capabilities.

[0023] S300, the enhanced color space feature representations are weighted and fused according to adaptive fusion weights to obtain the fused multi-color space features; In one specific embodiment of this application, S300 includes: S310, aggregate the feature representations from all enhanced color spaces to generate a global aggregated feature; the global aggregated feature is represented as follows: ; in, Represents global aggregated features. This represents the characteristics of the RGB color space. This represents the characteristics of the YUV color space. This represents the characteristics of the YCbCr color space. To quantitatively evaluate the effectiveness of each spatial branch under specific sample inputs, the module performs feature redistribution using a weight generator based on an attention mechanism. (Aggregate features) Firstly The convolutional layer performs local modeling of the spatial dimension, and then global average pooling is used to compress the spatial domain information to the channel domain, generating a highly generalized global state vector. :

[0024] S320, the global aggregated features are compressed and nonlinearly mapped to generate a set of adaptive weight coefficients that correspond one-to-one with the color space branches; Subsequently, to further capture the nonlinear interactions between the color space branches, this vector is fed into a bottleneck structure composed of fully connected layers to calculate the adaptive weight coefficients corresponding to each color space branch, expressed as: ; in, and Responsible for performing dimensionality-raising and lowering transformations to extract higher-order feature correlations. and Together, they constructed a bottleneck structure for the channel domain. It is responsible for performing dimensionality compression to remove feature redundancy through low-rank space modeling and learning the nonlinear interactions between color space branches, while It is responsible for remapping features back to three-dimensional space (corresponding to the weights of RGB, YUV, and YCbCr) to achieve saliency weight representation reconstruction for specific color gamut branches. The ReLU activation function is used. Depthwise separable convolution is introduced. The aim is to achieve independent weighted mapping at the channel level, effectively avoiding disordered coupling of cross-spatial information. Ultimately, this is achieved through the Sigmoid function. Map the output value to Interval, generate a three-dimensional weight vector .

[0025] S330, normalize the adaptive weight coefficients to obtain adaptive fusion weights; S340, multiply the enhanced color space feature representations corresponding to each color space branch by the corresponding adaptive fusion weights, and then perform a weighted summation to obtain the fused multi-color space features. The fused multi-color space features are represented as follows: ; in, This represents the adaptive weighting coefficients corresponding to the RGB color space. This represents the adaptive weighting coefficients corresponding to the YUV color space. This represents the adaptive weighting coefficient corresponding to the YCbCr color space.

[0026] The above adaptive fusion method enables dynamic adjustment of the contribution of features in different color spaces, avoiding information redundancy caused by simple splicing or fixed-weight fusion.

[0027] Since different color spaces contribute differently to liveness detection under varying imaging conditions, lighting environments, and attack methods, this invention dynamically weights and fuses features from different color spaces. Specifically, features from multiple color space branches are first aggregated to generate a global feature description reflecting the overall characteristics of the current input sample. Then, the global features are compressed using convolution and global pooling operations to obtain a compact feature vector representation. Based on this, a nonlinear mapping structure generates weight coefficients corresponding to each color space, representing the relative importance of each color space feature in the current sample. These weight coefficients are normalized to ensure their values ​​fall within a preset range. Finally, the feature representations of each color space are multiplied by their corresponding weight coefficients and summed using weighted methods to obtain the fused multi-color space feature representation. Through this adaptive fusion method, this invention can dynamically adjust the contribution of different color space features according to the actual characteristics of the input sample, avoiding redundant information caused by fixed weights or simple concatenation, and improving the discriminative power and robustness of the fused features.

[0028] S400, based on the pre-built memory bank, performs anomaly detection on the fused multi-color space features, and determines whether the input test face image is a live face image based on the detected anomaly score.

[0029] In one specific embodiment of this application, S400 includes: S410, calculate the distance between the fused multi-color space features and each prototype feature in the memory bank; wherein, the memory bank stores multiple prototype features representing the distribution of real human face features; In one specific embodiment of this application, the step of constructing the memory bank includes: a. Input a real face sample; b. Perform multi-color space transformation on the real face sample to obtain the feature representation of the real face sample in multiple color spaces, and extract multi-scale feature representation from the feature representation; c. Perform sparse self-attention enhancement on the multi-scale feature representations corresponding to real face samples to obtain enhanced feature representations for each color space. d. For the color space feature representations corresponding to real face samples, weighted fusion is performed according to adaptive fusion weights to obtain the fused multi-color space features; e. Extract local features from the multi-color space features corresponding to real face samples, and select representative features as prototypes through clustering or feature selection to form a memory bank for representing the distribution of real face features.

[0030] S420, Calculate the anomaly score based on the distance; the formula for calculating the anomaly score is as follows:

[0031] In the formula, , Represents the distance function. This indicates the characteristics of the merged multi-color space. The k-th prototype feature in the memory bank, where K is the total number of prototype features.

[0032] S430, compare the abnormal score with a preset threshold. If the score exceeds the threshold, the face image to be tested is determined to be an attack sample. If the score does not exceed the threshold, it is determined to be a real face image.

[0033] After completing the fusion of features in multiple color spaces, this invention uses a memory-based anomaly detection mechanism to determine the liveness of face samples.

[0034] During the training phase, the model is trained using only real face samples. First, local features are extracted from the real face samples and mapped into a feature space. A feature memory is constructed using clustering or feature filtering to store multiple prototype features representing the distribution of real face features. During model training, the feature vectors of real face samples are constrained to move closer to the prototype features in the feature memory, resulting in a relatively compact distribution of real face features in the feature space. During the inference phase, for a face sample to be detected, its corresponding local features are extracted, and the distance between these features and each prototype feature in the feature memory is calculated. An anomaly score is generated based on this distance information to measure the degree to which the sample deviates from the distribution of real face features. When the anomaly score exceeds a preset threshold, the face sample is determined to be an attack sample; when the anomaly score does not exceed the threshold, the face sample is determined to be a real face, thus completing the liveness detection process.

[0035] Secondly, this application provides a face liveness detection system that integrates multiple color space features, including: The multi-color space parallel feature extraction module is used to perform multi-color space transformation on the input face image to be tested, obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; The sparse self-attention feature enhancement module is used to perform sparse self-attention enhancement processing on the multi-scale feature representation to obtain the enhanced feature representations of each color space. The multi-color space adaptive fusion module is used to perform weighted fusion of the enhanced color space feature representations according to adaptive fusion weights to obtain the fused multi-color space features. The anomaly detection module based on the memory bank is used to perform anomaly detection on the fused multi-color space features based on the pre-built memory bank, and to determine whether the input face image to be tested is a live face image based on the detected anomaly score.

[0036] In one specific embodiment of this application, the multi-color space parallel feature extraction module includes a color space conversion unit and multiple feature extraction branch units; the sparse self-attention feature enhancement module includes a region partitioning unit and a local attention calculation unit; and the multi-color space adaptive fusion module includes an aggregation unit, a weight generation unit, and a weighted fusion unit. The color space conversion unit is used to convert the input face image to be tested into feature representations in RGB color space, YUV color space and YCbCr color space respectively; The multiple feature extraction branch units are used to extract feature maps from the feature representations under the RGB color space, YUV color space and YCbCr color space through independent deep convolutional neural network branches respectively; extract feature maps from multiple intermediate layers of the deep convolutional neural network of each branch respectively, and perform spatial scale alignment and channel concatenation on the extracted feature maps to form multi-scale feature representations corresponding to the RGB, YUV and YCbCr color spaces. The region division unit is used to divide the multi-scale feature map of each color space branch into multiple non-overlapping local regions in the spatial dimension. The local attention calculation unit independently performs self-attention calculation in each local region to enhance the correlation between features within the region and obtain enhanced local region features; all enhanced local region features are recombined to obtain enhanced color space feature representations.

[0037] The aggregation unit is used to aggregate the feature representations from all the enhanced color spaces to generate a global aggregated feature. The weight generation unit is used to compress and nonlinearly map the global aggregated features to generate a set of adaptive weight coefficients that correspond one-to-one with the color space branches; and to normalize the adaptive weight coefficients to obtain adaptive fusion weights. The weighted fusion unit is used to multiply the enhanced color space feature representations corresponding to each color space branch with the corresponding adaptive fusion weights, and then perform weighted summation to obtain the fused multi-color space features. The anomaly detection module based on the memory bank is used to calculate the distance between the fused multi-color space features and each prototype feature in the memory bank; wherein, the memory bank stores multiple prototype features representing the distribution of real face features; the anomaly score is calculated based on the distance; the anomaly score is compared with a preset threshold, and if it exceeds the threshold, the face image to be tested is determined to be an attack sample, and if it does not exceed the threshold, it is determined to be a real face image.

[0038] The overall implementation process of the face liveness detection of the present invention is as follows: Figure 2 As shown, for the face image samples to be detected, the system directly performs multi-color space modeling processing on the input face image. Since different color spaces have complementary characteristics in representing the brightness and chromaticity information of the face, to facilitate subsequent analysis and modeling of face features through deep learning models, this invention maps the input face image to multiple color space representations, preferably including RGB, YUV, and YCbCr color spaces. For face images in each color space, this invention constructs corresponding deep feature extraction networks to perform parallel feature extraction on images in different color spaces, and embeds the extracted features into fixed-dimensional feature representations. Each feature extraction network models the texture and structural information of the face image from different levels, and obtains multi-scale face feature representations through feature alignment and channel fusion operations, thereby fully characterizing the facial appearance features in different color spaces. After obtaining the feature representations corresponding to each color space, this invention introduces a local sparse self-attention mechanism in each feature branch to perform feature enhancement modeling on local regions in the face image. By modeling the correlation between features within local regions, the model's ability to perceive fine-grained anomalies caused by forgery attacks is enhanced, resulting in a multi-color space feature representation enhanced by local features. Subsequently, features from multiple color space branches are input into a multi-color space adaptive fusion module. This module adaptively calculates the fusion weights of different color space features based on the overall feature distribution of the input face sample, and dynamically weights and fuses the features of each color space to obtain a fused multi-color space feature representation that comprehensively reflects the discriminative information of the face sample in multiple color spaces. After feature fusion, the fused features are input into a memory-based anomaly detection module. During the training phase, the memory is constructed using real face samples to represent the distribution characteristics of real face features in the feature space; during the detection phase, the distance between the fused features of the face sample to be detected and the feature prototypes in the memory is calculated to obtain the corresponding anomaly score. Finally, based on the relationship between the abnormal score and the preset threshold, a liveness determination is performed on the face sample to be detected: when the abnormal score is lower than the preset threshold, the face sample is determined to be a real live face; when the abnormal score is higher than the preset threshold, the face sample is determined to be a fake attack sample, thereby completing the entire face liveness detection process.

[0039] Taking a facial recognition application system as an example, this system is deployed on the server side to perform liveness detection on facial images collected and uploaded by terminal devices, in order to prevent photo attacks, video playback attacks, or other spoofing attacks. In this embodiment, the facial liveness detection model adopts an anomaly detection framework, which is trained only using real facial samples and treats attack samples as abnormal data that deviates from the distribution of real facial features.

[0040] It is worth noting that the terms "first" and "second" in this application are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0041] The above description, in conjunction with specific preferred embodiments, provides a further detailed explanation of this application and should not be construed as limiting the specific implementation of this application to these descriptions. For those skilled in the art, various simple deductions or substitutions can be made without departing from the concept of this application, and all such modifications or substitutions should be considered within the scope of protection of this application.

Claims

1. A method for detecting facial liveness anomalies through multi-color space feature fusion, characterized in that, include: S100, Perform multi-color space transformation on the input face image to be tested to obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; S200, the multi-scale feature representation is subjected to sparse self-attention enhancement processing to obtain the enhanced feature representations of each color space; S300, the enhanced color space feature representations are weighted and fused according to adaptive fusion weights to obtain the fused multi-color space features; S400, based on the pre-built memory bank, performs anomaly detection on the fused multi-color space features, and determines whether the input test face image is a live face image based on the detected anomaly score.

2. The face liveness detection method based on multi-color space feature fusion according to claim 1, characterized in that, S100 includes: S110, convert the input face image to be tested into feature representations in RGB color space, YUV color space and YCbCr color space respectively; S120 extracts feature maps from the feature representations in the RGB color space, YUV color space, and YCbCr color space through independent deep convolutional neural network branches; S130, feature maps are extracted from multiple intermediate layers of the deep convolutional neural network in each branch, and the extracted feature maps are spatially scale aligned and channel-stitched to form multi-scale feature representations corresponding to RGB, YUV and YCbCr color spaces.

3. The face liveness detection method based on multi-color space feature fusion according to claim 1, characterized in that, S200 specifically includes: S210, the multi-scale feature map of each color space branch is divided into multiple non-overlapping local regions in the spatial dimension; S220, Self-attention calculation is performed independently in each local region to enhance the correlation between features within the region, thereby obtaining enhanced features in the local region; S230 reorganizes all local region enhancement features to obtain enhanced color space feature representations.

4. The face liveness detection method based on multi-color space feature fusion according to claim 1, characterized in that, The S300 includes: S310 aggregates the feature representations from all the enhanced color spaces to generate a global aggregated feature. S320, the global aggregated features are compressed and nonlinearly mapped to generate a set of adaptive weight coefficients that correspond one-to-one with the color space branches; S330, normalize the adaptive weight coefficients to obtain adaptive fusion weights; S340, multiply the enhanced color space feature representations corresponding to each color space branch by the corresponding adaptive fusion weights, and then perform weighted summation to obtain the fused multi-color space features.

5. The face liveness detection method based on multi-color space feature fusion according to claim 4, characterized in that, The global aggregation feature described in S310 is represented as follows: ； in, Represents global aggregated features. This represents the characteristics of the RGB color space. This represents the characteristics of the YUV color space. This represents the characteristics of the YCbCr color space. The adaptive weighting coefficients corresponding to each color space branch in S320 are expressed as follows: ； in, , = , This represents the normalization function, and the resulting weights are used to characterize the relative importance of different color space features in the current sample; The characteristics of the merged multi-color space in S340 are represented as follows: ； in, This represents the adaptive weighting coefficients corresponding to the RGB color space. This represents the adaptive weighting coefficients corresponding to the YUV color space. This represents the adaptive weighting coefficient corresponding to the YCbCr color space.

6. The face liveness detection method based on multi-color space feature fusion according to claim 1, characterized in that, The S400 includes: S410, calculate the distance between the fused multi-color space features and each prototype feature in the memory bank; wherein, the memory bank stores multiple prototype features representing the distribution of real human face features; S420, Calculate the anomaly score based on the distance; S430, compare the abnormal score with a preset threshold. If the score exceeds the threshold, the face image to be tested is determined to be an attack sample. If the score does not exceed the threshold, it is determined to be a real face image.

7. The face liveness detection method based on multi-color space feature fusion according to claim 6, characterized in that, The formula for calculating the anomaly score is as follows: In the formula, , Represents the distance function. This indicates the characteristics of the merged multi-color space. The k-th prototype feature in the memory bank, where K is the total number of prototype features.

8. The face liveness detection method based on multi-color space feature fusion according to claim 6, characterized in that, The steps for constructing the memory bank include: a. Input a real face sample; b. Perform multi-color space transformation on the real face sample to obtain the feature representation of the real face sample in multiple color spaces, and extract multi-scale feature representation from the feature representation; c. Perform sparse self-attention enhancement on the multi-scale feature representations corresponding to real face samples to obtain enhanced feature representations for each color space. d. For the color space feature representations corresponding to real face samples, weighted fusion is performed according to adaptive fusion weights to obtain the fused multi-color space features; e. Extract local features from the multi-color space features corresponding to real face samples, and select representative features as prototypes through clustering or feature selection to form a memory bank for representing the distribution of real face features.

9. A face liveness detection system based on multi-color space feature fusion, characterized in that, include: The multi-color space parallel feature extraction module is used to perform multi-color space transformation on the input face image to be tested, obtain the feature representation of the face image to be tested in multiple color spaces, and extract multi-scale feature representation from the feature representation; The sparse self-attention feature enhancement module is used to perform sparse self-attention enhancement processing on the multi-scale feature representation to obtain the enhanced feature representations of each color space. The multi-color space adaptive fusion module is used to perform weighted fusion of the enhanced color space feature representations according to adaptive fusion weights to obtain the fused multi-color space features. The anomaly detection module based on the memory bank is used to perform anomaly detection on the fused multi-color space features based on the pre-built memory bank, and to determine whether the input face image to be tested is a live face image based on the detected anomaly score.

10. The face liveness detection system based on multi-color space feature fusion according to claim 9, characterized in that, The multi-color space parallel feature extraction module includes a color space conversion unit and multiple feature extraction branch units; The sparse self-attention feature enhancement module includes a region partitioning unit and a local attention calculation unit; The multi-color space adaptive fusion module includes an aggregation unit, a weight generation unit, and a weighted fusion unit; The color space conversion unit is used to convert the input face image to be tested into feature representations in RGB color space, YUV color space and YCbCr color space respectively; The multiple feature extraction branch units are used to extract feature maps from the feature representations in the RGB color space, YUV color space and YCbCr color space through independent deep convolutional neural network branches respectively. Feature maps are extracted from multiple intermediate layers of the deep convolutional neural network in each branch, and the extracted feature maps are spatially scale aligned and channel-stitched to form multi-scale feature representations corresponding to RGB, YUV and YCbCr color spaces. The region division unit is used to divide the multi-scale feature map of each color space branch into multiple non-overlapping local regions in the spatial dimension. The local attention calculation unit independently performs self-attention calculation in each local region to enhance the correlation between features within the region and obtain enhanced local region features; all enhanced local region features are recombined to obtain enhanced color space feature representations. The aggregation unit is used to aggregate the feature representations from all the enhanced color spaces to generate a global aggregated feature. The weight generation unit is used to compress and nonlinearly map the global aggregated features to generate a set of adaptive weight coefficients that correspond one-to-one with the color space branches; and to normalize the adaptive weight coefficients to obtain adaptive fusion weights. The weighted fusion unit is used to multiply the enhanced color space feature representations corresponding to each color space branch with the corresponding adaptive fusion weights, and then perform weighted summation to obtain the fused multi-color space features. The anomaly detection module based on the memory bank is used to calculate the distance between the fused multi-color space features and each prototype feature in the memory bank; wherein, the memory bank stores multiple prototype features representing the distribution of real face features; the anomaly score is calculated based on the distance; the anomaly score is compared with a preset threshold, and if it exceeds the threshold, the face image to be tested is determined to be an attack sample, and if it does not exceed the threshold, it is determined to be a real face image.

Images