A hyperspectral face spoofing detection method based on waveband selection

Abstract

The application discloses a hyperspectral face spoofing detection method based on waveband selection, a clustering selection waveband is obtained in the waveband selection process, main wavebands containing hyperspectral face data information are acquired, and data redundancy is effectively eliminated; difference between reconstructed spectral information and priori spectral information of a face organ is mainly optimized; a hyperspectral face spoofing detection model is constructed, on the basis of a traditional visible light image classification network model, extraction capability of the model for spectral feature information and spatial spectral joint feature information is strengthened; the clustering selection waveband is sent to the hyperspectral face spoofing detection model, a hyperspectral face spoofing detection model with high accuracy and high detection speed is completed, and deployment and training difficulty of the model is reduced; the model is pre-trained, a problem of scarcity of hyperspectral true and false face data is effectively solved, generalization capability of the model is improved, a problem of overfitting of the model is avoided, and face spoofing detection accuracy is further improved.

Description

Technical Field

[0001] This invention relates to the fields of image processing technology and pattern recognition technology, and more specifically, to a hyperspectral face deception detection method based on band selection. Background Technology

[0002] Traditional RGB image-based facial spoofing detection techniques are susceptible to variations in pose, noise, lighting, and adversarial attacks, making them unsuitable for high-precision recognition. Hyperspectral imaging, on the other hand, accurately reflects the physical structure and chemical composition of different facial features, offering both spectral consistency and robustness. In biometric applications requiring high security and accuracy, an increasing number of applications are turning to hyperspectral imaging to mine facial features for face recognition.

[0003] Because hyperspectral facial images have extremely high dimensionality, the vast dimensionality of the data contains a large amount of redundant information, causing the curse of dimensionality. Therefore, traditional distance measurement methods, such as the Frobenius norm, may not be able to accurately distinguish the differences between real and simulated faces. In addition, the high dimensionality of facial data often leads to overfitting in machine learning models, resulting in a decline in classification and recognition performance.

[0004] Currently, some researchers have proposed using data dimensionality reduction methods to reduce the interference of redundant information on face recognition models. Chen Q, Sun J, Palade V et al. Hierarchical Clustering Based Band Selection Algorithm for Hyperspectral Face Recognition[J].IEEE Access,2019,PP:1-1.DOI:10.1109 / ACCESS.2019.2897213 discloses: calculating the similarity between bands using Kullback-Liebler divergence and performing clustering using a hierarchical clustering method; Di W, Zhang L, Zhang D, Pan Q. Studies on Hyperspectral Face Recognition in Visible Spectrum With Feature Band Selection[J].IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2010, 40(6):1354-1361.DOI:10.1109 / TSMCA.2010.2052603 discloses a band fusion technique that extracts bands with low redundancy and high representativeness from a hyperspectral face cube and reshapes it into a 2D matrix. Based on this, feature information fusion and analysis are completed to achieve face recognition.

[0005] Furthermore, in extracting features from hyperspectral face data, to obtain better differential representation and discriminative features, Wu M, Wei D, Zhang L et al. Hyperspectral Face Recognition with Patch-Based Low Rank Tensor Decomposition and PFFT... Symmetry, 2018, 10(12).DOI:10.3390 / sym10120714 discloses: extracting Gabor wavelet and directional gradient histogram features from the band images corresponding to each band in hyperspectral faces, and training a classifier based on cooperative representation; Xie Zhihua, Li Yi, Niu Jieyi. Hyperspectral face recognition combining block spectral band optimization and depth features [J]. Journal of Image and Graphics, 2021, 26(12): 2870-2878 discloses: extracting intra-band features based on convolutional neural network VGG12, fusing depth features of different spectral bands, extracting band image features and inter-band information features using a three-layer stacked autoencoder, and using a nearest neighbor classifier to complete the final detection and recognition.

[0006] For hyperspectral face images, traditional band selection algorithms mostly only measure simple linear or nonlinear relationships between adjacent bands, resulting in strong correlations among the selected bands. Furthermore, the band selection process neglects the role of key facial features such as the face, eyes, nose, and mouth in detecting fake faces, leading to weak expressive power in the selected bands. Moreover, current hyperspectral face deception detection models largely utilize traditional feature extraction operators or transfer learning algorithms from RGB image face detection and recognition algorithms, resulting in weak model representation capabilities and low utilization of spectral information, leading to low accuracy in face deception detection.

[0007] Therefore, there is an urgent need to provide a band-selective hyperspectral face deception detection method that can improve the accuracy of face deception detection. Summary of the Invention

[0008] In view of this, the present invention provides a hyperspectral face spoofing detection method based on band selection, comprising:

[0009] Band selection includes:

[0010] Collect raw hyperspectral data;

[0011] An attention mask is generated using an attention mechanism. The attention mask is then used to weight the original hyperspectral data, assigning weights to different bands to obtain encoded hyperspectral data.

[0012] Feature extraction and reconstruction of encoded hyperspectral data are performed using an autoencoder.

[0013] Feature extraction and reconstruction, including:

[0014] We used an autoencoder to mine spectral semantic features from encoded hyperspectral data and attempted to reconstruct the original hyperspectral data.

[0015] By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained. The correlation between the bands of the encoded hyperspectral data is calculated using the feature mask to obtain the band correlation coefficient matrix.

[0016] The K-means algorithm is used to cluster the encoded hyperspectral data to obtain the cluster selection bands;

[0017] Constructing a hyperspectral face deception detection model, including:

[0018] Transferring visible light image classification networks;

[0019] Modify the number of convolutional channels in the first three layers of the visible light image classification network. The number of convolutional channels in the first layer after modification is consistent with the number of convolutional channels selected by the band, and the number of convolutional channels in the second and third layers is increased proportionally.

[0020] Introducing an inverse residual structure to extract spatial and spectral features from hyperspectral face images;

[0021] A facial spatial spectral feature decoupling module was constructed to obtain a hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0022] Clustering-selected bands are fed into the hyperspectral face deception detection model to obtain an end-to-end hyperspectral face deception detection model based on band selection.

[0023] A band-selection-based hyperspectral face deception detection model was pre-trained using a hyperspectral face dataset;

[0024] Collect real and simulated hyperspectral facial data to train a hyperspectral face deception detection model based on band selection.

[0025] Optionally, raw hyperspectral face data may be acquired, including:

[0026] The original three-dimensional hyperspectral data is converted into a one-dimensional matrix R, which contains N bands and L pixels. Let X∈R^(L×N) be the original hyperspectral data.

[0027] Optionally, the feature mask M∈{0,1} L×N The feature learning mapping f satisfies the following relationship:

[0028]

[0029] in, To encode hyperspectral data.

[0030] Optional, band selection loss function L1:

[0031]

[0032] Where α and β are weighting parameters; Y i Prior spectral curves characterizing different facial organs; i is the i-th pixel of X corresponding to M.

[0033] Optionally, the band correlation coefficient matrix is ​​D, and its elements are calculated as follows:

[0034]

[0035] Where i is the i-th row vector of D, representing the correlation between the i-th band and other bands; j is the j-th vector of D, representing the correlation between the j-th band and other bands.

[0036] Optionally, the K-means algorithm can be used to cluster the hyperspectral data, including:

[0037] Random initialization: First, randomly select the data of the target band dimension from the encoded hyperspectral data as the initial cluster centers;

[0038] Band clustering: Using the band correlation coefficient matrix generated by the feature mask, the distance from each band of the encoded hyperspectral data to the cluster center of each band is calculated, and the band is assigned to the category of the nearest cluster center.

[0039] Calculate new band cluster centers: For each category of bands, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding hyperspectral data in their respective categories.

[0040] Iterative update: Repeat band clustering and calculate new band cluster centers until the new band cluster centers no longer change;

[0041] Band selection: In classification, the band with the largest weight ratio is selected as the feature band encoding hyperspectral data in each category. Clustering selects bands including multiple feature bands.

[0042] Optionally, a facial spatial spectral feature decoupling module is constructed, including:

[0043] We introduce facial spatial attention and facial spectral attention mechanisms, where the feature map is F and the facial spatial attention mapping is A. spatial And facial spectral attention mapping as A spectral :

[0044] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0045] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0046] Where σ represents the sigmoid function and f represents the convolution operation;

[0047] Decoupled feature map F′:

[0048]

[0049] in, This indicates element-wise multiplication.

[0050] Optionally, hyperspectral face datasets for pre-training band-selection-based hyperspectral face spoofing detection models include PolyU, CMU, and UWA.

[0051] Compared with existing technologies, the hyperspectral face spoofing detection method based on band selection provided by this invention achieves at least the following beneficial effects:

[0052] This invention provides a hyperspectral face deception detection method based on band selection, including a band selection process and a process of constructing a hyperspectral face deception detection model. The band selection process constructs an autoencoder based on an organ attention mechanism, uses the attention mechanism to generate an attention mask, and uses the attention mask to weight-encode the original hyperspectral data, assigning weights to different bands to obtain encoded hyperspectral data. The autoencoder is then used to mine spectral semantic features from the encoded hyperspectral data and attempts to reconstruct the original hyperspectral data. By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized, making... It can restore the original hyperspectral data to the greatest extent to obtain the feature mask; use the feature mask to calculate the correlation between the bands of the hyperspectral data to obtain the band correlation coefficient matrix; use the K-means algorithm to perform band clustering on the encoded hyperspectral data to obtain the clustering selection bands, and obtain the main bands containing hyperspectral face data information, effectively eliminating data redundancy; focus on optimizing the difference between the reconstructed spectral information and the prior spectrum of the face organs, so that the selected bands, on the basis of eliminating redundant information, pay more attention to the main feature bands of the face organs, thus the clustering selection bands contain the main facial feature information for face deception detection.

[0053] This invention provides a hyperspectral face deception detection method based on band selection, which further includes constructing a hyperspectral face deception detection model. Based on the traditional visible light image classification network model, the method enhances the model's ability to extract spectral feature information and spatial-spectral joint feature information. The construction of the hyperspectral face deception detection model involves first transferring and modifying the visible light image classification network to achieve feature extraction from higher spectral dimensions, avoiding data information loss during training, and mining spatial and spectral joint feature representations. Simultaneously, it unifies band selection with the hyperspectral face deception detection model, enabling end-to-end network training and improving recognition accuracy. In the feature learning network, an inverse residual structure is introduced to extract spatial and spectral features of the hyperspectral face image. A facial spatial-spectral feature decoupling module is constructed to obtain the hyperspectral face deception detection model, further enhancing the model's ability to extract spatial-spectral semantic features. The method involves: utilizing a hyperspectral face deception detection network to decouple the spatial spectral features of face data, resulting in higher-dimensional feature mappings capable of representing deep semantic information. This enables the hyperspectral face deception detection model to learn deep facial organ feature representations, improving its feature expression capabilities and detection accuracy. Clustering-selected bands are then fed into the hyperspectral face deception detection model, resulting in a high-accuracy, high-speed model that reduces deployment and training complexity. Pre-training the band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset effectively addresses the scarcity of real and fake hyperspectral face data, enhancing the model's generalization ability and preventing overfitting. Finally, collecting real and simulated hyperspectral face data allows for the training of the band-selection-based hyperspectral face deception detection model, achieving high-accuracy face deception detection. This invention constructs a complete framework for hyperspectral face deception detection based on band selection. When selecting bands, it emphasizes the important bands of different facial organs. It also designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective utilization of spatial and spectral information, and further improve the accuracy of face deception detection.

[0054] Of course, any product implementing this invention does not necessarily need to achieve all of the above technical effects simultaneously.

[0055] Other features and advantages of the invention will become clear from the following detailed description of exemplary embodiments of the invention with reference to the accompanying drawings. Attached Figure Description

[0056] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments of the invention and, together with their description, serve to explain the principles of the invention.

[0057] Figure 1This is a flowchart of a band-selective hyperspectral face deception detection method provided by the present invention;

[0058] Figure 2 This is a flowchart of the band selection process;

[0059] Figure 3 This is a flowchart of the process of constructing a hyperspectral face deception detection model;

[0060] Figure 4 This is a band selection structure diagram;

[0061] Figure 5 This is a diagram of the hyperspectral face deception detection network structure. Detailed Implementation

[0062] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of the invention.

[0063] The following description of at least one exemplary embodiment is merely illustrative and is in no way intended to limit the invention or its application or use.

[0064] Techniques, methods, and equipment known to those skilled in the art may not be discussed in detail, but where appropriate, they should be considered part of the specification.

[0065] In all the examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limitations. Therefore, other examples of exemplary embodiments may have different values.

[0066] It should be noted that similar labels and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be discussed further in subsequent figures.

[0067] Example 1

[0068] The following combination Figures 1 to 5 To illustrate a specific embodiment of the hyperspectral face spoofing detection method based on band selection provided by the present invention, the method includes S1: band selection.

[0069] S1: Band selection includes:

[0070] S11: Acquire raw hyperspectral data;

[0071] S12: Use an attention mechanism to generate an attention mask, use the attention mask to weight the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0072] S13: Use an autoencoder to extract and reconstruct features from encoded hyperspectral data;

[0073] Feature extraction and reconstruction, including:

[0074] S131: Use an autoencoder to mine spectral semantic features from encoded hyperspectral data and construct the original hyperspectral data;

[0075] S132: By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained.

[0076] S14: Calculate the correlation between bands of the encoded hyperspectral data using feature masks to obtain the band correlation coefficient matrix;

[0077] S15: Use the K-means algorithm to cluster the encoded hyperspectral data to obtain the cluster selection bands;

[0078] S2: Construct a hyperspectral face deception detection model, including:

[0079] S21: Transfer the visible light image classification network, modify the number of convolution channels in the first 3 layers of the visible light image classification network, the number of convolution channels in the modified first layer is consistent with the number of convolution channels in the selected band, and the number of convolution channels in the second and third layers is increased proportionally;

[0080] S22: Introduce inverse residual structure to extract spatial and spectral features of hyperspectral face images;

[0081] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0082] S3: The cluster-selected bands are sent to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0083] S4: Pre-train a band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset;

[0084] S5: Collect real face hyperspectral data and simulated face hyperspectral data to complete the training of a hyperspectral face deception detection model based on band selection.

[0085] Specifically, S1: band selection,

[0086] S1: Band selection includes:

[0087] S11: Acquire raw hyperspectral data;

[0088] It should be noted that S11: Acquiring the raw hyperspectral data involves converting the three-dimensional raw hyperspectral data into a one-dimensional matrix R, which contains N bands and L pixels. Let X∈R^(L×N) be the raw hyperspectral data, which lays the foundation for selecting the raw hyperspectral data using attention masks later.

[0089] S12: Use an attention mechanism to generate an attention mask, use the attention mask to weight the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0090] It should be noted that since the attention mechanism has the function of feature selection, the attention mechanism is used to calculate the relevant weights to generate the attention mask, thus completing the feature selection of the original hyperspectral data and obtaining the coded hyperspectral data after feature selection.

[0091] S13: Use an autoencoder to extract and reconstruct features from encoded hyperspectral data;

[0092] Feature extraction and reconstruction, including:

[0093] S131: Use an autoencoder to mine spectral semantic features from encoded hyperspectral data and construct the original hyperspectral data;

[0094] It should be noted that an autoencoder with skip connections is used to reconstruct and filter the encoded hyperspectral data. The autoencoder can achieve data compression and reconstruction in an unsupervised manner, and the skip connections can retain more information and enhance the network's generalization ability. The autoencoder is trained based on the enhanced representation bands representing facial organ features of the input to reconstruct the original hyperspectral data.

[0095] S132: By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained.

[0096] It should be noted that during the band selection process, the weight ratio of different bands in the encoded hyperspectral data is adjusted by minimizing the band selection loss function, so as to more accurately select important feature bands; reduce the difference between the reconstructed spectral information and the prior spectrum of the facial organs, so that the characterization bands focus more on the main feature bands of the facial organs on the basis of eliminating redundant information, thus the characterization bands contain the main facial feature information used for face deception detection.

[0097] S14: Calculate the correlation between encoded hyperspectral bands using feature masks to obtain the band correlation coefficient matrix;

[0098] It should be noted that the elements in the band correlation coefficient matrix are the difference values ​​between two bands (elements) of the encoded hyperspectral data (obtained by minimizing the loss function and optimizing the weight ratio of different bands in the encoded hyperspectral data to obtain the feature mask), thereby calculating the correlation between different bands in the encoded hyperspectral data;

[0099] S15: Use the K-means algorithm to cluster the encoded hyperspectral bands to obtain the cluster selection bands;

[0100] It should be noted that the difference values ​​of different bands in the encoded hyperspectral data are classified. Depending on the number of bands selected, the hyperspectral data is classified according to feature similarity. For example, two bands in the encoded hyperspectral data with high similarity are classified into the same category. This process obtains the characteristic bands containing hyperspectral face data, effectively eliminating data redundancy.

[0101] S2: Construct a hyperspectral face deception detection model, including:

[0102] S21: Transfer the visible light image classification network, modify the number of convolution channels in the first 3 layers of the visible light image classification network, the number of convolution channels in the modified first layer is consistent with the number of convolution channels in the selected band, and the number of convolution channels in the second and third layers is increased proportionally;

[0103] Optionally, the visible light image classification network can be a convolutional neural network based on RGB images. The number of convolutional channels in the second and third layers can be increased by six channels based on the number of convolutional channels in the modified first layer, or the number of convolutional channels in the modified first layer can be doubled.

[0104] It should be noted that only the number of convolutional channels in the first three layers of the visible light image classification network is modified, while the rest of the network structure remains unchanged, still using the original visible light image classification network structure. This lays the foundation for subsequent connection with the band selection process, avoids the loss of data information during training, mines joint spatial and spectral feature representations, and unifies band selection with the hyperspectral face deception detection model, achieving end-to-end network training and improving recognition accuracy. During the training of the hyperspectral face deception detection model, adjusting the convolutional kernel weights allows the modified detection and recognition network to learn data features that are beneficial for recognizing facial organs.

[0105] S22: Introduce inverse residual structure to extract spatial and spectral features of hyperspectral face images;

[0106] It should be noted that in the inverse residual structure, a lightweight dilated convolution is first performed on the input feature map. Using a 1×1 convolution, more feature representations are introduced, improving the network's spectral feature extraction capability. Then, a depthwise separable convolution is performed, and finally, a point convolution is used to reduce the number of convolution channels back to the initial number. This reduces the computational load of hyperspectral face deception detection image processing while further enhancing the network's ability to learn spectral features. Finally, the output feature map is added to the original input feature map, enabling the hyperspectral face deception detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face deception detection model.

[0107] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0108] It is understandable that by using a hyperspectral face deception detection network to decouple the spatial spectral features of face data, a higher-dimensional feature mapping that can represent deep semantic information is obtained; the features that help identify real and fake faces, such as the difference in gray value of feature mapping at the corners of the eyes, forehead, and lips, are emphasized.

[0109] S3: The cluster-selected bands are sent to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0110] It is understandable that connecting band selection with the hyperspectral face deception detection model to build a hyperspectral face deception detection model framework based on band selection, achieving end-to-end training, and unifying the band selection module and the deception detection module can effectively mine deep hyperspectral face data feature representation, improve recognition accuracy, and reduce the difficulty of model deployment and training.

[0111] S4: Pre-train a band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset;

[0112] Understandably, pre-training the model using existing hyperspectral facial data can enable the model to effectively mine the spatial spectral features of different facial organs, making up for the lack of real and simulated facial data in the actual training process; it can significantly improve the recognition accuracy of the deception detection model while avoiding the problem of overfitting.

[0113] S5: Collect real face hyperspectral data and simulated face hyperspectral data to complete the training of a hyperspectral face deception detection model based on band selection;

[0114] It is understandable that training a hyperspectral face deception detection model can effectively identify the similarity of real faces and effectively identify the differences between simulated faces and real faces.

[0115] It is understood that the hyperspectral face spoofing detection method based on band selection provided by this invention includes a band selection process and a hyperspectral face recognition model construction process. The band selection process constructs an autoencoder based on organ attention mechanism, generates an attention mask using the attention mechanism, and uses the attention mask to weight-encode the original hyperspectral data, assigning weights to different bands to obtain encoded hyperspectral data. The autoencoder is then used to mine spectral semantic features from the encoded hyperspectral data and attempts to reconstruct the original hyperspectral data. Finally, by minimizing the band selection loss function, the different bands of the encoded hyperspectral data are optimized. The weight ratios are adjusted to maximize the reconstruction of the original hyperspectral data, resulting in a feature mask. The feature mask is then used to calculate the correlation between the bands of the encoded hyperspectral data, yielding a band correlation coefficient matrix. The encoded hyperspectral data is then clustered using the K-means algorithm to obtain clustered selection bands, capturing the main bands containing hyperspectral facial data information and effectively eliminating data redundancy. The differences between the reconstructed spectral information and the prior spectra of facial organs are optimized, allowing the selected bands to focus more on the main feature bands of facial organs while eliminating redundant information. This ensures that the clustered selection bands contain the main facial feature information used for face deception detection.

[0116] This invention provides a hyperspectral face deception detection method based on band selection, which further includes constructing a hyperspectral face deception detection model. Based on the traditional visible light image classification network model, the method enhances the model's ability to extract spectral feature information and spatial-spectral joint feature information. The construction of the hyperspectral face deception detection model involves first transferring and modifying the visible light image classification network to achieve feature extraction from higher spectral dimensions, avoiding data information loss during training, and mining spatial and spectral joint feature representations. Simultaneously, it unifies band selection with the hyperspectral face deception detection model, enabling end-to-end network training and improving recognition accuracy. In the feature learning network, an inverse residual structure is introduced to extract spatial and spectral features of the hyperspectral face image. A facial spatial-spectral feature decoupling module is constructed to obtain the hyperspectral face deception detection model, further enhancing the model's ability to extract spatial-spectral semantic features. The method involves: utilizing a hyperspectral face deception detection network to decouple the spatial spectral features of face data, resulting in higher-dimensional feature mappings capable of representing deep semantic information. This enables the hyperspectral face deception detection model to learn deep facial organ feature representations, improving its feature expression capabilities and detection accuracy. Clustering-selected bands are then fed into the hyperspectral face deception detection model, resulting in a high-accuracy, high-speed model that reduces deployment and training complexity. Pre-training the band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset effectively addresses the scarcity of real and fake hyperspectral face data, enhancing the model's generalization ability and preventing overfitting. Finally, collecting real and simulated hyperspectral face data allows for the training of the band-selection-based hyperspectral face deception detection model, achieving high-accuracy face deception detection. This invention constructs a complete framework for hyperspectral face deception detection based on band selection. When selecting bands, it emphasizes the important bands of different facial organs. It also designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective utilization of spatial and spectral information, and further improve the accuracy of face deception detection.

[0117] Example 2

[0118] The following combination Figures 1 to 5 This embodiment illustrates a specific implementation of a band-selective hyperspectral face deception detection method provided by the present invention. Based on embodiment 1, this embodiment provides detailed limitations on the feature mask:

[0119] The feature mask M∈{0,1}^(L×N) and the feature learning map f satisfy the following relationship:

[0120]

[0121] in, To encode hyperspectral data.

[0122] It should be noted that the encoded hyperspectral data is The purpose is to compare with the original hyperspectral data X; by extracting and reconstructing features, a suitable feature mask M is found, and the characterization bands with appropriate weight ratios can effectively reconstruct the original hyperspectral data X, laying the foundation for comparing correlations.

[0123] Example 3

[0124] The following combination Figures 1 to 5 To illustrate a specific embodiment of the hyperspectral face spoofing detection method based on band selection provided by the present invention, this embodiment, based on embodiment 2, provides detailed limitations on the band selection loss function:

[0125] Band selection loss function L1:

[0126]

[0127] Where α and β are weighting parameters; Y i Prior spectral curves characterizing different facial organs; i is the i-th pixel of X corresponding to M;

[0128] It should be noted that an autoencoder with skip connections is used as the mapping f. The network is trained based on the input encoded hyperspectral data to accurately reconstruct the original hyperspectral data. This refers to the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This term is used to improve the sparsity of the M weight vector, thus better filtering out important feature bands; Y i Prior spectral curves characterizing different facial organs, i.e., existing spectral data of different facial organs. This allows the model to focus more on the main feature bands of facial organs while eliminating redundant information, making the representation bands more focused on the main facial feature bands used for face deception detection.

[0129] Example 4

[0130] The following combination Figures 1 to 5 This invention provides a specific embodiment of a hyperspectral face spoofing detection method based on band selection. This embodiment, based on Embodiment 3, provides a detailed explanation of the calculation of the correlation coefficient matrix and the use of the K-means algorithm for hyperspectral band clustering:

[0131] The band correlation coefficient matrix is ​​D, and its elements are calculated as follows:

[0132]

[0133] i Let be the i-th row vector of D, representing the correlation between the i-th band and other bands; let j be the j-th vector of D, representing the correlation between the j-th band and other bands.

[0134] It should be noted that by minimizing the band loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained. In fact, the weight ratio of M is adjusted and optimized. M is actually a binary mask generated by the attention mechanism. Therefore, when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio.

[0135] Understandably, the elements in the band correlation coefficient matrix are the difference values ​​between the i-th and j-th bands of the encoded hyperspectral data (obtained by minimizing the band loss function and optimizing the weight ratio of different bands in the encoded hyperspectral data, M). This allows for the calculation of the correlation between different bands in the encoded hyperspectral data. If two bands in the encoded hyperspectral data have a high similarity, they are classified as belonging to the same category. This process obtains the characteristic bands containing hyperspectral face data, effectively eliminating data redundancy.

[0136] Clustering of hyperspectral bands using the K-means algorithm includes:

[0137] Random initialization: First, randomly select the data of the target band dimension from the encoded hyperspectral data as the initial cluster centers;

[0138] Band clustering: Using the band correlation coefficient matrix generated by the feature mask, the distance from each band in the encoded hyperspectral data to the cluster center of each band is calculated, and the band is assigned to the category of the nearest band cluster center;

[0139] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding hyperspectral data in their respective categories.

[0140] Iterative update: Repeat band clustering and calculate new band cluster centers until the new band cluster centers no longer change;

[0141] Band selection: In classification, the band with the largest weight ratio is selected as the feature band encoding hyperspectral data in each category. Clustering selects bands including multiple feature bands.

[0142] It should be noted that the K-means algorithm, or k-means clustering, is a special case of using the expectation-maximization algorithm to solve Gaussian mixture models when the covariance of the normal distribution is the identity matrix and the posterior distribution of the latent variables is a set of Dirac functions. The design concept of the K-means clustering algorithm is intuitive and easy to understand and implement. The K-means clustering algorithm has fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability.

[0143] Example 5

[0144] The following combination Figures 1 to 5 To illustrate a specific embodiment of the hyperspectral face spoofing detection method based on band selection provided by the present invention, the method includes S1: band selection.

[0145] S1: Band selection includes:

[0146] S11: Acquire raw hyperspectral data;

[0147] Acquiring raw hyperspectral data involves converting the three-dimensional raw hyperspectral data into a one-dimensional matrix R, which contains N bands and L pixels. Let X∈R^(L×N) be the raw hyperspectral data, which lays the foundation for subsequent weighted encoding of the raw hyperspectral data using attention masks.

[0148] S12: Use an attention mechanism to generate an attention mask, use the attention mask to weight the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0149] S13: Use an autoencoder to extract and reconstruct features from encoded hyperspectral data;

[0150] Feature extraction and reconstruction, including:

[0151] S131: Use an autoencoder to mine spectral semantic features from encoded hyperspectral data and construct the original hyperspectral data;

[0152] S132: By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained.

[0153] The hyperspectral mask weights M∈{0,1}^(L×N) and the feature learning mapping f satisfy the following relationship:

[0154]

[0155] Where M represents the encoded hyperspectral data of the enhanced feature characterization band;

[0156] It should be noted that the encoded hyperspectral data is The purpose is to compare with the original facial spectrum X. By extracting and reconstructing features, a suitable hyperspectral weight mask M is found. Appropriate feature representation of band spectral information can effectively reconstruct the original hyperspectral data X, laying the foundation for comparison of correlation.

[0157] Band selection loss function L1:

[0158]

[0159] Where α and β are weighting parameters; Y i Prior spectral curves characterizing different facial organs; i is the i-th pixel of X corresponding to M;

[0160] It should be noted that an autoencoder with skip connections is used as the mapping f. The spectrum of f is selected and trained based on the input features to accurately reconstruct the original hyperspectral data. This refers to the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This term is used to improve the sparsity of the M weight vector, thus better filtering out important feature bands; Y i Prior spectral curves characterizing different facial organs, i.e., existing spectral data of different facial organs. This allows the model to focus more on the main feature bands of facial organs while eliminating redundant information, so that the representation bands retain the main facial feature information used for face deception detection.

[0161] S14: Calculate the correlation between hyperspectral bands using the optimized attention mask to obtain the correlation coefficient matrix;

[0162] The band correlation coefficient matrix is ​​D, and its elements are calculated as follows:

[0163]

[0164] i is the i-th row vector of D, representing the correlation between the i-th band and other bands; j is the j-th vector of D, representing the correlation between the j-th band and other bands.

[0165] It should be noted that by minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized to maximize the reconstruction of the original hyperspectral data and obtain the feature mask. In fact, this involves adjusting and optimizing the weight ratio of M, which is a binary mask generated by the attention mechanism. Therefore, when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio. Based on the weight mask matrix M, the correlation between the bands of the encoded spectral data is calculated to obtain the band correlation coefficient matrix.

[0166] It is understandable that the elements in the band correlation coefficient matrix are the difference values ​​between the i-th band and the j-th band of the hyperspectral data (obtained by minimizing the loss function and optimizing the weight ratio of different bands in the encoded hyperspectral data, M), thereby calculating the correlation between different bands in the encoded hyperspectral data.

[0167] S15: Use the K-means algorithm to cluster the encoded hyperspectral bands to obtain the cluster selection bands;

[0168] It should be noted that the difference values ​​of different bands in the encoded hyperspectral data are classified. Depending on the number of bands selected, the encoded hyperspectral data is classified according to feature similarity. For example, two bands in the encoded hyperspectral data with high similarity are classified into the same category. This process obtains the characteristic bands containing hyperspectral face data and effectively eliminates data redundancy.

[0169] Clustering of hyperspectral bands using the K-means algorithm includes:

[0170] Random initialization: First, randomly select the data of the target band dimension from the encoded hyperspectral data as the initial cluster centers;

[0171] Band clustering: Using the correlation coefficient matrix generated by the feature mask, the distance from each band in the encoded hyperspectral data to the cluster center of each band is calculated, and the band is assigned to the category of the nearest band cluster center;

[0172] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding hyperspectral data in their respective categories.

[0173] Iterative update: Repeat band clustering and calculate new band cluster centers until the new band cluster centers no longer change;

[0174] Band selection: In classification, the band with the largest weight ratio is selected as the feature band of hyperspectral data in each category. Clustering selects bands including multiple feature bands.

[0175] It should be noted that the K-means algorithm, or k-means clustering, is a special case of using the expectation-maximization algorithm to solve Gaussian mixture models when the covariance of the normal distribution is the identity matrix and the posterior distribution of the latent variables is a set of Dirac functions. The design concept of the K-means clustering algorithm is intuitive and easy to understand and implement. The K-means clustering algorithm has a fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability.

[0176] S2: Construct a hyperspectral face deception detection model, including:

[0177] S21: Transfer the visible light image classification network, modify the number of convolution channels in the first 3 layers of the visible light image classification network, the number of convolution channels in the modified first layer is consistent with the number of convolution channels in the selected band, and the number of convolution channels in the second and third layers is increased proportionally;

[0178] Optionally, the visible light image classification network can be a convolutional neural network based on RGB images; the number of convolutional channels in the second and third layers can be proportionally increased by six channels based on the modified number of convolutional channels in the first layer, or the number of convolutional channels in the modified first layer can be doubled.

[0179] It should be noted that only the number of convolutional channels in the first 3 layers of the visible light image classification network is modified, while the rest of the network structure remains unchanged. The original visible light image classification network structure is still used to lay the foundation for the subsequent connection with the band selection process, avoid the loss of data information during training and learning, mine the joint spatial and spectral feature representation, and unify the band selection with the hyperspectral face deception detection model to achieve end-to-end training of the network and improve the recognition accuracy.

[0180] S22: Introduce inverse residual structure to extract spatial and spectral features of hyperspectral face images;

[0181] It should be noted that in the inverse residual structure, a lightweight dilated convolution is first performed on the input feature map. Using a 1×1 convolution, more feature representations are introduced, improving the network's spectral feature extraction capability. Then, a depthwise separable convolution is performed, and finally, a point convolution is used to reduce the number of convolution channels back to the initial number. This reduces the computational load of hyperspectral face deception detection image processing while further enhancing the network's ability to learn spectral features. Finally, the output feature map is added to the original input feature map, enabling the hyperspectral face deception detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face deception detection model.

[0182] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0183] The facial spatial spectral feature decoupling module includes: introducing a facial spatial attention mechanism and a facial spectral attention mechanism, where the feature map is F and the facial spatial attention mapping is A. spatial And facial spectral attention mapping as A spectral :

[0184] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0185] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0186] Where σ represents the sigmoid function and f represents the convolution operation;

[0187] Decoupled feature map F′:

[0188]

[0189] in, This represents element-wise multiplication;

[0190] It should be noted that the attention mechanism is to weight the input values ​​before outputting them, giving greater weight to the parts that the network wants to pay attention to and less weight to the parts that the network does not want to pay attention to.

[0191] Facial spectral attention is divided into two parts: compression and activation. The compression part aims to compress the global facial spatial texture information, then learn features in the spectral dimension to form the importance of information in each spectral dimension. Finally, the activation part assigns different weights to the spectral information of the feature map. Average pooling and max pooling are performed on the feature map to learn the target degree information and discrimination information of facial organs, compress the spatial size, learn spectral features, and further feed them into a multilayer perceptron (MLP) to learn spectral discrimination features. Finally, attention is obtained using sigmoid mapping to enhance the learning of spectral features.

[0192] Facial spatial attention: This method transforms various deformed data in space and automatically captures important regional features in facial data, focusing on the representation and discrimination of feature locations. First, average pooling and max pooling are performed in the channel dimension to compress spectral information and make it focus on learning spatial features. Then, the feature maps generated are stitched together. Spatial features are further extracted using 7×7 convolutional kernels. Finally, attention is obtained using sigmoid mapping to enhance spatial feature learning.

[0193] Feature map F′ after spatial spectral feature decoupling: During the training process of the hyperspectral face deception detection model, the weights of the convolution kernels are adjusted so that the network can learn data features that are conducive to identifying the authenticity of faces. After the network training features are decoupled, a high-dimensional feature map that can represent deep semantic information is obtained. Features that help identify real and fake faces are emphasized, such as the difference in gray value of feature maps at the corners of the eyes, forehead, and lips.

[0194] S3: The cluster-selected bands are sent to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0195] S4: Pre-train a band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset;

[0196] S5: Collect real face hyperspectral data and simulated face hyperspectral data to complete the training of a hyperspectral face deception detection model based on band selection.

[0197] It is understood that the hyperspectral face spoofing detection method based on band selection provided by this invention includes a band selection process and a hyperspectral face recognition model construction process. The band selection process constructs an autoencoder based on organ attention mechanism, generates an attention mask using the attention mechanism, and uses the attention mask to weight-encode the original hyperspectral data, assigning weights to different bands to obtain encoded hyperspectral data. The autoencoder is then used to mine spectral semantic features from the encoded hyperspectral data and attempts to reconstruct the original hyperspectral data. Finally, by minimizing the band selection loss function, the different bands of the encoded hyperspectral data are optimized. The weight ratios are adjusted to maximize the reconstruction of the original hyperspectral data, resulting in a feature mask. The feature mask is then used to calculate the correlation between the bands of the encoded hyperspectral data, yielding a band correlation coefficient matrix. The encoded hyperspectral data is then clustered using the K-means algorithm to obtain clustered selection bands, capturing the main bands containing hyperspectral facial data information and effectively eliminating data redundancy. The differences between the reconstructed spectral information and the prior spectra of facial organs are optimized, allowing the selected bands to focus more on the main feature bands of facial organs while eliminating redundant information. This ensures that the clustered selection bands contain the main facial feature information used for face deception detection.

[0198] This invention provides a hyperspectral face deception detection method based on band selection, which further includes constructing a hyperspectral face deception detection model. Based on the traditional visible light image classification network model, the method enhances the model's ability to extract spectral feature information and spatial-spectral joint feature information. The construction of the hyperspectral face deception detection model involves first transferring and modifying the visible light image classification network to achieve feature extraction from higher spectral dimensions, avoiding data information loss during training, and mining spatial and spectral joint feature representations. Simultaneously, it unifies band selection with the hyperspectral face deception detection model, enabling end-to-end network training and improving recognition accuracy. In the feature learning network, an inverse residual structure is introduced to extract spatial and spectral features of the hyperspectral face image. A facial spatial-spectral feature decoupling module is constructed to obtain the hyperspectral face deception detection model, further enhancing the model's ability to extract spatial-spectral semantic features. The method involves: utilizing a hyperspectral face deception detection network to decouple the spatial spectral features of face data, resulting in higher-dimensional feature mappings capable of representing deep semantic information. This enables the hyperspectral face deception detection model to learn deep facial organ feature representations, improving its feature expression capabilities and detection accuracy. Clustering-selected bands are then fed into the hyperspectral face deception detection model, resulting in a high-accuracy, high-speed model that reduces deployment and training complexity. Pre-training the band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset effectively addresses the scarcity of real and fake hyperspectral face data, enhancing the model's generalization ability and preventing overfitting. Finally, collecting real and simulated hyperspectral face data allows for the training of the band-selection-based hyperspectral face deception detection model, achieving high-accuracy face deception detection. This invention constructs a complete framework for hyperspectral face deception detection based on band selection. When selecting bands, it emphasizes the important bands of different facial organs. It also designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective utilization of spatial and spectral information, and further improve the accuracy of face deception detection.

[0199] Example 6

[0200] The following combination Figures 1 to 5 To illustrate a specific embodiment of the hyperspectral face spoofing detection method based on band selection provided by the present invention, the method includes: S1: band selection,

[0201] S1: Band selection includes:

[0202] S11: Acquire raw hyperspectral data;

[0203] Acquiring raw hyperspectral data involves converting the three-dimensional raw hyperspectral data into a one-dimensional matrix R, which contains N bands and L pixels. Let X∈R^(L×N) be the raw hyperspectral data, which lays the foundation for subsequent weighted encoding of the raw hyperspectral data using attention masks.

[0204] S12: Use an attention mechanism to generate an attention mask, use the attention mask to weight the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0205] S13: Use an autoencoder to extract and reconstruct features from encoded hyperspectral data;

[0206] Feature extraction and reconstruction, including:

[0207] S131: Use an autoencoder to mine spectral semantic features from encoded hyperspectral data and construct the original hyperspectral data;

[0208] S132: By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained.

[0209] The hyperspectral mask weights M∈{0,1}^(L×N) and the feature learning mapping f satisfy the following relationship:

[0210]

[0211] Where M represents the encoded hyperspectral data of the enhanced feature characterization band;

[0212] It should be noted that the encoded hyperspectral data is The purpose is to compare with the original facial spectrum X. By extracting and reconstructing features, a suitable hyperspectral weight mask M is found. Appropriate feature representation of band spectral information can effectively reconstruct the original hyperspectral data X, laying the foundation for comparison of correlation.

[0213] Band selection loss function L1:

[0214]

[0215] Where α and β are weighting parameters; Y i Prior spectral curves characterizing different facial organs; i is the i-th pixel of X corresponding to M;

[0216] It should be noted that an autoencoder with skip connections is used as the mapping f. The spectrum of f is selected and trained based on the input features to accurately reconstruct the original hyperspectral data. This refers to the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This term is used to improve the sparsity of the M weight vector, thus better filtering out important feature bands; Y i Prior spectral curves characterizing different facial organs, i.e., existing spectral data of different facial organs. This allows the model to focus more on the main feature bands of facial organs while eliminating redundant information, so that the representation bands retain the main facial feature information used for face deception detection.

[0217] S14: Calculate the correlation between hyperspectral bands using the optimized attention mask to obtain the correlation coefficient matrix;

[0218] The band correlation coefficient matrix is ​​D, and its elements are calculated as follows:

[0219]

[0220] i is the i-th row vector of D, representing the correlation between the i-th band and other bands; j is the j-th vector of D, representing the correlation between the j-th band and other bands.

[0221] It should be noted that by minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized to maximize the reconstruction of the original hyperspectral data and obtain the feature mask. In fact, this involves adjusting and optimizing the weight ratio of M, which is a binary mask generated by the attention mechanism. Therefore, when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio. Based on the weight mask matrix M, the correlation between the bands of the encoded spectral data is calculated to obtain the band correlation coefficient matrix.

[0222] It is understandable that the elements in the band correlation coefficient matrix are the difference values ​​between the i-th band and the j-th band of the hyperspectral data (obtained by minimizing the loss function and optimizing the weight ratio of different bands in the encoded hyperspectral data, M), thereby calculating the correlation between different bands in the encoded hyperspectral data.

[0223] S15: Use the K-means algorithm to cluster the encoded hyperspectral bands to obtain the cluster selection bands;

[0224] It should be noted that the difference values ​​of different bands in the encoded hyperspectral data are classified. Depending on the number of bands selected, the encoded hyperspectral data is classified according to feature similarity. For example, two bands in the encoded hyperspectral data with high similarity are classified into the same category. This process obtains the characteristic bands containing hyperspectral face data and effectively eliminates data redundancy.

[0225] Clustering of hyperspectral bands using the K-means algorithm includes:

[0226] Random initialization: First, randomly select the data of the target band dimension from the encoded hyperspectral data as the initial cluster centers;

[0227] Band clustering: Using the correlation coefficient matrix generated by the feature mask, the distance from each band in the encoded hyperspectral data to the cluster center of each band is calculated, and the band is assigned to the category of the nearest band cluster center;

[0228] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding hyperspectral data in their respective categories.

[0229] Iterative update: Repeat band clustering and calculate new band cluster centers until the new band cluster centers no longer change;

[0230] Band selection: In classification, the band with the largest weight ratio is selected as the feature band of hyperspectral data in each category. Clustering selects bands including multiple feature bands.

[0231] It should be noted that the K-means algorithm, or k-means clustering, is a special case of using the expectation-maximization algorithm to solve Gaussian mixture models when the covariance of the normal distribution is the identity matrix and the posterior distribution of the latent variables is a set of Dirac functions. The design concept of the K-means clustering algorithm is intuitive and easy to understand and implement. The K-means clustering algorithm has a fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability.

[0232] S2: Construct a hyperspectral face deception detection model, including:

[0233] S21: Transfer the visible light image classification network, modify the number of convolution channels in the first 3 layers of the visible light image classification network, the number of convolution channels in the modified first layer is consistent with the number of convolution channels in the selected band, and the number of convolution channels in the second and third layers is increased proportionally;

[0234] Optionally, the visible light image classification network can be a convolutional neural network based on RGB images; the number of convolutional channels in the second and third layers can be proportionally increased by six channels based on the modified number of convolutional channels in the first layer, or the number of convolutional channels in the modified first layer can be doubled.

[0235] It should be noted that only the number of convolutional channels in the first 3 layers of the visible light image classification network is modified, while the rest of the network structure remains unchanged. The original visible light image classification network structure is still used to lay the foundation for the subsequent connection with the band selection process, avoid the loss of data information during training and learning, mine the joint spatial and spectral feature representation, and unify the band selection with the hyperspectral face deception detection model to achieve end-to-end training of the network and improve the recognition accuracy.

[0236] S22: Introduce inverse residual structure to extract spatial and spectral features of hyperspectral face images;

[0237] It should be noted that in the inverse residual structure, a lightweight dilated convolution is first performed on the input feature map. Using a 1×1 convolution, more feature representations are introduced, improving the network's spectral feature extraction capability. Then, a depthwise separable convolution is performed, and finally, a point convolution is used to reduce the number of convolution channels back to the initial number. This reduces the computational load of hyperspectral face deception detection image processing while further enhancing the network's ability to learn spectral features. Finally, the output feature map is added to the original input feature map, enabling the hyperspectral face deception detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face deception detection model.

[0238] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0239] The facial spatial spectral feature decoupling module includes: introducing a facial spatial attention mechanism and a facial spectral attention mechanism, where the feature map is F and the facial spatial attention mapping is A. spatial And facial spectral attention mapping as A spectral :

[0240] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0241] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0242] Where σ represents the sigmoid function and f represents the convolution operation;

[0243] Decoupled feature map F′:

[0244]

[0245] in, This represents element-wise multiplication;

[0246] It should be noted that the attention mechanism is to weight the input values ​​before outputting them, giving greater weight to the parts that the network wants to pay attention to and less weight to the parts that the network does not want to pay attention to.

[0247] Facial spectral attention is divided into two parts: compression and activation. The compression part aims to compress the global facial spatial texture information, then learn features in the spectral dimension to form the importance of information in each spectral dimension. Finally, the activation part assigns different weights to the spectral information of the feature map. Average pooling and max pooling are performed on the feature map to learn the target degree information and discrimination information of facial organs, compress the spatial size, learn spectral features, and further feed them into a multilayer perceptron (MLP) to learn spectral discrimination features. Finally, attention is obtained using sigmoid mapping to enhance the learning of spectral features.

[0248] Facial spatial attention: This method transforms various deformed data in space and automatically captures important regional features in facial data, focusing on the representation and discrimination of feature locations. First, average pooling and max pooling are performed in the channel dimension to compress spectral information and make it focus on learning spatial features. Then, the feature maps generated are stitched together. Spatial features are further extracted using 7×7 convolutional kernels. Finally, attention is obtained using sigmoid mapping to enhance spatial feature learning.

[0249] Feature map F′ after spatial spectral feature decoupling: During the training process of the hyperspectral face deception detection model, the weights of the convolution kernels are adjusted so that the network can learn data features that are conducive to identifying the authenticity of faces. After the network training features are decoupled, a high-dimensional feature map that can represent deep semantic information is obtained. Features that help identify real and fake faces are emphasized, such as the difference in gray value of feature maps at the corners of the eyes, forehead, and lips.

[0250] S3: The cluster-selected bands are sent to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0251] S4: Pre-train a band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset;

[0252] Hyperspectral face datasets include: PolyU, CMU, and UWA;

[0253] It should be noted that the hyperspectral face datasets PolyU, CMU, and UWA were selected sequentially, with the spectral bands uniformly reduced to 30. A self-supervised learning method was employed, using blurring, rotation, and flipping techniques to construct positive example data pairs for the same face image and negative example data pairs for different face images. The hyperspectral face deception detection network was trained to effectively identify the similarity of the same image and the differences between different images, and this was used as the pre-training weights for the band-selection-based hyperspectral face deception detection model. The model was then fine-tuned by combining collected real and simulated hyperspectral face data, thereby achieving the identification of real and fake faces.

[0254] Understandably, using three widely used hyperspectral face datasets, PolyU, CMU, and UWA, to pre-train the model can improve the generalization ability of the band selection-based hyperspectral face deception detection model, effectively make up for the lack of training data for real and fake hyperspectral faces, avoid model overfitting, and improve the accuracy of face deception detection.

[0255] S5: Collect real face hyperspectral data and simulated face hyperspectral data to complete the training of a hyperspectral face deception detection model based on band selection;

[0256] Specifically, S5: The hyperspectral face deception detection model based on band selection is trained using real face hyperspectral data and simulated face hyperspectral data. Let the true / false label of the face image be y. When the hyperspectral image is a real face, y = 1; when the hyperspectral image is a simulated face, y = 0. Then, the loss function L2 for face detection and recognition is:

[0257] L2 = -ylogp - (1-y)log(1-p),

[0258] Where p is the probability output by the network that the image is a real human face or a simulated human face;

[0259] Loss function L of the band-selective hyperspectral face deception detection model:

[0260] L = L1 + L2,

[0261] Understandably, by combining the collected hyperspectral data of real and simulated faces to fine-tune and train the model, a high-accuracy identification of real and fake faces can be achieved.

[0262] This invention provides a hyperspectral face spoofing detection method based on band selection, comprising a band selection process and a hyperspectral face spoofing detection model construction process. The band selection process constructs an autoencoder based on organ attention mechanism, generates an attention mask using the attention mechanism, and uses the attention mask to weight-encode the original hyperspectral data, assigning weights to different bands to obtain encoded hyperspectral data. The autoencoder is then used to mine spectral semantic features from the encoded hyperspectral data and attempts to reconstruct the original hyperspectral data. Finally, the weights of different bands in the encoded hyperspectral data are optimized by minimizing the band selection loss function. The proportions are adjusted to maximize the restoration of the original hyperspectral data, resulting in a feature mask. The feature mask is then used to calculate the correlation between the bands of the encoded hyperspectral data, yielding a band correlation coefficient matrix. The encoded hyperspectral data is then clustered using the K-means algorithm to obtain clustered selection bands, capturing the main bands containing hyperspectral facial data information and effectively eliminating data redundancy. The differences between the reconstructed spectral information and the prior spectra of facial organs are optimized, allowing the selected bands to focus more on the main feature bands of facial organs while eliminating redundant information. This ensures that the clustered selection bands contain the main facial feature information used for face deception detection.

[0263] This invention provides a hyperspectral face deception detection method based on band selection, which further includes constructing a hyperspectral face deception detection model. Based on the traditional visible light image classification network model, the method enhances the model's ability to extract spectral feature information and spatial-spectral joint feature information. The construction of the hyperspectral face deception detection model involves first transferring and modifying the visible light image classification network to achieve feature extraction from higher spectral dimensions, avoiding data information loss during training, and mining spatial and spectral joint feature representations. Simultaneously, it unifies band selection with the hyperspectral face deception detection model, enabling end-to-end network training and improving recognition accuracy. In the feature learning network, an inverse residual structure is introduced to extract spatial and spectral features of the hyperspectral face image. A facial spatial-spectral feature decoupling module is constructed to obtain the hyperspectral face deception detection model, further enhancing the model's ability to extract spatial-spectral semantic features. The method involves: utilizing a hyperspectral face deception detection network to decouple the spatial spectral features of face data, resulting in higher-dimensional feature mappings capable of representing deep semantic information. This enables the hyperspectral face deception detection model to learn deep facial organ feature representations, improving its feature expression capabilities and detection accuracy. Clustering-selected bands are then fed into the hyperspectral face deception detection model, resulting in a high-accuracy, high-speed model that reduces deployment and training complexity. Pre-training the band-selection-based hyperspectral face deception detection model using a hyperspectral face dataset effectively addresses the scarcity of real and fake hyperspectral face data, enhancing the model's generalization ability and preventing overfitting. Finally, collecting real and simulated hyperspectral face data allows for the training of the band-selection-based hyperspectral face deception detection model, achieving high-accuracy face deception detection. This invention constructs a complete framework for hyperspectral face deception detection based on band selection. When selecting bands, it emphasizes the important bands of different facial organs. It also designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective utilization of spatial and spectral information, and further improve the accuracy of face deception detection.

[0264] While specific embodiments of the invention have been described in detail by way of examples, those skilled in the art should understand that the examples are for illustrative purposes only and not intended to limit the scope of the invention. Those skilled in the art should understand that modifications can be made to the above embodiments without departing from the scope and spirit of the invention. The scope of the invention is defined by the appended claims.

Claims

1. A hyperspectral face spoofing detection method based on band selection, characterized in that, include: Band selection, the band selection includes: Collect raw hyperspectral data; An attention mask is generated using an attention mechanism, and the original hyperspectral data is weighted using the attention mask, with weights assigned to different bands to obtain encoded hyperspectral data. The encoded hyperspectral data is then subjected to feature extraction and reconstruction using an autoencoder. The feature extraction and the reconstruction include: The autoencoder is used to mine spectral semantic features from the encoded hyperspectral data in an attempt to reconstruct the original hyperspectral data. By minimizing the band selection loss function, the weight ratio of different bands in the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask can be obtained. The correlation between the bands of the coded hyperspectral data is calculated using the feature mask to obtain the band correlation coefficient matrix; The encoded hyperspectral data is clustered using the K-means algorithm to obtain cluster selection bands; a hyperspectral face deception detection model is constructed, including: Transferring visible light image classification networks; The number of convolutional channels in the first three layers of the visible light image classification network is modified. The number of convolutional channels in the first layer after modification is consistent with the number of convolutional channels in the band selection, and the number of convolutional channels in the second and third layers is increased proportionally. Introducing an inverse residual structure to extract spatial and spectral features from hyperspectral face images; A facial spatial spectral feature decoupling module is constructed to obtain the hyperspectral face deception detection model, further enhancing the model's ability to extract spatial spectral semantic features; The cluster-selected bands are sent to the hyperspectral face deception detection model to obtain an end-to-end hyperspectral face deception detection model based on band selection. A band-selection-based hyperspectral face deception detection model was pre-trained using a hyperspectral face dataset; Collect real and simulated hyperspectral facial data, train a band-selective hyperspectral facial deception detection model, and achieve high-accuracy facial deception detection.

2. The hyperspectral face spoofing detection method based on band selection according to claim 1, characterized in that, The raw hyperspectral face data was collected, including: The original three-dimensional hyperspectral data is converted into a one-dimensional matrix R, which contains N bands and L pixels. Let X∈R^(L×N) be the original hyperspectral data.

3. The hyperspectral face spoofing detection method based on band selection according to claim 1, characterized in that, Clustering of the hyperspectral data using the K-means algorithm includes: Random initialization: First, randomly select the data of the target band dimension size in the encoded hyperspectral data as the initial cluster centers; Band clustering: Using the band correlation coefficient matrix generated by the feature mask, calculate the distance from each band of the encoded hyperspectral data to each band cluster center, and assign the band to the category of the nearest cluster center; Calculate new band cluster centers: For each category of bands, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding the hyperspectral data in their respective categories. Iterative update: Repeat the band clustering and the calculation of new band cluster centers until the new band cluster centers no longer change; Determining bands: In the classification, the band with the largest weight ratio is selected as the feature band of the encoded hyperspectral data in each classification class. The clustering selection of bands includes multiple feature bands.

4. The hyperspectral face spoofing detection method based on band selection according to claim 1, characterized in that, The facial spatial spectral feature decoupling module includes: We introduce facial spatial attention and facial spectral attention mechanisms, where the feature map is F and the facial spatial attention mapping is A. spatial And the facial spectral attention mapping is A spectral : A spectral (F)=σ(MLP(AvgPool(F))+MLP(MaxPool(F))), A spatial (F)=σ(f([AvgPool(F);MaxPool(F)])), Where σ represents the sigmoid function and f represents the convolution operation; Decoupled feature map F′: in, This indicates element-wise multiplication.

5. The hyperspectral face spoofing detection method based on band selection according to claim 1, characterized in that, The hyperspectral face datasets used to pretrain the band-selection-based hyperspectral face deception detection model include PolyU, CMU, and UWA.

Images