Hyperspectral face spoofing detection method based on band selection

Abstract

The invention discloses a hyperspectral face spoofing detection method based on band selection, and the method comprises the steps: obtaining a clustering selection band in a band selection process, obtaining a main band containing hyperspectral face data information, and effectively eliminating data redundancy; emphatically optimizing the difference between the reconstructed spectral information and a human face organ prior spectrum; a hyperspectral face spoofing detection model is constructed, and on the basis of a traditional visible light image classification network model, the ability of the model to extract spectral feature information and spatial spectrum joint feature information is enhanced; the clustering selection wave band is sent to the hyperspectral face deception detection model, the hyperspectral face deception detection model with high precision and high detection speed is completed, and the deployment training difficulty of the model is reduced; the model is pre-trained, so that the problem of scarcity of hyperspectral true and false face data is effectively solved, the generalization ability of the model is improved, the problem of overfitting of the model is avoided, and the face deception detection precision is further improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology and pattern recognition technology, and more specifically, to a hyperspectral face deception detection method based on band selection. Background Art

[0002] Traditional RGB image-based simulated face spoofing detection technology is easily affected by posture changes, noise, lighting changes and adversarial attacks, and it is difficult to meet the needs of high-precision recognition. However, hyperspectral imaging accurately reflects the physical structure and chemical composition of different organs of the face, and has spectral consistency and robustness. In high-security biometric application scenarios with high recognition accuracy requirements, more and more applications are turning to hyperspectral imaging to mine facial representation features for face recognition.

[0003] Since hyperspectral face images have extremely high dimensions, there is a lot of redundant information in the huge dimensions of the data, which causes the problem of dimensionality curse. Therefore, traditional distance measurement methods, such as Frobenius norm, may not be able to distinguish the data differences between real faces and simulated faces with high accuracy; in addition, the high dimensionality of face data often leads to overfitting of machine learning models, resulting in reduced classification and recognition performance.

[0004] At present, some researchers have proposed to use data dimensionality reduction methods to reduce the interference of redundant information on face recognition models. Chen Q, Sun J, Palade V et al. Hierarchical Clustering Based Band Selection Algorithm for Hyperspectral Face Recognition [J]. IEEE Access, 2019, PP: 1-1. DOI: 10.1109 / ACCESS.2019.2897213 disclosed: Kullback-Liebler divergence is used to calculate the similarity between bands, and hierarchical clustering method is used for clustering; Di W, Zhang L, Zhang D, Pan Q. Studies on Hyperspectral Face Recognition in Visible Spectrum With Feature Band Selection [J]. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans, 2010, 40(6): 1354-1361. DOI: 10.1109 / TSMCA.2010.2052603 discloses: band fusion technology, extracting bands with low redundancy and high representativeness from a hyperspectral face cube and reshaping them into a 2D matrix, on this basis completing feature information fusion and analysis, thereby achieving face recognition.

[0005] In addition, in order to obtain better difference representation and discriminant features when extracting hyperspectral face data, Wu M, Wei D, Zhang L et al. Hyperspectral Face Recognition with Patch-Based Low RankTensor Decomposition and PFFT Symmetry, 2018, 10(12). DOI: 10.3390 / sym10120714. It is disclosed that: Gabor wavelet and oriented gradient histogram features are extracted from the band image corresponding to each band in the hyperspectral face, and a classifier is trained based on collaborative representation; Xie Zhihua, Li Yi, Niu Jieyi. Hyperspectral face recognition based on joint block spectral band optimization and deep features [J]. Journal of Image and Graphics, 2021, 26(12): 2870-2878. It is disclosed that: based on the convolutional neural network VGG12, intra-band features are extracted, deep features of different spectral bands are integrated, and band image features and inter-band information features are extracted using a three-layer stacked autoencoder. The nearest neighbor classifier is used to complete the final detection and recognition.

[0006] For hyperspectral face images, most traditional band selection algorithms can only measure simple linear or nonlinear relationships between adjacent bands, so the selected bands still have strong correlations. In addition, in the band selection process, the role of the main organ features of the face, eyes, nose, mouth, etc. in the detection of true and false face spoofing is ignored, making the selected bands weak in expression. In addition, for hyperspectral face spoofing detection models, most of the current methods use traditional feature extraction operators or migrate RGB image face detection and recognition algorithms, which makes the model representation ability weak and the utilization rate of spectral information low, resulting in low accuracy of face spoofing detection.

[0007] Therefore, there is an urgent need to provide a hyperspectral face spoofing detection method based on band selection that can improve the accuracy of face spoofing detection. Summary of the invention

[0008] In view of this, the present invention provides a hyperspectral face spoofing detection method based on band selection, comprising:

[0009] Band selection, band selection includes:

[0010] Collect raw hyperspectral data;

[0011] Use the attention mechanism to generate an attention mask, use the attention mask to weight-encode the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0012] Using autoencoders to extract and reconstruct features from encoded hyperspectral data;

[0013] Feature extraction and reconstruction, including:

[0014] Use autoencoders to mine spectral semantic features of encoded hyperspectral data and try to construct original hyperspectral data;

[0015] By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized, so that the original hyperspectral data can be restored to the greatest extent and the feature mask can be obtained;

[0016] The correlation between the bands of the encoded hyperspectral data is calculated using the feature mask to obtain the band correlation coefficient matrix;

[0017] The K-means algorithm is used to cluster the encoded hyperspectral data to obtain the cluster selection bands;

[0018] Build a hyperspectral face spoofing detection model, including:

[0019] Migrating visible light image classification networks;

[0020] Modify the number of convolution channels of the first three layers of the visible light image classification network. The number of convolution channels of the first layer after modification is consistent with the number of convolution channels selected by the band, and the number of convolution channels of the second and third layers is increased proportionally;

[0021] Introducing the inverse residual structure to extract the spatial and spectral features of hyperspectral face images;

[0022] Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face spoofing detection model, further enhancing the model's ability to extract spatial spectral semantic features;

[0023] The clustering selected bands are sent to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0024] The hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection;

[0025] Collect real face hyperspectral data and simulated face hyperspectral data, and complete the training of hyperspectral face spoofing detection model based on band selection.

[0026] Optionally, collect raw hyperspectral face data, including:

[0027] The three-dimensional original hyperspectral data is converted into a one-dimensional matrix R, where R contains N bands and L pixels. Let X∈R^(L×N) be the original hyperspectral data.

[0028] Optionally, feature mask M∈{0,1} L×N And the feature learning mapping f satisfies the following relationship:

[0029]

[0030] in, To encode hyperspectral data.

[0031] Optionally, band selection loss function L1:

[0032]

[0033] Among them, α and β are weight parameters; Y i A priori spectral curves characterizing different facial organs; || i is the i-th pixel of M corresponding to X.

[0034] Optionally, the band correlation coefficient matrix is ​​D, and its matrix elements are calculated as follows:

[0035]

[0036] Among them, I iis the i-th row vector of D, indicating the correlation between the i-th band and other bands; j is the j-th vector of D, indicating the correlation between the j-th band and other bands.

[0037] Optionally, use the K-means algorithm to cluster the hyperspectral data, including:

[0038] Random initialization: First, randomly select the data encoding the dimension size of the target band in the hyperspectral data as the initial cluster center;

[0039] Band clustering: Using the band correlation coefficient matrix generated by the feature mask, the distance from each band of the encoded hyperspectral data to the cluster center of each band is calculated, and the band is classified into the category of the cluster center closest to it;

[0040] Calculate new band cluster centers: For each category band, recalculate the band cluster center. The new cluster center is the mean of the band encoding the hyperspectral data in the category to which it belongs.

[0041] Iterative update: Repeat the band clustering and calculate the new band cluster center until the new band cluster center no longer changes;

[0042] Determine the bands: In the classification, the bands with the largest weight ratio are selected as the characteristic bands encoding the hyperspectral data in each classification class. The clustering selection bands include multiple characteristic bands.

[0043] Optionally, construct a facial spatial spectral feature decoupling module, including:

[0044] Introduce the facial spatial attention mechanism and the facial spectral attention mechanism, where the feature map is F and the facial spatial attention map is A spatial and the facial spectral attention map is A spectral :

[0045] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0046] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0047] Among them, σ represents the sigmoid function, and f represents the convolution operation;

[0048] Decoupled feature map F′:

[0049]

[0050] in, Represents element-wise multiplication.

[0051] Optionally, hyperspectral face datasets for pre-training band-selection based hyperspectral face spoofing detection models include: PolyU, CMU, and UWA.

[0052] Compared with the prior art, the hyperspectral face spoofing detection method based on band selection provided by the present invention achieves at least the following beneficial effects:

[0053] The invention provides a hyperspectral face spoofing detection method based on band selection, comprising a band selection process and a process of constructing a hyperspectral face spoofing detection model; the band selection process constructs an autoencoder based on an organ attention mechanism, uses the attention mechanism to generate an attention mask, uses the attention mask to weight-encode the original hyperspectral data, and assigns weights of different bands to obtain encoded hyperspectral data; uses the autoencoder to mine spectral semantic features of the encoded hyperspectral data, and attempts to construct the original hyperspectral data; and optimizes the weight ratio of different bands of the encoded hyperspectral data by minimizing the band selection loss function. It can restore the original hyperspectral data to the greatest extent and obtain the feature mask; use the feature mask to calculate the correlation between the hyperspectral data bands and obtain the band correlation coefficient matrix; cluster the encoded hyperspectral data bands through the K-means algorithm to obtain the clustering selected bands, obtain the main bands containing hyperspectral face data information, and effectively eliminate data redundancy; focus on optimizing the difference between the reconstructed spectral information and the prior spectrum of the facial organs, so that the selected bands pay more attention to the main characteristic bands of the facial organs on the basis of eliminating redundant information, so that the clustered selected bands contain the main facial feature information used for face spoofing detection.

[0054] The present invention provides a hyperspectral face deception detection method based on band selection, which also includes constructing a hyperspectral face deception detection model, and strengthening the model's ability to extract spectral feature information and spatial spectral joint feature information on the basis of a traditional visible light image classification network model; constructing a hyperspectral face deception detection model, first migrating the visible light image classification network and then modifying it to achieve feature extraction of higher spectral dimension data, avoid the loss of data information in the training and learning process, and mine spatial and spectral joint feature representations, while unifying the band selection with the hyperspectral face deception detection model to achieve network-to-network training and improve recognition accuracy; introducing an inverted residual structure in the feature learning network to extract the spatial and spectral features of the hyperspectral face image; constructing a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, and further strengthening the model's spatial spectral semantic feature extraction capability. force; after using the hyperspectral face spoofing detection network to complete the decoupling of the spatial spectral features of the face data, a higher-dimensional feature map that can represent the deep semantic information is obtained, so that the hyperspectral face spoofing detection model can learn the deep facial organ feature representation, and improve the feature expression ability and detection and recognition accuracy of the hyperspectral face spoofing detection model; the clustering selected bands are sent to the hyperspectral face spoofing detection model to complete a high-precision, high-detection speed hyperspectral face spoofing detection model, reducing the difficulty of model deployment and training; the hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection, which effectively solves the problem of scarcity of hyperspectral true and false face data, improves the generalization ability of the model, and avoids the problem of overfitting of the model; real face hyperspectral data and simulated face hyperspectral data are collected to complete the training of the hyperspectral face spoofing detection model based on band selection, and realize high-precision face spoofing detection. The present invention constructs a complete framework of a hyperspectral face spoofing detection method based on band selection. It emphasizes the important bands of different facial organs when selecting bands, and designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective use of spatial and spectral information, and further improve the accuracy of face spoofing detection.

[0055] Of course, any product implementing the present invention does not necessarily need to achieve all of the above technical effects at the same time.

[0056] Further features and advantages of the present invention will become apparent from the following detailed description of exemplary embodiments of the present invention with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1It is a flow chart of a method for detecting face deception based on band selection hyperspectral provided by the present invention;

[0059] Figure 2 is a flow chart of the band selection process;

[0060] Figure 3 is a flowchart of the process of building a hyperspectral face spoofing detection model;

[0061] Figure 4 It is a band selection structure diagram;

[0062] Figure 5 This is the structure diagram of the hyperspectral face spoofing detection network. DETAILED DESCRIPTION

[0063] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0064] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0065] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0066] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0067] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0068] Example 1

[0069] The following combination Figures 1 to 5 To illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention, comprising S1: band selection,

[0070] S1: Band selection includes:

[0071] S11: collect original hyperspectral data;

[0072] S12: Use the attention mechanism to generate an attention mask, use the attention mask to weight encode the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0073] S13: Feature extraction and reconstruction of encoded hyperspectral data using autoencoders;

[0074] Feature extraction and reconstruction, including:

[0075] S131: Use autoencoders to mine spectral semantic features of encoded hyperspectral data and construct original hyperspectral data;

[0076] S132: By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained;

[0077] S14: Calculate the correlation between the bands of the encoded hyperspectral data using the feature mask to obtain a band correlation coefficient matrix;

[0078] S15: cluster the encoded hyperspectral data using the K-means algorithm to obtain cluster selection bands;

[0079] S2: Build a hyperspectral face spoofing detection model, including:

[0080] S21: Migrate the visible light image classification network and modify the number of convolution channels of the first three layers of the visible light image classification network. The number of convolution channels of the modified first layer is consistent with the number of convolution channels of the selected band, and the number of convolution channels of the second and third layers is increased proportionally;

[0081] S22: Introduce the inverse residual structure to extract the spatial and spectral features of the hyperspectral face image;

[0082] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face spoofing detection model, and further enhance the model's spatial spectral semantic feature extraction capability;

[0083] S3: Send the clustering selected bands to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0084] S4: Pre-training the hyperspectral face spoofing detection model based on band selection using the hyperspectral face dataset;

[0085] S5: Collect real face hyperspectral data and simulated face hyperspectral data, and complete the training of hyperspectral face spoofing detection model based on band selection.

[0086] Specifically, S1: Band selection,

[0087] S1: Band selection includes:

[0088] S11: collect original hyperspectral data;

[0089] It should be noted that S11: collecting original hyperspectral data is to convert the three-dimensional original hyperspectral data into a one-dimensional matrix R, R contains N bands and L pixels, let X∈R^(L×N) be the original hyperspectral data, which lays the foundation for the subsequent selection of the original hyperspectral data using the attention mask;

[0090] S12: Use the attention mechanism to generate an attention mask, use the attention mask to weight encode the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0091] It should be noted that, since the attention mechanism has the function of feature selection, the attention mechanism is used to calculate the relevant weights to generate the attention mask, which completes the feature selection of the original hyperspectral data and obtains the encoded hyperspectral data after feature selection;

[0092] S13: Feature extraction and reconstruction of encoded hyperspectral data using autoencoders;

[0093] Feature extraction and reconstruction, including:

[0094] S131: Use autoencoders to mine spectral semantic features of encoded hyperspectral data and construct original hyperspectral data;

[0095] It should be noted that the encoded hyperspectral data is reconstructed and screened using an autoencoder with skip connections. The autoencoder can achieve data compression and reconstruction without supervision. The skip connection can retain more information and enhance the generalization ability of the network on this basis. The autoencoder is trained according to the input enhanced representation bands representing the characteristics of facial organs to reconstruct the original hyperspectral data.

[0096] S132: By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained;

[0097] It should be noted that in the band selection process, the weight ratio of different bands of the encoded hyperspectral data is adjusted by minimizing the band selection loss function to more accurately screen out important feature bands; the difference between the reconstructed spectral information and the prior spectrum of the facial organs is reduced, so that the characterization bands pay more attention to the main feature bands of the facial organs on the basis of eliminating redundant information, so that the characterization bands contain the main facial feature information used for face deception detection;

[0098] S14: Calculate the correlation between the encoded hyperspectral bands using the feature mask to obtain a band correlation coefficient matrix;

[0099] It should be noted that the elements in the band correlation coefficient matrix are the difference values ​​between two bands (elements) of the encoded hyperspectral data (the feature mask is obtained by minimizing the loss function and optimizing the weight ratio of different bands of the encoded hyperspectral data), so as to calculate the correlation between different bands in the encoded hyperspectral data;

[0100] S15: clustering the coded hyperspectral bands using the K-means algorithm to obtain clustering selected bands;

[0101] It should be noted that the difference values ​​of different bands of the encoded hyperspectral data are classified, and the hyperspectral data are classified according to the feature similarity according to the number of bands that need to be selected. For example, if the two bands in the encoded hyperspectral data bands have a large similarity, they are classified into one category, and the characteristic bands containing the hyperspectral face data are obtained, which effectively eliminates data redundancy;

[0102] S2: Build a hyperspectral face spoofing detection model, including:

[0103] S21: Migrate the visible light image classification network and modify the number of convolution channels of the first three layers of the visible light image classification network. The number of convolution channels of the modified first layer is consistent with the number of convolution channels of the selected band, and the number of convolution channels of the second and third layers is increased proportionally;

[0104] Optionally, the visible light image classification network may be a convolutional neural network based on RGB images, and the number of convolution channels of the second and third layers may be proportionally increased by six channel dimensions on the basis of the number of convolution channels of the modified first layer, or the number of convolution channels of the modified first layer may be directly doubled;

[0105] It should be noted that only the number of convolution channels of the first three layers of the visible light image classification network is modified, and the rest of the network structure remains unchanged. The original visible light image classification network structure is still used to lay the foundation for the subsequent connection with the band selection process, avoid the loss of data information in the training and learning process, and explore the joint feature representation of space and spectrum. At the same time, the band selection is unified with the hyperspectral face deception detection model to achieve network-to-network training and improve recognition accuracy. During the training process of the hyperspectral face deception detection model, the convolution kernel weights are adjusted, so that the modified detection and recognition network can learn data features that are conducive to identifying facial organs.

[0106] S22: Introduce the inverse residual structure to extract the spatial and spectral features of the hyperspectral face image;

[0107] It should be noted that in the inverted residual structure, a lightweight dilated convolution is first performed on the input feature map, using 1×1 convolution to introduce more feature expressions and improve the network's spectral feature extraction ability; then a depth-separable convolution is performed, and finally a point convolution is used to reduce the number of convolution channels back to the original number, reducing the amount of image processing computation for hyperspectral face spoofing detection while further strengthening the network's ability to learn spectral features; finally, the output feature map is added to the original input feature map to enable the hyperspectral face spoofing detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face spoofing detection model;

[0108] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face spoofing detection model, and further enhance the model's spatial spectral semantic feature extraction capability;

[0109] It can be understood that after decoupling the spatial spectral features of face data using the hyperspectral face spoofing detection network, a higher-dimensional feature map that can represent deep semantic information is obtained; the features that are helpful in identifying true and false faces, such as the grayscale value differences of feature maps at the corners of the eyes, forehead, lips, etc., are emphasized;

[0110] S3: Send the clustering selected bands to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0111] It can be understood that by connecting band selection with the hyperspectral face spoofing detection model, building a hyperspectral face spoofing detection model framework based on band selection, achieving end-to-end training, and completing the unification of the band selection module and the spoofing detection module, it is possible to effectively mine the deep hyperspectral face data feature representation, improve recognition accuracy, and reduce the difficulty of model deployment and training;

[0112] S4: Pre-training the hyperspectral face spoofing detection model based on band selection using the hyperspectral face dataset;

[0113] It is understandable that by completing the pre-training of the model with the existing hyperspectral face data, the model can effectively mine the spatial spectral characteristics of organs in different parts of the face, making up for the problem of insufficient real face and simulated face data in the actual training process; while greatly improving the recognition accuracy of the deception detection model, it can avoid the problem of overfitting of the model;

[0114] S5: Collect real human face hyperspectral data and simulated human face hyperspectral data, and complete the training of hyperspectral face spoofing detection model based on band selection;

[0115] It can be understood that training the hyperspectral face spoofing detection model can effectively identify the similarity of real faces and effectively identify the difference between simulated faces and real faces;

[0116] It can be understood that the hyperspectral face spoofing detection method based on band selection provided by the present invention includes a band selection process and a hyperspectral face spoofing detection model construction process hyperspectral face recognition model; the band selection process constructs an autoencoder based on the organ attention mechanism, uses the attention mechanism to generate an attention mask, uses the attention mask to weight-encode the original hyperspectral data, assigns weights to different bands, and obtains encoded hyperspectral data; uses the autoencoder to mine spectral semantic features of the encoded hyperspectral data, and attempts to construct the original hyperspectral data; optimizes the different bands of the encoded hyperspectral data by minimizing the band selection loss function. The weight ratio of the spectral information of the human face is set to restore the original hyperspectral data to the greatest extent and obtain the feature mask; the correlation between the bands of the encoded hyperspectral data is calculated using the feature mask to obtain the band correlation coefficient matrix; the encoded hyperspectral data is clustered by the K-means algorithm to obtain the clustering selected bands, and the main bands containing the hyperspectral face data information are obtained to effectively eliminate data redundancy; the difference between the reconstructed spectral information and the prior spectrum of the human face organs is optimized, so that the selected bands pay more attention to the main characteristic bands of the human face organs on the basis of eliminating redundant information, so that the clustered selected bands contain the main facial feature information for face spoofing detection.

[0117] The present invention provides a hyperspectral face deception detection method based on band selection, which also includes constructing a hyperspectral face deception detection model, and strengthening the model's ability to extract spectral feature information and spatial spectral joint feature information on the basis of a traditional visible light image classification network model; constructing a hyperspectral face deception detection model, first migrating the visible light image classification network and then modifying it to achieve feature extraction of higher spectral dimension data, avoid the loss of data information in the training and learning process, and mine spatial and spectral joint feature representations, while unifying the band selection with the hyperspectral face deception detection model to achieve network-to-network training and improve recognition accuracy; introducing an inverted residual structure in the feature learning network to extract the spatial and spectral features of the hyperspectral face image; constructing a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, and further strengthening the model's spatial spectral semantic feature extraction capability. force; after using the hyperspectral face spoofing detection network to complete the decoupling of the spatial spectral features of the face data, a higher-dimensional feature map that can represent the deep semantic information is obtained, so that the hyperspectral face spoofing detection model can learn the deep facial organ feature representation, and improve the feature expression ability and detection and recognition accuracy of the hyperspectral face spoofing detection model; the clustering selected bands are sent to the hyperspectral face spoofing detection model to complete a high-precision, high-detection speed hyperspectral face spoofing detection model, reducing the difficulty of model deployment and training; the hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection, which effectively solves the problem of scarcity of hyperspectral true and false face data, improves the generalization ability of the model, and avoids the problem of overfitting of the model; real face hyperspectral data and simulated face hyperspectral data are collected to complete the training of the hyperspectral face spoofing detection model based on band selection, and realize high-precision face spoofing detection. The present invention constructs a complete framework of a hyperspectral face spoofing detection method based on band selection. It emphasizes the important bands of different facial organs when selecting bands, and designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective use of spatial and spectral information, and further improve the accuracy of face spoofing detection.

[0118] Example 2

[0119] The following combination Figures 1 to 5 , to illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention. On the basis of Example 1, this embodiment defines the feature mask in detail:

[0120] The feature mask M∈{0,1}^(L×N) and the feature learning mapping f satisfy the following relationship:

[0121]

[0122] in, To encode hyperspectral data.

[0123] It should be noted that the encoded hyperspectral data is It is to compare with the original hyperspectral data X; by finding a suitable feature mask M through feature extraction and reconstruction, the characterization bands with appropriate weight ratios can effectively reconstruct the original hyperspectral data X, laying the foundation for comparing correlations;

[0124] Example 3

[0125] The following combination Figures 1 to 5 To illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention, based on Example 2, this embodiment defines the band selection loss function in detail:

[0126] Band selection loss function L1:

[0127]

[0128] Among them, α and β are weight parameters; Y i A priori spectral curves characterizing different facial organs; || i is the i-th pixel of M corresponding to X;

[0129] It should be noted that the autoencoder with jump connection is used as the mapping f, and the network is trained according to the input encoded hyperspectral data to accurately restore the original hyperspectral data, that is, That is, the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This item is used to improve the sparsity of the M weight vector and better filter out important feature bands; Y i A priori spectral curves representing different facial organs, i.e., existing spectral data of different facial organs; This allows the model to focus more on the main feature bands of facial organs on the basis of eliminating redundant information, and makes the characterization bands focus more on the main facial feature bands used for face spoofing detection.

[0130] Example 4

[0131] The following combination Figures 1 to 5 To illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention, this embodiment describes in detail the calculation of the correlation coefficient matrix and the use of K-means algorithm hyperspectral band clustering on the basis of Embodiment 3:

[0132] The band correlation coefficient matrix is ​​D, and the calculation method of its matrix elements is:

[0133]

[0134] I i is the i-th row vector of D, indicating the correlation between the i-th band and other bands; j is the j-th vector of D, indicating the correlation between the j-th band and other bands;

[0135] It should be noted that by minimizing the band loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and the feature mask is obtained; in fact, the weight ratio of M is adjusted and optimized. M is actually the binary mask generated by the attention mechanism, so when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio;

[0136] It can be understood that the elements in the band correlation coefficient matrix are the difference values ​​between the i-th band and the j-th band of the encoded hyperspectral data (M is obtained by minimizing the band loss function and optimizing the weight ratio of different bands of the encoded hyperspectral data), so as to calculate the correlation between different bands in the encoded hyperspectral data. If the similarity of two bands in the encoded hyperspectral data band is large, they are classified into one category, and the characteristic bands containing the hyperspectral face data are obtained, which effectively eliminates data redundancy.

[0137] The K-means algorithm is used to cluster hyperspectral bands, including:

[0138] Random initialization: First, randomly select the data encoding the dimension size of the target band in the hyperspectral data as the initial cluster center;

[0139] Band clustering: Using the band correlation coefficient matrix generated by the feature mask, calculate the distance from each band in the encoded hyperspectral data to the cluster center of each band, and assign the band to the category where the nearest band cluster center is located;

[0140] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding the hyperspectral data in the category to which they belong.

[0141] Iterative update: Repeat the band clustering and calculate the new band cluster center until the new band cluster center no longer changes;

[0142] Determine the band: In the classification, the band with the largest weight ratio is selected as the characteristic band for encoding hyperspectral data in each classification. The cluster selection band includes multiple characteristic bands;

[0143] It should be noted that the K-means algorithm, i.e., k-means clustering, is a special case of using the maximum expectation algorithm to solve the Gaussian mixture model when the covariance of the normal distribution is the unit matrix and the posterior distribution of the latent variable is a set of Dirac functions; the design concept of the K-means clustering algorithm is intuitive and clear, easy to understand and implement; the K-means clustering algorithm has fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability.

[0144] Example 5

[0145] The following combination Figures 1 to 5 To illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention, comprising S1: band selection,

[0146] S1: Band selection includes:

[0147] S11: collect original hyperspectral data;

[0148] Collecting the original hyperspectral data is to convert the three-dimensional original hyperspectral data into a one-dimensional matrix R, where R contains N bands and L pixels. Let X∈R^(L×N) be the original hyperspectral data, which lays the foundation for using the attention mask to encode the weight of the original hyperspectral data.

[0149] S12: Use the attention mechanism to generate an attention mask, use the attention mask to weight encode the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0150] S13: Feature extraction and reconstruction of encoded hyperspectral data using autoencoders;

[0151] Feature extraction and reconstruction, including:

[0152] S131: Use autoencoders to mine spectral semantic features of encoded hyperspectral data and construct original hyperspectral data;

[0153] S132: By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained;

[0154] The hyperspectral mask weight M∈{0,1}^(L×N) and the feature learning mapping f satisfy the following relationship:

[0155]

[0156] Where M is the encoded hyperspectral data of the enhanced feature characterization band;

[0157] It should be noted that the encoded hyperspectral data is In order to compare with the original facial spectrum X, a suitable hyperspectral weight mask M is found through feature extraction and reconstruction. The appropriate feature characterization band spectral information can effectively reconstruct the original hyperspectral data X, laying the foundation for comparing correlations;

[0158] Band selection loss function L1:

[0159]

[0160] Among them, α and β are weight parameters; Y i A priori spectral curves characterizing different facial organs; || i is the i-th pixel of M corresponding to X;

[0161] It should be noted that the autoencoder with jump connection is used as the mapping f, and the spectrum training f is selected according to the input features so that it can accurately restore the original hyperspectral data, that is, That is, the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This item is used to improve the sparsity of the M weight vector and better filter out important feature bands; Y i A priori spectral curves representing different facial organs, i.e., existing spectral data of different facial organs; The model focuses more on the main characteristic bands of facial organs on the basis of eliminating redundant information, so that the representation bands retain the main facial feature information for face spoofing detection;

[0162] S14: Calculate the correlation between hyperspectral bands using the optimized attention mask to obtain the correlation coefficient matrix;

[0163] The band correlation coefficient matrix is ​​D, and the calculation method of its matrix elements is:

[0164]

[0165] I i is the i-th row vector of D, indicating the correlation between the i-th band and other bands; j is the j-th vector of D, indicating the correlation between the j-th band and other bands;

[0166] It should be noted that by minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and the feature mask is obtained; in fact, the weight ratio of M is adjusted and optimized, and M is the binary mask generated by the attention mechanism. Therefore, when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio; according to the weight mask matrix M, the correlation between the encoded spectral data bands is calculated to obtain the band correlation coefficient matrix;

[0167] It can be understood that the elements in the band correlation coefficient matrix are the difference values ​​between the i-th band and the j-th band of the hyperspectral data (M obtained by minimizing the loss function and optimizing the weight ratio of different bands of the encoded hyperspectral data), so as to calculate the correlation between different bands in the encoded hyperspectral data;

[0168] S15: clustering the coded hyperspectral bands using the K-means algorithm to obtain clustering selected bands;

[0169] It should be noted that the difference values ​​of different bands of the encoded hyperspectral data are classified, and the encoded hyperspectral data are classified according to the feature similarity according to the number of bands that need to be selected. For example, if the two bands in the encoded hyperspectral data bands have a large similarity, they are classified into one category, and the characteristic bands containing the hyperspectral face data are obtained, which effectively eliminates data redundancy;

[0170] The K-means algorithm is used to cluster hyperspectral bands, including:

[0171] Random initialization: First, randomly select the data encoding the dimension size of the target band in the hyperspectral data as the initial cluster center;

[0172] Band clustering: Using the correlation coefficient matrix generated by the feature mask, the distance from each band in the encoded hyperspectral data to the cluster center of each band is calculated, and the band is classified into the category where the nearest band cluster center is located;

[0173] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding the hyperspectral data in the category to which they belong.

[0174] Iterative update: Repeat the band clustering and calculate the new band cluster center until the new band cluster center no longer changes;

[0175] Determine the band: In the classification, the band with the largest weight ratio is selected as the characteristic band of the hyperspectral data in each classification. The cluster selection band includes multiple characteristic bands;

[0176] It should be noted that the K-means algorithm, i.e., k-means clustering, is a special case of using the maximum expectation algorithm to solve the Gaussian mixture model when the covariance of the normal distribution is the unit matrix and the posterior distribution of the latent variable is a set of Dirac functions; the design concept of the K-means clustering algorithm is intuitive and clear, easy to understand and implement; the K-means clustering algorithm has fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability;

[0177] S2: Build a hyperspectral face spoofing detection model, including:

[0178] S21: Migrate the visible light image classification network and modify the number of convolution channels of the first three layers of the visible light image classification network. The number of convolution channels of the modified first layer is consistent with the number of convolution channels of the selected band, and the number of convolution channels of the second and third layers is increased proportionally;

[0179] Optionally, the visible light image classification network may be a convolutional neural network based on RGB images; the number of convolution channels of the second layer and the third layer may be proportionally increased by six channel dimensions on the basis of the number of convolution channels of the modified first layer, or the number of convolution channels of the modified first layer may be directly doubled;

[0180] It should be noted that only the number of convolution channels of the first three layers of the visible light image classification network is modified, and the rest of the network structure remains unchanged. The original visible light image classification network structure is still used to lay the foundation for the subsequent connection with the band selection process, avoid the loss of data information in the training and learning process, and explore the joint feature representation of space and spectrum. At the same time, the band selection is unified with the hyperspectral face deception detection model, realizing network-to-network training and improving recognition accuracy.

[0181] S22: Introduce the inverse residual structure to extract the spatial and spectral features of the hyperspectral face image;

[0182] It should be noted that in the inverted residual structure, a lightweight dilated convolution is first performed on the input feature map, using 1×1 convolution to introduce more feature expressions and improve the network's spectral feature extraction ability; then a depth-separable convolution is performed, and finally a point convolution is used to reduce the number of convolution channels back to the original number, reducing the amount of image processing computation for hyperspectral face spoofing detection while further strengthening the network's ability to learn spectral features; finally, the output feature map is added to the original input feature map to enable the hyperspectral face spoofing detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face spoofing detection model;

[0183] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face spoofing detection model, and further enhance the model's spatial spectral semantic feature extraction capability;

[0184] The facial spatial spectral feature decoupling module includes: introducing the facial spatial attention mechanism and the facial spectral attention mechanism, where the feature map is F and the facial spatial attention map is A spatial and the facial spectral attention map is A spectral :

[0185] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0186] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0187] Among them, σ represents the sigmoid function, and f represents the convolution operation;

[0188] Decoupled feature map F′:

[0189]

[0190] in, represents element-wise multiplication;

[0191] It should be noted that the attention mechanism is to weight the input value before outputting it, giving greater weight to the places where you want the network to pay attention, and giving smaller weight to the places where you don’t want the network to pay attention;

[0192] Facial spectral attention: It is divided into two parts: compression and excitation. The purpose of the compression part is to compress the global face spatial texture information, and then perform feature learning in the spectral dimension to form the importance of each spectral dimension information. Finally, the excitation part assigns different weights to the spectral information of the feature map; average pooling and maximum pooling are performed on the feature map respectively to learn the target degree information and discrimination information of the facial organs, compress the spatial size, learn the spectral features, and further send them to the multi-layer perceptron MLP to learn the spectral discrimination features. Finally, sigmoid mapping is used to obtain attention and strengthen the spectral feature learning;

[0193] Facial spatial attention: It transforms various deformation data in space and automatically captures important regional features in face data, focusing on characterizing the discriminative feature positions. It first averages and max-pools in the channel dimension to compress spectral information so that it focuses on learning spatial features, and then splices the feature maps they generate. It further extracts spatial features using a 7×7 convolution kernel, and finally uses sigmoid mapping to obtain attention and strengthen spatial feature learning.

[0194] Feature map F′ after decoupling of spatial and spectral features: During the training process of the hyperspectral face spoofing detection model, the convolution kernel weights are adjusted so that the network can learn data features that are helpful for identifying the authenticity of faces. After decoupling of network training features, a high-dimensional feature map that can represent deep semantic information is obtained; the features that are helpful for identifying authentic and fake faces are emphasized, such as the grayscale value differences of feature maps at the corners of the eyes, forehead, lips, etc.

[0195] S3: Send the clustering selected bands to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0196] S4: Pre-training the hyperspectral face spoofing detection model based on band selection using the hyperspectral face dataset;

[0197] S5: Collect real face hyperspectral data and simulated face hyperspectral data, and complete the training of the hyperspectral face spoofing detection model based on band selection.

[0198] It can be understood that the hyperspectral face spoofing detection method based on band selection provided by the present invention includes a band selection process and a hyperspectral face spoofing detection model construction process hyperspectral face recognition model; the band selection process constructs an autoencoder based on the organ attention mechanism, uses the attention mechanism to generate an attention mask, uses the attention mask to weight-encode the original hyperspectral data, assigns weights to different bands, and obtains encoded hyperspectral data; uses the autoencoder to mine spectral semantic features of the encoded hyperspectral data, and attempts to construct the original hyperspectral data; optimizes the different bands of the encoded hyperspectral data by minimizing the band selection loss function. The weight ratio of the spectral information of the human face is set to restore the original hyperspectral data to the greatest extent and obtain the feature mask; the correlation between the bands of the encoded hyperspectral data is calculated using the feature mask to obtain the band correlation coefficient matrix; the encoded hyperspectral data is clustered by the K-means algorithm to obtain the clustering selected bands, and the main bands containing the hyperspectral face data information are obtained to effectively eliminate data redundancy; the difference between the reconstructed spectral information and the prior spectrum of the human face organs is optimized, so that the selected bands pay more attention to the main characteristic bands of the human face organs on the basis of eliminating redundant information, so that the clustered selected bands contain the main facial feature information for face spoofing detection.

[0199] The present invention provides a hyperspectral face deception detection method based on band selection, which also includes constructing a hyperspectral face deception detection model, and strengthening the model's ability to extract spectral feature information and spatial spectral joint feature information on the basis of a traditional visible light image classification network model; constructing a hyperspectral face deception detection model, first migrating the visible light image classification network and then modifying it to achieve feature extraction of higher spectral dimension data, avoid the loss of data information in the training and learning process, and mine spatial and spectral joint feature representations, while unifying the band selection with the hyperspectral face deception detection model to achieve network-to-network training and improve recognition accuracy; introducing an inverted residual structure in the feature learning network to extract the spatial and spectral features of the hyperspectral face image; constructing a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, and further strengthening the model's spatial spectral semantic feature extraction capability. force; after using the hyperspectral face spoofing detection network to complete the decoupling of the spatial spectral features of the face data, a higher-dimensional feature map that can represent the deep semantic information is obtained, so that the hyperspectral face spoofing detection model can learn the deep facial organ feature representation, and improve the feature expression ability and detection and recognition accuracy of the hyperspectral face spoofing detection model; the clustering selected bands are sent to the hyperspectral face spoofing detection model to complete a high-precision, high-detection speed hyperspectral face spoofing detection model, reducing the difficulty of model deployment and training; the hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection, which effectively solves the problem of scarcity of hyperspectral true and false face data, improves the generalization ability of the model, and avoids the problem of overfitting of the model; real face hyperspectral data and simulated face hyperspectral data are collected to complete the training of the hyperspectral face spoofing detection model based on band selection, and realize high-precision face spoofing detection. The present invention constructs a complete framework of a hyperspectral face spoofing detection method based on band selection. It emphasizes the important bands of different facial organs when selecting bands, and designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective use of spatial and spectral information, and further improve the accuracy of face spoofing detection.

[0200] Example 6

[0201] The following combination Figures 1 to 5 To illustrate a specific embodiment of a hyperspectral face spoofing detection method based on band selection provided by the present invention, comprising: S1: band selection,

[0202] S1: Band selection includes:

[0203] S11: collect original hyperspectral data;

[0204] Collecting the original hyperspectral data is to convert the three-dimensional original hyperspectral data into a one-dimensional matrix R, where R contains N bands and L pixels. Let X∈R^(L×N) be the original hyperspectral data, which lays the foundation for using the attention mask to encode the weight of the original hyperspectral data.

[0205] S12: Use the attention mechanism to generate an attention mask, use the attention mask to weight encode the original hyperspectral data, assign weights to different bands, and obtain encoded hyperspectral data;

[0206] S13: Feature extraction and reconstruction of encoded hyperspectral data using autoencoders;

[0207] Feature extraction and reconstruction, including:

[0208] S131: Use autoencoders to mine spectral semantic features of encoded hyperspectral data and construct original hyperspectral data;

[0209] S132: By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained;

[0210] The hyperspectral mask weight M∈{0,1}^(L×N) and the feature learning mapping f satisfy the following relationship:

[0211]

[0212] Where M is the encoded hyperspectral data of the enhanced feature characterization band;

[0213] It should be noted that the encoded hyperspectral data is In order to compare with the original facial spectrum X, a suitable hyperspectral weight mask M is found through feature extraction and reconstruction. The appropriate feature characterization band spectral information can effectively reconstruct the original hyperspectral data X, laying the foundation for comparing correlations;

[0214] Band selection loss function L1:

[0215]

[0216] Among them, α and β are weight parameters; Y i Characterize the prior spectral curves of different facial organs; i is the i-th pixel of M corresponding to X;

[0217] It should be noted that the autoencoder with jump connection is used as the mapping f, and the spectrum training f is selected according to the input features so that it can accurately restore the original hyperspectral data, that is, That is, the difference between the reconstructed spectral information after band selection and the original hyperspectral data X; This item is used to improve the sparsity of the M weight vector and better filter out important feature bands; Y i A priori spectral curves representing different facial organs, i.e., existing spectral data of different facial organs; The model focuses more on the main characteristic bands of facial organs on the basis of eliminating redundant information, so that the representation bands retain the main facial feature information for face spoofing detection;

[0218] S14: Calculate the correlation between hyperspectral bands using the optimized attention mask to obtain the correlation coefficient matrix;

[0219] The band correlation coefficient matrix is ​​D, and the calculation method of its matrix elements is:

[0220]

[0221] I i is the i-th row vector of D, indicating the correlation between the i-th band and other bands; j is the j-th vector of D, indicating the correlation between the j-th band and other bands;

[0222] It should be noted that by minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and the feature mask is obtained; in fact, the weight ratio of M is adjusted and optimized, and M is the binary mask generated by the attention mechanism. Therefore, when calculating the correlation coefficient matrix D, the M used is the M (i.e., the feature mask) after minimizing the loss function and optimizing the weight ratio; according to the weight mask matrix M, the correlation between the encoded spectral data bands is calculated to obtain the band correlation coefficient matrix;

[0223] It can be understood that the elements in the band correlation coefficient matrix are the difference values ​​between the i-th band and the j-th band of the hyperspectral data (M obtained by minimizing the loss function and optimizing the weight ratio of different bands of the encoded hyperspectral data), so as to calculate the correlation between different bands in the encoded hyperspectral data;

[0224] S15: clustering the coded hyperspectral bands using the K-means algorithm to obtain clustering selected bands;

[0225] It should be noted that the difference values ​​of different bands of the encoded hyperspectral data are classified, and the encoded hyperspectral data are classified according to the feature similarity according to the number of bands that need to be selected. For example, if the two bands in the encoded hyperspectral data bands have a large similarity, they are classified into one category, and the characteristic bands containing the hyperspectral face data are obtained, which effectively eliminates data redundancy;

[0226] The K-means algorithm is used to cluster hyperspectral bands, including:

[0227] Random initialization: First, randomly select the data encoding the dimension size of the target band in the hyperspectral data as the initial cluster center;

[0228] Band clustering: Using the correlation coefficient matrix generated by the feature mask, the distance from each band in the encoded hyperspectral data to the cluster center of each band is calculated, and the band is classified into the category where the nearest band cluster center is located;

[0229] Calculate new cluster centers: For each category, recalculate the band cluster centers. The new cluster centers are the mean values ​​of the bands encoding the hyperspectral data in the category to which they belong.

[0230] Iterative update: Repeat the band clustering and calculate the new band cluster center until the new band cluster center no longer changes;

[0231] Determine the band: In the classification, the band with the largest weight ratio is selected as the characteristic band of the hyperspectral data in each classification. The cluster selection band includes multiple characteristic bands;

[0232] It should be noted that the K-means algorithm, i.e., k-means clustering, is a special case of using the maximum expectation algorithm to solve the Gaussian mixture model when the covariance of the normal distribution is the unit matrix and the posterior distribution of the latent variable is a set of Dirac functions; the design concept of the K-means clustering algorithm is intuitive and clear, easy to understand and implement; the K-means clustering algorithm has fast convergence speed, good clustering effect, simple adjustment of main parameters, high computational efficiency, and strong interpretability;

[0233] S2: Build a hyperspectral face spoofing detection model, including:

[0234] S21: Migrate the visible light image classification network and modify the number of convolution channels of the first three layers of the visible light image classification network. The number of convolution channels of the modified first layer is consistent with the number of convolution channels of the selected band, and the number of convolution channels of the second and third layers is increased proportionally;

[0235] Optionally, the visible light image classification network may be a convolutional neural network based on RGB images; the number of convolution channels of the second layer and the third layer may be proportionally increased by six channel dimensions on the basis of the number of convolution channels of the modified first layer, or the number of convolution channels of the modified first layer may be directly doubled;

[0236] It should be noted that only the number of convolution channels of the first three layers of the visible light image classification network is modified, and the rest of the network structure remains unchanged. The original visible light image classification network structure is still used to lay the foundation for the subsequent connection with the band selection process, avoid the loss of data information in the training and learning process, and explore the joint feature representation of space and spectrum. At the same time, the band selection is unified with the hyperspectral face deception detection model, realizing network-to-network training and improving recognition accuracy.

[0237] S22: Introduce the inverse residual structure to extract the spatial and spectral features of the hyperspectral face image;

[0238] It should be noted that in the inverted residual structure, a lightweight dilated convolution is first performed on the input feature map, using 1×1 convolution to introduce more feature expressions and improve the network's spectral feature extraction ability; then a depth-separable convolution is performed, and finally a point convolution is used to reduce the number of convolution channels back to the original number, reducing the amount of image processing computation for hyperspectral face spoofing detection while further strengthening the network's ability to learn spectral features; finally, the output feature map is added to the original input feature map to enable the hyperspectral face spoofing detection model to learn deep facial organ feature representations and improve the generalization ability of the hyperspectral face spoofing detection model;

[0239] S23: Construct a facial spatial spectral feature decoupling module to obtain a hyperspectral face spoofing detection model, and further enhance the model's spatial spectral semantic feature extraction capability;

[0240] The facial spatial spectral feature decoupling module includes: introducing the facial spatial attention mechanism and the facial spectral attention mechanism, where the feature map is F and the facial spatial attention map is A spatial and the facial spectral attention map is A spectral :

[0241] A spectral =σ((MLP(AvgPool(F)))+MLP(MaxPool(F))),

[0242] A spatial =σ(f([AvgPool(F);MaxPool(F)])),

[0243] Among them, σ represents the sigmoid function, and f represents the convolution operation;

[0244] Decoupled feature map F′:

[0245]

[0246] in, represents element-wise multiplication;

[0247] It should be noted that the attention mechanism is to weight the input value before outputting it, giving greater weight to the places where you want the network to pay attention, and giving smaller weight to the places where you don’t want the network to pay attention;

[0248] Facial spectral attention: It is divided into two parts: compression and excitation. The purpose of the compression part is to compress the global face spatial texture information, and then perform feature learning in the spectral dimension to form the importance of each spectral dimension information. Finally, the excitation part assigns different weights to the spectral information of the feature map; average pooling and maximum pooling are performed on the feature map respectively to learn the target degree information and discrimination information of the facial organs, compress the spatial size, learn the spectral features, and further send them to the multi-layer perceptron MLP to learn the spectral discrimination features. Finally, sigmoid mapping is used to obtain attention and strengthen the spectral feature learning;

[0249] Facial spatial attention: It transforms various deformation data in space and automatically captures important regional features in face data, focusing on characterizing the discriminative feature positions. It first averages and max-pools in the channel dimension to compress spectral information so that it focuses on learning spatial features, and then splices the feature maps they generate. It further extracts spatial features using a 7×7 convolution kernel, and finally uses sigmoid mapping to obtain attention and strengthen spatial feature learning.

[0250] Feature map F′ after decoupling of spatial and spectral features: During the training process of the hyperspectral face spoofing detection model, the convolution kernel weights are adjusted so that the network can learn data features that are helpful for identifying the authenticity of faces. After decoupling of network training features, a high-dimensional feature map that can represent deep semantic information is obtained; the features that are helpful for identifying authentic and fake faces are emphasized, such as the grayscale value differences of feature maps at the corners of the eyes, forehead, lips, etc.

[0251] S3: Send the clustering selected bands to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection;

[0252] S4: Pre-training the hyperspectral face spoofing detection model based on band selection using the hyperspectral face dataset;

[0253] Hyperspectral face datasets include: PolyU, CMU, and UWA;

[0254] It should be noted that the hyperspectral face datasets PolyU, CMU, and UWA were selected in turn, and the spectral bands were uniformly reduced to 30. At the same time, a self-supervised learning method was used to construct positive data pairs for the same face image using blurring, rotation, flipping and other methods, and negative data pairs were constructed using different face images. The hyperspectral face deception detection network was trained to enable the network to effectively identify the similarities of the same image and the differences between different images, which was used as the pre-training weight of the hyperspectral face deception detection model based on band selection. The model was fine-tuned and trained in combination with the collected hyperspectral real face and simulated face data, thereby realizing the recognition of true and false faces.

[0255] It is understandable that the model is pre-trained using three widely used hyperspectral face datasets, PolyU, CMU, and UWA, to improve the generalization ability of the hyperspectral face spoofing detection model based on band selection, effectively make up for the problem of insufficient training data for true and false hyperspectral faces, avoid overfitting of the model, and improve the accuracy of face spoofing detection;

[0256] S5: Collect real human face hyperspectral data and simulated human face hyperspectral data, and complete the training of hyperspectral face spoofing detection model based on band selection;

[0257] Specifically, S5: Use real face hyperspectral data and simulated face hyperspectral data to train the hyperspectral face spoofing detection model based on band selection. Let the true and false label of the face image be y. When the hyperspectral image is a real face, y=1, and when the hyperspectral image is a simulated face, y=0. Then the face detection recognition target loss function L2 is:

[0258] L2=-ylogp-(1-y)log (1-p),

[0259] Where p is the probability of the network output that the image is a real face or a simulated face;

[0260] Loss function L of the hyperspectral face spoofing detection model based on band selection:

[0261] L=L1+L2,

[0262] It can be understood that the model is fine-tuned and trained by combining the collected hyperspectral real face and simulated face data, so as to achieve high-precision recognition of real and fake faces;

[0263] The invention provides a hyperspectral face spoofing detection method based on band selection, comprising a band selection process and a hyperspectral face spoofing detection model construction process; the band selection process constructs an autoencoder based on an organ attention mechanism, uses the attention mechanism to generate an attention mask, uses the attention mask to weight-encode the original hyperspectral data, assigns weights to different bands, and obtains encoded hyperspectral data; uses the autoencoder to mine spectral semantic features of the encoded hyperspectral data, and attempts to construct the original hyperspectral data; optimizes the weights of different bands of the encoded hyperspectral data by minimizing the band selection loss function. ratio, so that it can restore the original hyperspectral data to the greatest extent and obtain the feature mask; use the feature mask to calculate the correlation between the encoded hyperspectral data bands and obtain the band correlation coefficient matrix; use the K-means algorithm to cluster the encoded hyperspectral data bands to obtain the clustering selected bands, obtain the main bands containing the hyperspectral face data information, and effectively eliminate data redundancy; focus on optimizing the difference between the reconstructed spectral information and the prior spectrum of the facial organs, so that the selected bands pay more attention to the main characteristic bands of the facial organs on the basis of eliminating redundant information, so that the clustered selected bands contain the main facial feature information for face spoofing detection.

[0264] The present invention provides a hyperspectral face deception detection method based on band selection, which also includes constructing a hyperspectral face deception detection model, and strengthening the model's ability to extract spectral feature information and spatial spectral joint feature information on the basis of a traditional visible light image classification network model; constructing a hyperspectral face deception detection model, first migrating the visible light image classification network and then modifying it to achieve feature extraction of higher spectral dimension data, avoid the loss of data information in the training and learning process, and mine spatial and spectral joint feature representations, while unifying the band selection with the hyperspectral face deception detection model to achieve network-to-network training and improve recognition accuracy; introducing an inverted residual structure in the feature learning network to extract the spatial and spectral features of the hyperspectral face image; constructing a facial spatial spectral feature decoupling module to obtain a hyperspectral face deception detection model, and further strengthening the model's spatial spectral semantic feature extraction capability. force; after using the hyperspectral face spoofing detection network to complete the decoupling of the spatial spectral features of the face data, a higher-dimensional feature map that can represent the deep semantic information is obtained, so that the hyperspectral face spoofing detection model can learn the deep facial organ feature representation, and improve the feature expression ability and detection and recognition accuracy of the hyperspectral face spoofing detection model; the clustering selected bands are sent to the hyperspectral face spoofing detection model to complete a high-precision, high-detection speed hyperspectral face spoofing detection model, reducing the difficulty of model deployment and training; the hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection, which effectively solves the problem of scarcity of hyperspectral true and false face data, improves the generalization ability of the model, and avoids the problem of overfitting of the model; real face hyperspectral data and simulated face hyperspectral data are collected to complete the training of the hyperspectral face spoofing detection model based on band selection, and realize high-precision face spoofing detection. The present invention constructs a complete framework of a hyperspectral face spoofing detection method based on band selection. It emphasizes the important bands of different facial organs when selecting bands, and designs a powerful spatial and spectral feature extraction method for hyperspectral face data to obtain deep semantic features, realize the effective use of spatial and spectral information, and further improve the accuracy of face spoofing detection.

[0265] Although some specific embodiments of the present invention have been described in detail by way of example, it will be appreciated by those skilled in the art that the above examples are for illustration only and are not intended to limit the scope of the present invention. It will be appreciated by those skilled in the art that the above embodiments may be modified without departing from the scope and spirit of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A hyperspectral face spoofing detection method based on band selection, characterized in that: include: Band selection, the band selection includes: Collect raw hyperspectral data; Using an attention mechanism to generate an attention mask, using the attention mask to weight encode the original hyperspectral data, assigning weights to different bands, and obtaining encoded hyperspectral data; Using an autoencoder to extract and reconstruct features of the encoded hyperspectral data; The feature extraction and the reconstruction include: Using the autoencoder to mine spectral semantic features of the encoded hyperspectral data, and attempting to construct the original hyperspectral data; By minimizing the band selection loss function, the weight ratio of different bands of the encoded hyperspectral data is optimized so that the original hyperspectral data can be restored to the greatest extent and a feature mask is obtained; Calculating the correlation between the bands of the encoded hyperspectral data using the feature mask to obtain a band correlation coefficient matrix; The encoded hyperspectral data are clustered using a K-means algorithm to obtain cluster selection bands; a hyperspectral face deception detection model is constructed, including: Migrating visible light image classification networks; Modify the number of convolution channels of the first three layers of the visible light image classification network, the number of convolution channels of the modified first layer is consistent with the number of convolution channels selected by the band, and the number of convolution channels of the second and third layers is increased proportionally; Introducing the inverse residual structure to extract the spatial and spectral features of hyperspectral face images; Constructing a facial spatial spectral feature decoupling module to obtain the hyperspectral face deception detection model, further enhancing the spatial spectral semantic feature extraction capability of the model; Sending the cluster selected bands to the hyperspectral face spoofing detection model to obtain an end-to-end hyperspectral face spoofing detection model based on band selection; The hyperspectral face dataset is used to pre-train the hyperspectral face spoofing detection model based on band selection; Collect real face hyperspectral data and simulated face hyperspectral data, and complete the training of hyperspectral face spoofing detection model based on band selection.

2. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: Collecting the original hyperspectral face data includes: The three-dimensional original hyperspectral data is converted into a one-dimensional matrix R, where R contains N bands and L pixels, and X∈R^(L×N) is the original hyperspectral data.

3. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: The feature mask M∈{0,1} L×N And the feature learning mapping f satisfies the following relationship: in, The encoded hyperspectral data.

4. The hyperspectral face spoofing detection method based on band selection according to claim 3 is characterized in that: The band selection loss function L1: Among them, α and β are weight parameters; Y i Characterize the prior spectral curves of different facial organs; i is the i-th pixel of M corresponding to X.

5. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: The band correlation coefficient matrix is ​​D, and the calculation method of its matrix elements is: Among them, I i is the i-th row vector of D, indicating the correlation between the i-th band and other bands; j is the j-th vector of D, indicating the correlation between the j-th band and other bands.

6. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: Clustering the hyperspectral data using a K-means algorithm includes: Random initialization: firstly, randomly select the data of the dimension size of the target band in the encoded hyperspectral data as the initial clustering center; Band clustering: using the band correlation coefficient matrix generated by the feature mask, calculating the distance between each band of the encoded hyperspectral data and each band cluster center, and assigning the band to the classification of the cluster center closest to it; Calculate new band cluster centers: for each category band, recalculate the band cluster centers, and the new cluster centers are the mean values ​​of the bands of the encoded hyperspectral data in the category to which they belong; Iterative update: repeating the band clustering and calculating the new band cluster center until the new band cluster center no longer changes; Determine the band: in the classification, select the band with the largest weight ratio as the characteristic band of the encoded hyperspectral data in each classification class, and the cluster selection band includes a plurality of the characteristic bands.

7. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: The construction of the facial spatial spectral feature decoupling module comprises: The facial spatial attention mechanism and the facial spectral attention mechanism are introduced, where the feature map is F and the facial spatial attention map is A spatial And the face spectral attention map is A spectral : A spectral =v((MLP(AvgPool(F))+MLP(MaxPool(F))), A spatial =σ(f([AvgPool(F);MaxPool(F)])), Among them, σ represents the sigmoid function, and f represents the convolution operation; Decoupled feature map F′: in, Represents element-wise multiplication.

8. The hyperspectral face spoofing detection method based on band selection according to claim 1 is characterized in that: The hyperspectral face datasets for pre-training the band selection-based hyperspectral face spoofing detection model include: PolyU, CMU and UWA.

Images