A palmprint recognition method based on deep learning and multi-feature fusion

Abstract

This invention discloses a palmprint recognition method based on deep learning and multi-feature fusion, belonging to the field of image processing technology. This method first preprocesses the collected palmprint image through histogram equalization, Gaussian low-pass filtering, Laplacian edge detection, adaptive threshold binarization, corrosion, and connected region merging. This improves the extraction of palmprint main lines and wrinkles, thereby increasing the accuracy of palmprint recognition. In the feature extraction stage, a ResNet network and a Gabor filter are used for joint extraction. Finally, the fused features are input into a KNN for recognition, significantly improving the accuracy and stability of palmprint recognition.

Description

Technical Field

[0001] The present invention relates to the field of biometric technology, and in particular to a palmprint recognition method based on deep learning and multi-feature fusion. Background Art

[0002] Palmprint recognition technology is a branch of biometrics. It leverages advanced computer science techniques with optical and acoustic sensing, as well as biometric principles, to accurately authenticate individuals based on the unique physiological structure of the palm—the skin texture of the hand. Compared to facial and fingerprint recognition, palmprint recognition can identify the palm's structure and internal features with a simple wave of the hand. This offers unique convenience and hygiene advantages, providing a better user experience and making it more adaptable to certain environments, such as those requiring epidemic prevention, than facial recognition. This technology focuses on distinct and differentiated elements on the palm, such as main lines, wrinkles, fine texture endings, and bifurcations. These elements form a complex and stable personalized feature map. However, ensuring the long-term stability and reliability of palmprint recognition systems, especially considering subtle changes in palmprint patterns caused by aging or physiological fluctuations (such as subtle adjustments in vein distribution and soft tissue morphology), places higher demands on feature extraction and matching algorithms. Therefore, feature extraction and recognition are particularly critical for palmprint recognition. Given the richness and uniqueness of palmprint features, palmprint recognition must ensure high discrimination between individuals.

[0003] In palmprint recognition systems, texture features play a crucial role, providing both microscopic and macrostructural information within palmprint images. Mainline features, in particular, are a typical and distinct texture pattern within palmprints. Their effective and precise extraction is directly related to the accuracy and efficiency of the recognition system. Mainline features contain rich individual identification information and can significantly enhance the uniqueness and reliability of identity authentication.

[0004] To efficiently utilize this texture information, Gabor filters are widely used as an image processing technique. Drawing on the biological principles of the human visual system's perception of image texture and edges, the core concept of their design is to mimic the receptive field characteristics of neurons in the visual cortex. This filter operates linearly, combining the spatial locality of the Gaussian function with the temporal or frequency characteristics of the sine function, resulting in an efficient filtering algorithm that captures both local spatial structure and texture frequency information. Specifically, the Gabor filter applies a one-dimensional Gaussian function as a window weight on a two-dimensional plane and then convolves it with sinusoidal plane waves of varying orientations and scales, thereby extracting image texture features at varying locations, orientations, and scales. This design makes the Gabor filter extremely robust to variations in light intensity, enabling it to accurately depict texture details and edge contours in images even under complex lighting conditions.

[0005] In 1985, Professor John Daugman innovatively generalized the Gabor function to two dimensions, constructing the 2D Gabor filter. This groundbreaking work revealed a remarkable fact: the 2D Gabor filter not only achieves the optimal resolution bounds set by the Heisenberg uncertainty principle in both the time and frequency domains, but also exhibits striking consistency with the signal reception patterns of mammalian retinal neurons, responding simultaneously to specific directional and frequency characteristics. Although the basis functions used by the Gabor filter lack perfect orthogonality, meaning that theoretically a finite number of filters cannot fully cover the entire possible image feature space, Daugman pointed out that, from a functional perspective, Gabor filters actually share certain similarities with wavelet filters. Both can analyze image information at multiple resolutions and in multiple directions, providing powerful technical support for image analysis and recognition. As a result, Gabor filters play a key role in various biometric recognition technologies, such as face recognition, fingerprint recognition, and palmprint recognition, and are widely used in image segmentation, image coding, texture analysis, and many other fields.

[0006] ResNet18 is a highly regarded classic deep convolutional neural network model in the field of deep learning. Its most prominent feature and contribution lies in its innovative introduction of a residual learning framework, which effectively addresses the vanishing and exploding gradient problems that can occur during deep neural network training. A core component of ResNet18's architectural design is the so-called "Residual Block." Each residual block consists of at least two convolutional layers, each typically followed by batch normalization and an activation function (such as ReLU). Together, these operations provide a deep abstraction and transformation of input features. However, unlike the traditional stacking of convolutional layers, the residual block is unique in its introduction of a "skip connection" mechanism. Skip connections allow information in the network to bypass some or all of the nonlinear transformation layers, passing directly from input to output. This allows the network to learn the residual between input and output during training, rather than the absolute output value. This approach greatly simplifies the network's optimization objectives, allowing the network to maintain effective propagation of low-level features and sufficient learning of high-level features while increasing its depth, thereby avoiding the problem of gradient vanishing and improving the network's convergence speed and training efficiency. Summary of the Invention

[0007] Purpose of the invention: The present invention aims to provide a palmprint recognition method based on deep learning and multi-feature fusion, which is used to improve the degree of palmprint main line extraction and wrinkle extraction, increase the accuracy of palmprint recognition, and achieve a significant improvement in the accuracy and stability of palmprint recognition.

[0008] Technical solution: A palmprint recognition method based on deep learning and multi-feature fusion, including the following steps:

[0009] S1. Improve the degree and accuracy of palmprint main line extraction and fold extraction through preprocessing;

[0010] The preprocessing method firstly improves the overall contrast of the image by performing histogram equalization, then performs Gaussian blur on the palmprint image by using a Gaussian low-pass filter, and then strengthens the palmprint texture and edge features based on Laplacian edge detection; then adopts adaptive threshold binarization to obtain a binary image, and then performs an erosion operation on the binary image to remove isolated and scattered noise points and burr lines to highlight the main lines of the palmprint; then performs connected region extraction to obtain backbone information in the palmprint image, and finally merges the extracted connected regions by calculating the average value of the pixel sums of all connected regions to obtain a final line image;

[0011] S2. Construct a set of Gabor filters for palmprint image feature extraction, which consists of Gabor filters of different frequencies and directions to achieve maximum coverage of texture features in the palmprint image. Then, convolve the image with each Gabor filter to extract the feature vector of each image.

[0012] S3. Extract features from the original palmprint image using a Resnet network. The Resnet network includes introducing skip connections to add input features to output features to solve the gradient vanishing and gradient exploding problems.

[0013] S4, fusing the features extracted in step S2 and step S3 to generate a comprehensive feature vector, and then classifying the comprehensive feature vector using the K-nearest neighbor algorithm;

[0014] Feature fusion methods include PCA dimensionality reduction fusion, Deep CCA fusion, or ASSF fusion;

[0015] S5. For a new palmprint sample, by calculating its feature distance with each sample in the training set, the nearest K neighbors are selected, and the most common category among these K neighbors is used as the predicted category to complete the palmprint recognition task.

[0016] Furthermore, in step S1, the Gaussian blurring process of the palmprint image using a Gaussian low-pass filter is performed according to the following expression:

[0017]

[0018] Where u and v represent the coordinates of the pixel relative to the center of the kernel, and σ represents the standard deviation of the Gaussian function, which determines the width and blur of the Gaussian kernel. The larger the σ value, the more obvious the blur effect; the smaller the σ value, the less obvious the blur effect. D(u,v) 2 =u 2 +v 2 , represents the feature depth.

[0019] The Gabor filter in step S2 includes using Fourier transform to improve the convolution speed during the convolution operation on the image. The spatial domain of the Gabor filter in two-dimensional space is described as follows:

[0020]

[0021] Among them, σ x and σ y Represents the spatial scale factors of horizontal and vertical directions, ω f and θ f Respectively represent the center frequency and azimuth;

[0022] The above formula can be decomposed into sine filter and cosine filter:

[0023]

[0024]

[0025] Furthermore, the Resnet network described in step S3 includes an input layer, a convolutional layer, and a pooling layer, thereby performing a convolution operation on the input data to extract features, and simultaneously downsampling the feature map to reduce the amount of computation and retain important features;

[0026] The ResNet network also includes a residual block and a residual connection layer. The residual connection layer uses a skip connection to add input features to output features, thereby solving the gradient vanishing and gradient exploding problems.

[0027] The global average pooling layer of the Resnet network performs global average pooling on the feature map, reducing the size of the feature map of each channel to 1x1;

[0028] In this step, since the depth of the feature map is fixed, this layer can convert the feature map into a vector of fixed length to prepare for subsequent dimensionality reduction and classification. Specifically, for each depth slice in the feature map, the average value of all spatial positions of the slice is calculated to generate a 1xD vector, where D represents the depth of the feature map and each element corresponds to the original feature. Figure 1 The average value of the channels.

[0029] Furthermore, the PCA dimensionality reduction fusion described in step S4 is to perform PCA dimensionality reduction on the two feature sets extracted in steps S2 and S3, and then concatenate the low-dimensional feature vectors by column to form a comprehensive feature vector. Column concatenation refers to directly connecting the two feature vectors in the column direction to form a new feature vector. If one feature vector is (a1, a2, a3) and the other feature vector is (b1, b2, b3), then after column concatenation, the result is (a1, a2, a3, b1, b2, b3).

[0030] Furthermore, the Deep CCA fusion described in step S4 regards the low-level features obtained by Gabor filtering and the high-level features extracted by ResNet18 as two different modal data, and then constructs a deep neural network to perform nonlinear transformation on the Gabor features and ResNet features respectively, maximizing the typical correlation coefficient of the two sets of transformed features, so that the features output by the network not only maintain the expressive power within each modality, but also enhance the correlation between the two.

[0031] The ASSF fusion described in step S4 first performs dimensionality reduction processing on the extracted image features through PCA and normalizes them. Then, the module is selected in the ASSF learning model, and the image features extracted in steps S2 and S3 are weighted selected. The weights are dynamically adjusted according to the features of the input image. According to the calculated weights, the two feature subspaces are linearly combined to form a new fused feature space to generate a comprehensive feature vector.

[0032] In the above method, the extracted features are reduced in dimension by PCA, and the distance between the original sample point and the projected reconstructed sample point is:

[0033]

[0034] Where x i is the original sample point, W is the projection matrix, z i is the projection of the sample in the low-dimensional coordinate system, and the projection matrix that minimizes the above formula is what we are looking for.

[0035] Beneficial Effects: This invention provides a method for improving the efficiency of palmprint image recognition, designing a feature extraction and recognition method suitable for multi-classification tasks. Currently, the application scope of palmprint recognition models is increasingly broad, mainly extracting and identifying the main line position. The method of this invention can largely preserve the original palmprint features and improve the accuracy of palmprint recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments are briefly introduced below.

[0037] Figure 1 A schematic diagram of the flow of the palmprint recognition method of the present invention;

[0038] Figure 2 This is an example structure diagram of the present invention performing Gabor filtering for palmprint recognition;

[0039] Figure 3 Schematic diagram of the Resnet network for deep learning method of the present invention;

[0040] Figure 4 This is a schematic diagram of the main lines and wrinkle positions of the palmprint image extracted by the present invention. DETAILED DESCRIPTION

[0041] The technical solutions in the embodiments of the present invention will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present invention.

[0042] Currently, palmprint recognition tends to use ROI to extract the main area, then perform normalization and smoothing. Gabor filters are used for feature extraction, and algorithms such as SVM, KNN, and Euclidean distance are often used for feature recognition. This combination of methods has been widely used, but the resulting extracted main lines may be discontinuous or wrinkle extraction may fail. Furthermore, while these algorithms can achieve good judgment when using classification algorithms such as KNN and SVM, their overall generalization ability is low, and the same accuracy cannot be guaranteed for subsequent datasets.

[0043] In general, the present invention provides a palmprint recognition method based on deep learning and multi-feature fusion. This method collects palmprint images, performs preprocessing processes such as filtering enhancement, edge detection, and binarization, and then uses a Resnet18 network and a Gabor filter to extract multi-dimensional features from the image. The extracted features are then fused as a comprehensive feature vector for recognition using the KNN algorithm, achieving significant improvements in the accuracy and stability of palmprint recognition. Specifically, the present invention first collects a user's palmprint image and preprocesses the image, including histogram equalization, Gaussian low-pass filtering, Laplacian edge detection, adaptive threshold binarization, and connected component analysis. This removes image noise, increases contrast, and highlights features to form a continuous palmprint line pattern. The output is a better image that can mark the main lines and wrinkle locations, thereby improving the accuracy and reliability of palmprint recognition. Then, the Resnet convolutional network model and the Gabor filter in the deep learning network model are used to perform feature extraction on the preprocessed palmprint image, extracting the complex texture, detailed features, and spatial structure information in the palmprint. The features extracted from the two are then fused through the PCA dimensionality reduction and splicing method to produce a comprehensive feature vector. The training and optimization process of the deep learning model enables the system to automatically learn and identify the uniqueness of the palmprint, thereby achieving high-precision feature extraction; Gabor filter extraction can extract the multi-directional and multi-scale texture features of the palmprint by performing convolution calculations on the image. This not only includes the texture features captured by traditional methods, but also incorporates the high-level features mined by deep learning methods. Finally, the present invention uses the fused feature vector as the input of the KNN algorithm for palmprint recognition. Fast and accurate palmprint recognition is achieved. The palmprint recognition technology based on deep learning and multi-feature fusion described in the present invention has the advantages of high recognition accuracy, good stability, and strong adaptability. It is suitable for various security authentication, identity recognition and other scenarios and has high practical application value.

[0044] Specifically, the method of the present invention includes the following steps:

[0045] S1. First, use the histogram equalization operation to improve the overall contrast of the image and make the grayscale distribution of the image more uniform.

[0046] Taking into account the uneven characteristics of palm print images, the following formula is used to use the Gaussian low-pass filter to perform Gaussian blur on the image and filter out the high-frequency components in the image, thereby making the image smooth while retaining the general characteristics of the image.

[0047]

[0048] The next step is to use Laplacian edge detection to enhance the palmprint texture and edge features, highlight the palmprint line details, and obtain the edge information of the image.

[0049] Next, thresholding is applied to the image, using adaptive threshold binarization to convert it into a binary image. Erosion is then performed on the binary image to remove scattered noise and fine lines, retaining only the essential lines. Connected regions are extracted from this processed binary image, removing connected regions smaller than 40 pixels, removing burrs and isolated pixels, and extracting the backbone information in the image. Finally, the extracted connected regions are merged by averaging the pixel sums of all connected regions to produce the final line image.

[0050] S2. Feature extraction stage

[0051] First, we use a two-dimensional Gabor filter for feature extraction. We construct a set of Gabor filters based on the bandwidth, frequency, and other parameters selected for the Gabor filter. We select multiple Gabor filters of varying frequencies and orientations to cover the various texture features in the image. We then convolve the image with each Gabor filter. Using Fourier transforms to increase convolution speed, we extract the feature vectors for each image.

[0052] The two-dimensional Gabor filter space domain is described as follows:

[0053]

[0054] This can be decomposed into a sine filter and a cosine filter:

[0055]

[0056]

[0057] Taking the cosine filter as an example, the Fourier transform is as follows:

[0058]

[0059] S3 and Resnet networks for feature extraction

[0060] ResNet extracts deeper features from the original image. It consists of an input layer; convolutional and pooling layers: These layers perform convolution operations on the input data to extract features. They also downsample the feature maps to reduce computational effort while preserving important features; residual blocks; residual connection layers: These layers introduce skip connections that add input and output features, helping to address vanishing and exploding gradients; and global average pooling layers: These layers perform global average pooling on the feature maps, reducing the size of each channel's feature map to 1x1 and converting them into fixed-length vectors to prepare for subsequent dimensionality reduction and classification.

[0061] S4. PCA is then used to reduce the dimensionality of the two extracted features and the two features are directly merged to complete the feature fusion stage. This generates a comprehensive feature vector that incorporates both the texture features captured by traditional methods and the high-level features mined by deep learning methods.

[0062]

[0063] The above formula is the distance between the original sample point and the sample point reconstructed by projection, where x i is the original sample point, W is the projection matrix, z i is the projection of the sample in the low-dimensional coordinate system, and the projection matrix that minimizes the above formula is what we are looking for.

[0064] S5. Finally, the K-nearest neighbor algorithm is used to classify the vector after feature fusion. Given a new palmprint sample, its feature distance with each sample in the training set is calculated, the nearest K neighbors are selected, and the most common category among these K neighbors is used as the predicted category to complete the palmprint recognition task.

[0065] It is further pointed out that for step S4, the following two methods can also be used:

[0066] ①DEEP CCA

[0067] Gabor filters can extract texture features from palmprint images at different angles and scales, while the residual structure of the ResNet18 network can extract deep semantic features from the image. The low-level features obtained by Gabor filtering and the high-level features extracted by ResNet18 are treated as two different modal data and feature fusion is performed.

[0068] During the feature fusion stage, the Deep CCA method is used. Specifically, by constructing a deep neural network, nonlinear transformations are performed on both Gabor features and ResNet features, maximizing the canonical correlation coefficients (CCCs) between the two sets of transformed features. This ensures that the features output by the network not only maintain the expressive power within each modality but also enhance the correlation between the two. By fusing the features extracted by Gabor filtering and ResNet18 through Deep CCA, the accuracy and robustness of palmprint recognition can be improved while preserving the local texture details of the palmprint and leveraging the powerful abstraction capabilities of deep learning.

[0069] ②ASSF (Adaptive Spatial Feature Fusion)

[0070] A Gabor filter is used to extract texture features from the palmprint image, and a ResNet18 model is used to extract deep features from the image. PCA is then used to reduce the dimensionality of the extracted features (and perform normalization). The selection module learned by ASSF performs a weighted selection of the two feature sets. This selection process is adaptive, dynamically adjusting the weights based on the characteristics of the input image. Based on the calculated weights, the two feature subspaces are linearly combined to form a new fused feature space. This fused feature space is then used for classification or recognition tasks.

[0071] ASSF provides a flexible and efficient way to integrate palmprint features extracted using Gabor filters and deep learning models. It adaptively learns the importance of different levels of features at each location to avoid spatial conflicts. ASSF is also simple to implement and has minimal additional computational cost. Because the optimal fusion search operation is differentiable, it can be easily learned using the backpropagation (BP) algorithm.

[0072] In order to make the features of the present invention more obvious and easy to understand, the following Figure 1-4 The following examples are given.

[0073] A palmprint recognition method based on deep learning and multi-feature fusion, such as Figure 1 As shown, the palmprint image sample set is divided into a training set and a test set. Perform the following steps:

[0074] Step 1: Perform image preprocessing on the original image, including histogram equalization, Gaussian low-pass filter filtering, Laplacian edge detection, adaptive threshold binarization, etc., and then perform corrosion operation, connected region analysis, and merge connected regions to merge adjacent and related connected regions to form a continuous palmprint line image.

[0075] Histogram equalization, as an effective image processing technique, is particularly suitable for enhancing the overall contrast of images with limited dynamic range or uneven grayscale distribution. By redistributing pixel grayscale values, the equalized image exhibits a more uniform grayscale distribution, revealing more details hidden in shadowed or overexposed areas. The Gaussian filter prioritizes low-frequency components, effectively smoothing images and reducing high-frequency noise in palmprint images, such as salt-and-pepper noise or random noise with a Gaussian distribution. The Laplacian operator, an edge detection method based on the second-order derivative of image grayscale, excels at capturing sudden grayscale changes, known as edge features. The Laplacian operator produces a distinct response at locations with abrupt grayscale gradient changes. Due to its isotropic properties, it can reliably detect edge features regardless of edge direction, achieving direction-independent edge detection, which is highly effective in the subsequent extraction of main lines and wrinkles. Adaptive thresholding targets local image features, customizing the threshold for each pixel rather than applying a global standard. This method performs exceptionally well in scenes with uneven lighting, complex backgrounds, or significant grayscale differences within the image. It can accurately distinguish foreground from background, significantly improving the adaptability and accuracy of image processing. Palmprint images are subjected to operations such as erosion, connected region analysis, and connected region merging. In practical applications, this step is often used to optimize image segmentation results, reduce erroneous segmentation, and meet specific image analysis or object recognition requirements. The present invention utilizes this step in the preprocessing of palmprint images, effectively extracting the main line structure of the palmprint, facilitating subsequent palmprint image recognition.

[0076] Step 2: The image preprocessed in step 1 is filtered using a Gabor filter. The image is convolved with multiple Gabor filters of different scales and directions to extract multi-directional and multi-scale texture features from the palmprint.

[0077] The Gabor filter is a linear filter used for edge extraction. Its frequency and direction expression are similar to those of the human visual system. It can provide good direction selection and scale selection characteristics, and is insensitive to changes in illumination, making it very suitable for texture feature analysis. In the spatial domain, a two-dimensional Gabor filter is the product of a sinusoidal plane wave and a Gaussian kernel function. The former is a tuning function, and the latter is a window function. Therefore, the Gabor transform is also called a window Fourier transform (short-time Fourier transform), which helps us pay more attention to the features within the local range of the signal. In this example, a sinusoidal two-dimensional Gabor filter is used to extract features. In the palmprint image recognition process, it can extract rich local features from complex texture information by filtering the image at multiple scales and directions, such as the main lines of the palmprint, branch points, breakpoints, detail points and other unique details. After further processing, these features can be used to construct feature vectors for efficient matching and recognition. By adjusting the center frequency, bandwidth and direction of the filter, it can adapt to palmprint images of various complexities and qualities, thereby improving the stability and accuracy of the recognition system. The example structure of the present invention for Gabor filtering for palmprint recognition is as follows: Figure 2 shown.

[0078] In step 3, the preprocessed image in step 1 is input into the Resnet18 network. Through this deep learning method, deeper feature representations are extracted from the original image. These features are usually highly abstract and can effectively distinguish complex patterns of different palm prints.

[0079] The basic structure of the Resnet18 network is as follows Figure 3 As shown, it includes the input layer, convolution layer: a total of 4 convolution layers, each convolution layer uses a 3x3 convolution kernel and a ReLU activation function to extract local features of the image. Residual block: a total of 8 residual blocks, each residual block consists of two convolution layers and a jump connection, which is used to solve the problems of gradient disappearance and gradient explosion in deep convolutional neural networks. Global average pooling layer: perform global average pooling on the feature map and convert the feature map into a one-dimensional vector. Fully connected layer: contains a fully connected layer of size 1000 for classification output. Output layer: uses the softmax activation function to generate a probability distribution of 1000 categories. In our example, the Resnet18 network uses pre-trained weights to reduce training time, only makes some fine-tuning to the model, removes the fully connected layer of the last layer of the model, and retains only the previous layers to obtain the feature vector. Specifically, the preprocessed palmprint image is sent to the model for forward propagation to extract features, and the output of the last layer is selected as the feature vector, combined with Figure 4 .

[0080] Step 4, feature fusion, involves performing PCA dimensionality reduction on the features of the palmprint images extracted in steps 2 and 3, then directly concatenating and combining them to generate a comprehensive feature vector. This combines the texture features captured by traditional methods with the high-level features mined by deep learning methods. Both feature sets are preprocessed (normalization, missing value filling, etc.), and then PCA dimensionality reduction is performed separately. This reduces feature dimensionality and captures key information. The low-dimensional feature vectors of the two feature sets after PCA dimensionality reduction are concatenated column by column to form a comprehensive feature vector. This integrates key information extracted by deep learning and Gabor filters, improving overall feature representation and classification performance.

[0081] Step 5: Use the simpler KNN algorithm to classify the new sample according to the fused features, select the k nearest neighbors, and use the most common category among these k neighbors as the predicted category to complete the palmprint recognition task.

Claims

1. A palmprint recognition method based on deep learning and multi-feature fusion, characterized in that: The steps include: S1. Improve the degree and accuracy of palmprint main line extraction and fold extraction through preprocessing; The preprocessing method firstly improves the overall contrast of the image by performing histogram equalization, then performs Gaussian blur on the palmprint image by using a Gaussian low-pass filter, and then strengthens the palmprint texture and edge features based on Laplacian edge detection; then adopts adaptive threshold binarization to obtain a binary image, and then performs an erosion operation on the binary image to remove isolated and scattered noise points and burr lines to highlight the main lines of the palmprint; then performs connected region extraction to obtain backbone information in the palmprint image, and finally merges the extracted connected regions by calculating the average value of the pixel sums of all connected regions to obtain a final line image; S2. Construct a set of Gabor filters for palmprint image feature extraction, which consists of Gabor filters of different frequencies and directions to achieve maximum coverage of texture features in the palmprint image. Then, convolve the image with each Gabor filter to extract the feature vector of each image. The Gabor filter includes using Fourier transform to improve the convolution speed during the convolution operation on the image. The spatial domain of the Gabor filter in two-dimensional space is described as follows: Among them, σ x and σ y Represents the spatial scale factors of horizontal and vertical directions, ω f and θ f Respectively represent the center frequency and azimuth; The above formula is decomposed into sine filter and cosine filter: S3. Extract features from the original palmprint image using a Resnet network. The Resnet network includes introducing skip connections to add input features to output features to solve the gradient vanishing and gradient exploding problems. S4, fusing the features extracted in step S2 and step S3 to generate a comprehensive feature vector, and then classifying the comprehensive feature vector using the K-nearest neighbor algorithm; Feature fusion methods include PCA dimensionality reduction fusion, Deep CCA fusion, or ASSF fusion; S5. For a new palmprint sample, by calculating its feature distance with each sample in the training set, the nearest K neighbors are selected, and the most common category among these K neighbors is used as the predicted category to complete the palmprint recognition task.

2. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1 is characterized in that: In step S1, the Gaussian low-pass filter is used to perform Gaussian blur processing on the palmprint image according to the following expression: Where u and v represent the coordinates of the pixel relative to the center of the kernel, σ represents the standard deviation of the Gaussian function, which determines the width and blur of the Gaussian kernel. 2 =u 2 +v 2 ; The larger the σ value, the more obvious the blurring effect, and the smaller the σ value, the less obvious the blurring effect.

3. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1 is characterized in that: The Resnet network described in step S3 includes an input layer, a convolutional layer, and a pooling layer, thereby performing a convolution operation on the input data to extract features and downsample the feature map to reduce the amount of calculation and retain important features; The ResNet network also includes a residual block and a residual connection layer. The residual connection layer uses a skip connection to add input features to output features, thereby solving the gradient vanishing and gradient exploding problems. The global average pooling layer of the Resnet network performs global average pooling on the feature map, reducing the size of the feature map of each channel to 1x1; In this step, since the depth of the feature map is fixed, this layer can convert the feature map into a vector of fixed length to prepare for subsequent dimensionality reduction and classification. Specifically, for each depth slice in the feature map, the average value of all spatial positions of the slice is calculated to generate a 1xD vector, where D represents the depth of the feature map and each element corresponds to the average value of a channel of the original feature map.

4. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1 is characterized in that: The PCA dimensionality reduction fusion described in step S4 is to perform PCA dimensionality reduction on the two feature sets extracted in step S2 and step S3, and then concatenate the low-dimensional feature vectors by column to form a comprehensive feature vector.

5. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1 is characterized in that: The Deep CCA fusion described in step S4 is to regard the low-level features obtained by Gabor filtering and the high-level features extracted by ResNet18 as two different modal data, and then perform nonlinear transformation on the Gabor features and ResNet features respectively by constructing a deep neural network, maximizing the typical correlation coefficient of the two sets of transformed features, so that the features output by the network not only maintain the expressive power within each modality, but also enhance the correlation between the two.

6. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1 is characterized in that: The ASSF fusion described in step S4 first performs dimensionality reduction processing on the extracted image features through PCA and normalizes them. Then, the module is selected in the ASSF learning model, and the image features extracted in steps S2 and S3 are weighted selected. The weights are dynamically adjusted according to the features of the input image. According to the calculated weights, the two feature subspaces are linearly combined to form a new fused feature space to generate a comprehensive feature vector.

7. The palmprint recognition method based on deep learning and multi-feature fusion according to claim 1, 4 or 6, characterized in that: The extracted features are reduced in dimension by PCA, and the distance between the original sample point and the projected reconstructed sample point is: Where x i is the original sample point, W is the projection matrix, z i is the projection of the sample in the low-dimensional coordinate system, and the projection matrix that minimizes the above formula is what we are looking for.

Images