Knuckle print identification method based on dynamic personalized federal learning

Abstract

The invention discloses a knuckle print recognition method based on dynamic personalized federal learning, and relates to the technical field of mode recognition, image processing and privacy protection. The method comprises the following steps of local training, aggregation and testing. The existing knuckle print recognition technology has two core defects: (1) the existing deep learning-based method depends on centralized training, and sensitive biological data privacy leakage is easily caused; and (2) the existing traditional personalized federal learning-based method adopts a fixed model matching strategy, only considers the personalized recognition performance and ignores the generalization ability, so that the model generalization ability and the personalized suitability are difficult to balance. According to the method, privacy leakage of original biological characteristic data transmission is avoided, meanwhile, a client bearing heterogeneous knuckle print data can have a personalized model for accurate recognition, and meanwhile, the personalized model has generalization ability stably exerted in a cross-client and cross-dataset scene by means of a dynamically updated model matching strategy.

Description

Technical Field

[0001] This invention relates to pattern recognition, image processing technology, and privacy protection technology. Specifically, each client possesses knuckle print images from different datasets and uses these images for local training. The local model parameters are then uploaded to a server for federated aggregation to obtain a global model. This global model is then distributed to each client for gradient analysis and parameter difference analysis, and uploaded back to the server to dynamically update the local model weights in the personalized model. Finally, a personalized model is generated for each client. This model accurately adapts to the local knuckle print dataset and possesses strong cross-scene generalization capabilities, making it suitable for cross-dataset knuckle print recognition tasks. This invention further improves the model's recognition accuracy and generalization stability while effectively protecting the privacy of the client's biological data. Background Technology

[0002] In the digital age, the accuracy and reliability requirements for identity authentication are constantly increasing, and biometric recognition is gradually becoming the mainstream method of identity verification. Although facial and fingerprint recognition technologies are widely used, they each have significant drawbacks: facial recognition carries the risk of privacy leaks, while fingerprints are easily damaged and forged, and are susceptible to environmental interference. In contrast, knuckleprint recognition technology exhibits unique advantages: firstly, it boasts high recognition accuracy due to its unique and highly discriminative texture features, which, combined with optimized algorithms, enable efficient matching across different scenarios; secondly, it offers strong stability, as the features are not easily affected by external environmental interference and remain stable throughout life, adapting to diverse collection conditions; and thirdly, it provides high security, as the features are difficult to forge and the collection process is contactless, reducing the risk of information leakage. Therefore, knuckleprint recognition has been successfully applied in various fields such as identity authentication, device unlocking, and security verification.

[0003] Research on knuckleprint recognition mainly includes three technical approaches: global methods for statistical feature extraction, local methods for feature description, and automatic feature extraction methods based on deep learning. Traditional global and local methods rely excessively on expert experience and manual design, making it difficult to adapt to diverse knuckleprint images. While deep learning methods offer superior performance, they require centralized data processing, which can easily lead to privacy and security issues. To protect data privacy, researchers have developed technical solutions such as differential privacy, homomorphic encryption, and federated learning. Among these, federated learning, through local training and server aggregation, protects data privacy while building a high-performance global model. However, existing federated learning methods face two main challenges in knuckleprint recognition: First, knuckleprint images from different acquisition environments exhibit data heterogeneity, making it difficult for the global model to adapt to various clients; second, although a personalized model can be constructed through weighted fusion of local and global models, the fixed weight ratio cannot dynamically adapt to the model evolution process, ignoring the heterogeneity at the feature level and limiting the overall system performance. Therefore, developing a novel personalized federated learning method that can effectively handle data heterogeneity and adaptively adjust model aggregation weights is of great significance for promoting the development of knuckleprint recognition technology. Summary of the Invention

[0004] This invention provides a fingerprint recognition method based on dynamic personalized federated learning, and proposes a server-side dynamically updated aggregation strategy to further achieve high accuracy and strong generalization of fingerprint recognition and privacy protection in heterogeneous data scenarios.

[0005] To implement the above method, the specific steps are as follows:

[0006] Step 100: Each client divides its local knuckle print dataset into a training set, a test set, and a subset of datasets. It then uses the training set data to train a local model under the constraints of a multi-view loss function and uploads the local model parameters to the server.

[0007] Step 200: The server receives the local models from all clients, generates a global model through federated averaging (FedAvg), and distributes it to each client. The client performs gradient analysis on its local subset based on the global model, and uploads the gradients and parameter changes of the global model compared to the local models to the server. The server adjusts the proportion of local models in each client's personalized model through a dynamic update mechanism. Each client uses the updated proportion to perform personalized federated learning (PFL) to generate a personalized model for the next round of training.

[0008] Step 300, the testing phase, is used throughout the process to ultimately evaluate the recognition performance and generalization ability of each client-specific personalized model.

[0009] In this process, each client possesses fingerprint images from different acquisition environments and with different distributions. Step 100, which involves dividing these images into training sets, test sets, and subsets and performing local training to obtain a local model, includes:

[0010] First, each client divides its local knuckle dataset into a training set, a test set, and a sub-dataset. To match the heterogeneous data scenario in reality, each client uses an independent dataset, which exhibits heterogeneity in terms of the number of samples collected, the collection environment, and the collection devices.

[0011] Then, each client performs local training using a local training set. First, a comprehensive competition network (CCNet, which is used by all clients for random initialization, as described in "Comprehensive competition mechanism in palmprint recognition") is randomly initialized. Then, it is trained for one communication round (containing three cycles) under the constraints of the multi-view loss function proposed in this invention, thereby obtaining the local model corresponding to each client, and these local models are uploaded to the server.

[0012] The process 200, in which the server receives all local models and obtains the global model through federated aggregation, then distributes them to different clients for gradient and parameter analysis and uploads them back to the server, dynamically updates the aggregated weights of the personalized models on the server side, and finally each client uses the updated aggregated weights to aggregate and obtain the personalized model, includes:

[0013] The server receives local model parameters uploaded by all clients and uses a federated averaging method (FedAvg, "Communication-efficient learning of deep networks from decentralized data") to weight and aggregate all local model parameters, generating a global model and distributing the parameters to each client. Each client receives the global model and calculates the gradient of the global model on a subset of data and the parameter changes of the global model compared to the local models. Then, it uploads the gradient and parameter changes to the server. The server receives the gradient and parameter changes from each client and uses the dynamic update mechanism proposed in this invention to update the proportion m of the local model in each client's personalized model ([0,1], m of each client is dynamically updated in each round of federated communication to continuously balance personalized performance and generalization performance), and distributes it to the corresponding client. Finally, the client uses the updated local model proportion to perform personalized federated aggregation (PFL) to obtain a personalized model for the next round of local training. This process is repeated until all federated communication is completed.

[0014] Step 300 in the performance evaluation process for the final model includes:

[0015] Each client uses the personalized model obtained after all federated communications to perform personalized performance testing and generalization performance testing. Specifically, for personalized performance testing: each client uses its personalized model to extract features from the local training and test sets and performs matching, then calculates the Euclidean distance, selects the label corresponding to the feature vector with the smallest distance as the predicted label, and compares it with the true label to calculate the accuracy (ACC). In addition, different thresholds are set to calculate the true positive rate (TPR) and false positive rate (FPR) to obtain the equal error rate (EER, which is the value when 1-TPR=FPR). For generalization performance testing: personalized models from other clients are used to test on the training and test sets of this client, and the accuracy (ACC) and equal error rate (EER) are calculated respectively, following the same procedure as the personalized performance testing steps.

[0016] The present invention has the following advantages:

[0017] (1) The final personalized model is applicable to both local knuckle print datasets and knuckle print datasets from other sources. That is, this invention is applicable to heterogeneous data scenarios with different collection environments and different distributions.

[0018] (2) This invention is applicable to different tasks: knuckleprint recognition task or verification task, and has good accuracy (ACC) or error rate (EER) performance indicators.

[0019] (3) It has strong generalization ability. By simply adjusting the hyperparameter ratio update rate α, the extraction method of this invention can be applied to other similar biometric recognition, such as palm print recognition, palm vein recognition and finger vein recognition. Attached Figure Description

[0020] Figure 1 This diagram compares the differences and advantages of the present invention with traditional knuckleprint recognition methods and traditional knuckleprint recognition methods based on personalized federated learning.

[0021] Figure 2 This is a flowchart of the fingerprint recognition method based on dynamic personalized federated learning according to the present invention.

[0022] Figure 3 This is a schematic diagram of the ROI images of the two knuckle print datasets used in the experimental process of this invention.

[0023] Figure 4 This is a schematic diagram of two palmprint dataset ROI images used to verify the generalization of this invention to other biometric recognition methods.

[0024] Figure 5 This is a schematic diagram of the hand back image and ROI image used in the open-set knuckle print recognition scenario of this invention.

[0025] Figure 6 For the present invention in Figure 2 ROC curves tested on the dataset

[0026] Figure 7 This is a graph showing the relationship between the dynamic update rate α of different hyperparameters and the test ACC performance in this invention.

[0027] Figure 8 This is a schematic diagram illustrating the dynamic update of the personalized model proportion in the two client datasets of this invention.

[0028] Figure 9 The graph shows the convergence curves of the training process for the two client datasets in this invention. Detailed Implementation

[0029] The method will be described in detail below with reference to the accompanying drawings.

[0030] This application provides a knuckleprint recognition method based on dynamic personalized federated learning, which achieves high-performance and strong generalization knuckleprint recognition and verification in federated scenarios.

[0031] Figure 1 This application demonstrates the differences and advantages of its proposed method compared to traditional knuckleprint recognition methods. Unlike traditional centralized recognition methods based on deep learning, which suffer from privacy leaks during data transmission, this invention introduces a federated learning mechanism to effectively protect privacy. Furthermore, compared to traditional personalized federated learning-based knuckleprint recognition methods that rely on fixed aggregation weights and optimization methods that prioritize personalized performance while neglecting generalization performance, this invention employs a dynamically updated aggregation weight method to effectively improve both personalized and generalization performance.

[0032] Figure 2 The flowchart of the knuckleprint recognition method based on dynamic personalized federated learning of this application is shown.

[0033] This application proposes a knuckleprint recognition method based on dynamic personalized federated learning, the specific steps of which are as follows:

[0034] Step 100: Each client divides its local knuckle print dataset into a training set, a test set, and a sub-dataset. Based on the training set, it trains a local model under the constraint of a multi-view loss function. After training, the local model parameters are uploaded to the server.

[0035] Step 200: The server receives the local models from all clients, generates a global model through federated averaging (FedAvg), and distributes it to each client. Each client performs gradient analysis and model parameter change analysis on its local subset based on the global model, and uploads the gradients and parameter changes of the global model compared to the local models to the server. The server adjusts the proportion of local models in each client's personalized model through a dynamic update mechanism. The client uses the updated proportions to perform personalized federated learning (PFL) to generate a personalized model for the next round of training.

[0036] Step 300, the testing phase, is used in the entire process to ultimately evaluate the recognition performance and generalization ability of each client-specific personalized model.

[0037] Step 100 includes the following sub-steps:

[0038] Sub-step 110: Each client has a dataset, and the ROI images of different datasets are as follows: Figure 3 As shown, each client divides its local knuckle print dataset into a training set, a test set, and a subset. Three images are randomly selected for each category as the training set, and the remaining images are used as the test set. The dataset from the first batch of each training round is selected as the subset.

[0039] Sub-step 120: This invention proposes a novel loss function, the multi-view loss function, to improve local models (feature extraction networks, Comprehensive Competition Networks, CCNet). The feature discrimination ability of the model parameters of the i-th client local model. The multi-view loss function formula is:

[0040]

[0041] in This represents the feature-based discrimination constraint loss. Indicates the loss of basic supervision, It comprises three components that optimize feature extraction capabilities from the perspectives of boundary loss, center loss, and triplet loss, respectively, and their expressions are as follows:

[0042]

[0043] Specifically, The analytical approach using the perspective of enhancing inter-category relationships is defined as follows:

[0044]

[0045] Where s (=64) is the scaling factor, δ (=0.5) represents the angle marginal coefficient, θ represents the angle measure between the feature vector and its corresponding class weight vector, and y corresponds to the true class label. To enhance intra-class compactness, we use center loss. Its expression is as follows:

[0046]

[0047] Where μ c f represents the central feature of class c. n Let the feature of the nth sample be represented. It includes all samples belonging to class c, where C is the total number of classes. Triplet loss. The expression for optimizing the distance between samples is as follows:

[0048]

[0049] Where x is the reference sample, y represents a positive sample similar to x, and z represents a negative sample dissimilar to x. The marginal hyperparameter β (=1.0) is determined empirically and can effectively control the separability of positive and negative sample pairs. Let represent the Euclidean distance, and Q represent the feature dimension. On the other hand, to ensure the learning ability of basic features, the basic loss... It combines classification learning and contrastive learning, and its expression is as follows:

[0050]

[0051] α1 and α2 are two hyperparameters used to balance these two constraints, set to α1 = 0.8 and α2 = 0.2. Classification Loss Component Provides basic supervision, expressed as follows:

[0052]

[0053] Where N represents the batch size, y n,c Represents the true label (if the nth sample belongs to the cth class, then y n,c =1, otherwise 0), p n,c Represents the predicted probability. Compare the loss components. It is used to facilitate feature similarity learning, and its calculation method is as follows:

[0054]

[0055] in Represents the sample index set, This represents the set of positive samples that belong to the same category as sample n. The cardinality of the set (i.e., the number of elements in the set). It contains the indices of all samples except sample n. n and f p These are the feature embeddings for the anchor samples and the positive samples, respectively. The temperature parameter t (=0.07) is used to control the concentration of the distribution.

[0056] All clients train local models using the aforementioned comprehensive loss constraints and upload the local model parameters to the server.

[0057] Step 200 includes the following sub-steps:

[0058] Step 210: The server receives the parameters θ1 of the local model ① uploaded by all clients, and then uses the federated averaging algorithm to obtain the parameters θ2 of the global model ②. The calculation method is as follows:

[0059]

[0060] in Here, θ2 represents the parameters of the local model ① for the i-th client, where I represents the total number of clients. After this, the server will distribute the parameters of the global model ② to each client.

[0061] Step 220: Each client performs gradient analysis by computing the gradient on the subset of data using the global model ②. The gradient of the i-th client's subset of data... It can be represented as:

[0062]

[0063] in Represents classification loss The gradient of x i y represents the knuckle print image in the first batch of data from the i-th client. i This represents the corresponding label. Simultaneously, each client calculates the parameter differences between the global model ② and its own local model ①. Subsequently, each client calculates its gradient. Differences from parameters Uploaded to the server.

[0064] Step 230: To balance personalization and generalization capabilities, we propose a dynamic update mechanism. Through this mechanism, the server updates the proportion of the local model in the previous personalized model ③ and distributes it to the corresponding clients. For the dynamic update mechanism, a personalization contribution item is first constructed, which takes into account the direction of the adjustment of the local model proportion. Distribution characteristics of client-specific datasets The compatibility between them is expressed as follows:

[0065]

[0066] Then, a generalization contribution term was constructed, which takes into account the direction of adjustment of the local model's proportion. Distribution characteristics of other client datasets The compatibility between them is expressed as follows:

[0067]

[0068] in This represents the average gradient for all clients except the current client. Next, the personalized contribution and the generalized contribution are merged, and the proportion of local models in the personalized model is updated using the following formula:

[0069]

[0070] Where α represents the local model percentage update rate. The total contribution term is represented by the clip(w,0,1) function, which constrains w to the interval [0,1]. and These represent the proportion of the local model for the i-th client in the nth and n+1th rounds of communication, respectively.

[0071] Step 240: Finally, each client updates the proportion of the local model through aggregation. The parameters θ2 of the global model ② and the parameters of the local model ① Each individualized model is obtained ③, as follows:

[0072]

[0073] in For the model parameters of the personalized model ③ of the i-th client, each client uses its own personalized model ③ and repeats steps 120-230 until all federated communications are completed.

[0074] Step 300:

[0075] For the testing process: Each client uses the personalized model obtained after all federated communications are completed to perform personalized performance testing and generalization performance testing. Specifically, for personalized performance testing: Each client uses the personalized model to extract features from the local training set and test set and performs matching. in For each image in the training set, the feature vector is... For a feature vector of an image in the test set, the label corresponding to the feature vector with the smallest distance in the training set is selected as the predicted label, and compared with the true label to calculate the accuracy (ACC). This performance is used for the recognition task. In addition, the True Positive Rate (TPR) and False Positive Rate (FPR) are calculated to obtain the Equal Error Rate (EER, which is the value when 1-TPR = FPR), which serves as the performance for the validation task. Specifically, features are extracted from the training set and all client test sets and matched, then the Euclidean distance is calculated. The Euclidean distance is normalized to a matching score. Then, more than 1000 uniformly distributed thresholds are generated between the minimum and maximum matching scores (match scores above the threshold indicate a correct match, and scores below the threshold indicate an incorrect match), and the correct acceptance rate at each threshold is calculated. Where TP is the number of correctly identified members of the same class, FN is the number of correctly identified members of the same class but different members, and the false acceptance rate is... Where FP represents the number of outliers identified as belonging to the same class, TN represents the number of outliers identified as outliers, and the equal error rate is the value when 1-TPR = FPR. For generalization performance testing: personalized models from other clients are tested on the training and test sets of this client, and the accuracy (ACC) and equal error rate (EER) are calculated respectively. The specific process is the same as the personalized performance testing steps. This invention re-randomizes the training and test sets and performs 5 repeated experiments, i.e., retraining the personalized model and testing it. The average and standard deviation (STD) of the accuracy (ACC) of the 5 tests are calculated, and the best equal error rate (EER) among the 5 tests is selected as the final performance.

[0076] The knuckle print images used in this invention were obtained from two publicly available databases: the Data-fkp dataset and the PolyU-fkp dataset. The Data-fkp dataset contains a total of 5,850 knuckle print images, collected from 585 different fingers of 195 subjects. Each participant provided 10 samples from their index, middle, and ring fingers. This database contains knuckle print regions of interest (ROIs) uniformly cropped to 90×200 pixels. The PolyU-fkp dataset contains 7,920 knuckle print images, collected from 165 individuals in two independent acquisition sessions. In each session, each subject contributed 6 images from the index and middle fingers of both hands, totaling 48 images from four fingers per participant. This database contains images uniformly cropped to 110×220 pixels. The ROI images from both datasets are shown below. Figure 3 As shown. To ensure consistent federated training, all images are uniformly set to a size of 128×128 pixels.

[0077] The experimental environment was a PC, and the experiment was implemented based on the PyTorch framework, using the Adam optimizer for training with a learning rate of 0.001. The batch size, number of communication rounds, and local training cycles were set to 512, 100, and 3, respectively. The experimental environment configuration was: Intel(R) Xeon(R) Gold 5218 CPU @ 2.30GHz processor, 62GB of RAM, and an NVIDIA GTX 3090 GPU.

[0078] To test the recognition performance of this invention, we compared it with existing mainstream knuckleprint recognition methods, including DDBFL, JDFL, JLCMHC, SDMFC, RSLSR, and CCNet. Table 1 shows the accuracy ± standard deviation (ACC ± STD) results of different methods on two knuckleprint datasets.

[0079] Table 1. Accuracy (ACC ± Standard Deviation (STD)) of different centralized identification methods on two knuckleprint datasets.

[0080]

[0081] First, to further verify the recognition performance of our method, we conducted comparative experiments with mainstream federated learning methods such as FedAvg, FedPer, FedProx, FedBN, and FedFV, as well as the 'w / oFL' method which trains using only local datasets without federated learning. The experimental details are as follows: the constraint term μ of FedProx was set to 0.01, and the proportion of the local model in FedFV was m. i The loss function was fixed at 0.5, and all comparison methods used the loss function settings adopted in this invention to verify the effectiveness of the federated learning framework proposed in this invention. The experiments were divided into two scenarios: "intra-dataset" (identification of different methods on the same client dataset) and "cross-dataset" (identification of different methods on cross-client datasets). Table 2 shows the accuracy ± standard deviation (ACC ± STD) results of different methods on the two knuckleprint datasets. In the intra-dataset scenario, other federated methods performed worse than non-federated methods due to data heterogeneity. This invention, however, effectively alleviated the heterogeneity problem with its dynamic update mechanism, achieving recognition accuracies of 99.95% and 96.92% on the Data-fkp and PolyU-fkp datasets, respectively, demonstrating strong personalization capabilities. In the cross-dataset scenario, this invention maintained excellent generalization performance, with recognition accuracies of 96.82% and 99.97% for "Data-fkp→PolyU-fkp" and "PolyU-fkp→Data-fkp," respectively, significantly outperforming other federated learning methods. These results indicate that this invention has good performance and generalization capabilities in recognition tasks. Meanwhile, the verification performance of the present invention in the two scenarios described above was evaluated, and the specific results are shown in Table 3. It can be observed that the present invention still has strong competitiveness compared with other methods. Figure 6 The ROC curves under different methods further validated the good verification capability of this invention.

[0082] Table 2. Accuracy (ACC ± Standard Deviation (STD)) of different federated learning methods on two knuckleprint datasets.

[0083]

[0084] Table 3. Equal error rates (EER) of different federated learning (FL) methods on two palmprint datasets.

[0085]

[0086] Second, to further explore the generalization ability of the present invention, two sets of experiments were conducted: cross-modal generalization experiment and open set finger joint recognition experiment.

[0087] For cross-modal generalization experiments, we conducted experiments on two publicly available palmprint datasets: the IITD dataset and the CASIA dataset. The IITD dataset contains 2,601 palmprint images collected from 460 different hands, with 5 to 7 images per hand. The CASIA dataset consists of 5,502 palmprint images covering 312 different categories, with 8 to 17 images per category. All images were uniformly set to a size of 128×128 pixels, and other settings remained consistent with the knuckleprint experiment settings. Figure 4 The table shows partial palmprint region of interest (ROI) images from these two databases. Table 4 lists the recognition results of this invention and other federated learning-based methods on these two palmprint datasets. It is clear from Table 4 that, whether in intra-dataset scenarios or cross-dataset scenarios, the recognition accuracy of this invention is still higher than other methods. This result verifies that this invention possesses strong cross-modal generalization ability, making it well-suited for practical applications of palmprint recognition.

[0088] For open-set knuckleprint recognition, personalized models were trained using the Data-fkp and PolyU-fkp databases respectively, and then tested on the Hong Kong Polytechnic University contactless knuckleprint image database (PolyUfkp_v1.0). PolyUfkp_v1.0 contains two sub-databases (denoted as Minor-fkp and Major-fkp, respectively), with images collected from the "minor region" (Minor-fkp) and "major region" (Major-fkp) of knuckleprints. This database contains a total of 2515 knuckleprint images from 503 subjects, with each subject providing 5 sample images, of which the first 3 were used for training and the remaining 2 for testing. Figure 5 The table shows some images of the back of the hand and ROIs from the PolyUfkp_v1.0 dataset. Table 5 records the accuracy (ACC) of different federated learning methods in the open-set knuckle print experiment. It can be observed that in most scenarios, the accuracy of the proposed method is higher than that of other federated learning (FL) based methods, indicating that the proposed method has strong generalization and adaptability.

[0089] Table 4. Accuracy (ACC ± Standard Deviation (STD)) of different federated learning methods in cross-modal experiments.

[0090]

[0091] Table 5. Accuracy (ACC) of different federated learning methods in the open-set knuckle print experiment.

[0092]

[0093] Third, to further explore the effectiveness of the loss constraints and different contribution components in the dynamic update mechanism used in this invention, ablation experiments were conducted.

[0094] To investigate the role of each loss constraint term in formula (1), we conducted an ablation experiment: while retaining other loss terms, we removed one of the loss terms one by one. Table 6 presents the identification accuracy (ACC ± standard deviation, STD) results of the ablation experiment. It can be seen that removing the basic loss... Recognition performance drops significantly in both intra-dataset and cross-dataset scenarios, with the decline being particularly pronounced on the PolyU-fkp dataset. This confirms... Its importance as a core learning objective lies in its ability to support both task-specific feature learning and domain-invariant feature learning. Furthermore, removing feature loss... The recognition accuracy will decrease slightly, which indicates that It plays an auxiliary role and has the ability to distinguish fine-grained features.

[0095] Table 6. The impact of different loss constraints on recognition performance accuracy (ACC ± standard deviation, STD) (%)

[0096]

[0097] For the personalized contribution term D in the system evaluation formula (11) local With the generalization contribution term D in formula (12) cross To assess the impact of this invention on recognition performance, we conducted ablation experiments: removing any one of the contributing factors (while retaining the other) to verify its effect. Table 7 records the recognition accuracy (ACC ± standard deviation, STD) (%) under different settings. It can be observed that removing either the personalized or generalized contributing factor leads to a decrease in model performance in most scenarios. This result indicates that these two contributing factors are significant for both intra-dataset and cross-dataset recognition of knuckle prints.

[0098] Table 7. The impact of different contribution factors on recognition performance accuracy (ACC ± standard deviation, STD) (%)

[0099]

[0100] Fourth, the invention provides an in-depth analysis of the hyperparameter settings, training process, and computational and communication overhead of the dynamic personalization mechanism proposed in this invention.

[0101] Specifically, Figure 7The impact of the hyperparameter dynamic update rate α on the recognition performance of the present invention in two scenarios ("within the dataset" and "across the dataset") on two knuckleprint datasets is demonstrated. It can be observed that when α = 0.030, the present method can achieve the best recognition performance in most scenarios, so this setting was adopted in subsequent experiments.

[0102] Figure 8 This shows the local model percentage of the personalized models on the two knuckleprint datasets during training. The dynamic update process, Figure 9 The paper demonstrates the changes in loss and accuracy during local training on two clients as the number of communication rounds increases. It can be observed that both datasets converge around 10 rounds, which fully verifies the effectiveness of the invention in heterogeneous data scenarios.

[0103] Table 8 details the computational and communication overhead of this invention. The entire system's training time is 233 minutes, and the time consumed per sample during the sample identity reasoning stage is extremely short, requiring only 0.046 seconds. This clearly demonstrates that this invention has excellent application scenarios for identity verification. Furthermore, the maximum single communication model size of the entire federated learning framework in this invention is 244MB, indicating that this invention can be applied to communication-constrained scenarios in real life and achieve good recognition results.

[0104] Table 8. Analysis of Computational and Communication Overhead

[0105]

[0106] The knuckleprint recognition method based on dynamic personalized federated learning in this invention has the following advantages:

[0107] (1) This invention effectively enhances data privacy protection by introducing federated learning technology into the fingerprint recognition process.

[0108] (2) This invention proposes an effective solution to the problem that the strong heterogeneity of different client datasets in the federated scenario leads to a significant reduction in model performance and generalization ability.

[0109] (3) The present invention has strong generalization ability. The method of the present invention can be applied to other similar biometric identification, such as palm print recognition, finger vein recognition and palm vein recognition.

Claims

1. A fingerprint recognition method based on dynamic personalized federated learning, characterized in that, Includes the following steps: Step 100: Each client divides its local knuckle print dataset into a training set, a test set, and a subset of datasets. It then uses the training set data to train a local model under the constraints of a multi-view loss function and uploads the local model parameters to the server. Step 200: The server receives the local models from all clients, generates a global model using federated averaging (FedAvg), and distributes it to each client; the client performs gradient analysis and model parameter analysis on its local subset based on the global model, and uploads the gradient and model parameter changes to the server; The server adjusts the proportion of local models in the personalized models of each client through a dynamic update mechanism. The client then uses the updated proportions to generate a personalized model for the next round of training using Personalized Federated Learning (PFL). Step 300: Testing phase. Finally, evaluate the performance of the personalized model generated for each client using this method.

2. The method according to claim 1, characterized in that, Step 100 includes: Each client has a dataset. Each client divides its local dataset into a training set, a test set, and a subset. Three images are randomly selected for each category as the training set, and the remaining images are used as the test set. The first batch of data from the training set during each round of training is taken as the subset. A novel multi-view loss function is proposed to improve the feature discrimination ability of the local model CCNet. Here are the model parameters for the i-th client-side local model; the multi-view loss function formula is: in This represents the feature-based discrimination constraint loss. Indicates the loss of basic supervision, It comprises three components that optimize feature extraction capabilities from the perspectives of boundary loss, center loss, and triplet loss, respectively, and their expressions are as follows: Specifically, The analytical approach using the perspective of enhancing inter-category relationships is defined as follows: Where s = 64 is the scaling factor, δ = 0.5 represents the angle marginal coefficient; θ represents the angle measurement between the feature vector and its corresponding class weight vector, and y corresponds to the true class label; To enhance intra-class compactness, a central loss method is adopted. Its expression is as follows: Where μ c f represents the central feature of class c. n Let the feature of the nth sample be represented. Includes all samples belonging to class c, where C is the total number of classes; triplet loss The expression for optimizing the distance between samples is as follows: Where x is the reference sample, y represents a positive sample similar to x, and z represents a negative sample dissimilar to x; the marginal hyperparameter β (=1.0) is determined empirically and can effectively control the separability of positive and negative sample pairs; Let represent the Euclidean distance, and Q represent the feature dimension; on the other hand, to ensure the learning ability of basic features, the basic loss... It combines classification learning and contrastive learning, and its expression is as follows: α1 and α2 are two hyperparameters used to balance these two constraints, set to α1 = 0.8 and α2 = 0.2; classification loss component Provides basic supervision, expressed as follows: Where N represents the batch size, y n,c Represents the true label; if the nth sample belongs to the cth class, then y n,c =1, otherwise 0, p n,c Represents the predicted probability; compares the loss components. It is used to facilitate feature similarity learning, and its calculation method is as follows: in Represents the sample index set, This represents the set of positive samples that belong to the same category as sample n. The cardinality of the set is the number of elements in the set. Includes the indexes of all samples except sample n; f n and f p These are the feature embeddings for anchor samples and positive samples, respectively; the temperature parameter t = 0.07 is used to control the concentration of the distribution. All clients train local models using the aforementioned comprehensive loss constraints and upload the local model parameters to the server.

3. The method according to claim 1, characterized in that: Step 200 includes: The server receives the parameters θ1 of the local model ① uploaded by all clients, and then uses the federated averaging algorithm to obtain the parameters θ2 of the global model ②. The calculation method is as follows: in Let θi be the parameters of the local model ① for the i-th client, where I represents the total number of clients. After this, the server distributes the parameters θ2 of the global model ② to each client. Each client receives the global model parameters and performs gradient analysis by calculating the gradient on a subset of data using the global model ②. The gradient of the i-th client's subset of data... It can be represented as: in Represents classification loss gradient, x i y represents the knuckle print image in the i-th client subset. i This indicates the corresponding label; simultaneously, each client calculates the parameter differences between the global model ② and its own local model ①. Subsequently, each client calculates its gradient. Differences from parameters The data is uploaded to the server; the server accepts the differences in gradients and model parameters and executes the proposed dynamic update mechanism; through this mechanism, the server updates the local model proportions in the previous personalized model and distributes them to the corresponding clients; for the dynamic update mechanism, a personalized contribution term is first constructed, which takes into account the direction of local model proportion adjustment. Distribution characteristics of this client's dataset The compatibility between them is expressed as follows: Then, from a cross-client perspective, the generalization contribution of the local model proportion is set as follows: in This represents the average gradient of all clients except the current client; subsequently, the personalized contribution and the generalized contribution are merged, and the local model proportion is updated using the following formula: Where α is the percentage update rate. Representing the total contribution, the clip(w,0,1) function constrains w to the interval [0,1]. and These represent the proportion of the local model for the i-th client in the nth and n+1th rounds of communication, respectively; ultimately, each client's local model proportion is updated through fusion. The parameters θ2 of the global model ② and the parameters of the local model ① Each personalized model is obtained using PFL③, and the PFL formula is: in, These are the parameters of the personalized model ③ for the i-th client.

4. The method according to claim 1, characterized in that: Step 300 includes: For the testing process: Each client uses the personalized model obtained after all federated communications are completed to perform personalized performance testing and generalization performance testing. Specifically, for personalized performance testing: Each client uses the personalized model to extract features from the local training set and test set respectively and performs matching. in For each image in the training set, the feature vector is... For a feature vector of an image in the test set, the label corresponding to the feature vector with the smallest distance in the training set is selected as the predicted label, and compared with the true label to calculate the accuracy (ACC). This performance is used to evaluate the performance for the recognition task. Additionally, the correct acceptance rate (TPR) and incorrect acceptance rate (FPR) are calculated to obtain the equal error rate (EER), which is the value when 1 - TPR = FPR, used to evaluate the performance for the validation task. Features are extracted from the training set and all client test sets and matched, then the Euclidean distance is calculated. The Euclidean distance is normalized to a matching score. Then, more than 1000 uniformly distributed thresholds are generated between the minimum and maximum matching scores. A matching score greater than the threshold indicates a correct match, and a score less than the threshold indicates a wrong match. The correct acceptance rate is calculated for each threshold. Where TP is the number of correctly identified members of the same class, FN is the number of correctly identified members of the same class but different members, and the false acceptance rate is... Where FP is the number of outliers identified as belonging to the same class, TN is the number of outliers identified as outliers, and the equal error rate is the value when 1-TPR=FPR; For generalization performance testing: the personalized model of other clients is used to test on the training and test sets of this client, and the accuracy ACC and equal error rate EER are calculated respectively. The specific process is the same as the personalized performance testing steps; the training and test sets are randomly re-divided and 5 repeated experiments are conducted, that is, the personalized model is retrained and tested, and the average and standard deviation of the accuracy of the 5 tests are calculated. The best equal error rate among the 5 tests is selected as the final performance.

Images