A personnel biometric information acquisition method based on deep learning

Abstract

The application discloses a personnel biological feature information collection method based on deep learning, relates to the field of artificial intelligence and biological feature recognition technology, and significantly improves the authentication coverage ability of special groups; aiming at physiological degradation problems such as fingerprint wear and tear and iris blur of the old, a cross-modal graph neural network is used to construct the dynamic association of fingerprint minutiae points and iris texture blocks, even if the single-mode feature is seriously incomplete, effective fusion features can still be generated through geometric alignment and attention weight compensation; the system robustness of a complex scene is enhanced; a spatial embedding layer maps the fingerprint coordinates to the iris polar coordinate system, and eliminates the geometric deviation between modes caused by the collection angle and the offset of the diseased area; a residual learning unit relieves the deep network training degradation through feature iteration optimization, and has self-adaptive compensation ability for imaging defects such as low contrast and local occlusion.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and biometric recognition technology, and in particular to a method for collecting human biometric information based on deep learning. Background Technology

[0002] Currently, the medical insurance system widely uses multimodal biometric recognition technologies such as fingerprints and irises to verify the identity of insured persons in order to improve service security and efficiency. In the medical insurance window service scenario, the elderly population generally suffers from fingerprint surface wear and blurred iris texture due to lesions such as cataracts due to natural physiological degeneration. These physiological characteristics make it difficult for conventional collection equipment to completely extract effective biometric features. Especially in the dual-modal collaborative authentication process, the failure of a single modality collection will lead to the interruption of the entire verification.

[0003] Existing solutions mainly revolve around hardware enhancement and algorithm optimization. Some systems use high-sensitivity fingerprint sensors and infrared-enhanced iris cameras to improve recognition rates by increasing physical acquisition accuracy. At the same time, a multimodal backup mechanism is introduced to allow alternative biometric features to be used when a single feature verification fails. Some solutions also build incomplete biometric feature completion models based on generative adversarial networks (GANs) to generate complete feature vectors using existing features. However, hardware upgrades are limited by equipment costs and aging-in adaptation cycles, while generative models rely on large-scale, high-quality training data, which faces the problem of sample scarcity in actual deployment in grassroots medical insurance institutions.

[0004] It is evident that current technologies still struggle to address extreme scenarios where both modalities fail simultaneously. When an elderly person's fingerprints are severely worn and they also have iris lesions, hardware upgrades are limited by physical characteristics and cannot acquire basic biometric images. Generative algorithms, on the other hand, cannot initiate the reconstruction process due to a lack of effective input features. This forces window services to switch to manual verification channels, increasing the risk of identity theft and reducing the efficiency of services for vulnerable groups. Although current integrated data collection devices integrate multiple modules, the independent architecture of each module cannot achieve real-time cross-modal collaborative compensation, becoming a key bottleneck restricting the inclusiveness of services. Summary of the Invention

[0005] In view of the aforementioned existing problems, the present invention is proposed.

[0006] This invention provides a deep learning-based method for collecting biometric information of personnel, which solves the problem that existing medical insurance biometric authentication lacks an effective solution for scenarios where both fingerprint and iris recognition fail for the elderly, leading to a decline in service efficiency and security.

[0007] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0008] In a first aspect, embodiments of the present invention provide a method for collecting human biometric information based on deep learning, comprising,

[0009] Step S1: Obtain fingerprint and iris images;

[0010] Step S2: Perform quality assessment and standardization preprocessing on the image to obtain fingerprint usability score and iris usability score;

[0011] Step S3: Extract the fingerprint minutiae set based on the fingerprint image, and generate the iris texture block set in the polar coordinate domain based on the iris image;

[0012] Step S4: Construct a heterogeneous graph containing fingerprint minutiae and iris texture blocks, and perform feature propagation and fusion through a cross-modal graph neural network to obtain a fused feature vector;

[0013] Step S5: Compare the fused feature vector with the registration template, and output the authentication result based on the comparison score and a preset threshold.

[0014] Specifically, when the quality assessment of any modality is below a threshold, a quality gating strategy is used to assign reduced or masked aggregation weights to the graph nodes of that modality.

[0015] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, the fingerprint minutiae are the coordinates and orientation information of the ridge termination point and the bifurcation point in the fingerprint image.

[0016] Iris texture blocks are grayscale or texture feature blocks obtained by fan-shaped partitioning of the denoised and normalized iris region.

[0017] The quality assessment includes a comprehensive score for acquisition integrity, sharpness, contrast, occlusion and reflection interference, and forms a configurable threshold.

[0018] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, in step S4, the edge connections of the heterogeneous graph are determined by a cross-modal spatial embedding layer, which is used to align the image coordinates of fingerprint minutiae to the iris polar coordinate reference system and calculate cross-modal adjacency relationships; both intramodal and cross-modal edges carry edge features based on spatial proximity and local texture similarity.

[0019] In step S4, the mapping method from fingerprint coordinates to the iris polar coordinate reference system is as follows:

[0020] Any fingerprint minutiae (x) f ,t f Mapping to the polar coordinate reference frame of the iris, obtaining (ρ, θ) to align with the center of the iris texture patch, the mapping is performed after a similarity transformation and then converted to polar coordinates:

[0021]

[0022]

[0023] Where ρ is the normalized radius, θ is the polar angle in rad, Φ is the spatial embedding map, and x f ,y f Let x0 and y0 be the x and y coordinates of the fingerprint minutiae in the fingerprint image coordinates, in pixels; S is the anisotropic scaling matrix; R(θ0) is the rotation matrix, where θ0 is the reference direction, in rad; x0 and y0 are the centroid coordinates of the fingerprint minutiae set, in pixels; r0 is the equivalent radius of the effective fingerprint region, in pixels; atan2(·,·) is the two-parameter arctangent function, returning the polar angle, in rad; and y′ and x′ are the intermediate transformation coordinates, in pixels.

[0024] In the formula:

[0025] θ0=atan2(∑ k sinψ k ,∑ k cosψ k ),

[0026] Where θ0 is the reference direction, in rad, ∑ k sinψ k With ∑ k cosψ k These are the sine and cosine components accumulated at index k, respectively, ψ k Let be the orientation angle of the kth detail point, in rad, where k is the index of the detail point;

[0027] Cross-modal adjacency determination based on spatial distance and local texture differences: In a unified reference frame, cross-modal edge weights are calculated for any fingerprint minutiae u and the iris texture block center v, and the adjacency set is constrained by a threshold and Top-K. The backbone weights are multiplicatively coupled with distance and texture terms and a quality gating is introduced.

[0028]

[0029] Among them, w uv For cross-modal adjacency weights, d uv This is a normalized distance measure, dimensionless, obtained by weighting the difference in polar radius and the difference in angle, σ. d For distance scale parameters, Δ t (u,v) is a measure of local texture difference, σ t For texture scale parameters, g u ,g vThe quality gating factor takes values ​​[0,1], and u,v are cross-modal node indices;

[0030] In the formula:

[0031]

[0032] in, ρ is the square of the spatial distance. u ,ρ v Let θ be the polar radius of the two nodes. u ,θ v The polar angle between the two nodes, in rad, λ θ Angular weighting coefficient, in rad. -2 ;

[0033] The adjacency establishment rule is: when w uv ≥τ w When v is a Top-K neighbor of u, an edge is connected, where τ w is the edge weight threshold, and K is the maximum number of cross-modal connections for a single node, in units of nodes.

[0034] As a preferred embodiment of the deep learning-based method for collecting biometric information of personnel described in this invention, in step S4, the graph neural network includes an attention-based message passing mechanism, calculates a similarity scalar for fingerprint minutiae and iris texture blocks, and scales the aggregation weights; the similarity includes cosine similarity based on feature vectors and a weighted term based on edge features;

[0035] In step S4, after establishing cross-modal adjacency, the similarity between the feature vectors of fingerprint minutiae and iris texture blocks is measured and used as a prior term for attention scoring. The basic similarity is in cosine form.

[0036]

[0037] Among them, c uv h represents the cosine similarity between node pairs. u ,h v Let be the feature vectors of nodes u and v respectively, and <·,·> represent the inner product operation, and ‖·‖ represent the L2 norm;

[0038] Introducing temperature scaling to adjust the sharpness of the similarity distribution:

[0039] s uv =exp(β) T c uv ),

[0040] Among them, s uv For the similarity score after temperature scaling, β TThe reciprocal of the temperature coefficient is dimensionless, and exp(·) is an exponential function;

[0041] Before graph message passing, gated attention weights are calculated for each candidate adjacency, and the output representation is aggregated in the multi-head case; for the h-th attention head, soft normalization with prior bias is performed using the following formula:

[0042]

[0043] in, Let be the attention weight of the h-th head-to-edge (u,v). These are the query and key vectors, respectively. Indicates transpose, d h γ is the dimension of the key vector of this head. s The similarity prior weight coefficient is dimensionless, s uv For the temperature-scaled similarity score obtained from the above formula, γ w w is the a priori coefficient of the edge weights, dimensionless. uv The cross-modal adjacency weights obtained in step S4, ε w G is the numerical stability constant, dimensionless. u ,g v v is the quality gating factor, taking values ​​in the range [0,1], and v′ is the adjacency set. Any neighbor index in, Let be the set of candidate adjacencies of node u;

[0044] The linear aggregation quadratic expansion of the multi-head output is as follows:

[0045]

[0046] in, Let H be the update representation of node u, H be the number of attention heads, and π be the value of π. h V is the head aggregation coefficient, which is non-negative and sums to 1. (h) Let h be the value mapping matrix of the h-th head. v is the input feature vector of the neighbor node v.

[0047] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, in step S4, the cross-modal graph neural network includes a residual learning unit, which uses shortcut connections to sum the features of the previous layer nodes with the residuals after nonlinear transformation, so as to alleviate the degradation of deep training and enhance the robustness to low-quality samples.

[0048] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, the following steps are taken: before authentication, liveness detection is performed on fingerprint and iris images, and a rejection is directly output when the liveness detection fails; when authentication is successful, the summary of the fused features and the comparison log are encrypted and stored according to compliance policies.

[0049] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, when the quality assessment of two modalities is simultaneously below the threshold or the comparison score between the fused feature and the registered template is in the gray zone, a degradation strategy is triggered, including: re-collection prompt, short-term caching and reuse of the most recent valid collection, or outputting an identifier that requires manual review.

[0050] As a preferred embodiment of the deep learning-based method for collecting human biometric information described in this invention, the cross-modal graphical neural network is pre-trained offline and deployed for inference on a window device. During the training phase, a dataset containing multiple age groups and multiple imaging conditions is used, and the loss objective includes a weighted combination of classification loss and metric learning loss. During the inference phase, only forward computation is performed.

[0051] Secondly, embodiments of the present invention provide a computer device, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps of the above-described method for collecting and authenticating human biometric information based on deep learning.

[0052] Thirdly, embodiments of the present invention provide a computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the steps of the above-described method for collecting and authenticating human biometric information based on deep learning.

[0053] The beneficial effects of this invention are as follows: It significantly improves the authentication coverage for special groups. Addressing physiological degeneration issues such as fingerprint wear and iris blurring in the elderly, it utilizes a cross-modal graphical neural network to construct a dynamic association between fingerprint minutiae and iris texture blocks. Even with severely incomplete single-modal features, effective fusion features can still be generated through geometric alignment and attention weight compensation. A quality gating strategy further shields against interference from invalid features, avoiding overall authentication interruptions caused by poor local quality in traditional solutions.

[0054] This invention enhances the robustness of systems in complex scenarios. The spatial embedding layer maps fingerprint coordinates to the iris polar coordinate system, eliminating intermodal geometric deviations caused by acquisition perspective and lesion area offset. The residual learning unit mitigates deep network training degradation through feature iteration optimization and has adaptive compensation capabilities for imaging defects such as low contrast and local occlusion. When the quality of both modalities is insufficient, the degradation strategy automatically switches to the manual review channel to ensure service continuity.

[0055] This invention reduces deployment costs and privacy risks. It supports inference on an integrated device, eliminating the need to upload original biological images; it uses fused feature vectors to replace original data for authentication, mitigating the risk of leakage from centralized storage of biological information. The offline pre-trained model is compatible with domestically produced hardware platforms, allowing grassroots medical insurance windows to upgrade their authentication without replacing their sensors with high-precision ones. Attached Figure Description

[0056] To more clearly illustrate the technical solutions of the embodiments of this application, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of this application and should not be regarded as a limitation on the scope of this application.

[0057] Figure 1 This is a flowchart illustrating the deep learning-based method for collecting human biometric information in Example 1. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0059] All terms used in this application (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0060] For example, the terms “first” and “second” used in this application are only used to distinguish and describe similar objects, to differentiate the first object from another object, and are not used to describe a specific order or sequence, nor should they be interpreted as indicating or implying relative importance.

[0061] This application proposes a method for collecting human biometric information based on deep learning, combining... Figure 1 As shown, the method includes:

[0062] Step S1: Obtain fingerprint and iris images;

[0063] In this embodiment, data acquisition is completed synchronously on the integrated device at the window. The acquisition process includes three stages: acquisition initiation, alignment guidance, and result confirmation. The criteria for successful acquisition are simultaneously obtaining a single frame of fingerprint and iris image of measurable quality or reaching a preset retry limit. Furthermore, the default timeout for acquisition control is set to 1.5s, with an adjustable range of 0.5–5s. The default number of retry attempts for a single modality is 2, with an adjustable range of 0–3. These criteria are determined based on the window service latency limit and the cooperation level of personnel in the pilot environment. Optionally, when the device is in a high-concurrency queue, the acquisition interval is at least 200ms by default to avoid sensor overheating and false triggering. In the event of alignment failure, occlusion, or strong reflection interference, the minimum executable strategy is to prompt for repositioning once and record one failure event. When consecutive failures reach the retry limit, a degradation strategy is implemented.

[0064] Step S2: Perform quality assessment and standardization preprocessing on the image to obtain fingerprint availability score and iris availability score;

[0065] Furthermore, the quality assessment uses sharpness, contrast, occlusion ratio, and reflection interference as basic indicators, and uniformly maps them to a usability score of [0,1]. In this embodiment, the quality threshold is 0.40 by default, with an adjustable range of 0.20–0.60. The threshold is selected based on the balance point between the target false rejection rate and false acceptance rate on the development set. Similarly, the standardized preprocessing includes grayscale normalization, size alignment, and geometric correction. Size alignment by default unifies the effective areas of the fingerprint and iris to a preset reference scale (with consistent equivalent resolution), and the adjustable range is ±20% of the reference scale. If any modality preprocessing fails or the score is invalid, the minimum executable strategy is to mark the modality as low-quality and hand it over to subsequent quality gating processing without interrupting the process.

[0066] Step S3: Extract the fingerprint minutiae set based on the fingerprint image, and generate the iris texture block set in the polar coordinate domain based on the iris image;

[0067] In this embodiment, fingerprint minutia extraction employs a conventional process of ridge tracing and minutiae selection, with local ridge quality and minimum ridge distance as constraints. Iris texture blocks are equally divided into fan-shaped block sets in the polar coordinate domain along the circumferential and radial directions, each with its own grayscale or texture descriptor. For example, the default maximum number of minutiae is 120, with an adjustable range of 60–180; the default number of iris fan-shaped blocks is 32, with an adjustable range of 16–64. These values ​​are determined based on a performance-complexity tradeoff of the model on the development set. Optionally, when excessive minutiae density or texture blocks cause the subsequent graph size to exceed the budget, downsampling is prioritized based on quality scoring and spatial balance to maintain stable computational load. In the case of missing tests, an empty set is entered into subsequent gating with zero weight.

[0068] Step S4: Construct a heterogeneous graph containing fingerprint minutiae and iris texture blocks, and perform feature propagation and fusion through a cross-modal graph neural network to obtain a fused feature vector;

[0069] Similarly, the node features of the heterogeneous graph are composed of local texture descriptions, spatial locations, and quality scores, while the edge features consist of spatial proximity and local texture differences. The default number of layers in the graph neural network is 3, with an adjustable range of 2–4 layers. The default hidden dimension per layer is 128, with an adjustable range of 64–256. The default number of attention heads is 4, with an adjustable range of 2–8. The default number of cross-modal Top-K connections per node is 6, with an adjustable range of 4–12. The above configurations are jointly calibrated based on inference latency budget and offline verification results. In this embodiment, the default upper limit of the graph size is 300 nodes and 3000 edges. When the upper limit is exceeded, low-quality nodes and weak edges are pruned first. The minimum executable approach when resources are insufficient is to degenerate to a single-layer message passing and retain half of the Top-K connections.

[0070] Step S5: Compare the fused feature vector with the registered template, and output the authentication result based on the comparison score and the preset threshold.

[0071] In this embodiment, the comparison targets the registration template of the same person, outputs a single score, and compares it with a threshold to determine whether it passes or fails. Furthermore, the threshold is calibrated by default according to the target false acceptance rate set on the development set, with an adjustable range for fine-tuning around small intervals of equal error intersections. Threshold updates are performed quarterly by default or in advance when drift monitoring is triggered. For example, a gray zone bandwidth is set to trigger a review; the gray zone width is 2%–5% of the score range by default to reduce the impact of fluctuations in edge samples. If the template is missing or damaged, the minimum executable action is to prompt for re-sampling and return a flag indicating that manual review is required.

[0072] Specifically, when the quality assessment of any modality is below a threshold, a quality gating strategy is used to assign reduced or masked aggregation weights to the graph nodes of that modality.

[0073] Similarly, quality gating is applied simultaneously at the node and edge levels: at the node level, linear or piecewise suppression is applied to message aggregation weights based on availability scores, while at the edge level, the weights are adjusted downwards based on the product of the scores of the two node endpoints. In this embodiment, the default suppression lower limit is set to two levels: mild suppression and complete masking. The adjustable range covers continuous suppression, and the gating curve is obtained by fitting a validation set. Furthermore, to avoid information isolation caused by complete masking of a single modality, the remaining adjacency weights are renormalized within the modality before aggregation to maintain numerical stability. If both modalities are simultaneously of low quality, a degradation strategy is prioritized over continued aggregation.

[0074] In one embodiment, fingerprint minutiae are the coordinates and orientation information of the ridge termination point and bifurcation point in the fingerprint image;

[0075] Iris texture blocks are grayscale or texture feature blocks obtained by fan-shaped partitioning of the denoised and normalized iris region.

[0076] The quality assessment includes a comprehensive score for acquisition integrity, sharpness, contrast, occlusion and reflection interference, and forms a configurable threshold;

[0077] In one embodiment, in step S4, the edge connectivity of the heterogeneous graph is determined by a cross-modal spatial embedding layer. The spatial embedding layer is used to align the image coordinates of fingerprint minutiae to the iris polar coordinate reference system and calculate cross-modal adjacency relationships. Both intramodal and cross-modal edges carry edge features based on spatial proximity and local texture similarity.

[0078] In step S4, the mapping method from fingerprint coordinates to the iris polar coordinate reference system is as follows:

[0079] Any fingerprint minutiae (x) f ,t f Mapping to the polar coordinate reference frame of the iris, obtaining (ρ, θ) to align with the center of the iris texture patch, the mapping is performed after a similarity transformation and then converted to polar coordinates:

[0080]

[0081]

[0082] Where ρ is the normalized radius, θ is the polar angle in rad, Φ is the spatial embedding map, and x f ,y f Let x0 and y0 be the x and y coordinates of the fingerprint minutiae in the fingerprint image coordinates, in pixels; S is the anisotropic scaling matrix; R(θ0) is the rotation matrix, where θ0 is the reference direction, in rad; x0 and y0 are the centroid coordinates of the fingerprint minutiae set, in pixels; r0 is the equivalent radius of the effective fingerprint region, in pixels; atan2(·,·) is the two-parameter arctangent function, returning the polar angle, in rad; and y′ and x′ are the intermediate transformation coordinates, in pixels.

[0083] In the formula:

[0084] θ0=atan2(∑ k sinψ k ,∑ k cosψ k ),

[0085] Where θ0 is the reference direction, in rad, ∑ k sinψ k With ∑ k cosψ k These are the sine and cosine components accumulated at index k, respectively, ψ kLet be the orientation angle of the kth detail point, in rad, where k is the index of the detail point;

[0086] Cross-modal adjacency determination based on spatial distance and local texture differences: In a unified reference frame, cross-modal edge weights are calculated for any fingerprint minutiae u and the iris texture block center v, and the adjacency set is constrained by a threshold and Top-K. The backbone weights are multiplicatively coupled with distance and texture terms and a quality gating is introduced.

[0087]

[0088] Among them, w uv For cross-modal adjacency weights, d uv This is a normalized distance measure, dimensionless, obtained by weighting the difference in polar radius and the difference in angle, σ. d For distance scale parameters, Δ t (u,v) is a measure of local texture difference (such as 1 minus cosine similarity or chi-square distance), σ t For texture scale parameters, g u ,g v The quality gating factor is derived from the modal / node quality score in step S2, and takes the value [0,1]. u and v are cross-modal node indices.

[0089] In the formula:

[0090]

[0091] in, ρ is the square of the spatial distance. u ,ρ v Let θ be the polar radius of the two nodes. u ,θ v The polar angle between the two nodes, in rad, λ θ Angular weighting coefficient, in rad. -2 ;

[0092] The adjacency establishment rule is: when w uv ≥τ w When v is a Top-K neighbor of u, an edge is connected, where τ w K represents the edge weight threshold, and K is the maximum number of cross-modal connections for a single node, in units of nodes.

[0093] Specifically, this step revolves around cross-modal alignment and adjacency determination. First, a unified reference system is constructed using similarity transformation and polar coordinates to avoid geometric deviations caused by inconsistencies between fingerprints and irises in terms of viewpoint, scale, and starting angle. The reference direction is obtained by statistically analyzing the directions of minutiae in the circumferential direction to reduce the impact of local anomalies on overall alignment.

[0094] Under a unified coordinate system, spatial proximity is measured by the composite distance of polar radius and polar angle, and the influence of circumferential deviation on the overall distance is adjusted by angular weights, so that the matching sensitivity of different circumferential regions can be controlled. In the texture dimension, a difference metric based on local descriptors is introduced to supplement scenes with insufficient geometric similarity, so that effective cross-modal candidates can still be screened in the case of low contrast or partial occlusion. Quality gating feeds the score of step S2 forward into the edge weights to suppress the negative effect of inferior nodes on message passing. Finally, sparse and stable cross-modal adjacency is formed by threshold and Top-K dual constraints, providing a reliable graph structure for subsequent attention message passing and fusion.

[0095] In this embodiment, the reference direction, scale matrix, rotation matrix, centroid, and effective region radius involved in the spatial embedding mapping are all derived from the geometric statistics of the current samples or produced by the preprocessing in the previous stage, and typical values ​​are robustly estimated from the development set. Furthermore, the symbols for distance scale, texture scale, and angular weights in cross-modal adjacency are determined on the validation set through grid search or Bayesian optimization before training, typically limited to an interval that balances sparsity and recall. Optionally, the edge threshold and Top-K are fine-tuned online in small steps during scene switching to adapt to different lighting and crowd conditions. If the mapping fails or the coordinates are abnormal, the degradation strategy is to retain only same-modal adjacencies and reduce the cross-modal prior weights in subsequent attention stages.

[0096] In one embodiment, in step S4, the graph neural network includes an attention-based message passing mechanism to calculate a similarity scalar for fingerprint minutiae and iris texture blocks and scale the aggregation weights; the similarity includes cosine similarity based on feature vectors and a weighted term based on edge features;

[0097] In step S4, after establishing cross-modal adjacency, the similarity between the feature vectors of fingerprint minutiae and iris texture blocks is measured and used as a prior term for attention scoring. The basic similarity is in cosine form.

[0098]

[0099] Among them, c uv h represents the cosine similarity between node pairs. u ,h v Let be the feature vectors of nodes u and v respectively, and <·,·> represent the inner product operation, and ‖·‖ represent the L2 norm;

[0100] Introducing temperature scaling to adjust the sharpness of the similarity distribution:

[0101] s uv =exp(β) T c uv ),

[0102] Among them, suv For the similarity score after temperature scaling, β T The reciprocal of the temperature coefficient is dimensionless, and exp(·) is an exponential function;

[0103] Before graph message passing, gated attention weights are calculated for each candidate adjacency, and the output representation is aggregated in the multi-head case; for the h-th attention head, soft normalization with prior bias is performed using the following formula:

[0104]

[0105] in, Let be the attention weight of the h-th head-to-edge (u,v). These are the query and key vectors, respectively. Indicates transpose, d h γ is the dimension of the key vector of this head. s The similarity prior weight coefficient is dimensionless, s uv For the temperature-scaled similarity score obtained from the above formula, γ w w is the a priori coefficient of the edge weights, dimensionless. uv The cross-modal adjacency weights obtained in step S4, ε w G is the numerical stability constant, dimensionless. u ,g v v is the quality gating factor, taking values ​​in the range [0,1], and v′ is the adjacency set. Any neighbor index in, Let be the set of candidate adjacencies of node u;

[0106] The linear aggregation quadratic expansion of the multi-head output is as follows:

[0107]

[0108] in, Let H be the update representation of node u, H be the number of attention heads, and π be the value of π. h V is the head aggregation coefficient, which is non-negative and sums to 1. (h) Let h be the value mapping matrix of the h-th head. v The input feature vector of the neighbor node v;

[0109] Specifically, this step revolves around cross-modal similarity priors and attention weights. First, cosine similarity is obtained using a cosine metric. Then, temperature scaling is used to alter the concentration of the distribution, giving highly similar samples a significant advantage in subsequent scoring while preventing low-similarity samples from being completely suppressed. Next, three types of information are introduced into the attention calculation: first, the query-key matching terms, reflecting the linear separability of feature alignment; second, prior terms obtained from similarity, guiding attention towards candidates with consistent geometry and texture; and third, edge weight biases from cross-modal adjacency construction, giving additional gains to connections that are spatially adjacent and locally texture-matched. Quality gating embeds node scores as factors into the numerator and denominator, reducing the impact of low-quality samples on normalized weights. Multi-head aggregation, through weighted summation of attention results from different subspaces, forms a more discriminative representation, providing stable input for subsequent residual units and fusion comparisons.

[0110] Similarly, the similarity prior and edge weight bias in attention scoring undergo temperature scaling and numerical stabilization before normalization. The temperature coefficient and stability constant are calibrated by the development set and kept within a safe range to avoid gradient explosion. For example, the aggregation coefficients of multi-head outputs are learned or fixed according to the constraint of non-negativity and summing to 1, with uniform aggregation being used by default. Optionally, when there is extreme sparseness in adjacency or excessively sharp similarity distribution, a robust processing of a default head discard ratio of 0.25 is performed, and the adjacency threshold is temporarily relaxed by a small step. If numerical overflow or invalid values ​​occur, the minimum feasible approach is to fall back to the previous layer of features and recalculate the current layer weights.

[0111] In one embodiment, in step S4, the cross-modal graph neural network includes a residual learning unit that uses shortcut connections to sum the features of the previous layer nodes with the residuals after nonlinear transformation, in order to alleviate deep training degradation and enhance robustness to low-quality samples.

[0112] In this embodiment, the residual learning unit uses the identity shortcut by default, and uses the dimension-matching linear projection shortcut to align the number of channels when necessary; the network depth is 3 layers by default, with an adjustable range of 2–4 layers; the residual injection ratio is 1:1 by default, with an adjustable range of 0.5:1–1.5:1, and the configuration is determined by both training stability and inference latency; similarly, to suppress noise amplification, a lightweight normalization or gating coefficient layer can be added before the residuals, and the gating coefficient is appropriately reduced on low-quality samples. If residual accumulation leads to numerical divergence, the fallback strategy is to temporarily close the deepest residual path.

[0113] In one embodiment, a liveness detection is performed on the fingerprint and iris images before authentication, and a rejection is directly output if the liveness detection fails; when authentication is successful, the summary of the fused features and the comparison log are encrypted and stored according to a compliance policy.

[0114] In this embodiment, liveness detection uses lightweight features such as texture consistency and micro-motion response as criteria. The judgment threshold is calibrated on the development set by default based on the target false rejection rate and can be adjusted within a small range according to the actual scenario. Furthermore, encrypted storage follows a local key hosting strategy and records the minimum necessary fields, with the default retention period not exceeding the minimum period required by business compliance. Optionally, when liveness detection fails consecutively beyond the upper limit, a degradation process is directly initiated, and retry within the current transaction is prohibited. In case of storage or encryption / decryption anomalies, the minimum feasible approach is to not write to the log and return a rejection.

[0115] In one embodiment, when the quality assessment of both modalities is below the threshold or the comparison score between the fusion feature and the registered template is in the gray zone, a degradation strategy is triggered, including: re-acquisition prompt, short-term caching and reuse of the most recent valid acquisition, or outputting an identifier that requires manual review.

[0116] Furthermore, the default cooldown time for re-collection is 3 seconds, adjustable from 1 to 10 seconds; the default validity period for short-term buffers is 5 minutes, adjustable from 1 to 15 minutes, and they automatically expire after the validity period. Similarly, when a manual review flag is triggered, the system only outputs the minimum information required for review and terminates subsequent automatic comparisons. Optionally, when the queue is congested, buffer reuse is prioritized to reduce window blocking. If the buffer is missing or unavailable, a prompt to re-collect once will be displayed before proceeding to manual review.

[0117] In one embodiment, the cross-modal graphical neural network is pre-trained offline and deployed for inference on a window device. During the training phase, a dataset containing multiple age groups and imaging conditions is used, and the loss objective includes a weighted combination of classification loss and metric learning loss. During the inference phase, only forward computation is performed. In this embodiment, the default update cycle for pre-training is once per quarter, with an adjustable range of on-demand hotfix or semi-annual update. Regression testing is completed on an independent validation set before weight distribution. Similarly, the latency budget for the inference end defaults to no more than 300ms, with an adjustable range of 100–500ms, and the memory usage defaults to no more than 200MB, with an adjustable range of 100–400MB. The above metrics are set based on the device's computing power and window throughput target. Optionally, when version rollback or hot update fails, the minimum executable approach is to enable the previous available version and record a rollback event.

[0118] This embodiment also provides a computer device, including: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize a deep learning-based method for collecting human biometric information as proposed in the above embodiment.

[0119] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0120] This embodiment also provides a storage medium storing a computer program that, when executed by a processor, implements a deep learning-based method for collecting human biometric information as proposed in the above embodiment. The storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random Access Memory (SRAM), Electrically Erasable Programmable Read-Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Red-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0121] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0122] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are meant to be within the scope of this application and form different embodiments. For example, all the embodiments above can be used in any combination. The information disclosed in this background section is intended only to enhance the understanding of the general background of this application and should not be construed as an admission or in any way implying that such information constitutes prior art known to those skilled in the art.

Claims

1. A method for collecting human biometric information based on deep learning, characterized in that, include, Step S1: Obtain fingerprint and iris images; Step S2: Perform quality assessment and standardization preprocessing on the image to obtain fingerprint usability score and iris usability score; Step S3: Extract the fingerprint minutiae set based on the fingerprint image, and generate the iris texture block set in the polar coordinate domain based on the iris image; Step S4: Construct a heterogeneous graph containing fingerprint minutiae and iris texture blocks, and perform feature propagation and fusion through a cross-modal graph neural network to obtain a fused feature vector; In step S4, the edge connections of the heterogeneous graph are determined by the cross-modal spatial embedding layer, which is used to align the image coordinates of fingerprint minutiae to the iris polar coordinate reference system and calculate cross-modal adjacency relationships; both intramodal and cross-modal edges carry edge features based on spatial proximity and local texture similarity. In step S4, the mapping method from fingerprint coordinates to the iris polar coordinate reference system is as follows: any fingerprint detail Mapped to the polar coordinate reference frame of the iris, we obtain Aligned with the center of the iris texture block, the mapping is performed by converting to polar coordinates after a similarity transformation: ， ， in, For the normalized radius, Polar angle, unit: rad. For spatial embedding mapping, These are the x and y coordinates of the fingerprint minutiae in the fingerprint image, in pixels. The anisotropic scaling matrix, For rotation matrix, Reference direction, unit: rad. The centroid coordinates of the fingerprint minutiae set, in pixels. The effective radius of the fingerprint area, in pixels. This is a two-parameter arctangent function that returns the polar angle in rad. The coordinates are intermediate transformation coordinates, in pixels; In the formula: ， in, Reference direction, unit: rad. and By index The sum of sine and cosine components, For the first The direction angle of a detailed point, in rad. Index for details; Cross-modal adjacency determination based on spatial distance and local texture differences: In a unified reference frame, for any fingerprint minutiae... Center of iris texture block Calculate cross-modal edge weights and use the threshold and Top- Constrained adjacency sets, the backbone weights employ multiplicative coupling of distance and texture terms and introduce quality gating: ， in, For cross-modal adjacency weights, This is a normalized distance measure, dimensionless, obtained by weighting the difference in polar radius and the difference in angle. For distance scale parameters, As a measure of local texture difference, For texture scale parameters, This is the quality gate factor, with a value of [value missing]. For cross-modal node indexing; In the formula: ， in, It is in the form of the square of the spatial distance. Let be the polar radius of the two nodes. The polar angle between the two nodes, in rad. Angular weighting coefficient, unit ; The adjacency establishment rule is: when and belong Top- When neighbors are connected, among them, The edge weight threshold, The maximum number of cross-modal connections for a single node, expressed in units of connections. In step S4, the graph neural network includes an attention-based message passing mechanism, calculates a similarity scalar for fingerprint minutiae and iris texture blocks, and scales the aggregation weights; the similarity includes cosine similarity based on feature vectors and a weighted term based on edge features; In step S4, after establishing cross-modal adjacency, the similarity between the feature vectors of fingerprint minutiae and iris texture blocks is measured and used as a prior term for attention scoring. The basic similarity is in cosine form. ， in, Let be the cosine similarity between node pairs. They are nodes eigenvectors, For inner product operations, It is a 2-norm; Introducing temperature scaling to adjust the sharpness of the similarity distribution: ， in, The similarity score is calculated after temperature scaling. It is the reciprocal of the temperature coefficient, dimensionless. It is an exponential function; Before graph message passing, gated attention weights are calculated for each candidate adjacency, and the output representation is aggregated in the multi-head case; for the th For each attention head, soft normalization with prior bias is performed using the following formula: ， in, For the first head to side Attention weights These are the query and key vectors, respectively. Indicates transpose. This represents the dimension of the key vector of the header. These are the prior weighting coefficients for similarity, which are dimensionless. The temperature-scaled similarity score obtained from the above formula, These are the a priori coefficients of the edge weights, which are dimensionless. The cross-modal adjacency weights obtained in step S4, The numerical stability constant is dimensionless. This is the quality gate factor, with a value range of [value range missing]. , adjacency set Any neighbor index in, For nodes The candidate adjacency set; The linear aggregation quadratic expansion of the multi-head output is as follows: ， in, For nodes The update indicates that, To focus on the number of heads, The aggregation coefficient of the head is non-negative and its sum is 1. For the first The value mapping matrix of the head, For neighboring nodes The input feature vector; Step S5: Compare the fused feature vector with the registration template, and output the authentication result based on the comparison score and a preset threshold. Specifically, when the quality assessment of any modality is below a threshold, a quality gating strategy is used to assign reduced or masked aggregation weights to the graph nodes of that modality.

2. The method for collecting personnel biometric information based on deep learning as described in claim 1, characterized in that, Fingerprint minutiae are the coordinates and orientation information of the ridge termination points and bifurcation points in a fingerprint image. Iris texture blocks are grayscale or texture feature blocks obtained by fan-shaped partitioning of the denoised and normalized iris region. The quality assessment includes a comprehensive score for acquisition integrity, sharpness, contrast, occlusion and reflection interference, and forms a configurable threshold.

3. The method for collecting personnel biometric information based on deep learning as described in claim 1, characterized in that, In step S4, the cross-modal graph neural network includes a residual learning unit, which uses shortcut connections to sum the features of the previous layer nodes with the residuals after nonlinear transformation, in order to alleviate deep training degradation and enhance robustness to low-quality samples.

4. The method for collecting personnel biometric information based on deep learning as described in claim 1, characterized in that, Before authentication, liveness detection is performed on fingerprint and iris images, and rejection is directly output if the liveness detection fails; when authentication is successful, the summary of the fused features and the comparison log are encrypted and stored in accordance with compliance policies.

5. The method for collecting personnel biometric information based on deep learning as described in claim 1, characterized in that, When the quality assessment of both modalities is below the threshold or the comparison score between the fused feature and the registered template is in the gray zone, a degradation strategy is triggered, including: prompting for re-acquisition, short-term caching and reusing the most recent valid acquisition, or outputting a label that requires manual review.

6. The method for collecting personnel biometric information based on deep learning as described in claim 1, characterized in that, The cross-modal graphical neural network is pre-trained offline and deployed for inference on the window device. During the training phase, a dataset containing multiple age groups and multiple imaging conditions is used, and the loss objective includes a weighted combination of classification loss and metric learning loss. During the inference phase, only forward computation is performed.

7. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the deep learning-based method for collecting and authenticating human biometric information as described in any one of claims 1 to 6.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by the processor, it implements the steps of the deep learning-based method for collecting and authenticating human biometric information as described in any one of claims 1 to 6.

Images