Chip card-oriented multi-mode biological feature fusion identity authentication method and system

Abstract

The embodiment of the invention provides a chip card-oriented multi-mode biological feature fusion identity authentication method and system. The method comprises the following steps of: acquiring multi-mode biological feature data of fingerprints, facial images and iris textures of a user in real time; generating an encrypted feature template through the multi-modal biological feature data, and transmitting the encrypted feature template to a chip card in groups; the data fusion time sequence of each modal feature is dynamically adjusted by monitoring the communication delay of the chip card and the real-time feedback of the biological feature encryption overhead; and executing multi-modal feature matching according to the adjusted time sequence, and outputting an identity authentication result. Through the scheme of the embodiment of the invention, the problem of how to regulate and control the multi-modal data fusion time sequence according to the real-time feedback of the chip card communication delay and the biological characteristic encryption overhead so as to solve the problem of authentication response timeout can be solved.

Description

Technical Field

[0001] This application relates to the field of information security technology, specifically to a multimodal biometric fusion identity authentication method and system for chip cards. Background Technology

[0002] In existing technologies, with the continuous improvement of information security requirements, the limitations of single biometric features (such as fingerprints, faces, or irises) in identity authentication are becoming increasingly prominent. Multimodal biometric fusion authentication is gradually becoming the mainstream solution for high-security scenarios (such as financial payments, government terminals, and border control). However, when implementing multimodal biometric fusion on the chip card (SE, IC card, or eID carrier) side, multiple serial steps need to be completed sequentially, including sensor acquisition, preprocessing, feature extraction, template splicing, encryption encapsulation, and interactive signature with the card's security domain. This results in an exponential increase in computational load and time consumption, leading to terminal transaction timeouts, retry degradation, or even transaction failures, seriously hindering the large-scale commercial deployment of multimodal biometric fusion authentication on resource-constrained chip cards. Summary of the Invention

[0003] In view of this, the present disclosure provides a multimodal biometric fusion authentication method for chip cards, which at least partially solves the problems existing in the prior art.

[0004] Multimodal biometric fusion authentication methods for chip cards include:

[0005] Real-time acquisition of multimodal biometric data of users, including fingerprints, facial images, and iris textures;

[0006] The multimodal biometric data is used to generate encrypted feature templates, which are then grouped and transmitted to the chip card.

[0007] By monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead, the data fusion timing of each modality is dynamically adjusted;

[0008] Perform multimodal feature matching based on the adjusted time sequence and output the identity authentication result.

[0009] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0010] Get the current communication delay D;

[0011] Obtain the current biometric encryption cost E;

[0012] The fusion timing adjustment factor α is calculated based on the following formula: α = k1 × (D / T) + k2 × (E / B), where T is the expected communication delay threshold, B is the expected encryption overhead threshold, and k1 and k2 are weighting coefficients.

[0013] The multimodal feature matching order is reordered based on the magnitude of α.

[0014] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0015] Obtain the historical communication delay D_avg;

[0016] Get the current communication delay D_cur;

[0017] Calculate the communication delay rate of change ΔD = |D_cur-D_avg| / D_avg;

[0018] The timing adjustment is determined based on the following formula: if ΔD > β, the timing adjustment strategy is triggered; otherwise, the original order is maintained, where β is the set rate of change threshold.

[0019] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0020] Get the average data volume Q within the current time slice;

[0021] Get the current encryption processing time T_enc;

[0022] The encryption load state is determined based on the formula: if Q / T_enc > γ, then enter the high load mode and reduce the data input priority of other modes; otherwise, maintain the normal process. In the formula, γ is the system load critical point, which represents the upper limit of the system's performance under high pressure.

[0023] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0024] The transmission delay of the collected fingerprint data is D_fingerprint;

[0025] The processing latency of the acquired facial image is D_face;

[0026] The upload latency for capturing iris textures is Di_iris;

[0027] The main timing order is determined by comparing T_total = max(D_fingerprint, D_face, D_iris) with η, where η is the upper limit of the acceptable maximum response time, and T_total is the system's maximum tolerable latency, ensuring that timeouts will not occur.

[0028] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0029] Calculate the actual contribution of each mode f_i = (1-ρ_i) / σ_i, where ρ_i is the matching failure rate of that mode and σ_i is the feature difference variance;

[0030] The order of matching is determined by the size of f_i;

[0031] Introduce a moving average mechanism: α_t = α_{t-1} × ω + f_i × (1 ω), where ω is the smoothing coefficient;

[0032] If α_t < ε, then the system enters an inefficient state, and the subsequent process is adjusted to frequency reduction. Here, f_i reflects the reliability and contribution of a single mode in the system, and ε is the inefficient state trigger threshold.

[0033] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0034] Collect the success rate S_n of the most recent N communications;

[0035] Calculate the encryption latency volatility Var(E);

[0036] The communication stability score is determined using the formula C_score = λ × S_n + (1-λ) × (1 / Var(E)).

[0037] If C_score < δ, enter conservative mode and select the most stable modality for priority matching, where λ is the score allocation coefficient and δ is the stability criterion.

[0038] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0039] Record the time T_match consumed for feature matching in each frame;

[0040] Calculate the remaining available time of the system, R_time;

[0041] Definition condition: If T_match ≥ R_time × ζ, then asynchronous merging is used;

[0042] If T_match < R_time × ζ, then a synchronous merging method is used to ensure a complete verification process, where ζ is a reserved ratio coefficient, representing the allowed remaining time and space.

[0043] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0044] Set the modal response window W[i] to represent the expected range of response time for the corresponding mode;

[0045] Calculate the timeout tendency index I[i] = (D[i]-W[i]) / W[i] for each mode, where D[i] represents the actual acquired response time of that mode;

[0046] Determine whether to enable the alternative path strategy based on I[i].

[0047] If I[i] > φ, then skip this mode and proceed directly to the next mode, where φ is the timeout warning coefficient.

[0048] Preferably, the step of dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the encryption overhead of biometrics further includes:

[0049] Construct the communication load curve F(D) = a × e^{b × D};

[0050] Construct the encrypted payload curve G(E) = c × ln(E + d);

[0051] The scheduling strategy is generated by combining the two load curves: the scheduling strategy is the matching sequence that minimizes ∑(F(D) + G(E));

[0052] The timing is adjusted according to the optimal sequence to ensure that timeouts are not caused by resource bottlenecks. Here, a, b, c, and d are system parameters, F(D) describes the trend of communication load changing with delay, and G(E) represents the relationship between encryption time and load.

[0053] This application also provides a multimodal biometric fusion identity authentication system for chip cards, including:

[0054] The acquisition module collects multimodal biometric data of users in real time, including fingerprints, facial images, and iris textures.

[0055] The encryption module generates an encryption feature template from the multimodal biometric data and transmits it in groups to the chip card;

[0056] The adjustment module dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead.

[0057] The output module performs multimodal feature matching based on the adjusted time sequence and outputs the identity authentication result.

[0058] This disclosure provides a multimodal biometric fusion authentication method for chip cards, comprising: real-time acquisition of multimodal biometric data of user fingerprints, facial images, and iris textures; generation of encrypted feature templates from the multimodal biometric data and grouping and transmitting them to the chip card; dynamic adjustment of the data fusion timing of each modality by monitoring the real-time feedback of chip card communication latency and biometric encryption overhead; and execution of multimodal feature matching according to the adjusted timing to output authentication results. The solution provided by this disclosure addresses the problem of authentication response timeout by adjusting the multimodal data fusion timing based on real-time feedback of chip card communication latency and biometric encryption overhead. Attached Figure Description

[0059] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the various drawings denote the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings depict only some embodiments disclosed in this application and should not be construed as limiting the scope of this application.

[0060] Figure 1 This is a flowchart of a multimodal biometric fusion authentication method for chip cards;

[0061] Figure 2 This is a block diagram of a multimodal biometric fusion identity authentication system for chip cards. Detailed Implementation

[0062] In the following description, only certain exemplary embodiments are briefly described. As those skilled in the art will recognize, the described embodiments can be modified in various ways without departing from the spirit or scope of this application. Therefore, the drawings and description are considered to be exemplary in nature and not restrictive.

[0063] With the deep integration of smart cards and biometric technology, multimodal identity authentication systems are widely used in financial payments, access control, and other fields. However, existing technologies generally treat chip card communication and biometric encryption as independent processes, neglecting their temporal coupling. On the one hand, half-duplex communication between the chip card and the terminal is affected by protocol frame length, radio frequency interference, and electric field strength, often resulting in unpredictable millisecond-level jitter. On the other hand, biometric template encryption (such as irreversible transformation and key derivation of fingerprints and faces) must be completed within the card's security domain. Its computational overhead increases exponentially with the feature dimension, encryption algorithm strength, and anti-side-channel masking strategy, resulting in processing times ranging from tens to hundreds of milliseconds. When the system adopts a fixed fusion timing (such as completing biometric encryption before starting card communication), the cumulative delay of any step directly compresses the overall authentication time window.

[0064] Next, please refer to the appendix. Figure 1 This invention describes a multimodal biometric fusion authentication method for chip cards. The method includes:

[0065] S101: Real-time acquisition of multimodal biometric data including user fingerprints, facial images, and iris texture. Specifically, multiple sensors, such as a high-resolution fingerprint scanner, a high-definition camera, and an infrared camera, can be integrated into the device to acquire the user's fingerprint, facial, and iris information respectively, and these data are unified into a digital data format for subsequent processing and analysis. In practice, the system begins to simultaneously acquire information from these three modalities when the user touches the fingerprint sensor, faces the camera, and places their eyes in the iris acquisition device. For example, when a user logs into a payment application on a mobile terminal, the device automatically activates the acquisition modules for all three biometric features, ensuring that all data is acquired within the same time window.

[0066] S102: An encrypted feature template is generated from the multimodal biometric data and transmitted to the chip card in groups. In this stage, the collected biometric information undergoes preprocessing, feature extraction, and encryption to form a secure encrypted feature template, which is typically protected using symmetric or asymmetric encryption algorithms. Then, according to different biometric categories, the corresponding feature data is grouped to reduce the computational burden on the chip card during processing and improve communication efficiency. For example, in one embodiment, the system generates independent encrypted features for fingerprint, facial, and iris data respectively, and then divides them into three different batches and sends them to the chip card for storage and verification. This method avoids the resource consumption problems caused by centralized data processing.

[0067] S103: By monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead, the data fusion timing of each modality is dynamically adjusted. This step relies on an embedded monitoring mechanism within the system, which can track in real-time the communication latency required by the chip card when receiving data and the processing resources consumed during encryption operations. Based on this feedback information, the algorithm will rearrange the input order and waiting time of each modality's data to adapt to the current processing capabilities of the chip card. Specifically, if it is detected that the data transmission time of a certain modality's biometric data is too long or the encryption computing resources are insufficient, the system will prioritize scheduling low-latency or low-overhead modality information, thereby reducing the response time in the overall authentication process.

[0068] Assuming this method is applied to a mobile payment scenario requiring high security, users authenticate payments using a mobile phone integrated with a chip card. The initial default fusion sequence is set to "fingerprint → face → iris". During the authentication process, the system monitors two key performance indicators in real time: first, the communication latency for data exchange with the chip card (e.g., signal interference between the phone and the card reader causes the latency to surge from the normal 10ms to 80ms); second, the overhead of the chip card in processing biometric encryption (e.g., encrypting a high-resolution facial feature template requires 80% of the chip card's CPU resources, while encrypting a fingerprint feature template only requires 20%). When the system detects excessively high communication latency through real-time feedback, or detects that the chip card is experiencing resource strain due to processing a certain modality (such as facial) encryption task, it triggers a dynamic adjustment mechanism. For example, the system will immediately assess the current status of each modality: if the fingerprint sensor is ready and its data volume is small and encryption overhead is low, while the facial image is still undergoing liveness detection due to poor ambient light and has a heavy encryption burden, the system will prioritize the transmission and encryption of the fingerprint feature, and temporarily adjust the fusion sequence from the default "fingerprint → face → iris" to "fingerprint → iris → face", thereby bypassing the current bottleneck and ensuring the startup speed and overall efficiency of the authentication process.

[0069] Specifically, the method in this application continuously receives latency data from the chip card communication interface and resource utilization data from the chip card encryption coprocessor. When a certain indicator (such as latency > 50ms or encryption overhead > 60%) exceeds a preset threshold, the algorithm recalculates the priority of each modality. For example, it lowers the priority of modalities with large data volumes and complex encryption calculations (such as high-precision facial images), while increasing the priority of modalities with small data volumes, fast processing speeds, and low encryption requirements (such as fingerprints or iris codes). Based on the new priority ranking, the system reconstructs the feature data transmission and fusion queue in real time. This dynamic scheduling ensures that when chip card resources are limited or the communication environment is poor, the system can prioritize the fusion of "lightweight" and fast-responding features for initial matching. If the initial matching confidence is high enough, authentication can even be completed ahead of time without waiting for all modalities to finish processing. Ultimately, this method overcomes the problem of poor adaptability of fixed-time fusion in complex environments, significantly improves the real-time performance, robustness, and user experience of the identity authentication system, and fully utilizes the limited computing and communication resources of the chip card.

[0070] S104: Perform multimodal feature matching based on the adjusted time series and output the identity authentication result. In this step, the adjusted time series data is sequentially fed into feature matching algorithms, such as template matching, support vector machines, or deep neural networks, to calculate the similarity score between each modality feature and the registered information. These scores are then combined to determine whether authentication is successful. Through reasonable time series control, the system avoids the timeout of a single biometric feature processing from affecting the overall system's response efficiency, thus effectively solving the authentication response timeout problem. For example, in practical applications, if the data for a certain modality is temporarily not loaded due to communication fluctuations, the system will temporarily skip this part, prioritize the comparison of other completed data, and finally supplement the incomplete information according to weighted allocation, ensuring the entire authentication process is efficient and reliable.

[0071] The beneficial effects of this technical solution are that by intelligently controlling the real-time feedback of chip card communication latency and biometric encryption overhead, the timing of multimodal biometric data processing is optimized, the overall response speed and stability of identity authentication are improved, the user experience is enhanced, and the probability of abnormal system interruption is reduced. It is suitable for scenarios with high security requirements such as payment and access control.

[0072] Next, specific embodiments of the present invention will be described, which dynamically adjust the data fusion timing of each modality feature by monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead.

[0073] Specifically, the current communication latency D can be obtained first to measure the data transmission speed between the chip card and the terminal device. A lower communication latency D indicates faster transmission. For example, in a chip card system using both fingerprint and iris scanning, if network fluctuations cause the communication latency to exceed the expected threshold T, it will affect overall efficiency.

[0074] Next, the current biometric encryption cost E is obtained, representing the computational resources or time cost required to complete a single biometric match. E is typically determined by algorithm complexity and hardware performance; ideally, E should be kept within a reasonable range to avoid computational bottlenecks. For example, if the cost of certain biometric matches is found to be excessive during the encryption process, such as high-precision algorithms for face recognition, the value of E may increase significantly.

[0075] The fusion timing adjustment factor α is then calculated based on the formula α = k1 × (D / T) + k2 × (E / B), where T is the expected communication latency threshold, B is the expected encryption overhead threshold, and k1 and k2 are coefficients that adjust the weights of communication latency and encryption overhead, typically set between 0.1 and 1.0. α is used to comprehensively evaluate the impact of both on the fusion timing. This formula is designed to achieve dynamic optimization, improving user experience while ensuring security.

[0076] Finally, the multimodal feature matching order is reordered based on the magnitude of α to ensure that features with fast response and low overhead are processed first. For example, when α is large, the fingerprint can be verified before the iris, making the overall authentication process smoother and more efficient. This mechanism effectively balances communication efficiency and computational overhead, improving the system's adaptability and stability in different environments. In a more specific embodiment, the multimodal data fusion timing can be adjusted when the fusion timing adjustment factor α is greater than a predetermined threshold.

[0077] In a more specific embodiment, the system pre-sets multiple time-series strategy thresholds corresponding to the adjustment factor α, for example, setting thresholds α1=1.2 and α2=1.5. During the authentication process, the system calculates the α value in real time and compares it with these thresholds to trigger different sorting logics—if α < 1.2, it indicates a light system load, and the default matching order prioritizing authentication accuracy, "face → fingerprint → iris," is used; if the α value is detected to rise to 1.2 ≤ α < 1.5, it indicates a significant increase in communication latency or encryption overhead, and the system activates an efficiency-first strategy, prioritizing the matching of fingerprint feature templates with smaller data volume and faster encryption calculations, adjusting the sorting to "fingerprint → face → iris"; if α further increases to α ≥ A value of 1.5 indicates that system resources are extremely strained. At this point, the system will only schedule the fingerprint and iris features with the least computational load for fast matching, and temporarily skip the most complex facial features. The order will be optimized to "fingerprint → iris". If the confidence of the joint matching of the first two is high enough, the authentication result will be output directly. This will minimize the delay of the authentication process while ensuring security, and realize intelligent and adaptive time-series scheduling under resource-constrained conditions.

[0078] Next, an example of another embodiment of the present invention is described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0079] First, obtain the historical communication latency D_avg. This parameter reflects the average communication response time of the chip card over a certain period of time and is used as the basis for judging the current communication status.

[0080] Next, obtain the current communication latency D_cur. This refers to the actual communication response time between the chip card and the server at a specific moment, used to compare real-time performance with historical average levels. This value directly reflects the instantaneous communication state of the system and can be used to assess stability and the probability of abnormal situations.

[0081] The communication delay variation rate ΔD = |D_cur - D_avg| / D_avg is calculated. This formula quantifies the deviation of the current communication delay from the historical average, avoiding the influence of absolute value fluctuations on decision-making. ΔD generally ranges from 0 to 0.2. Exceeding the threshold β (e.g., setting β to 0.2) indicates a decrease in communication stability, requiring optimization measures.

[0082] The timing adjustment is determined based on a formula: if ΔD > β, a timing adjustment strategy is triggered; otherwise, the original order is maintained. This mechanism can proactively optimize the processing order of modal data under unstable conditions, improving the overall system's response efficiency and authentication reliability.

[0083] For example, in one embodiment, when the chip card performs dual-modal authentication using fingerprint and iris scanning, if it detects a sudden increase in the current communication latency from 30ms to 60ms, resulting in ΔD = 1.0, which is much higher than the set β value of 0.2, the system automatically prioritizes transmitting the more stable fingerprint information, reducing the risk of authentication failure due to communication instability. This solution improves the adaptability and authentication efficiency of the chip card under heterogeneous network conditions.

[0084] Next, an example of another embodiment of the present invention is described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0085] Specifically, in this example, the average data volume Q within the current time slice is first obtained. Q represents the total amount of multimodal biometric data received by the chip card within a fixed time period, measured in bytes or data packets. This step is used to measure the data input pressure.

[0086] Next, obtain the current encryption processing time T_enc. T_enc is the average time required for the chip card to complete one multimodal data encryption operation, in milliseconds. This parameter reflects the system's current processing capacity and resource allocation status.

[0087] Then, the encryption load status is determined based on the formula: if Q / T_enc > γ, then the system enters high-load mode, reducing the data input priority of other modes; otherwise, the normal process is maintained. Here, γ is the system load critical point, representing the upper limit of system performance under high pressure. Thus, together with the adjustment factor α and the communication delay change rate ΔD mentioned above, a multi-dimensional, three-dimensional system status monitoring network is formed.

[0088] Specifically, in this application, the system uses a sliding time window (e.g., the most recent 100 milliseconds) as a period to obtain the average data volume Q within the current time slice (i.e., the average amount of data transmitted to the chip card's fingerprint, facial, and iris feature templates during this period, e.g., a total of 15KB), and simultaneously obtains the current encryption processing time T_enc (i.e., the time taken for the chip card's encryption coprocessor to actually process this 15KB of data, e.g., 5 milliseconds). Subsequently, the system calculates the ratio of Q / T_enc, which essentially reflects the real-time data throughput of the chip card's encryption module (3KB / ms in this example). The system compares this real-time throughput with a preset system load threshold γ (e.g., 5KB / ms, which is experimentally determined and represents the upper limit of the chip card's performance while ensuring stability and response time): if Q / T_enc > γ, it means that the current data inflow rate has exceeded the chip card's encryption processing capacity limit, and the system immediately enters a high-load mode. In this mode, the system intelligently reduces the data input priority of other modalities. For example, if high-resolution facial image data is identified as the main cause of an excessively large Q value, the system will postpone the transmission of subsequent facial feature templates and place them later in the fusion sequence. At the same time, it will prioritize the allocation of communication and computing resources to fingerprint features with smaller data volume and faster encryption, thereby quickly alleviating congestion. Otherwise, when the real-time throughput does not exceed γ, the system maintains the normal process and processes all modalities in the predetermined order.

[0089] Next, the steps of another embodiment of the present invention are described, which involve dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0090] In this embodiment, firstly, the transmission latency D_fingerprint of the fingerprint data is collected, which measures the time required for data transmission from the fingerprint acquisition module to the chip card. Secondly, the processing latency D_face of the facial image is collected, representing the time required for preprocessing and feature extraction of the facial image. Next, the upload latency D_iris of the iris texture is collected, reflecting the time taken for the iris information to be uploaded to the chip card. The main timing order is determined according to the formula T_total = max(D_fingerprint, D_face, D_iris) ≤ η, where η is the upper limit of the acceptable maximum response time. In this formula, T_total represents the maximum latency that the system can tolerate, ensuring that the overall process is completed within the user's acceptable time range. Using the maximum latency of each modality feature as the basis for judging the total system latency ensures that the overall timeout will not be caused by excessive latency of a certain modality.

[0091] Specifically, when initiating authentication, the system concurrently collects the transmission latency of fingerprint data (D_fingerprint, e.g., the time required to read and preprocess fingerprint features from the sensor, 30 milliseconds), the processing latency of facial image acquisition (D_face, e.g., the time required for liveness detection and feature extraction, 80 milliseconds), and the upload latency of iris texture acquisition (D_iris, e.g., the time required to capture and compress the iris code via a near-infrared camera, 50 milliseconds). The system then calculates T_total = max(30, 80, 50) = 80 milliseconds, taking the maximum value among the three as the estimated maximum time required to complete all data preparation under the current environment. The system compares this T_total value with a preset acceptable maximum response time limit η (for example, the system requires the authentication process to give a result within 100 milliseconds, so η is set to 100 milliseconds). If T_total ≤ η, such as 80ms ≤ 100ms in the example above, the system determines that the current environment is good and can maintain a primary timing sequence that prioritizes authentication accuracy or user experience, i.e., in order of decreasing latency, such as "face → iris → fingerprint" according to user operating habits. If T_total > η (for example, if the facial liveness detection time surges to 120ms due to dim ambient light, then T_total = 120ms > η = 100ms), the system must immediately activate an emergency strategy. It will select a primary timing sequence that prioritizes speed, such as prioritizing the modality with the fastest data preparation and the lowest latency (fingerprint in this example), i.e., in order of increasing latency, forming the sequence "fingerprint → iris → face", to ensure that the core authentication process can start as soon as possible and avoid overall authentication timeout.

[0092] Unlike the passive adjustments based on "current" or "historical" loads mentioned above, this configuration allows D_fingerprint, D_face, and D_iris to be estimated during the data acquisition phase, representing a proactive approach to resource scheduling. This enables the system to make timing decisions early in the feature fusion and matching process. It allows the system to identify bottleneck modes and pre-plan optimal strategies for subsequent communication and computing resource allocation, thus avoiding a chain reaction of adverse effects caused by a mode "stuttering" later in the process. Furthermore, this embodiment focuses on estimating the actual time consumption during the data preparation phase, providing an initial, objectively delayed "master timing" blueprint for the entire dynamic adjustment process. The scheme described above, on the other hand, is responsible for finer adjustments during the subsequent transmission and encryption phases.

[0093] Next, an embodiment of the present invention is further described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0094] In this embodiment, firstly, the actual contribution of each modality is calculated as f_i = (1 - ρ_i) / σ_i, where ρ_i represents the matching failure rate of the modality, ranging from [0, 1], and σ_i is the feature variance of the modality, reflecting the stability of the feature distribution. This formula combines the matching success rate (1 - ρ_i) with the feature consistency (σ_i) to comprehensively evaluate the effectiveness of each modality. Its optimal value is f_i as large as possible, meaning that the modality has high reliability and discriminative power. Next, the priority matching order is determined according to the size of f_i, ensuring that modalities with high contribution are processed first, thereby improving the overall matching efficiency. Subsequently, a moving average mechanism is introduced: α_t = α_{t-1} × ω + f_i × (1 - ω), where ω is a smoothing coefficient, ranging from [0, 1], used to balance the influence of current and historical data and prevent misjudgment due to instantaneous fluctuations. When α_t is less than ε (i.e., the inefficient state trigger threshold), the system is judged to be inefficient and enters the frequency reduction processing mode to reduce resource consumption and response time.

[0095] Specifically, the system first calculates the actual contribution f_i of each modality. For example, for the fingerprint modality, the system calculates its recent matching failure rate ρ_i (e.g., 5%) and the variance σ_i of the feature extraction results (e.g., 0.1), and substitutes these values ​​into the formula f_i = (1-ρ_i) / σ_i to calculate its contribution (in this example, f_fingerprint = (1-0.05) / 0.1 = 9.5). This value f_i comprehensively reflects the reliability and contribution of a single modality in the system; a higher value indicates that the modality is more stable and reliable. Subsequently, the system introduces a moving average mechanism, that is, it updates a smoothing value α_t representing the overall effectiveness of the modality according to the formula α_t = α_{t-1} × ω + f_i × (1-ω), where ω is a smoothing coefficient (e.g., 0.9), used to reduce the fluctuation of a single calculation, so that α_t can reflect the long-term trend of the contribution (e.g., in this calculation, α_t = 8.6 × 0.9 + 9.5). (×0.1 = 8.69); The system compares this smoothed α_t value with a preset inefficiency trigger threshold ε (e.g., 8.5). If α_t < ε, it determines that the modality has been in an inefficient state (e.g., its long-term contribution has declined due to user fingerprint wear). The system then enters the inefficient state and adjusts subsequent processes to reduce frequency, such as lowering the sampling resolution of the modality, reducing the number of feature comparisons, or reducing its weight in the fusion decision, thereby concentrating limited computing resources on other modalities with higher contributions. This setting avoids overly aggressive strategy adjustments due to accidental fluctuations in the single authentication environment (e.g., brief light and shadow interference with facial recognition), ensuring that system decisions are based on more stable and reliable long-term data, and improving the robustness and accuracy of the adjustment strategy.

[0096] Next, an example of another embodiment of the present invention is described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0097] First, the success rate S_n of the most recent N communications is collected. S_n represents the proportion of times the chip card successfully completed communication in the most recent N communications, and its range is from 0 to 1. The higher the value of S_n, the better the overall stability of the system. For example, if the chip card successfully returns the verification result in multiple biometric verification attempts, then S_n is close to 1.

[0098] Secondly, calculate the encryption time volatility Var(E). Var(E) is the standard deviation of the encryption time data for the most recent encryption processes, reflecting the uncertainty or abnormalities in the encryption process. The larger the value, the more unstable the encryption time. For example, when the encryption time increases due to high system load at certain times, Var(E) will increase accordingly.

[0099] Next, the communication stability score is determined using the formula C_score = λ × S_n + (1 - λ) × (1 / Var(E)). In this formula, λ is the score allocation coefficient, typically ranging from 0 to 1, used to balance the success rate and encryption stability. If λ is close to 1, more emphasis is placed on the success rate; if it is close to 0, more emphasis is placed on encryption stability. 1 / Var(E) reflects encryption efficiency, avoiding misjudgments of the system state due to extreme fluctuations.

[0100] Then, if C_score < δ, the system enters conservative mode, prioritizing the matching of only the most stable modality. δ is a set threshold used to determine whether a more cautious strategy is needed. For example, when C_score is below a critical value, the system can prioritize the fingerprint modality with the highest stability to reduce the risk of false identification and improve processing speed.

[0101] This technical solution can respond to changes in system status in real time, improve the reliability and flexibility of identity authentication, and effectively cope with changes in network conditions and hardware load.

[0102] Next, the steps of the present invention are further described to dynamically adjust the data fusion timing of each modality feature by monitoring the real-time feedback of the communication delay of the chip card and the biometric encryption overhead.

[0103] First, record the time T_match consumed for feature matching in each frame. This step measures the actual time spent on each feature comparison, serving as a basis for subsequent decisions. Next, calculate the remaining available system time R_time. This step determines how much executable time the current task has left based on the overall system processing time constraints. Then, define the following conditions: if T_match ≥ R_time × ζ, use asynchronous merging; if T_match < R_time × ζ, use synchronous merging to ensure a complete verification process. This condition determines whether different data fusion strategies are needed to adapt to real-time requirements under different loads.

[0104] ζ is the reserved ratio coefficient, representing the allowable remaining time space. It is generally set in the range of 0.8 to 0.95, with the optimal value being around 0.9 based on the actual system performance. Setting this parameter ensures that data fusion operations do not time out due to excessive processing time, thus improving system stability and user experience.

[0105] Specifically, during the matching process, the system continuously records the time T_match consumed for feature matching in each frame (e.g., 15 milliseconds for completing one fingerprint feature comparison) and calculates the remaining available time R_time (e.g., if the system's total authentication time limit is set at 100 milliseconds, and 70 milliseconds have already elapsed, then R_time = 30 milliseconds). The system then compares T_match with R_time × ζ, which takes into account buffer space, where ζ is a reserve ratio coefficient (e.g., set to 0.8), representing the allowed remaining time space to cope with unforeseen minor fluctuations. The system defines conditions and makes decisions: If T_match ≥ R_time × ζ (i.e., 15ms ≥ 30ms × 0.8 = 24ms, which does not hold true in this example), it means that the time required to complete the next full match is likely to take up almost all or even exceed the remaining time budget. The system will adopt an asynchronous merging method. For example, after the fingerprint matching is completed, it will not wait for the iris matching result which takes a long time to calculate, but will immediately merge the fingerprint result with the previously cached facial result and output a preliminary authentication conclusion. If T_match < R_time × ζ (as shown in this example), it indicates that the time budget is relatively sufficient. The system adopts a synchronous merging method to ensure the complete verification process. That is, it waits for the matching results of all predetermined modalities to return step by step before making a unified decision in order to pursue the highest authentication accuracy.

[0106] Next, an embodiment of the present invention is further described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0107] In this embodiment, the real-time bandwidth utilization rate B_usage is collected first. This parameter reflects the current load status of the communication link. B_usage ranges from 0 to 1, where 0 indicates that the bandwidth is completely idle and 1 indicates that it is running at full capacity.

[0108] Next, we analyze the user's historical device models and protocol support status, M_support. M_support[i] represents the compatibility score of the i-th mode on the user device, ranging from 0 to 1, where 0 indicates incompatibility and 1 indicates full compatibility. This parameter is used to evaluate the stability and efficiency of different modes on a specific hardware platform.

[0109] Construct the priority matrix: P_matrix[i] = B_usage × M_support[i]. This formula calculates the execution priority of each mode by combining bandwidth consumption and compatibility, ensuring that modes with lower bandwidth consumption and higher compatibility are executed first, thereby improving the overall system responsiveness.

[0110] The execution order is selected after sorting by P_matrix[i]. This step schedules tasks based on priority, optimizes data fusion timing, and reduces communication bottlenecks and authentication latency.

[0111] Specifically, the system first collects the real-time bandwidth utilization rate B_usage (for example, by monitoring the current wireless channel quality, the ratio of available bandwidth to total bandwidth is calculated, such as 60%). This value will be used as the bandwidth consumption weight, directly reflecting the current network congestion level. At the same time, the system analyzes the user's historical device model and protocol support status M_support, and assigns a compatibility score to the sensor or module used for each mode (for example, for an older mobile phone model, its fingerprint sensor driver is stable, M_support[fingerprint] = 0.9; while its facial recognition module is an early version, M_support[face] = 0.6; and it is not equipped with an iris module, M_support[iris] = 0). Subsequently, the system constructs a priority matrix: P_matrix[i] = B_usage × M_support[i], which calculates the priority score for each modality (in this example, P_fingerprint = 0.6 × 0.9 = 0.54, P_face = 0.6 × 0.6 = 0.36, P_iris = 0.6 × 0 = 0). Finally, the system sorts the data according to P_matrix[i] and selects the execution order, that is, it determines the fingerprint modality to be processed first, followed by the facial modality, and completely skips the iris modality.

[0112] Next, the steps of another embodiment of the present invention are described, which involve dynamically adjusting the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0113] First, set the modal response window W[i], which represents the expected range of response time for the corresponding modality. W[i] is used to quantify the average response time range of each biometric modality under normal conditions. It is usually set to the average of historical data plus or minus a certain percentage to ensure that the system is neither too sensitive nor misinterprets normal responses.

[0114] Then, the timeout tendency index I[i] = (D[i] - W[i]) / W[i] is calculated for each modality. Here, D[i] represents the actual collected response time of that modality, W[i] is the pre-set response window, and I[i] reflects whether the actual response of the modality exceeds expectations; a negative value indicates early completion, and a positive value indicates exceeding the threshold. A larger positive value increases the likelihood of timeout, therefore, a reasonable φ value needs to be set for early warning judgment.

[0115] Next, it is determined whether to enable the alternative path strategy based on I[i]. φ is the timeout warning coefficient, which is generally a positive number between 0.2 and 0.5. Too high a value may mistakenly block valid modes, while too low a value may cause response delays. When I[i] > φ, the mode is temporarily blocked, and the process jumps to the next higher priority mode to reduce the total response time and ensure the continuity of the process.

[0116] For example, in one embodiment, due to device busyness, D[i] for a certain biometric modality (such as fingerprint) is significantly higher than W[i], causing I[i] to reach 0.6, exceeding φ's 0.5. Based on this, the system skips the fingerprint authentication step and directly proceeds to face comparison, thereby shortening the overall authentication time and improving user experience. This technical solution can effectively cope with real-time environmental changes and improve the system's robustness and response efficiency.

[0117] Next, an embodiment of the present invention is described, which dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the communication latency of the chip card and the biometric encryption overhead.

[0118] First, construct the communication load curve F(D) = a × e^{b × D}. Here, D represents the communication latency, and a and b are system parameters used to describe the trend of communication load changing with latency. Parameter a is typically in the range of 0.1 to 1.0, representing the initial communication burden; b is in the range of 0.01 to 0.1, reflecting the degree of impact of latency on the load. For example, if the chip card exhibits a significant performance degradation under high latency conditions, this formula can quickly reflect the change.

[0119] Next, we construct the encryption load curve G(E) = c × ln(E + d). E represents the time or complexity of the encryption operation, while c and d are system parameters. c ranges from 0.5 to 2.0, and d ranges from 1 to 10, representing the basic burden of the encryption operation. This formula uses the natural logarithm to illustrate the decreasing rate of increase in encryption overhead with increasing load. For example, when performing multiple encryption tasks, the G(E) function can effectively simulate the state where the encryption time gradually approaches saturation.

[0120] Then, a scheduling strategy is generated by combining the two load curves: a matching sequence with the objective of minimizing ∑(F(D) + G(E)). This strategy aims to balance the use of communication and encryption resources, avoiding system timeouts caused by excessive load in one type. For example, if the current encryption overhead is high, the workload of the communication module is reduced first, and the fusion order is adjusted accordingly.

[0121] Finally, the timing is adjusted based on the optimal sequence to ensure reasonable resource allocation and prevent timeouts.

[0122] Specifically, the system first constructs a communication load curve F(D) = a × e^{b × D}, where F(D) describes the trend of communication load changing with latency. This exponential function form (where a and b are system parameters) accurately depicts the nonlinear relationship where even a slight increase in communication latency D leads to a sharp deterioration in system communication load (for example, when latency D increases from 10ms to 20ms, the calculated result of F(D) may surge from 1.0 to 2.5). Simultaneously, the system constructs an encryption load curve G(E) = c × ln(E + d), where G(E) represents the relationship between encryption time and load. This logarithmic function form (where c and d are system parameters) reflects that as encryption overhead E increases, the resulting system load G(E) increases, but the rate of increase gradually slows down, which aligns with the marginal effect of encryption tasks running on the chip card. Subsequently, the system integrates the two load curves to generate a scheduling strategy. Its core objective is to find a modality matching sequence such that the sum of ∑(F(D) + G(E)) for all modes in the sequence is minimized. For example, the system enumerates multiple possible sequences such as "fingerprint → face → iris" and "face → fingerprint → iris," and calculates the sum of the communication and encryption load curve values ​​for each sequence, ultimately selecting the sequence with the lowest total load. The system adjusts the timing based on this optimal sequence, thus mathematically ensuring the lightest overall system load and preventing timeouts due to resource bottlenecks.

[0123] The multimodal biometric fusion authentication method for chip cards of the present invention includes: firstly, real-time acquisition of multimodal biometric data such as user fingerprints, facial images, and iris textures; and secondly, conversion of these raw data into encrypted feature templates using a high-performance algorithm. Next, the generated feature templates are grouped according to certain rules and transmitted to the chip card's internal storage module. Subsequently, the system obtains real-time feedback information by monitoring the chip card's communication latency and biometric encryption processing time. Based on this feedback, the system dynamically adjusts the data acquisition and transmission sequence of each biometric modality to optimize the overall fusion process time allocation. Finally, multimodal feature matching calculations are performed based on the adjusted sequence to ensure high-accuracy authentication within a limited time. To address the issue of authentication response timeout, this method introduces an adaptive control mechanism, dynamically optimizing the data fusion order and execution rhythm by real-time monitoring of the chip card's communication and encryption performance. For example, when high communication latency is detected, the system can prioritize processing feature modes with lower computational overhead but higher matching reliability, or adjust the data packetization strategy to balance the transmission load; when encryption overhead is too high, the accuracy requirements of other modes can be appropriately reduced to ensure that key modes complete authentication in a timely manner, thereby effectively reducing authentication response time while ensuring security, and improving the overall system response efficiency and user experience.

[0124] In addition, this application also discloses a multimodal biometric fusion identity authentication system 100 for chip cards, the system comprising:

[0125] The acquisition module 101 acquires multimodal biometric data of the user's fingerprints, facial images, and iris textures in real time;

[0126] Encryption module 102 generates encrypted feature templates from the multimodal biometric data and transmits them in groups to the chip card;

[0127] The adjustment module 103 dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead.

[0128] Output module 104 performs multimodal feature matching based on the adjusted time sequence and outputs the identity authentication result.

[0129] The functions and implementation methods of each module in the multimodal biometric fusion identity authentication system 100 for chip cards have been referenced above. Figure 1 The description has already been provided, so I will not repeat it here.

[0130] The above description is the preferred embodiment of this application. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of this invention, and these improvements and modifications should also be considered within the scope of protection of this application.

Claims

1. A multimodal biometric fusion authentication method for chip cards, characterized in that, include: Real-time acquisition of multimodal biometric data of users, including fingerprints, facial images, and iris textures; The multimodal biometric data is used to generate encrypted feature templates, which are then grouped and transmitted to the chip card. By monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead, the data fusion timing of each modality is dynamically adjusted; Perform multimodal feature matching based on the adjusted time sequence and output the identity authentication result.

2. The multimodal biometric fusion authentication method for chip cards according to claim 1, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Get the current communication delay D; Obtain the current biometric encryption cost E; The fusion timing adjustment factor α is calculated based on the following formula: α = k1×(D / T) + k2 × (E / B), where T is the expected communication delay threshold, B is the expected encryption overhead threshold, and k1 and k2 are weighting coefficients. The multimodal feature matching order is reordered based on the magnitude of α.

3. The multimodal biometric fusion authentication method for chip cards according to claim 1, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Obtain the historical communication delay D_avg; Get the current communication delay D_cur; Calculate the communication delay rate of change ΔD = |D_cur-D_avg| / D_avg; The timing adjustment is determined based on the following formula: if ΔD > β, the timing adjustment strategy is triggered; otherwise, the original order is maintained, where β is the set rate of change threshold.

4. The multimodal biometric fusion authentication method for chip cards according to claim 3, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Get the average data volume Q within the current time slice; Get the current encryption processing time T_enc; The encryption load state is determined based on the formula: if Q / T_enc > γ, then enter the high load mode and reduce the data input priority of other modes; otherwise, maintain the normal process. In the formula, γ is the system load critical point, which represents the upper limit of the system's performance under high pressure.

5. The multimodal biometric fusion authentication method for chip cards according to claim 1, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: The transmission delay of the collected fingerprint data is D_fingerprint; The processing latency of the acquired facial image is D_face; The upload latency for capturing iris textures is Di_iris; The main timing order is determined by comparing T_total = max(D_fingerprint, D_face, D_iris) with η, where η is the upper limit of the acceptable maximum response time, and T_total is the maximum tolerable latency of the system, ensuring that there will be no timeout.

6. The multimodal biometric fusion authentication method for chip cards according to claim 5, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Calculate the actual contribution of each mode f_i = (1-ρ_i) / σ_i, where ρ_i is the matching failure rate of that mode and σ_i is the feature difference variance; The order of matching is determined by the size of f_i; Introduce a moving average mechanism: α_t = α_{t-1} × ω + f_i × (1 ω), where ω is the smoothing coefficient; If α_t < ε, then the system enters an inefficient state, and the subsequent process is adjusted to frequency reduction. Here, f_i reflects the reliability and contribution of a single mode in the system, and ε is the inefficient state trigger threshold.

7. The multimodal biometric fusion authentication method for chip cards according to claim 1, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Collect the success rate S_n of the most recent N communications; Calculate the encryption latency volatility Var(E); The communication stability score is determined using the formula C_score = λ × S_n + (1-λ) × (1 / Var(E)). If C_score < δ, enter conservative mode and select the most stable modality for priority matching, where λ is the score allocation coefficient and δ is the stability criterion.

8. The multimodal biometric fusion authentication method for chip cards according to claim 7, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Record the time T_match consumed for feature matching in each frame; Calculate the remaining available time of the system, R_time; Definition condition: If T_match ≥ R_time × ζ, then asynchronous merging is used; If T_match < R_time × ζ, then a synchronous merging method is used to ensure a complete verification process, where ζ is a reserved ratio coefficient, representing the allowed remaining time and space.

9. The multimodal biometric fusion authentication method for chip cards according to claim 1, characterized in that, The method of dynamically adjusting the data fusion timing of each modality feature through real-time feedback of monitoring the communication latency of the chip card and the encryption overhead of biometric features further includes: Set the modal response window W[i] to represent the expected range of response time for the corresponding mode; Calculate the timeout tendency index I[i] = (D[i]-W[i]) / W[i] for each mode, where D[i] represents the actual acquired response time of that mode; Determine whether to enable the alternative path strategy based on I[i]. If I[i] > φ, then skip this mode and proceed directly to the next mode, where φ is the timeout warning coefficient.

10. A multimodal biometric fusion identity authentication system for chip cards, characterized in that, include: The acquisition module collects multimodal biometric data of users in real time, including fingerprints, facial images, and iris textures. The encryption module generates an encryption feature template from the multimodal biometric data and transmits it in groups to the chip card; The adjustment module dynamically adjusts the data fusion timing of each modality feature by monitoring the real-time feedback of the chip card's communication latency and biometric encryption overhead. The output module performs multimodal feature matching based on the adjusted time sequence and outputs the identity authentication result.

Images