A Deep Learning-Based Method for Accelerating Gate Identity Recognition

Abstract

This invention discloses a deep learning-based method for accelerating gate identity recognition. To address the problem of high end-to-end latency caused by frame merging waiting and queuing congestion due to asynchronous arrival of visible light, infrared, and depth modes, this invention uses end-to-end queue simulation training and joint loss to jointly optimize joint arrival-quality distribution prediction, single-modal feature extraction, delay-aware dual-channel attention fusion, differentiable early stopping, and similarity retrieval. Based on adaptive decision-making based on waiting cost and target latency quantile, this invention achieves the technical effect of significantly reducing average latency and tail latency, and improving early retrieval rate and throughput while ensuring the accuracy of identity recognition and the reliability of liveness detection.

Description

Technical Field

[0001] This invention relates to the field of intelligent security access control technology, and in particular to a deep learning-based method for accelerating gate identity recognition. Background Technology

[0002] In smart security entrances and exits scenarios such as subways, industrial parks, and office buildings, deep learning-based gate identification and liveness detection systems have been widely deployed.

[0003] To improve robustness under complex lighting and occlusion conditions, the industry generally adopts multimodal acquisition such as visible light, infrared or near-infrared and depth, and completes the database similarity retrieval and access determination through detection, tracking, face alignment, single-modal feature extraction and multimodal fusion.

[0004] Existing solutions mostly employ fixed-time-window frame merging or synchronization strategies that wait for all modalities to arrive. Fusion methods mainly focus on early fusion, late fusion, or attention-weighted methods, with training objectives emphasizing recognition accuracy and liveness reliability.

[0005] However, existing technologies still have shortcomings:

[0006] 1. There is a lack of explicit modeling for multimodal asynchronous arrival and timestamp jitter. Fixed frame merging or full-modal synchronization brings additional waiting time. Under congestion conditions, the end-to-end average latency and tail latency are too high and it is difficult to adaptively adjust according to equipment status and on-site queuing load.

[0007] 2. During the integration phase, the focus is generally only on content similarity and quality score, lacking cost awareness of latency. It is difficult to incorporate arrival probability, waiting budget and quality uncertainty into the weight generation. Early decision strategies rely more on experience thresholds, and the ability to balance accuracy and latency is insufficient.

[0008] 3. The training link is mostly optimized in modules, without incorporating time costs into the end-to-end joint loss. Early stopping or early retrieval is usually non-differentiable, and there is a lack of queue process modeling and simulation calibration, resulting in a mismatch between parameters and the on-site throughput target, which limits the passage efficiency under high load.

[0009] Therefore, a method for accelerating gate identity recognition that can overcome the shortcomings of the existing technology is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0010] One objective of this invention is to propose a deep learning-based method for accelerating gate identification. Addressing the problems of frame merging waiting caused by asynchronous arrivals of multiple modalities (visible light, infrared, and depth), high end-to-end latency due to queuing congestion, and a lack of time cost awareness in fusion and early decision-making, this invention proposes a joint optimization scheme that combines arrival and quality distribution prediction, quality uncertainty assessment, waiting cost calculation, delay-aware dual-channel attention fusion, and differentiable early stopping, along with end-to-end queue simulation and joint loss. This invention achieves the technical effect of maintaining identification accuracy and liveness reliability under target latency quantile constraints while significantly reducing average latency and tail latency, improving early retrieval rate and throughput, and adapting to device status and on-site load.

[0011] A deep learning-based method for accelerating gate identity recognition according to an embodiment of the present invention is characterized by comprising:

[0012] S1. Based on historical visible light frames, infrared frames, depth frames, timestamp sequences, historical device status, and historical queuing load, jointly optimize the joint arrival quality distribution prediction network, single-modal feature extraction network, waiting cost calculation unit, delay-aware dual-channel attention fusion network, differentiable early stop module, and similarity retrieval module, and output the parameters of each module, and set the time window length, passage decision threshold, and target delay quantile.

[0013] S2. Maintain time series cache based on time window length, collect and update visible light frames, infrared frames and depth frames and record timestamps, synchronously obtain device status and calculate queue depth and waiting budget based on on-site queuing situation;

[0014] S3. Based on the parameters of the joint arrival quality distribution prediction network, generate arrival time distribution parameters, quality scores and quality uncertainties using the timestamp sequence, device status and three-modal frames mentioned above, respectively.

[0015] S4. Based on the parameters of the single-modal feature extraction network, extract visible light single-modal identity features, infrared single-modal identity features and depth single-modal identity features from the three-modal frames respectively, and simultaneously generate corresponding visible light single-modal liveness cues, infrared single-modal liveness cues and depth single-modal liveness cues.

[0016] S5. Based on the parameters of the waiting cost calculation unit, combined with the arrival time distribution parameters, quality uncertainty, queue depth, waiting budget and target delay quantile, generate the waiting cost parameters;

[0017] S6. Based on the parameters of the delayed-aware dual-channel attention fusion network, the content channel weights are generated in the content channel based on the feature similarity between each single-modal identity feature and the corresponding liveness cue. The cost channel weights are generated in the cost channel based on the waiting cost parameter and the arrival time distribution parameter. The gating coefficients are generated based on the network parameters and the waiting cost parameter. The weights of the two channels are weighted and synthesized to obtain the fusion weight. Based on this weighting, the fusion identity feature and the fusion liveness score are obtained.

[0018] S7. Based on the parameters of the differentiable early stop module, combined with the fused identity features, fused liveness score, waiting cost parameters and target delay quantile, generate an early stop determination result under the time cost constraint;

[0019] S8. When searching for the early stop judgment result, calculate the identity similarity score of the base database based on the similarity retrieval module parameters and the fused identity features. Generate the passage decision result based on the passage decision threshold and the fused liveness score. Generate the gate opening control command when the release conditions are met.

[0020] Optionally, step S1 specifically includes:

[0021] Based on historical visible light frames, historical infrared frames, historical depth frames and their timestamp sequences, combined with historical device status and historical queuing load, an end-to-end queue simulation training process is established, which includes arrival process modeling, service process modeling and waiting budget update. The simulation training process characterizes the impact of multimodal asynchronous arrival and critical path computation overhead on end-to-end latency.

[0022] According to the end-to-end queue simulation training process, a joint loss of time cost term and recognition accuracy term is set. The time cost term is used to measure the deviation of end-to-end latency from the target latency quantile and to penalize the case of exceeding the waiting budget. The recognition accuracy term is used to simultaneously constrain identity similarity accuracy and liveness detection reliability, thereby forming a training objective for joint optimization.

[0023] Based on the joint loss, end-to-end joint optimization is performed on the joint arrival quality distribution prediction network, single-modality feature extraction network, delay-aware dual-channel attention fusion network, differentiable early stopping module, similarity retrieval module and waiting cost calculation unit, and training convergence is achieved through parameter gradient update;

[0024] After training convergence, the parameters of the joint arrival quality distribution prediction network, the single-modal feature extraction network, the delay-aware dual-channel attention fusion network, the differentiable early stop module, the similarity retrieval module, and the waiting cost calculation unit are output. The time window length, passage decision threshold, and target delay quantile are set according to the statistical results of historical queuing load.

[0025] Terminology definition:

[0026] The historical visible light frames are a set of historical image frames acquired and stored by a visible light camera during the training or parameter setting phase, used to statistically analyze arrival and quality distribution.

[0027] The historical infrared frames are a collection of historical image frames acquired and stored by infrared or near-infrared sensors during the training or parameter setting phase.

[0028] The historical depth frames are a collection of historical depth maps / point cloud frames acquired and stored by depth sensors (structured light, ToF, or binoculars, etc.) during the training or parameter setting phase.

[0029] The timestamp sequence is an ordered set of time records corresponding to each modal frame, used to describe the acquisition time and relative timing relationship. The time reference can be the device's local clock or the synchronous system clock.

[0030] The historical device status is a collection of historical data reflecting the operating status of the acquisition and computing devices, including at least processor utilization, memory usage, temperature, power consumption, sensor frame rate, exposure / gain settings, network and storage latency, etc.

[0031] The historical queuing load is a set of historical statistical data reflecting the gate passage queue situation, including at least queue length, arrival rate, service rate, parallelism, congestion level and its time distribution;

[0032] The joint arrival quality distribution prediction network is a deep learning model that predicts the arrival time distribution of multimodal frames based on timestamp sequences and device status, and evaluates frame quality and its uncertainty. Its output includes arrival time distribution parameters, quality score and quality uncertainty.

[0033] The single-modal feature extraction network is a deep learning model that extracts identity features and generates liveness cues for visible light, infrared and depth frames respectively. The identity features are feature vectors suitable for similarity retrieval, and the liveness cues are numerical representations used for fusion and judgment.

[0034] The waiting cost calculation unit is a calculation module that jointly measures the arrival probability, expected arrival time, quality reliability weight, congestion weight and delay deviation to generate waiting cost parameters. The implementation method can be a learnable neural network or a parameterized function.

[0035] The delayed-aware dual-channel attention fusion network is a network structure that generates fusion weights based on the content channel and the cost channel and outputs fused identity features and fused liveness scores. The content channel generates weights based on the similarity between identity features and liveness cues and the quality score, while the cost channel generates weights based on the waiting cost parameter and the arrival time distribution parameter and with reference to the quality uncertainty. The results of the two channels are combined through a gating coefficient.

[0036] The differentiable early stopping module is a decision module that outputs an early retrieval or continued waiting decision under time cost constraints, and its decision function can participate in end-to-end training through gradients. The implementation methods include continuous gating functions or expected risk minimization approximations.

[0037] The similarity retrieval module is a module that calculates identity similarity scores in the base database based on fused identity features and outputs retrieval results. The implementation methods include, but are not limited to, indexing and searching based on cosine distance, Euclidean distance, or metric learning.

[0038] The time window length is a time range parameter used to maintain the time series cache and perform arrival distribution prediction and fusion decision-making, and can be adaptively set according to historical queuing load statistics.

[0039] The passage decision threshold is a threshold parameter for determining passage based on identity similarity score and liveness score. It can be a single threshold or a combination of multiple thresholds, and supports static or adaptive setting.

[0040] The target delay quantile is a quantile index (such as P90, P95 or P99) used to measure the end-to-end delay target, and is used to construct the time cost term and the boundary for early decision-making.

[0041] The end-to-end queue simulation training process is an integrated simulation-training process that simulates multimodal arrival processes, service processes, and waiting budget updates during the training phase to evaluate end-to-end latency and optimize model parameters accordingly.

[0042] The arrival process modeling is a statistical or generative modeling of the arrival time of each modal frame, including the characterization of asynchronous arrival, timestamp offset and jitter;

[0043] The service process modeling is a modeling of the time consumption and parallelism of computational steps such as detection, feature extraction, fusion, and retrieval, used to reflect the computational overhead of the critical path;

[0044] The waiting budget is a constraint parameter for the upper limit of the acceptable waiting time for a single session, which is dynamically adjusted according to the queue status and the target latency quantile.

[0045] The waiting budget update is a rule or algorithm that dynamically adjusts the waiting budget based on the real-time or historical queue status and the target delay quantile.

[0046] The time cost term is a loss component in the joint loss used to measure the deviation of end-to-end delay from the target delay quantile and to penalize cases that exceed the waiting budget.

[0047] The recognition accuracy term is a loss component in the joint loss used to simultaneously constrain the identity similarity accuracy and the reliability of liveness determination, including but not limited to classification, metric learning, or calibration-related losses.

[0048] The joint loss is the overall training objective formed by weighted combination of the time cost term and the recognition accuracy term, which is used for end-to-end joint optimization.

[0049] The critical path computation overhead is the sum of the time consumed by the slowest computation path that determines the end-to-end latency, and includes at least detection, feature extraction, fusion, retrieval, and I / O latency.

[0050] The statistical results of the historical queuing load are the statistical and distribution estimates of indicators such as historical queue length, arrival rate, service rate, and tail delay quantile, which are used to set the time window length, passage decision threshold, and target delay quantile.

[0051] Optionally, step S2 specifically includes:

[0052] Maintain the time series cache based on the time window length, ensuring that the length of the time series cache does not exceed the time window length;

[0053] Visible light frames, infrared frames, and depth frames are acquired using a visible light camera, an infrared sensor, and a depth sensor, and corresponding timestamps are recorded during acquisition. The visible light timestamp sequence, infrared timestamp sequence, and depth timestamp sequence are updated in the time series buffer based on the timestamps.

[0054] Simultaneously, acquire equipment status and calculate queue depth and waiting budget based on on-site queuing conditions;

[0055] Output visible light frames, infrared frames, depth frames, visible light timestamp sequences, infrared timestamp sequences, depth timestamp sequences, device status, queue depth, and waiting budget.

[0056] Terminology definition:

[0057] The time series cache is a bounded data structure used during the inference phase to store each modal frame and its corresponding timestamp for a period not exceeding the length of the time window. It can be a circular buffer or a sliding window.

[0058] The visible light frame is an image frame acquired by a visible light camera during the inference phase for identity recognition and liveness detection;

[0059] The infrared frame is an image frame acquired by an infrared or near-infrared sensor during the inference phase for identity recognition and liveness determination.

[0060] The depth frame is a depth image or point cloud frame acquired by a depth sensor during the inference phase for identity recognition and liveness detection.

[0061] The timestamp is a collection time marker that corresponds one-to-one with each modal frame, used to describe the collection time and the relative temporal relationship;

[0062] The visible light timestamp sequence is an ordered set of visible light frame timestamps recorded in chronological order within the time series buffer;

[0063] The infrared timestamp sequence is an ordered set of infrared frame timestamps recorded in chronological order within the time sequence buffer;

[0064] The depth timestamp sequence is an ordered set of depth frame timestamps recorded in chronological order within the time series buffer;

[0065] The device status is a set of indicators reflecting the current operating status of the acquisition and computing equipment, used for arrival distribution correction and waiting budget calculation;

[0066] The on-site queuing situation is a set of real-time passage queue status information at the gate, used to estimate arrival rate, service rate and congestion level;

[0067] The queue depth is a measure of the length of the current waiting queue or the number of sessions, used to characterize the degree of congestion and participate in the waiting budget calculation.

[0068] Optionally, step S3 specifically includes:

[0069] Based on the parameters of the joint arrival quality distribution prediction network, time series modeling is performed on visible light timestamp sequences, infrared timestamp sequences, and depth timestamp sequences. The offset and jitter of timestamps are adaptively corrected in combination with the device status. Arrival time distribution parameters are generated within the time range defined by the time window length to characterize the arrival probability distribution and statistical characteristics of each mode within this time range.

[0070] Simultaneously, based on the parameters of the joint arrival quality distribution prediction network, the visible light frame, infrared frame, and depth frame are quality evaluated, a quality score is generated, and a quality uncertainty is generated to characterize the credibility of the quality score.

[0071] Terminology definition:

[0072] The time series modeling is a process or model for modeling the sequence dependencies of visible light timestamp sequences, infrared timestamp sequences and depth timestamp sequences, used to characterize asynchronous arrival patterns, periodicity and cross-modal correlations.

[0073] The timestamp offset is the systematic deviation of each modal timestamp relative to a unified time reference, used to characterize constant or slowly varying time alignment errors;

[0074] The timestamp jitter is the random fluctuation of the timestamp within a short time scale, which originates from the uncertainty of collection, transmission or processing;

[0075] The adaptive correction is an algorithm that dynamically estimates and corrects timestamp offset and jitter based on device status and historical statistics, so as to align the time base of each mode.

[0076] The time range defined by the time window length is a sliding window interval with reference to the current time or session, and its width is determined by the time window length.

[0077] The arrival time distribution parameters are a set of parameters used to characterize the probability distribution of arrival time for each mode within the time range, including at least arrival probability, expected arrival time, variance or scaling parameter, quantile parameter, and component weights when using a mixture model.

[0078] The arrival probability distribution is the probability density or discrete probability distribution of the arrival time of each mode within the time range, used to calculate arrival-related statistics;

[0079] The statistical characteristics are descriptive indicators derived from the arrival time distribution parameters, including at least the expected value, variance, and quantiles.

[0080] The quality assessment is a process or model for scoring the imaging quality and effectiveness of visible light frames, infrared frames, and depth frames in recognition and liveness tasks.

[0081] The quality score is a numerical score that reflects the quality level of a single frame under the current task, and is used to participate in fusion and weight adjustment;

[0082] The quality uncertainty is a measure of the reliability of the quality score, reflecting the variance or confidence interval of the score, and is used to weight the contribution of the quality score in subsequent calculations with confidence.

[0083] Optionally, step S4 specifically includes:

[0084] Based on the parameters of the single-modal feature extraction network, forward reasoning is performed on visible light frames to generate visible light single-modal identity features and visible light single-modal liveness cues;

[0085] Based on the parameters of the single-modal feature extraction network, forward reasoning is performed on the infrared frames to generate infrared single-modal identity features and infrared single-modal liveness cues.

[0086] Based on the parameters of the unimodal feature extraction network, forward inference is performed on the depth frame to generate deep unimodal identity features and deep unimodal liveness cues. The identity features are numerical representations suitable for similarity retrieval, and the liveness cues are numerical representations used for subsequent fusion.

[0087] Terminology definition:

[0088] The three-modal frames are sets of frames collected by visible light, infrared and depth sensors respectively within the same time window of the current recognition process, used to generate modal features and liveness cues;

[0089] The forward inference is a process of calculating input frames to output features and clues under fixed network parameters, without involving parameter training or updating;

[0090] The parameters of the single-modal feature extraction network are model weights and configuration parameters used for forward inference, which are used to determine the generation method of features and liveness cues;

[0091] The visible light single-modal identity feature is the identity feature output on the visible light frame by the single-modal feature extraction network;

[0092] The infrared single-modal identity feature is the identity feature output on the infrared frame by the single-modal feature extraction network;

[0093] The deep single-modal identity feature is the identity feature output by the single-modal feature extraction network on the depth frame;

[0094] The visible light single-modal liveness cues are numerical cues for liveness determination output by the single-modal feature extraction network on the visible light frame;

[0095] The infrared single-modal liveness cues are numerical cues for liveness determination output by the single-modal feature extraction network on the infrared frame.

[0096] The deep unimodal liveness cues are numerical cues for liveness determination output by the unimodal feature extraction network on the depth frame.

[0097] Optionally, step S5 specifically includes:

[0098] Based on the arrival time distribution parameters, the arrival probability of each mode is obtained within the waiting budget time range, and the expected arrival time is calculated. At the same time, the delay deviation is measured based on the target delay quantile.

[0099] A quality confidence weight is generated based on the quality uncertainty, which is used to adjust the contribution of the arrival probability of each mode to the waiting cost;

[0100] Congestion weights are generated based on queue depth to characterize the incremental impact of queuing load on waiting costs;

[0101] The waiting cost parameters are generated by jointly measuring the arrival probability, expected arrival time, delay deviation metric, quality reliability weight, and congestion weight using the parameters of the waiting cost calculation unit.

[0102] Terminology definition:

[0103] The waiting budget defines a time range that is an upper limit time interval determined by the waiting budget with reference to the current time or session, and is used to statistically analyze and measure the arrival events of each modality within this interval.

[0104] The arrival probability of each modality is the probability value of the arrival of the modality frame obtained by integrating or accumulating the arrival time distribution parameters within the time limit defined by the waiting budget.

[0105] The expected arrival time is the expected value or conditional expectation of the arrival time of this mode calculated based on the arrival time distribution parameters within the time limit defined by the waiting budget.

[0106] The delay deviation metric is a numerical indicator used to quantify the deviation of the current or predicted end-to-end delay from the target delay quantile, and may include a penalty component for exceeding the waiting budget.

[0107] The quality reliability weight is a weighting coefficient calculated based on the quality uncertainty, used to adjust the contribution of the arrival probability of each mode to the waiting cost. The greater the quality uncertainty, the smaller the weight.

[0108] The congestion weight is a weight coefficient calculated based on queue depth and related queuing indicators. It is used to characterize the incremental impact of congestion status on waiting costs. The higher the degree of congestion, the greater the weight.

[0109] The joint metric is a calculation process or function that maps arrival probability, expected arrival time, delay deviation metric, quality reliability weight and congestion weight into a cost representation through a waiting cost calculation unit, and is not limited to linear or specific form;

[0110] The waiting cost parameter is a cost representation generated by the joint metric for subsequent fusion and early decision-making, and includes at least a numerical parameter that assesses the time cost of continuing to wait and the relative cost trade-off between immediate retrieval.

[0111] Optionally, step S6 specifically includes:

[0112] Based on the parameters of the time-delayed perception dual-channel attention fusion network, content channel weights are generated in the content channel based on the feature similarity of visible light monomodal identity features, infrared monomodal identity features, deep monomodal identity features and their corresponding monomodal liveness cues, and the content channel weights are adjusted according to the quality score.

[0113] In the cost channel, a cost channel weight is generated based on the waiting cost parameter and the arrival time distribution parameter, and with reference to the quality uncertainty. This weight is used to characterize the impact of waiting cost and arrival probability on fusion.

[0114] Subsequently, gating coefficients are generated based on the parameters of the delay-aware dual-channel attention fusion network and the waiting cost parameter. The content channel weights and cost channel weights are then weighted and synthesized to form the fusion weights.

[0115] Based on the fusion weight, the identity features of each single modality and the liveness clues of each single modality are weighted and converged to generate fused identity features and fused liveness scores.

[0116] Terminology definition:

[0117] The content channel is a branch in the time-delayed perception dual-channel attention fusion network that generates attention weights based on the feature similarity between each single-modal identity feature and its corresponding single-modal liveness cue, combined with quality information, and is used to characterize the impact of content relevance on fusion.

[0118] The content channel weight is a set of modal weight coefficients output by the content channel, used to measure the content contribution of each modality in the fusion.

[0119] The feature similarity is a numerical indicator that measures the consistency between each modality's identity features and its corresponding liveness cues, and includes at least cosine similarity, inner product, or the output of a learnable metric function;

[0120] The quality adjustment is a mapping strategy that scales, recalibrates, or suppresses / amplifies the weights of content channels based on the quality score, and can be a linear, piecewise, or neural network mapping.

[0121] The cost channel is a branch in the delay-aware dual-channel attention fusion network that generates weights based on waiting cost parameters and arrival time distribution parameters and with reference to quality uncertainty. It is used to characterize the impact of time cost and arrival probability on fusion.

[0122] The cost channel weight is a set of modal weight coefficients output by the cost channel, used to measure the cost contribution of each modality in the fusion.

[0123] The gating coefficient is a control coefficient calculated based on network parameters and waiting cost parameters. It is used to adjust the combined ratio of content channel weight and cost channel weight. The value is preferably in the range of [0,1] and can be a scalar or a vector.

[0124] The weighted synthesis is a calculation process that combines the content channel weights and cost channel weights linearly or nonlinearly according to the gating coefficients to generate the fusion weights.

[0125] The fusion weight is a set of coefficients obtained by weighting the two-channel weights and using them to weight each single-modal identity feature and each single-modal liveness clue.

[0126] The weighted convergence is a computational process that sums the weights or aggregates the attention of each single-modal identity feature and each single-modal liveness clue based on the fusion weight to obtain the fusion output;

[0127] The fused identity feature is a unified identity feature representation for similarity retrieval generated by weighted aggregation;

[0128] The fused liveness score is a confidence value for whether the current object is a live object, generated by weighted aggregation, and is used for passage determination and early stop decision.

[0129] Optionally, step S7 specifically includes:

[0130] Based on the parameters of the differentiable early stop module, combined with the fused identity features and fused liveness score, the time cost of continuing to wait is measured, and the recognition benefit of immediate retrieval is measured.

[0131] The decision boundary for early decision-making is determined based on the waiting cost parameter and the target delay quantile.

[0132] Under the constraints of the above-mentioned measurement and judgment boundaries, the differentiable early stop module generates an early stop judgment result. When the early stop judgment result indicates an early decision, the fused identity features and fused liveness score are confirmed for use in step S8. When the early stop judgment result indicates to continue waiting, the current session is maintained and the data updated by step S2 in the next time window is waited for.

[0133] Terminology definition:

[0134] The time cost of continuing to wait is the expected time cost of choosing not to trigger the retrieval at the current moment and waiting for subsequent data. It is estimated based on the waiting cost parameter and includes a penalty for exceeding the target delay quantile or waiting budget.

[0135] The recognition benefit of immediate retrieval is the expected recognition and passage determination benefit that can be obtained by triggering the retrieval at the current moment. Based on the fusion of identity features and the fusion of liveness score estimation, it can reflect the comprehensive gain of identity matching accuracy and liveness passage probability.

[0136] The decision boundary is a decision threshold or decision function used to compare the recognition benefits and time costs. It is adaptively determined based on the waiting cost parameter and the target delay quantile, and can be a scalar threshold or a segmentation curve on the benefit-cost plane.

[0137] The time cost constraint is a restriction imposed on the feasible decision space when generating the early stop determination result, requiring that the expected end-to-end delay does not exceed the target delay quantile and / or does not exceed the waiting budget;

[0138] The early stop determination result is a binary or probabilistic decision result output by the differentiable early stop module, which is used to indicate whether to make an early decision or continue to wait.

[0139] The advance decision is the decision to directly enter the retrieval and passage determination process at the current moment without waiting for subsequent frames;

[0140] The decision to continue waiting is to not trigger a search at the current moment, maintain the processing state, and wait for data updates in the next time window.

[0141] The current session is a processing instance for a single inspected object from the time it is captured until the passage determination is completed. It is used to continuously associate various modal data and decision states within multiple time windows.

[0142] The immediate retrieval refers to the operation of directly performing similarity calculation on the base database using fused identity features after making an advance decision, which is used to trigger the subsequent passage determination process.

[0143] Optionally, step S8 specifically includes:

[0144] When the early stop judgment result indicates that the retrieval is triggered, the similarity of the gate database is calculated based on the parameters of the similarity retrieval module and the fused identity features to generate an identity similarity score;

[0145] The identity similarity score is determined based on the passage decision threshold and the fused liveness score, and the passage decision result is generated.

[0146] When the passage decision meets the release conditions, a gate opening control command is generated.

[0147] Terminology definition:

[0148] The gate database is a storage set of registered identity information and its feature representation used for access determination. It includes at least identity identifiers, feature vectors, and optional permission / blacklist / whitelist attributes for similarity retrieval and access control.

[0149] The similarity calculation is a computational process that measures each identity feature in the gate database based on the fused identity features to generate an identity similarity score. The measurement method is not limited to cosine, Euclidean or learnable measurement functions.

[0150] The identity similarity score is a numerical indicator that reflects the degree of similarity between the fused identity features and the target identity features in the gate database. It is used to participate in the passage determination, and the range of values ​​and the measurement normalization method are not limited.

[0151] The passage decision result is the passage status output after judging the identity similarity score based on the passage decision threshold and the fused liveness score. It includes at least the result form of allowing passage, denying passage, or requiring manual review.

[0152] The passage conditions are a comprehensive set of conditions that must be met to determine whether passage is allowed. These conditions include at least an identity similarity score that reaches or exceeds the passage decision threshold and a liveness score that meets the liveness requirement. They can also be combined with security constraints such as blacklists / permission policies.

[0153] The gate opening control command is a control signal sent to the gate controller to execute the gate opening action when the passage decision result meets the release conditions. It includes at least parameters such as gate opening trigger, duration or validity period.

[0154] The beneficial effects of this invention are:

[0155] 1. Significantly reduce end-to-end latency and tail latency: By jointly predicting arrival time and quality distribution, calculating waiting costs, and fusion of latency-aware dual-channel attention, and adaptively performing early retrieval under the constraint of differentiable early stopping, and making decisions based on queue depth and waiting budget, the system effectively reduces the time overhead caused by frame merging waiting and congestion while ensuring recognition accuracy and liveness reliability, thereby improving throughput.

[0156] 2. Improve the robustness of fusion and the reliability of recognition: The weights are adjusted by introducing quality scores and quality uncertainty. The content channel is based on the similarity between single-modal identity features and liveness cues, while the cost channel generates weights based on waiting costs and arrival probabilities. The results of the two channels are combined by gating coefficients, so that the fused identity features and fused liveness scores can still be stably output under conditions of modal quality fluctuations, modal missingness, or changes in device status, thereby reducing false recognition and missed recognition.

[0157] 3. End-to-end optimization with adaptive parameters and field load: Through end-to-end queue simulation training, the arrival process, service process and waiting budget are incorporated into the joint loss. The time cost term and the recognition accuracy term are jointly constrained. With the help of differentiable early stopping, the entire link can be trained, so that the parameters of each module match the field queuing load statistics, reducing the cost of manual threshold parameter tuning and improving the efficiency of system deployment and migration. Attached Figure Description

[0158] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0159] Figure 1 This is a flowchart of a deep learning-based gate identity recognition acceleration method proposed in this invention. Detailed Implementation

[0160] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0161] refer to Figure 1 A deep learning-based method for accelerating gate identity recognition, characterized by comprising:

[0162] S1. Based on historical visible light frames, infrared frames, depth frames, timestamp sequences, historical device status, and historical queuing load, jointly optimize the joint arrival quality distribution prediction network, single-modal feature extraction network, waiting cost calculation unit, delay-aware dual-channel attention fusion network, differentiable early stop module, and similarity retrieval module, and output the parameters of each module, and set the time window length, passage decision threshold, and target delay quantile.

[0163] S2. Maintain time series cache based on time window length, collect and update visible light frames, infrared frames and depth frames and record timestamps, synchronously obtain device status and calculate queue depth and waiting budget based on on-site queuing situation;

[0164] S3. Based on the parameters of the joint arrival quality distribution prediction network, generate arrival time distribution parameters, quality scores and quality uncertainties using the timestamp sequence, device status and three-modal frames mentioned above, respectively.

[0165] S4. Based on the parameters of the single-modal feature extraction network, extract visible light single-modal identity features, infrared single-modal identity features and depth single-modal identity features from the three-modal frames respectively, and simultaneously generate corresponding visible light single-modal liveness cues, infrared single-modal liveness cues and depth single-modal liveness cues.

[0166] S5. Based on the parameters of the waiting cost calculation unit, combined with the arrival time distribution parameters, quality uncertainty, queue depth, waiting budget and target delay quantile, generate the waiting cost parameters;

[0167] S6. Based on the parameters of the delayed-aware dual-channel attention fusion network, the content channel weights are generated in the content channel based on the feature similarity between each single-modal identity feature and the corresponding liveness cue. The cost channel weights are generated in the cost channel based on the waiting cost parameter and the arrival time distribution parameter. The gating coefficients are generated based on the network parameters and the waiting cost parameter. The weights of the two channels are weighted and synthesized to obtain the fusion weight. Based on this weighting, the fusion identity feature and the fusion liveness score are obtained.

[0168] S7. Based on the parameters of the differentiable early stop module, combined with the fused identity features, fused liveness score, waiting cost parameters and target delay quantile, generate an early stop determination result under the time cost constraint;

[0169] S8. When searching for the early stop judgment result, calculate the identity similarity score of the base database based on the similarity retrieval module parameters and the fused identity features. Generate the passage decision result based on the passage decision threshold and the fused liveness score. Generate the gate opening control command when the release conditions are met.

[0170] In this specific embodiment, S1 specifically refers to:

[0171] First, based on historical visible light frames, historical infrared frames, historical depth frames, and their timestamp sequences, and combined with historical device status and historical queuing load, an end-to-end queue simulation training process is constructed to explicitly represent multimodal asynchronous arrival, timestamp offset and jitter, critical path computational overhead, and dynamic constraints on waiting budget. At the session level, a differentiable simulation of the arrival and service processes is performed using a rolling window. For each training batch, the simulator generates an end-to-end delay estimate under a given target delay quantile constraint and drives joint loss optimization. The differentiable mapping from input to end-to-end delay is expressed as a function:

[0172] ;

[0173] in This represents the end-to-end latency estimate for a single session. This represents a differentiable simulation mapping that includes arrival process modeling, service process modeling, and waiting for budget updates. The parameters represent the joint arrival quality distribution prediction network. These represent the parameters of a single-modal feature extraction network. These represent the parameters of a delay-aware dual-channel attention fusion network. The parameters represent the differentiable early stop module. These represent the parameters of the similarity retrieval module. This indicates the parameters for the waiting cost calculation unit. This represents the training sample input consisting of a sequence of three-modal frames and their timestamps. This indicates the historical or simulated device status and queuing load characteristics. Indicates the length of the time window. This indicates the quantile level of the target delay quantile. Indicates the wait budget for the current session;

[0174] Based on this, a joint loss consisting of a time cost term and a recognition accuracy term is defined to drive end-to-end training, taking... For positive part operators, the joint loss is written as follows:

[0175] ;

[0176] in Indicates the scalar value of the joint loss. This represents the weighting coefficients of the time cost term and the recognition accuracy term. This represents the penalty weight for delays exceeding the target quantile. This indicates the penalty weight for exceeding the waiting budget. This represents the target latency quantile threshold determined by historical load statistics. This represents the loss term used to constrain the accuracy of identity similarity (e.g., a scalar of the instantiation result of metric learning or classification loss). The tradeoff coefficient representing the loss of life. This represents a loss term (e.g., a scalar of the instantiation result of binary classification or calibration loss) used to constrain the reliability of liveness determination.

[0177] During training, random mini-batch optimization is used, and all module parameters are updated synchronously via gradient backpropagation, which can be written as:

[0178] ;

[0179] in This represents the set of parameters to be optimized. Represents the learning rate scalar. This represents the gradient vector of the joint loss with respect to the set of parameters;

[0180] In terms of implementation details, arrival process modeling uses a joint arrival quality distribution prediction network to learn the arrival probability, expected arrival time, and scale parameters of each modality based on timestamp sequences and device states, and outputs quality scores and quality uncertainties. Service process modeling obtains the time distribution of detection, feature extraction, fusion, and retrieval through historical measurements and network computational estimation. Waiting budget updates are dynamically adjusted based on simulation queue depth and target quantiles to limit session-level waiting time. During training, perturbation sampling is performed under different loads and device states to enhance robustness, and the end-to-end delay quantiles and recognition metrics of the validation set are used as convergence criteria. After training convergence, the output is... and The time window length is set by combining historical queuing load statistics. Passage decision threshold Compared with the target delay quantile ,in This represents the threshold parameter for jointly determining identity similarity and liveness score, which is used for collaborative applications of online access decisions and early termination determination.

[0181] In this specific embodiment, S2 specifically refers to:

[0182] At the current moment, the system performs bounded maintenance and updates on the acquisition and timestamps of each modality with a preset time window length. Specifically, it maintains the time series cache to ensure that the cache length does not exceed the time window, and synchronously records the timestamps and generates the corresponding timestamp sequences when acquiring visible light, infrared and depth frames. At the same time, it obtains the device status and calculates the queue depth and waiting budget based on the on-site queuing situation for subsequent steps to call.

[0183] To ensure that the constraint "length does not exceed the time window" has an operable update criterion, the current time is defined as... The time window length is Time series cache is The visible light frame and its timestamp pair are ( Infrared frames and their timestamp pairs are Depth frames and their timestamps are ,in Indicates the first One visible light frame, Indicates the corresponding visible light timestamp, Indicates the first One infrared frame, Indicates the corresponding infrared timestamp, Indicates the first Each depth frame Indicates the corresponding depth timestamp, Indicates at time Time series caching Indicates the current system time, This indicates the length of the time window used to maintain the cache. The cache update rule is written as follows:

[0184] ;

[0185] Under this rule, the timestamp sequence of the three modalities is continuously updated in chronological order within the sliding window;

[0186] To measure on-site queuing congestion and allocate waiting time budgets accordingly, queue depth is defined as... The cumulative number of arrivals within the window is The cumulative number of services completed within the window is To determine the maximum value, the queue depth is calculated as follows:

[0187] ;

[0188] in Indicates at time queue depth, Indicates in The cumulative number of arriving sessions within the statistical interval at the right endpoint. Indicates in This represents the cumulative number of completed service sessions within the statistical interval at the right endpoint.

[0189] Further, a waiting budget is set based on the congestion status, and the waiting budget is defined as follows: The maximum acceptable waiting limit is The baseline waiting time is The congestion adjustment coefficient is For the minimum value operator, wait for the budget to be written:

[0190] ;

[0191] in Indicates at time Acceptable waiting time for the current session This indicates the waiting limit set based on scenario safety and experience constraints. This indicates the baseline latency time set based on historical load and target latency strategy. This represents the adjustment coefficient used to reflect the impact of congestion levels on budget increments;

[0192] During operation, the system acquires visible light frames, infrared frames, and depth frames in real time and records their timestamps. Based on the aforementioned caching rules, it maintains the visible light timestamp sequence, infrared timestamp sequence, and depth timestamp sequence. It synchronously reads the device status vector to reflect operational status such as processor utilization, memory usage, and sensor frame rate, and outputs data including the latest visible light frame, latest infrared frame, latest depth frame, tri-modal timestamp sequence, device status, and queue depth. With waiting budget The results are used for subsequent arrival distribution prediction, cost calculation, and fusion decision-making.

[0193] In this specific embodiment, S3 specifically refers to:

[0194] The system performs time-series modeling and adaptive correction of timestamp sequences and device states based on a trained joint arrival quality distribution prediction network, and performs quality assessment of three-modal frames in parallel to generate quality scores and quality uncertainties: First, time base alignment and jitter correction are performed, and then the three-modal frames are aligned and corrected. The original timestamp of the frame is adaptively mapped to a corrected timestamp by the device state, written as:

[0195] ;

[0196] in Representing modes The The correction timestamp scalar for each frame, Representing modes The The original timestamp scalar of each frame, Represents the device state vector Adaptive estimation of modes The timestamp systematic offset scalar, The device state vector at the current moment, Representing modes The scalar of timestamp jitter estimation for each frame Representing visible light / infrared / depth modes in order. Indicates the frame order index within the current window;

[0197] The arrival time distribution is parametrically modeled within a time window of limited length, let the current time be... The time window length is Then the mode In the interval The arrival probability can be written as:

[0198] ;

[0199] in Representing modes Arrival probability scalar within the window Indicates For expectations, The normal distribution of variance over time variable probability density function on Representing modes Expected arrival time parameter scalar Representing modes Arrival time scale parameter scalar, Time scalar representing the integral variable, Represents the scalar value at the current time. This represents a scalar value indicating the length of the time window, which forms the set of arrival time distribution parameters used for subsequent steps. ;

[0200] Simultaneously, a quality evaluation header is executed on each frame within the window to output a quality score and uncertainty, denoted as:

[0201] ;

[0202] in Representing modes The The quality score scalar of each frame in the current task. This represents the scalar of quality uncertainty corresponding to the quality score. Indicated by network parameters Modes of weights Quality assessment mapping function The parameter set of the joint arrival quality distribution prediction network. Representing modes The One input frame tensor;

[0203] The above outputs are filtered and aggregated within the window to form arrival time distribution parameters, frame-by-frame quality scores, and quality uncertainties for the three modes, which are then sent to the subsequent cost calculation and fusion decision module.

[0204] In this specific embodiment, S4 specifically refers to:

[0205] The system employs a single-modal feature extraction network for forward reasoning on visible light, infrared, and depth frames respectively to obtain identity feature vectors suitable for similarity retrieval and liveness cues for subsequent fusion determination; to illustrate the core computational relationship, the generation of identity features is denoted as... And record the generation of living cues as ;

[0206] in Representing modes Within the window The identity feature vector of the frame (dimension: (column vectors) Representing modes Identity feature forward mapping function Representing modes The Input frame tensor, Representing modes The parameter set of the single-modal feature extraction network Representing modes In the Frame liveness cue scalar, Representing modes Live cue forward mapping function, Representing visible light, infrared and depth modes respectively. Indicates the frame sequence index within the current time window, The dimension representing the identity feature vector;

[0207] In the actual implementation, in order to balance real-time performance and robustness, A lightweight convolutional backbone and attention head are used to extract multi-scale identity features to adapt to asynchronous arrival and modal quality fluctuations. The output is sigmoid-calibrated to form a liveness confidence score with interval constraints, which can be confidently integrated with the quality score. All three networks perform forward inference with fixed parameters and do not involve training or updates. The output is... The data is fed into the subsequent delayed-perception dual-channel attention fusion and early stop determination module in the form of data. This is used to measure the consistency between identity features and liveness cues in the content channel and to generate fusion weights by combining waiting costs and arrival distribution in the cost channel.

[0208] In this specific embodiment, S5 specifically includes:

[0209] Within the budgeted waiting timeframe, the system jointly measures arrival probability, expected arrival time, delay deviation metric, quality reliability weight, and congestion weight to generate waiting cost parameters, which serve as the cost signal for subsequent fusion and early stopping. First, within the current time window, the system calculates the arrival probability and conditionally expected arrival time of each modality within the budgeted timeframe to characterize "whether it will arrive" and "how long to wait." The modality set is denoted as... These refer to the visible light, infrared, and depth modes, respectively, and the current system time is... Waiting for the budget to be The probability density function of the arrival time is: ,in For time integration variables, For modality Expected arrival time parameter For modality Given the arrival time scale parameter, the arrival probability and conditionally expected arrival time within the budget window are denoted as follows:

[0210] and ;

[0211] in Representing modes In the interval Arrival probability scalar within, Representing modes In the interval The expected arrival time scalar under the given conditions;

[0212] Subsequently, a quality reliability weight is generated based on the quality uncertainty to suppress the contribution of high-uncertainty quality to the waiting cost, allowing the in-window mode to... The number of frames is , No. Frame quality uncertainty is The aggregated uncertainty value is:

[0213] ;

[0214] And obtained by exponential confidence weighting ,in For modality The mean scalar of window uncertainty For modality Quality reliability weight scalar This is the adjustment coefficient for the quality uncertainty;

[0215] To characterize the incremental impact of on-site congestion on waiting costs and the target latency constraint, the queue depth is defined as... The congestion adjustment coefficient is The congestion weight is And the time delay deviation is measured using the positive part operator:

[0216] ;

[0217] in For a moment Congestion weight scalar, For a moment queue depth scalar, A scalar measure of the time delay deviation relative to the target time delay quantile. For end-to-end delay prediction scalars based on simulation or historical estimation, For the target delay quantile threshold scalar, For finding the maximum value operator;

[0218] Based on this, the waiting cost calculation unit jointly measures the arrival probability, conditionally expected arrival time, quality reliability weight, congestion weight, and delay deviation, and outputs the waiting cost value for each mode, written as:

[0219] ;

[0220] in For modality The waiting cost scalar The weighting coefficients for the probability of arrival, The weighting coefficients for the expected arrival time term, The weighting coefficients for the time delay deviation term, and the three factors together with... and Together they form the parameter set of the waiting cost calculation unit;

[0221] Finally, the waiting costs of the three modes are combined into a waiting cost parameter vector. This is for use by the delay-aware dual-channel attention fusion and differentiable early stop module.

[0222] In this specific embodiment, S6 specifically refers to:

[0223] Driven by a time-delay-aware dual-channel attention fusion network, the system generates weights for trimodal identity features and liveness cues in the content channel and cost channel, respectively, and then performs weighted synthesis using gating coefficients to output fused identity features and fused liveness scores.

[0224] First, calculate the content consistency score in the content channel and obtain a normalized weight after quality adjustment, written as:

[0225] ;

[0226] in Representing modes Content channel weight scalar, The Softmax normalization operator is represented on the modality set. Indicates the scaling factor of the content channel. Representing modes Content consistency score scalar Indicates the fusion of network parameters For the learnable metric function of weights, Representing modes The identity feature column vector (obtained by time-series aggregation within a time window) Representing modes The live cue scalar (obtained by temporal convergence within a time window). Representing modes Quality rating scale The dimension of the identity feature vector Representing visible light, infrared and depth modes respectively. This represents the set of parameters for a delay-aware dual-channel attention fusion network.

[0227] Subsequently, a normalized weight is formed by integrating waiting costs, arrival probability, and quality uncertainty in the cost channel, written as:

[0228] ;

[0229] in Representing modes Cost channel weight scalar The scaling factor representing the waiting cost item. Representing modes The waiting cost scalar (obtained from the waiting cost parameter in step S5), The scaling factor representing the arrival probability term, Representing modes The arrival probability scalar in the current time range (obtained from the arrival distribution parameters in step S3). The scaling factor representing the mass uncertainty term. Representing modes The mass uncertainty scalar;

[0230] To perform adaptive synthesis between the two channels, the gating coefficient is defined as follows:

[0231] ;

[0232] in Indicates time scalar of gating coefficients Represents the Sigmoid function, The scaling factor representing the gating factor, The scalar representing the mean of the current session's trimodal wait cost. This represents a scalar representing the cost reference threshold for gating. Represent the wait cost scalars for visible light, infrared, and depth modes, respectively. Represents the current system time scalar;

[0233] The fusion weights are generated under the influence of the gating coefficients, written as:

[0234] ;

[0235] in Representing modes The fusion weight scalar;

[0236] Finally, the trimodal identity features and liveness clues are weighted and converged using fusion weights to obtain the fused identity features and fused liveness score, written as follows:

[0237] and ;

[0238] in Represents a column vector of fused identity features. The above Softmax is defined as a scalar representing the fusion of liveness fractions. And all scaling factors Compared with reference threshold All are included in the set of learnable or configurable parameters of the fusion network. middle.

[0239] In this specific embodiment, S7 specifically refers to:

[0240] The system utilizes a differentiable early stop module to compare the expected risk and benefit of two candidate actions, "continue waiting" and "immediate retrieval," and outputs a probabilistic decision of early / continue. The module reads the fused identity features and fused liveness score obtained in step S6, as well as the waiting cost parameters generated in step S5, and combines them with the target latency quantile and waiting budget to form a time constraint. First, the time cost of continuing to wait is defined as:

[0241] ;

[0242] in Indicates at time The expected time cost scalar of choosing to continue waiting This represents the weighting coefficient for the average waiting cost. The scalar representing the mean of the three-modal waiting cost. Represent the waiting cost scalars for visible light, infrared, and depth modes, respectively, and form a waiting cost parameter vector. Represents the transpose operator, Indicates the weighting coefficient for time delay deviation, A scalar representing the time delay deviation relative to the target time delay quantile threshold. This represents the end-to-end delay prediction scalar obtained based on simulation or historical estimation. Represents the target delay quantile threshold scalar, This indicates the weighting coefficient for the over-budget item. Represents the positive part operator, Indicates time Waiting budget scalar for the current session Represents the current system time scalar;

[0243] The recognition benefit of immediate retrieval is then defined as:

[0244] ;

[0245] in Indicates at time Expected recognition revenue scalar that triggers the retrieval The weighting coefficients representing the benefits of identity similarity This indicates the revenue forecast header of the differentiable early stopping module with parameters. The predicted similarity scalar obtained by mapping fused identity features A learnable mapping function representing revenue prediction. Represents a column vector of fused identity features. Represents the dimension of the identity feature vector. Weighting coefficients representing the gains from liveness credibility. Represents the fractional scalar of fusion living individuals;

[0246] To form differentiable advance decisions under time constraints, a sigmoid-gated method is used to generate advance probabilities, written as:

[0247] ;

[0248] in Indicates time Scalar of advance search probability Represents the Sigmoid function, The scalar representing the independent variable of a function Indicates the scaling factor of the gating, This represents the decision boundary scalar used to reflect the target delay quantile and the waiting budget constraint. and These represent the weighting coefficients for the quantile constraint and the budget constraint, respectively.

[0249] In online inference, the decision based on probability and threshold can be written as:

[0250] ;

[0251] in The binary result representing the early stopping determination. This indicates that the characteristic function takes the value 1 if the condition is true, and 0 otherwise. Represents the advance threshold scalar used for probability determination, when Confirming the integration of identity features With fusion of living fractions For subsequent retrieval and access determination calls The current session is maintained and the data is awaited in the next time window to reassess the costs and benefits.

[0252] In this specific embodiment, S8 specifically refers to:

[0253] When the differential early stop module's judgment result indicates that a retrieval has been triggered, the system calls the similarity retrieval module to calculate the similarity of the gate's database and, combined with the passage decision threshold and the fused liveness score, provides passage judgment and gate opening control instructions. First, the fused identity features and database storage features are normalized to obtain a numerically stable similarity score, written as:

[0254] and ;

[0255] in Represents the normalized fused identity feature column vector. This represents the fused identity feature column vector obtained from step S6. The second norm of a vector Indicates the bottom of the library Normalized feature column vectors of records Indicates the bottom of the library The original feature column vector of each record Indicates the index number of the base database record and This indicates the total number of records in the base database and that all feature vectors have the same dimension. ;

[0256] Subsequently, cosine similarity was used for retrieval calculation and candidate selection, written as follows:

[0257] and And record ;

[0258] in Indicates the fusion feature and the first Similarity scalar of base database features Represents the vector transpose operator, The index number of the entry with the highest similarity. Represents the independent variable that maximizes the objective function. Represents the Top-1 similarity score scalar;

[0259] To jointly determine the identity matching result with the liveness confidence level and access policy, let the identity threshold for the access decision be . The liveness threshold is The fusion of living fractions is The strategy indicator is The common judgment is written as follows:

[0260] ;

[0261] in A binary result indicating whether passage is permitted. This indicates that the characteristic function takes the value 1 if the condition within the parentheses is true, and 1 otherwise. This represents the threshold for determining the pass rate based on identity similarity scores. This indicates the threshold for determining the number of live individuals. This represents the fusion liveness fraction obtained from step S6. The policy permission indicator quantity bound to the candidate identity is determined by the permission configuration and the blacklist / whitelist.

[0262] To ensure consistency with the gate controller interface, the judgment result is mapped to a control parameter set, and the current system time is defined as... The duration of the gate opening is as follows: Then the control instructions are written as follows:

[0263] ;

[0264] in Indicates at time The control parameter set output to the gate controller The time controller performs gate opening and holds. Time The controller maintains the gate closed state and can trigger prompts and log recording processes.

[0265] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

[0266] This invention transforms the trade-off between "whether to continue waiting for missing modalities" and "the recognition benefits of immediate retrieval" into a measurable and trainable trade-off process through the synergistic effects of joint arrival-quality distribution prediction, waiting cost calculation, delay-aware dual-channel attention fusion, and differentiable early stopping. On the one hand, it models and corrects the arrival time distribution based on timestamp sequences and device status, and outputs quality scores and quality uncertainties, enabling the system to predict the arrival probability and expected arrival time of each modality within a given time window. On the other hand, it combines queue depth and waiting budget to form a waiting cost parameter, which is used as a cost channel signal and participates in the fusion weight generation along with the content channel (identity features / liveness cue similarity, quality score). Simultaneously, the differentiable early stopping module provides an early retrieval decision constrained by the target delay quantile. The above link is trained through end-to-end queue simulation and joint loss with time cost terms, aligning the model parameters with the on-site throughput target. This mechanism reduces the end-to-end average and tail latency caused by frame merging waiting and congestion amplification, and improves the early retrieval rate and throughput while maintaining the accuracy of identity recognition and the reliability of liveness determination.

[0267] In terms of algorithm structure, this invention makes targeted improvements to address the specific technical problem of "asynchronous arrival + congestion":

[0268] First, by introducing explicit arrival time distribution and quality uncertainty prediction, and combining it with dynamic waiting budget, a cost representation that can be perceived as "when to wait, how long to wait, and whether it is worthwhile" is formed.

[0269] Secondly, attention fusion is adopted in parallel with content channel and cost channel, and the weights of the two channels are adaptively synthesized with gating coefficients to avoid the failure of traditional fusion based solely on content similarity under high load or modality loss.

[0270] Third, the design allows for differential early stopping, transforming early decision-making from an experience threshold into a learnable time-precision tradeoff, enabling the decision boundary to adaptively shift with the target delay quantile and the field load.

[0271] Fourth, construct a queue simulation that includes arrival / service processes and critical path overhead, and incorporate it into joint loss training to achieve end-to-end collaborative optimization of multiple modules.

[0272] These improvements collectively ensure that the system can still stably output fused identity features and liveness scores even when there are fluctuations in device status, variations in modal quality, or partial modal loss, and achieve the predetermined passage target with lower average and tail latency.

Claims

1. A deep learning-based method for accelerating gate identity recognition, characterized in that, include: S1. Based on historical visible light frames, infrared frames, depth frames, timestamp sequences, historical device status, and historical queuing load, jointly optimize the joint arrival quality distribution prediction network, single-modal feature extraction network, waiting cost calculation unit, delay-aware dual-channel attention fusion network, differentiable early stop module, and similarity retrieval module, and output the parameters of each module, and set the time window length, passage decision threshold, and target delay quantile. S2. Maintain time series cache based on time window length, collect and update visible light frames, infrared frames and depth frames and record timestamps, synchronously obtain device status and calculate queue depth and waiting budget based on on-site queuing situation; S3. Based on the parameters of the joint arrival quality distribution prediction network, generate arrival time distribution parameters, quality scores and quality uncertainties using the timestamp sequence, device status and three-modal frames mentioned above, respectively. S4. Based on the parameters of the single-modal feature extraction network, extract visible light single-modal identity features, infrared single-modal identity features and depth single-modal identity features from the three-modal frames respectively, and simultaneously generate corresponding visible light single-modal liveness cues, infrared single-modal liveness cues and depth single-modal liveness cues. S5. Based on the parameters of the waiting cost calculation unit, combined with the arrival time distribution parameters, quality uncertainty, queue depth, waiting budget and target delay quantile, generate the waiting cost parameters; S6. Based on the parameters of the delayed-aware dual-channel attention fusion network, the content channel weights are generated in the content channel based on the feature similarity between each single-modal identity feature and the corresponding liveness cue. The cost channel weights are generated in the cost channel based on the waiting cost parameter and the arrival time distribution parameter. The gating coefficients are generated based on the network parameters and the waiting cost parameter. The weights of the two channels are weighted and synthesized to obtain the fusion weight. Based on this weighting, the fusion identity feature and the fusion liveness score are obtained. S7. Based on the parameters of the differentiable early stop module, combined with the fused identity features, fused liveness score, waiting cost parameters and target delay quantile, generate an early stop determination result under the time cost constraint; S8. When searching for the early stop judgment result, calculate the identity similarity score of the base database based on the similarity retrieval module parameters and the fused identity features. Generate the passage decision result based on the passage decision threshold and the fused liveness score. Generate the gate opening control command when the release conditions are met. Specifically, S1 is: Based on historical visible light frames, historical infrared frames, historical depth frames and their timestamp sequences, combined with historical device status and historical queuing load, an end-to-end queue simulation training process is established, which includes arrival process modeling, service process modeling and waiting budget update. The simulation training process characterizes the impact of multimodal asynchronous arrival and critical path computation overhead on end-to-end latency. According to the end-to-end queue simulation training process, a joint loss of time cost term and recognition accuracy term is set. The time cost term is used to measure the deviation of end-to-end latency from the target latency quantile and to penalize the case of exceeding the waiting budget. The recognition accuracy term is used to simultaneously constrain identity similarity accuracy and liveness detection reliability, thereby forming a training objective for joint optimization. Based on the joint loss, end-to-end joint optimization is performed on the joint arrival quality distribution prediction network, single-modality feature extraction network, delay-aware dual-channel attention fusion network, differentiable early stopping module, similarity retrieval module and waiting cost calculation unit, and training convergence is achieved through parameter gradient update; After training convergence, the parameters of the joint arrival quality distribution prediction network, the single-modal feature extraction network, the delay-aware dual-channel attention fusion network, the differentiable early stop module, the similarity retrieval module, and the waiting cost calculation unit are output. The time window length, passage decision threshold, and target delay quantile are set according to the statistical results of historical queuing load.

2. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S2 specifically refers to: Maintain the time series cache based on the time window length, ensuring that the length of the time series cache does not exceed the time window length; Visible light frames, infrared frames, and depth frames are acquired using a visible light camera, an infrared sensor, and a depth sensor, and corresponding timestamps are recorded during acquisition. The visible light timestamp sequence, infrared timestamp sequence, and depth timestamp sequence are updated in the time series buffer based on the timestamps. Simultaneously, acquire equipment status and calculate queue depth and waiting budget based on on-site queuing conditions; Output visible light frames, infrared frames, depth frames, visible light timestamp sequences, infrared timestamp sequences, depth timestamp sequences, device status, queue depth, and waiting budget.

3. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S3 specifically refers to: Based on the parameters of the joint arrival quality distribution prediction network, time series modeling is performed on visible light timestamp sequences, infrared timestamp sequences, and depth timestamp sequences. The offset and jitter of timestamps are adaptively corrected in combination with the device status. Arrival time distribution parameters are generated within the time range defined by the time window length to characterize the arrival probability distribution and statistical characteristics of each mode within this time range. Simultaneously, based on the parameters of the joint arrival quality distribution prediction network, the visible light frame, infrared frame, and depth frame are quality evaluated, a quality score is generated, and a quality uncertainty is generated to characterize the credibility of the quality score.

4. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S4 specifically refers to: Based on the parameters of the single-modal feature extraction network, forward reasoning is performed on visible light frames to generate visible light single-modal identity features and visible light single-modal liveness cues; Based on the parameters of the single-modal feature extraction network, forward reasoning is performed on the infrared frames to generate infrared single-modal identity features and infrared single-modal liveness cues. Based on the parameters of the unimodal feature extraction network, forward inference is performed on the depth frame to generate deep unimodal identity features and deep unimodal liveness cues. The identity features are numerical representations suitable for similarity retrieval, and the liveness cues are numerical representations used for subsequent fusion.

5. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S5 specifically refers to: Based on the arrival time distribution parameters, the arrival probability of each mode is obtained within the waiting budget time range, and the expected arrival time is calculated. At the same time, the delay deviation is measured based on the target delay quantile. A quality confidence weight is generated based on the quality uncertainty, which is used to adjust the contribution of the arrival probability of each mode to the waiting cost; Congestion weights are generated based on queue depth to characterize the incremental impact of queuing load on waiting costs; The waiting cost parameters are generated by jointly measuring the arrival probability, expected arrival time, delay deviation metric, quality reliability weight, and congestion weight using the parameters of the waiting cost calculation unit.

6. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S6 specifically refers to: Based on the parameters of the time-delayed perception dual-channel attention fusion network, content channel weights are generated in the content channel based on the feature similarity of visible light monomodal identity features, infrared monomodal identity features, deep monomodal identity features and their corresponding monomodal liveness cues, and the content channel weights are adjusted according to the quality score. In the cost channel, a cost channel weight is generated based on the waiting cost parameter and the arrival time distribution parameter, and with reference to the quality uncertainty. This weight is used to characterize the impact of waiting cost and arrival probability on fusion. Subsequently, gating coefficients are generated based on the parameters of the delay-aware dual-channel attention fusion network and the waiting cost parameter. The content channel weights and cost channel weights are then weighted and synthesized to form the fusion weights. Based on the fusion weight, the identity features of each single modality and the liveness clues of each single modality are weighted and converged to generate fused identity features and fused liveness scores.

7. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S7 specifically refers to: Based on the parameters of the differentiable early stop module, combined with the fused identity features and fused liveness score, the time cost of continuing to wait is measured, and the recognition benefit of immediate retrieval is measured. The decision boundary for early decision-making is determined based on the waiting cost parameter and the target delay quantile. Under the constraints of the above-mentioned measurement and judgment boundaries, the differentiable early stop module generates an early stop judgment result. When the early stop judgment result indicates an early decision, the fused identity features and fused liveness score are confirmed for use in step S8. When the early stop judgment result indicates to continue waiting, the current session is maintained and the data updated by step S2 in the next time window is waited for.

8. The deep learning-based gate identification acceleration method according to claim 1, characterized in that, S8 specifically refers to: When the early stop judgment result indicates that the retrieval is triggered, the similarity of the gate database is calculated based on the parameters of the similarity retrieval module and the fused identity features to generate an identity similarity score; The identity similarity score is determined based on the passage decision threshold and the fused liveness score, and the passage decision result is generated. When the passage decision meets the release conditions, a gate opening control command is generated.

Images