Gait and facial feature fusion recognition method and system of deep learning architecture

Abstract

The application is suitable for the field of biometric identification technology, and provides a gait and facial feature fusion identification method and system of deep learning architecture, the method comprises the following steps: monitoring the card swiping event of the subway gate, analyzing the card ID information and generating a dynamic time window signal; collecting the plantar pressure distribution and facial video of the card swippers and processing to obtain the spatio-temporal alignment foot pressure time series data and the anti-occlusion facial image; reconstructing the continuous gait phase feature according to the foot pressure time series data, inversely restoring the three-dimensional facial biometric feature under the extrusion state by using the anti-occlusion facial image, and integrating to obtain the human biometric feature vector; performing confidence evaluation on the human biometric feature vector and performing dynamic weighted fusion to obtain the multi-modal feature, and then generating the final identity authentication result through the multi-round voting mechanism within the time window. The application realizes high-precision and low-misrecognition-rate fusion identification in a complex scene, and provides a stable and reliable solution for the intelligent management of the subway gate.

Description

Technical Field

[0001] This invention relates to the field of biometrics, specifically to a gait and facial feature fusion recognition method and system based on a deep learning architecture. Background Technology

[0002] Gait and facial feature fusion recognition is a method that combines two biometric technologies. This fusion leverages the fact that gait recognition can still work effectively at long distances and low resolutions, and the advantage of facial recognition in providing high-precision recognition at close range.

[0003] Current subway turnstile biometric systems have certain technical shortcomings. At the physical level, the inevitable gait pauses that occur when passengers swipe their cards disrupt the continuous gait sequence. This leads to a sharp increase in the feature extraction error rate of existing gait recognition models based on temporal convolutional networks, while conventional interpolation compensation methods distort key motion features. Simultaneously, the combined mechanical forces generated by crowds squeezing within turnstile channels during peak hours cause instantaneous distortions in facial geometry that exceed the processing capabilities of traditional deformation models, significantly reducing the practical value of facial recognition in real-world scenarios.

[0004] More importantly, mainstream fusion architectures typically assume that gait and facial features are continuously synchronized in time and space. However, in real-world turnstile scenarios, there is a significant time shift between the gait pause phase and the moment of maximum facial compression, which easily leads to intermodal conflicts. The average time from sensor acquisition to completing the recognition decision exceeds the acceptable passage time window for passengers, mainly due to the failure to establish a hard real-time correlation mechanism among card swiping behavior, changes in mechanical pressure, and biometric feature acquisition. Therefore, a gait and facial feature fusion recognition method and system based on a deep learning architecture is proposed to address the aforementioned problems. Summary of the Invention

[0005] In view of the shortcomings of the existing technology, the purpose of this invention is to provide a gait and facial feature fusion recognition method and system based on a deep learning architecture, so as to solve the problems existing in the above-mentioned background technology.

[0006] This invention is implemented as follows: a gait and facial feature fusion recognition method based on a deep learning architecture, the method comprising the following steps:

[0007] The system monitors card swiping events at subway turnstiles, parses card ID information, and generates dynamic time window signals. The card swiping events are obtained by reading the signals from the transportation card using a card reader.

[0008] When the dynamic time window signal is acquired, the foot pressure distribution and facial video of the card swiper are collected and processed to obtain spatiotemporally aligned foot pressure time series data and anti-occlusion facial image. The foot pressure distribution is collected by a high-density array of pressure sensors distributed in the gate passage, and the facial video is collected by a multi-view camera device.

[0009] Continuous gait phase features are reconstructed based on foot pressure time-series data. Three-dimensional facial biometrics under compression state are inversely restored using anti-occlusion facial images. The results are then integrated to obtain a standardized human biometric vector.

[0010] The confidence level of human biometric vectors is evaluated and dynamically weighted and fused to obtain multimodal features. Then, the final identity authentication result is generated through a multi-round voting mechanism within a time window, and the gate passage is controlled.

[0011] As a further aspect of the present invention: the step of monitoring card swiping events at subway turnstiles, parsing card ID information, and generating dynamic time window signals specifically includes:

[0012] The high-frequency carrier and modulation data in the card swiping signal are separated by a card reader, and the card ID information is obtained by decoding the modulation data using an envelope detector.

[0013] Based on card ID information combined with real-time queue density images, the optimal acquisition time is predicted through a convolutional neural network, and a dynamic time window instruction for capturing variable biometric features is generated. The real-time queue density images are obtained by a camera on top of the gate.

[0014] The dynamic time window command is converted into a nanosecond-level photoelectric synchronization signal and transmitted to an array-type pressure sensor and a multi-view camera device to control their precise start-stop timing.

[0015] Monitor the time overlap of card swiping events across multiple channels and dynamically allocate collection priorities to ensure conflict-free card swiping signal processing under high-density passenger flow.

[0016] As a further aspect of the present invention: the step of collecting the foot pressure distribution and facial video of the cardholder and processing them to obtain spatiotemporally aligned foot pressure time-series data and anti-occlusion facial images specifically includes:

[0017] The foot pressure distribution is detected by an array of pressure sensors in the gate passage, converted into a charge distribution map, and then processed by a charge amplifier to obtain foot pressure time series data.

[0018] Anti-occlusion facial images are acquired using a multi-view camera device, which can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution.

[0019] Synchronized dual-modal data is obtained by performing temporal interpolation compensation on foot pressure time-series data and anti-occlusion facial images based on gate vibration sensor data;

[0020] The output voltage of the array-type pressure sensor is monitored in real time, and when the output voltage is saturated, the charge shunt protection circuit is triggered.

[0021] As a further aspect of the present invention: the steps of reconstructing continuous gait phase features based on foot pressure time-series data, inversely restoring three-dimensional facial biometrics under compression conditions using anti-occlusion facial images, and integrating and processing to obtain standardized human biometric vectors specifically include:

[0022] Based on foot pressure time-series data, the motion phase interrupted during gait stagnation is calculated using the second-order differential equation of foot pressure trajectory to obtain a continuous gait cycle feature vector.

[0023] Based on the occlusion-resistant facial image, the corrected three-dimensional facial topology is obtained by inversely calculating the facial compression deformation using a biomechanical elasticity model.

[0024] By fusing gait cycle feature vectors with 3D facial topology, noise reduction is performed using a conditional adversarial network.

[0025] A standardized human biometric vector is obtained by extracting features from the denoised data using a spatiotemporal pyramid network.

[0026] As a further aspect of the present invention: the step of inversely calculating the corrected three-dimensional facial topology based on the anti-occlusion facial image using a biomechanical elasticity model specifically includes:

[0027] Based on clinical data, the local elastic modulus was extracted from unobstructed facial images to obtain a Young's modulus reference value, which is correlated with age and gender.

[0028] By calculating the distribution of compressive force using the displacement of marker points in an anti-occlusion facial image and optical flow, the stress state of deep tissues can be inferred to obtain a spatiotemporally continuous mechanical sensing signal.

[0029] Based on Young's modulus reference value and mechanical sensing signals, the dynamic displacement field is obtained through three-dimensional nonlinear wave equation and explicit integration algorithm.

[0030] Generative adversarial networks are used to combine dynamic displacement fields with an initial model to generate a corrected 3D facial topology, where the initial model is the initial state of the 3D facial topology.

[0031] As a further aspect of the present invention: the step of evaluating the confidence level of human biometric vectors and performing dynamic weighted fusion to obtain multimodal features, and then generating the final identity authentication result through a multi-round voting mechanism within a time window, specifically includes:

[0032] The real-time confidence weights of gait and face are calculated by analyzing the gait stagnation index and facial deformation entropy in human biometric vectors.

[0033] A gating attention mechanism is used to dynamically aggregate gait and facial features to obtain fused multimodal features;

[0034] Preliminary identity authentication results are obtained by weighted majority voting on the fusion decision of multimodal features within a time window;

[0035] When a decision conflict is detected, the final identity authentication result is output using a Bayesian inference engine.

[0036] Another object of the present invention is to provide a gait and facial feature fusion recognition system based on a deep learning architecture, the system comprising:

[0037] The signal response module is used to monitor card swiping events at subway turnstiles, parse card ID information, and generate dynamic time window signals. The card swiping events are obtained by reading the signal from the transportation card through a card reader.

[0038] The micro-motion capture module is used to acquire the foot pressure distribution and facial video of the card swiper when a dynamic time window signal is obtained, and process them to obtain spatiotemporally aligned foot pressure timing data and anti-occlusion facial image. The foot pressure distribution is acquired by a high-density array of pressure sensors distributed in the gate passage, and the facial video is acquired by a multi-view camera device.

[0039] The feature reconstruction module is used to reconstruct continuous gait phase features based on foot pressure time series data, and to inversely restore the three-dimensional facial biometric features under the compression state using anti-occlusion facial images, and integrate and process them to obtain a standardized human biometric feature vector.

[0040] The dynamic decision-making module is used to assess the confidence level of human biometric vectors and perform dynamic weighted fusion to obtain multimodal features. Then, it generates the final identity authentication result through a multi-round voting mechanism within a time window and controls the gate passage.

[0041] As a further aspect of the present invention: the signal response module includes:

[0042] The signal processing unit is used to separate the high-frequency carrier and modulation data in the card swiping signal through the card reader, and to decode the modulation data using the envelope detector to obtain the card ID information.

[0043] The dynamic window generation unit is used to predict the optimal acquisition time based on card ID information and real-time queue density image through a convolutional neural network, and generate a dynamic time window instruction for variable biometric capture. The real-time queue density image is obtained by a camera on top of the gate.

[0044] A hardware triggering unit is used to convert the dynamic time window instruction into a nanosecond-level photoelectric synchronization signal and transmit it to the array-type pressure sensor and the multi-view camera device to control the precise start-stop sequence of the two.

[0045] The conflict arbitration unit is used to monitor the time overlap of card swiping events across multiple channels and dynamically allocate collection priorities to ensure conflict-free card swiping signal processing under high-density passenger flow.

[0046] As a further aspect of the present invention: the micro-motion capture module includes:

[0047] The foot pressure sensing unit is used to sense the foot pressure distribution through an array of pressure sensors in the gate passage, convert it into a charge distribution map, and obtain foot pressure time series data after processing by a charge amplifier.

[0048] A multispectral imaging unit is used to acquire occlusion-resistant facial images through a multi-view camera device, which can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution.

[0049] The data alignment unit is used to perform temporal interpolation compensation on foot pressure time-series data and anti-occlusion facial images based on gate vibration sensor data to obtain synchronized dual-modal data;

[0050] The overload protection unit is used to monitor the output voltage of the array pressure sensor in real time, so that when the output voltage is saturated, the charge shunt protection circuit is triggered.

[0051] As a further aspect of the present invention: the feature reconstruction module includes:

[0052] The gait phase field reconstruction unit is used to calculate the motion phase interrupted during gait stagnation based on foot pressure time series data and the second-order differential equation of foot pressure trajectory to obtain a continuous gait period feature vector.

[0053] The deformation mechanics solution unit is used to inversely calculate the corrected three-dimensional facial topology based on the anti-occlusion facial image and using a biomechanical elastic model;

[0054] A multimodal noise reduction unit is used to fuse gait period feature vectors with three-dimensional facial topology and perform noise reduction using a conditional adversarial network.

[0055] The feature extraction unit is used to extract features from the denoised data using a spatiotemporal pyramid network to obtain standardized human biometric vectors.

[0056] Compared with the prior art, the beneficial effects of the present invention are:

[0057] This invention, by monitoring card-swiping events and generating dynamic time window signals, can accurately capture the temporal sequence of passenger behavior within the turnstile channel, solving the problem of continuous sequence breaks caused by gait stagnation in traditional methods and avoiding motion feature distortion. After collecting foot pressure distribution data from a high-density array of pressure sensors and acquiring anti-occlusion facial images from multi-view cameras, it inversely reconstructs three-dimensional facial biometrics under compression conditions, effectively overcoming the problem of instantaneous distortion of facial geometry caused by crowd compression during peak hours. Furthermore, this invention, through dynamic weighted fusion of multimodal features and the introduction of a multi-round voting mechanism within the time window, not only alleviates modal conflicts caused by the spatiotemporal offset between gait stagnation and facial compression, but also significantly shortens the time from sensor acquisition to recognition decision through a hard real-time association mechanism linking card-swiping behavior, mechanical pressure changes, and biometric feature acquisition, ensuring identity authentication is completed within an acceptable passage time window for passengers. In summary, this invention achieves high-precision, low-false-recognition-rate fusion recognition in complex scenarios, providing a stable and reliable solution for the intelligent management of subway turnstiles. Attached Figure Description

[0058] Figure 1 This is a flowchart of a gait and facial feature fusion recognition method based on a deep learning architecture.

[0059] Figure 2 This is a flowchart illustrating the generation of dynamic time window signals in a gait and facial feature fusion recognition method based on a deep learning architecture.

[0060] Figure 3 This is a flowchart illustrating the process of obtaining spatiotemporally aligned foot pressure time-series data and occlusion-resistant facial images in a gait and facial feature fusion recognition method based on a deep learning architecture.

[0061] Figure 4 This is a flowchart illustrating the standardized human biometric vector obtained in a gait and facial feature fusion recognition method based on a deep learning architecture.

[0062] Figure 5 This is a flowchart illustrating the corrected 3D facial topology obtained in a gait and facial feature fusion recognition method based on a deep learning architecture.

[0063] Figure 6 This is a flowchart illustrating the generation of the final authentication result in a gait and facial feature fusion recognition method based on a deep learning architecture.

[0064] Figure 7 This is a schematic diagram of a gait and facial feature fusion recognition system based on a deep learning architecture.

[0065] Figure 8 This is a schematic diagram of the signal response module of a gait and facial feature fusion recognition system based on a deep learning architecture.

[0066] Figure 9 This is a schematic diagram of the micro-motion capture module of a gait and facial feature fusion recognition system based on a deep learning architecture.

[0067] Figure 10 This is a schematic diagram of the feature reconstruction module of a gait and facial feature fusion recognition system based on a deep learning architecture. Detailed Implementation

[0068] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0069] The specific implementation of the present invention will be described in detail below with reference to specific embodiments.

[0070] like Figure 1 As shown, this embodiment of the invention provides a gait and facial feature fusion recognition method based on a deep learning architecture, the method comprising the following steps:

[0071] S100 monitors card swiping events at subway turnstiles, parses card ID information, and generates dynamic time window signals. The card swiping events are obtained by reading the signal from the transportation card using a card reader.

[0072] S200, when the dynamic time window signal is acquired, the foot pressure distribution and facial video of the card swiper are collected and processed to obtain spatiotemporally aligned foot pressure time series data and anti-occlusion facial image. The foot pressure distribution is collected by a high-density array of pressure sensors distributed in the gate passage, and the facial video is collected by a multi-view camera device.

[0073] S300 reconstructs continuous gait phase features based on foot pressure time-series data, and uses anti-occlusion facial images to inversely restore three-dimensional facial biometrics under compression conditions, integrating and processing them to obtain standardized human biometric vectors.

[0074] S400 assesses the confidence level of human biometric vectors and performs dynamic weighted fusion to obtain multimodal features. Then, it generates the final identity authentication result through a multi-round voting mechanism within a time window and controls the gate passage.

[0075] It's important to note that a dynamic time window is a time period that adjusts in real-time based on card-swiping events, used for synchronized data acquisition and processing. For example, when a passenger swipes their card, the system triggers a time window (e.g., 10 seconds) during which it collects plantar pressure and facial video data, ensuring a strict correlation between the data and the card-swiping behavior. The array-type pressure sensor consists of an array of multiple high-density pressure sensors covering the turnstile passageway area. Compared to single-point sensors, array-type sensors can capture the spatial distribution of plantar pressure with high precision, supporting the reconstruction of continuous gait phase characteristics.

[0076] In this embodiment of the invention, by monitoring card-swiping events and generating dynamic time window signals, the invention can accurately capture the temporal sequence of passenger behavior within the turnstile channel, solving the problem of continuous sequence breakage caused by gait stagnation in traditional methods and avoiding motion feature distortion. After collecting foot pressure distribution data from a high-density array of pressure sensors and acquiring anti-occlusion facial images from multi-view cameras, the invention reverse-engineers the three-dimensional facial biometrics under compression conditions, effectively overcoming the problem of instantaneous distortion of facial geometry caused by crowd compression during peak hours. Furthermore, by dynamically weighting and fusing multimodal features and introducing a multi-round voting mechanism within the time window, the invention not only alleviates the modal conflict caused by the spatiotemporal offset between the gait stagnation phase and the facial compression moment, but also significantly shortens the time from sensor acquisition to recognition decision through a hard real-time association mechanism linking card-swiping behavior, mechanical pressure changes, and biometric feature acquisition, ensuring identity authentication is completed within the passenger's acceptable passage time window. In summary, this invention achieves high-precision, low-false-recognition-rate fusion recognition in complex scenarios, providing a stable and reliable solution for the intelligent management of subway turnstiles.

[0077] like Figure 2 As shown, in a preferred embodiment of the present invention, the step of monitoring the card swiping events of the subway turnstile, parsing the card ID information, and generating a dynamic time window signal specifically includes:

[0078] S101 separates the high-frequency carrier and modulation data in the card swiping signal using a card reader, and uses an envelope detector to decode the modulation data to obtain the card ID information.

[0079] S102, based on card ID information combined with real-time queue density image, predict the optimal collection time through convolutional neural network and generate dynamic time window instructions for variable biometric capture, wherein the real-time queue density image is obtained by camera on top of gate.

[0080] S103, the dynamic time window instruction is converted into a nanosecond-level photoelectric synchronization signal and transmitted to the array-type pressure sensor and the multi-view camera device to control the precise start-stop timing of the two.

[0081] S104 monitors the time overlap of card swiping events across multiple channels and dynamically allocates collection priorities to ensure conflict-free card swiping signal processing under high-density passenger flow.

[0082] In this embodiment of the invention, the card reader separates the high-frequency carrier and modulated data, and extracts the card ID information using an envelope detector. This process ensures rapid parsing of the card swiping signal. Combining the card ID information with the real-time queue density image captured by the camera at the top of the gate, a convolutional neural network (CNN) predicts the optimal acquisition time (e.g., during peak hours when queues are dense, the CNN may predict a shorter time window of 3 seconds, while during off-peak hours it may be extended to 5 seconds). A variable biometric capture time window instruction is dynamically generated to ensure that the acquisition time adaptively matches the passenger flow status. After the dynamic time window instruction is converted into a nanosecond-level photoelectric synchronization signal, it precisely controls the start-stop timing of the array-type pressure sensor and the multi-view camera equipment. The system monitors the time overlap of card-swiping events across multiple channels and handles conflicts under high-density passenger flow using dynamic priority allocation algorithms (such as timestamp sorting or queue scheduling strategies). For example, when two passengers swipe their cards simultaneously, the system prioritizes the signal of the passenger closer to the gate to avoid data overlay or processing delays. This approach not only solves the problem of biometric data acquisition distortion caused by gait stagnation or crowd compression in traditional gates, but also ensures conflict-free and efficient passage during peak hours through nanosecond-level synchronization signals and dynamic priority allocation, thereby significantly improving the real-time performance and reliability of the gate system.

[0083] like Figure 3 As shown, in a preferred embodiment of the present invention, the step of collecting the plantar pressure distribution and facial video of the cardholder and processing them to obtain spatiotemporally aligned plantar pressure time-series data and anti-occlusion facial images specifically includes:

[0084] S201 uses array-type pressure sensors in the gate passage to sense the distribution of plantar pressure, converts it into a charge distribution map, and obtains the plantar pressure time sequence data after processing by a charge amplifier.

[0085] S202, Acquire anti-occlusion facial images through a multi-view camera device, wherein the multi-view camera device can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution;

[0086] S203, based on the gate vibration sensor data, perform time-domain interpolation compensation on foot pressure time-series data and anti-occlusion facial images to obtain synchronized dual-modal data;

[0087] S204 monitors the output voltage of the array-type pressure sensor in real time, triggering the charge shunt protection circuit when the output voltage is saturated.

[0088] In this embodiment of the invention, an array of pressure sensors within the turnstile passageway converts the pressure distribution of a passenger's foot into a charge distribution map using the piezoelectric or piezoresistive effects. For example, when a passenger walks, pressure changes in different areas of the foot trigger deformation of nanowires or piezoresistors in the sensor, generating a charge signal proportional to the pressure. These charge signals are then processed by a charge amplifier, reducing cable capacitance interference through the Miller effect, and outputting low-impedance foot pressure timing data, thereby accurately capturing key phase features such as ground contact and swaying during the gait cycle. Simultaneously, a multi-view camera device generates an occlusion-resistant multispectral facial image through the coordinated acquisition of visible and near-infrared spectra. When a passenger's face may be partially obscured due to hair, a mask, or a side profile, the multi-view camera can jointly capture facial texture (e.g., skin texture under visible light) and subcutaneous vascular distribution (e.g., blood oxygenation signal under infrared light) at different wavelengths, and use a multispectral image fusion algorithm to complete the obscured area, generating a high-confidence facial feature vector. To achieve spatiotemporal alignment between foot pressure time-series data and facial images, a gate vibration sensor (such as an accelerometer) is needed to monitor mechanical vibration signals in real time. However, this requires combining frequency-domain zero-padding with time-domain interpolation to compensate for timing deviations caused by differences in sensor sampling rates or data loss. For example, if the pressure sensor samples at 100Hz and the camera at 30Hz, the vibration sensor data can be used as a reference signal. Foot pressure data for intermediate frames is generated by interpolating the vibration amplitude at adjacent time points, ensuring strict synchronization between the two data points on the time axis. Furthermore, by monitoring the output voltage of the array-type pressure sensor in real time, a charge shunt protection circuit is triggered when the voltage approaches saturation. For example, if a passenger's weight is too high, causing the sensor output voltage to approach the maximum dynamic range of the charge amplifier, the MOSFET in the shunt protection circuit will automatically turn on, releasing excess charge through a low-impedance path to prevent sensor damage due to overload. This mechanism ensures long-term stable operation of the sensor under high-density passenger flow, avoiding system-wide recognition interruption due to a single point of failure. In summary, this embodiment solves the technical bottlenecks of traditional methods in terms of occlusion, spatiotemporal asynchrony, and hardware overload by synergistically combining charge signal processing, multispectral image fusion, temporal interpolation compensation, and charge protection circuits, providing a high-accuracy solution for intelligent feature acquisition of subway turnstiles.

[0089] like Figure 4 As shown, in a preferred embodiment of the present invention, the steps of reconstructing continuous gait phase features based on foot pressure time-series data, inversely reconstructing three-dimensional facial biometrics under compression conditions using anti-occlusion facial images, and integrating and processing to obtain standardized human biometric vectors specifically include:

[0090] S301, based on foot pressure time series data, uses the second-order differential equation of foot pressure trajectory to calculate the motion phase interrupted during gait stagnation and obtain a continuous gait cycle feature vector.

[0091] S302, based on anti-occlusion facial images, uses a biomechanical elasticity model to inversely calculate facial compression deformation and obtain the corrected three-dimensional facial topology;

[0092] S303 integrates gait cycle feature vectors with three-dimensional facial topology and uses conditional adversarial networks for noise reduction.

[0093] S304 uses a spatiotemporal pyramid network to extract features from the denoised data to obtain standardized human biometric vectors.

[0094] In this embodiment of the invention, based on foot pressure time-series data, the system uses the second-order differential equation of foot pressure trajectory (which uses a second-order differential equation to describe and predict the motion trajectory of the plantar pressure center during walking) to calculate the motion phase interrupted during gait pauses. For example, when a passenger pauses briefly due to gate congestion, the second-order differential equation can be used to calculate the dynamic relationship between foot pressure acceleration and velocity (as shown in equation...). ,in This represents the second derivative of position with respect to time. The foot pressure distribution function, Using a proportionality constant, the phase continuity of the interrupted phase is predicted, thereby reconstructing the discrete foot pressure signal into a continuous gait cycle feature vector, which includes parameters such as stride frequency and stride length. Anti-occlusion facial images (such as Wild2Avatar multi-view modeling technology) utilize biomechanical elasticity models (such as nonlinear elastic equations) to... Where σ is stress, E is Young's modulus, and ε is strain, facial compression deformation is calculated inversely. For example, during peak hours, passengers experience facial geometric distortion due to crowd compression. The system inputs multi-view, multispectral images (such as visible light texture and near-infrared vascular distribution) into the elastic model to simulate the reverse deformation of deep tissues under stress, ultimately correcting it to a standard three-dimensional facial topology. Subsequently, by fusing gait cycle feature vectors with the three-dimensional facial topology, denoising can be performed using a conditional adversarial network. For example, the original data may be distorted due to sensor noise or lighting interference. The conditional adversarial network evaluates the authenticity of the data through a discriminator, while the generator uses complementary information from gait and face to generate a more robust feature representation. The spatiotemporal pyramid network performs multi-granularity analysis on the denoised data: in the time dimension, it extracts the temporal dependence of the gait cycle (such as the variation pattern of step frequency) through a sliding window; in the spatial dimension, it captures the spatial correlation between facial key points (such as the tip of the nose and the distance between the eyes) and foot pressure thermal zones through pyramidal convolutional kernels, ultimately outputting a standardized human biometric vector. Therefore, this process not only solves the limitations of traditional methods in occlusion and motion interruption, but also achieves high-confidence identity authentication in complex scenarios through the collaborative optimization of physical modeling and deep learning.

[0095] like Figure 5 As shown, in a preferred embodiment of the present invention, the step of inversely calculating the corrected three-dimensional facial topology based on the anti-occlusion facial image using a biomechanical elasticity model specifically includes:

[0096] S312, Combining clinical data, the Young's modulus reference value is obtained by extracting the local elastic modulus based on the unobstructed facial image, and the Young's modulus reference value is associated with age and gender;

[0097] S322, by calculating the distribution of compressive force through the displacement of marker points in the anti-occlusion facial image and optical flow method, the stress state of deep tissue is inferred to obtain a spatiotemporally continuous mechanical sensing signal;

[0098] S332, based on Young's modulus reference value and mechanical sensing signals, obtains the dynamic displacement field through three-dimensional nonlinear wave equation and explicit integration algorithm;

[0099] S342, using a generative adversarial network to combine the dynamic displacement field with the initial model to generate a corrected three-dimensional facial topology, wherein the initial model is the initial state of the three-dimensional facial topology.

[0100] In this embodiment of the invention, the aim is to address the facial deformation distortion caused by crowds pushing and shoving in high-density scenarios such as subway turnstiles. By combining clinical data (such as skin and subcutaneous tissue stiffness measurements of different age groups and genders in medical imaging studies), the system extracts local deformation features of key areas (such as the cheekbone, nose, and jaw) from unobstructed facial images and estimates a Young's modulus baseline value associated with individual age and gender. For example, young men typically have firmer facial tissue, so their Young's modulus (a physical quantity measuring material stiffness) may be set to a higher value, while older women may have a lower value due to collagen loss. This baseline value provides personalized material parameters for subsequent mechanical modeling. By using marker point displacement analysis and optical flow to track the minute movement trajectories of the facial surface during compression, and analyzing the gradient and direction of these displacements, the distribution of the compressive force acting on the facial surface is deduced. Furthermore, by combining the principles of elasticity, the stress state of deeper tissues such as subcutaneous muscles and fat is inferred, forming a spatiotemporally continuous mechanical sensing signal that evolves over time. Based on the known Young's modulus reference value and the calculated mechanical sensing signal, a three-dimensional nonlinear wave equation (a partial differential equation describing the propagation of stress waves in an elastic medium, such as...) is used. , This indicates the density of facial tissues. It is the second partial derivative of the displacement field with respect to time, representing the acceleration at a point in the medium; It is the stress tensor The divergence; It refers to the density of external force per unit volume, in a facial compression scenario. The extrusion force distribution derived from optical flow (i.e., the spatiotemporally continuous mechanical sensing signal) simulates the complex deformation behavior of soft tissue under non-uniform extrusion force. This equation is numerically solved using an explicit integration algorithm (such as the central difference method), progressively calculating the dynamic displacement field of each voxel (3D pixel) on the face during the extrusion process—that is, the movement path of each point in three-dimensional space. To perfectly integrate this physically simulated displacement field with the realistic facial appearance and to handle any subtle errors or unnaturalness that may exist in the simulation, a Generative Adversarial Network (GAN) is introduced. The calculated dynamic displacement field is used as a conditional input to an initial 3D facial model. The GAN's generator draws a corrected 3D structure that is both physically accurate and highly realistic, while the discriminator continuously evaluates the realism of the generated result, ultimately outputting a corrected 3D facial topology. For example, when a passenger's face is partially dented due to pressure from a nearby passenger's backpack, the system can infer the internal force based on the surface deformation, simulate tissue rebound using personalized Young's modulus, and generate a high-fidelity 3D model that removes compression artifacts and restores the true facial contours through GAN, providing a reliable basis for subsequent identity recognition.

[0101] like Figure 6 As shown, in a preferred embodiment of the present invention, the steps of evaluating the confidence level of human biometric vectors and dynamically weighting and fusing them to obtain multimodal features, and then generating the final identity authentication result through a multi-round voting mechanism within a time window, specifically include:

[0102] S401 calculates the real-time confidence weights of gait and face by analyzing the gait stagnation index and facial deformation entropy in human biometric vectors.

[0103] S402 utilizes a gated attention mechanism to dynamically aggregate gait and facial features to obtain fused multimodal features;

[0104] S403, based on weighted majority voting on the fusion decision of multimodal features within a time window, a preliminary identity authentication result is obtained;

[0105] S404: When a decision conflict is detected, the final identity authentication result is output using the Bayesian inference engine.

[0106] In this embodiment of the invention, real-time confidence weights are calculated by analyzing the gait stagnation index and facial deformation entropy. For example, when a passenger suddenly stops due to gate congestion, the gait signal may become abnormal; the reliability is quantified by calculating the gait stagnation index. If the face experiences localized indentation or texture distortion due to pressure from nearby passengers, the system assesses the degree of distortion using facial deformation entropy. These two indicators jointly determine the real-time confidence weights for gait and facial modalities. A gating attention mechanism is used to dynamically aggregate gait and facial features. The gating mechanism (such as the gating unit in LSTM or GRU) allocates attention to different modalities based on the current confidence weight. For example, when the facial weight is high, the system enhances the accuracy of facial feature extraction while suppressing the gait features affected by stagnation. Preliminary identity authentication results are generated based on multiple rounds of weighted majority voting within a time window. The system divides the multimodal features within several consecutive seconds into a time window (e.g., 100 frames within 5 seconds) and performs identity matching (e.g., similarity calculation with database templates) on the weighted fused features of each frame. For example, within a time window, the probability of a gait modality matching user A is 70% (weight 20%), and the probability of a facial modality matching user A is 90% (weight 80%). Therefore, the weighted total probability is 0.7 × 0.2 + 0.9 × 0.8 = 0.86. The system will statistically analyze the weighted probabilities of all frames within the time window. If the probability exceeds a threshold (e.g., 85%) and most frames point to the same user, a preliminary authentication result will be output. For decision conflicts, such as a gait matching user A but a facial match matching user B, the Bayesian inference engine will be activated. This engine combines the prior probability of the user's historical behavior (e.g., user A used the gate more frequently during that time period than user B) with the current modality weight to correct the final decision. It should be noted that the Bayesian inference engine is a common technique for solving uncertainty problems, especially performing well in scenarios with sparse data, requiring dynamic updates, or needing risk quantification; therefore, it will not be described in detail here.

[0107] like Figure 7 As shown, embodiments of the present invention also provide a gait and facial feature fusion recognition system based on a deep learning architecture, the system comprising:

[0108] The signal response module 100 is used to monitor card swiping events at the subway turnstile, parse card ID information, and generate dynamic time window signals. The card swiping events are obtained by reading the signal from the transportation card through a card reader.

[0109] The micro-motion capture module 200 is used to acquire the foot pressure distribution and facial video of the card swiper when a dynamic time window signal is obtained, and process them to obtain spatiotemporally aligned foot pressure timing data and anti-occlusion facial image. The foot pressure distribution is acquired by a high-density array of pressure sensors distributed in the gate passage, and the facial video is acquired by a multi-view camera device.

[0110] The feature reconstruction module 300 is used to reconstruct continuous gait phase features based on foot pressure time series data, and to reverse restore the three-dimensional facial biometrics under the compression state using anti-occlusion facial images, and to integrate and process them to obtain a standardized human biometric vector.

[0111] The dynamic decision module 400 is used to evaluate the confidence of human biometric vectors and perform dynamic weighted fusion to obtain multimodal features. Then, it generates the final identity authentication result through a multi-round voting mechanism within a time window and controls the gate passage.

[0112] In this embodiment of the invention, by monitoring card-swiping events and generating dynamic time window signals, the timing of passenger behavior within the gate channel can be accurately captured, solving the problem of continuous sequence breaks caused by gait stagnation in traditional methods and avoiding motion feature distortion. After collecting foot pressure distribution data from a high-density array of pressure sensors and acquiring anti-occlusion facial images from multi-view cameras, the three-dimensional facial biometrics under compression conditions are inversely reconstructed, effectively overcoming the problem of instantaneous distortion of facial geometry caused by crowd compression during peak hours. Furthermore, by dynamically weighting and fusing multimodal features and introducing a multi-round voting mechanism within the time window, this invention not only alleviates the modal conflict caused by the spatiotemporal offset between the gait stagnation phase and the facial compression moment, but also significantly shortens the time from sensor acquisition to recognition decision through a hard real-time association mechanism linking card-swiping behavior, mechanical pressure changes, and biometric acquisition, ensuring that identity authentication is completed within the passenger's acceptable passage time window.

[0113] like Figure 8 As shown, in a preferred embodiment of the present invention, the signal response module 100 includes:

[0114] Signal processing unit 101 is used to separate the high-frequency carrier and modulation data in the card swiping signal through the card reader, and to decode the modulation data using an envelope detector to obtain card ID information;

[0115] The dynamic window generation unit 102 is used to predict the optimal acquisition time based on card ID information and real-time queue density image through a convolutional neural network, and generate a dynamic time window instruction for variable biometric capture. The real-time queue density image is obtained by a camera on top of the gate.

[0116] The hardware triggering unit 103 is used to convert the dynamic time window instruction into a nanosecond-level photoelectric synchronization signal and transmit it to the array-type pressure sensor and the multi-view camera device to control the precise start-stop sequence of the two.

[0117] The conflict arbitration unit 104 is used to monitor the time overlap of card swiping events in multiple channels and dynamically allocate the collection priority to ensure that there is no conflict in the card swiping signal processing under high-density passenger flow.

[0118] like Figure 9 As shown, in a preferred embodiment of the present invention, the micro-motion capture module 200 includes:

[0119] The foot pressure sensing unit 201 is used to sense the foot pressure distribution through the array of pressure sensors in the gate passage, convert it into a charge distribution map, and obtain foot pressure time series data after processing by a charge amplifier.

[0120] The multispectral imaging unit 202 is used to acquire anti-occlusion facial images through a multi-view camera device, which can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution.

[0121] The data alignment unit 203 is used to perform temporal interpolation compensation on foot pressure time-series data and anti-occlusion facial images based on gate vibration sensor data to obtain synchronized dual-modal data.

[0122] The overload protection unit 204 is used to monitor the output voltage of the array pressure sensor in real time, so that when the output voltage is saturated, the charge shunt protection circuit is triggered.

[0123] like Figure 10 As shown, in a preferred embodiment of the present invention, the feature reconstruction module 300 includes:

[0124] The gait phase field reconstruction unit 301 is used to calculate the motion phase interrupted during gait stagnation based on foot pressure time series data and the second-order differential equation of foot pressure trajectory to obtain a continuous gait period feature vector.

[0125] The deformation mechanics calculation unit 302 is used to inversely calculate the corrected three-dimensional facial topology based on the anti-occlusion facial image and using a biomechanical elastic model to calculate the facial compression deformation.

[0126] The multimodal noise reduction unit 303 is used to fuse gait period feature vectors with three-dimensional facial topology and perform noise reduction processing using a conditional adversarial network.

[0127] The feature extraction unit 304 is used to extract features from the denoised data using a spatiotemporal pyramid network to obtain a standardized human biometric vector.

[0128] All actions taken to obtain information or data in this application were carried out in accordance with the relevant data protection laws and policies of the country where the application is located and with the authorization of the relevant users.

[0129] The above description only details the preferred embodiments of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0130] It should be understood that although the steps in the flowcharts of the various embodiments of the present invention are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the various embodiments may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least a portion of the sub-steps or stages of other steps.

[0131] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM), etc.

[0132] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the disclosure in the specification and embodiments. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

Claims

1. A gait and facial feature fusion recognition method based on a deep learning architecture, characterized in that, The method includes the following steps: The system monitors card swiping events at subway turnstiles, parses card ID information, and generates dynamic time window signals. The card swiping events are obtained by reading the signals from the transportation card using a card reader. When the dynamic time window signal is acquired, the foot pressure distribution and facial video of the card swipe are collected and processed to obtain spatiotemporally aligned foot pressure time series data and anti-occlusion facial image. The foot pressure distribution is collected by a high-density array of pressure sensors distributed in the gate passage, and the facial video is collected by a multi-view camera device. Continuous gait phase features are reconstructed based on foot pressure time-series data, and three-dimensional facial biometrics under compression state are inversely restored using anti-occlusion facial images. The results are then integrated to obtain a standardized human biometric vector. The confidence level of human biometric vectors is evaluated and dynamically weighted and fused to obtain multimodal features. Then, the final identity authentication result is generated through a multi-round voting mechanism within a time window, and the gate passage is controlled. The steps of reconstructing continuous gait phase features based on foot pressure time-series data, inversely reconstructing three-dimensional facial biometrics under compression conditions using anti-occlusion facial images, and integrating these steps to obtain standardized human biometric vectors specifically include: Based on foot pressure time-series data, the motion phase interrupted during gait stagnation is calculated using the second-order differential equation of foot pressure trajectory to obtain a continuous gait cycle feature vector. Based on the occlusion-resistant facial image, the corrected three-dimensional facial topology is obtained by inversely calculating the facial compression deformation using a biomechanical elasticity model. By fusing gait cycle feature vectors with 3D facial topology, noise reduction is performed using a conditional adversarial network. A standardized human biometric vector is obtained by extracting features from the denoised data using a spatiotemporal pyramid network.

2. The gait and facial feature fusion recognition method based on the deep learning architecture according to claim 1, characterized in that, The steps of monitoring card swiping events at subway turnstiles, parsing card ID information, and generating dynamic time window signals specifically include: The high-frequency carrier and modulation data in the card swiping signal are separated by a card reader, and the card ID information is obtained by decoding the modulation data using an envelope detector. Based on card ID information combined with real-time queue density images, the optimal acquisition time is predicted through a convolutional neural network, and a dynamic time window instruction for capturing variable biometric features is generated. The real-time queue density images are obtained by a camera on top of the gate. The dynamic time window command is converted into a nanosecond-level photoelectric synchronization signal and transmitted to an array-type pressure sensor and a multi-view camera device to control their precise start-stop timing. Monitor the time overlap of card swiping events across multiple channels and dynamically allocate collection priorities to ensure conflict-free card swiping signal processing under high-density passenger flow.

3. The gait and facial feature fusion recognition method based on the deep learning architecture according to claim 1, characterized in that, The steps of collecting and processing the foot pressure distribution and facial video of the cardholder to obtain spatiotemporally aligned foot pressure time-series data and occlusion-resistant facial images specifically include: The foot pressure distribution is detected by an array of pressure sensors in the gate passage, converted into a charge distribution map, and then processed by a charge amplifier to obtain foot pressure time series data. Anti-occlusion facial images are acquired using a multi-view camera device, which can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution. Synchronized dual-modal data is obtained by performing temporal interpolation compensation on foot pressure time-series data and anti-occlusion facial images based on gate vibration sensor data; The output voltage of the array-type pressure sensor is monitored in real time, and when the output voltage is saturated, the charge shunt protection circuit is triggered.

4. The gait and facial feature fusion recognition method based on the deep learning architecture according to claim 1, characterized in that, The step of inversely calculating the corrected three-dimensional facial topology based on the anti-occlusion facial image using a biomechanical elasticity model specifically includes: Based on clinical data, the local elastic modulus was extracted from unobstructed facial images to obtain a Young's modulus reference value, which is correlated with age and gender. By calculating the distribution of compressive force using the displacement of marker points in an anti-occlusion facial image and optical flow, the stress state of deep tissues can be inferred to obtain a spatiotemporally continuous mechanical sensing signal. Based on Young's modulus reference value and mechanical sensing signals, the dynamic displacement field is obtained through three-dimensional nonlinear wave equation and explicit integration algorithm. Generative adversarial networks are used to combine dynamic displacement fields with an initial model to generate a corrected 3D facial topology, where the initial model is the initial state of the 3D facial topology.

5. The gait and facial feature fusion recognition method based on the deep learning architecture according to claim 1, characterized in that, The steps of evaluating the confidence level of human biometric vectors and dynamically weighting and fusing them to obtain multimodal features, and then generating the final identity authentication result through a multi-round voting mechanism within a time window, specifically include: The real-time confidence weights of gait and face are calculated by analyzing the gait stagnation index and facial deformation entropy in human biometric vectors. A gating attention mechanism is used to dynamically aggregate gait and facial features to obtain fused multimodal features; Preliminary identity authentication results are obtained by weighted majority voting on the fusion decision of multimodal features within a time window; When a decision conflict is detected, the final identity authentication result is output using a Bayesian inference engine.

6. A gait and facial feature fusion recognition system based on a deep learning architecture, characterized in that, The system includes: The signal response module is used to monitor card swiping events at subway turnstiles, parse card ID information, and generate dynamic time window signals. The card swiping events are obtained by reading the signal from the transportation card through a card reader. The micro-motion capture module is used to acquire the foot pressure distribution and facial video of the card swiper when a dynamic time window signal is obtained, and process them to obtain spatiotemporally aligned foot pressure timing data and anti-occlusion facial image. The foot pressure distribution is acquired by a high-density array of pressure sensors distributed in the gate passage, and the facial video is acquired by a multi-view camera device. The feature reconstruction module is used to reconstruct continuous gait phase features based on foot pressure time series data, and to inversely restore the three-dimensional facial biometric features under the compression state using anti-occlusion facial images, and integrate and process them to obtain a standardized human biometric feature vector. The dynamic decision-making module is used to evaluate the confidence of human biometric vectors and perform dynamic weighted fusion to obtain multimodal features. Then, it generates the final identity authentication result through a multi-round voting mechanism within a time window and controls the gate passage. The feature reconstruction module includes: The gait phase field reconstruction unit is used to calculate the motion phase interrupted during gait stagnation based on foot pressure time series data and the second-order differential equation of foot pressure trajectory to obtain a continuous gait period feature vector. The deformation mechanics solution unit is used to inversely calculate the corrected three-dimensional facial topology based on the anti-occlusion facial image and using a biomechanical elastic model; A multimodal noise reduction unit is used to fuse gait period feature vectors with three-dimensional facial topology and perform noise reduction using a conditional adversarial network. The feature extraction unit is used to extract features from the denoised data using a spatiotemporal pyramid network to obtain standardized human biometric vectors.

7. The gait and facial feature fusion recognition system based on the deep learning architecture according to claim 6, characterized in that, The signal response module includes: The signal processing unit is used to separate the high-frequency carrier and modulation data in the card swiping signal through the card reader, and to decode the modulation data using the envelope detector to obtain the card ID information. The dynamic window generation unit is used to predict the optimal acquisition time based on card ID information and real-time queue density image through a convolutional neural network, and generate a dynamic time window instruction for variable biometric capture. The real-time queue density image is obtained by a camera on top of the gate. A hardware triggering unit is used to convert the dynamic time window instruction into a nanosecond-level photoelectric synchronization signal and transmit it to the array-type pressure sensor and the multi-view camera device to control the precise start-stop sequence of the two. The conflict arbitration unit is used to monitor the time overlap of card swiping events across multiple channels and dynamically allocate collection priorities to ensure conflict-free card swiping signal processing under high-density passenger flow.

8. The gait and facial feature fusion recognition system based on the deep learning architecture according to claim 6, characterized in that, The micro-motion capture module includes: The foot pressure sensing unit is used to sense the foot pressure distribution through an array of pressure sensors in the gate passage, convert it into a charge distribution map, and obtain foot pressure time series data after processing by a charge amplifier. A multispectral imaging unit is used to acquire occlusion-resistant facial images through a multi-view camera device, which can output a fused multispectral image by acquiring facial texture features and subcutaneous blood vessel distribution. The data alignment unit is used to perform temporal interpolation compensation on foot pressure time-series data and anti-occlusion facial images based on gate vibration sensor data to obtain synchronized dual-modal data; The overload protection unit is used to monitor the output voltage of the array pressure sensor in real time, so that when the output voltage is saturated, the charge shunt protection circuit is triggered.

Images