Intelligent tablet multi-modal biometric continuous authentication method and system

Abstract

The application relates to the field of information security and biometric recognition technology, and discloses a kind of intelligent tablet multi-modal biometric continuous authentication method and system.The method comprises the following steps: synchronously collecting physiological, behavioral and environmental context data; extracting heterogeneous feature vectors; through a cross-modal spatio-temporal attention fusion module combined with dynamic weights of environmental background, realizing context-adaptive feature deep fusion; based on a sliding time window, generating a continuous authentication score and comparing it with a dynamic threshold to perform permission control.The system includes multi-source data acquisition, feature extraction processing, cross-modal fusion, continuous authentication decision and security control execution module.Through the above scheme, the application realizes high-precision, low-latency, environment-adaptive continuous identity authentication, and improves security protection capability and user experience.

Description

Technical Field

[0001] This invention belongs to the field of information security and biometric recognition technology, specifically relating to a method and system for continuous authentication of multimodal biometric features on a smart tablet. Background Technology

[0002] With the rapid development of mobile internet technology, mobile terminals such as smart tablets have deeply penetrated various core application scenarios, including business offices, online education, and financial transactions. Ensuring the absolute security of sensitive data and user privacy within these terminals has become a focus of industry attention. In existing security defense systems, biometric recognition technology, with its unique and uncopyable physiological characteristics, has become a key technology for identity access control and permission management on smart tablets.

[0003] Continuous authentication technology, as a security mechanism capable of real-time, implicit identity verification throughout the entire lifecycle of a user's device use, has demonstrated enormous application potential in enhancing the proactive defense capabilities of terminals. This technology typically integrates physiological features such as facial recognition and voiceprints, as well as behavioral features such as touch and keystrokes, aiming to build a dynamic security environment that ensures continuous and legitimate authorization without frequent user intervention, preventing unauthorized replacement or malicious takeover of devices during use.

[0004] Existing biometric authentication technologies still face numerous challenges in practical applications: single-modal recognition schemes lack robustness in complex environments, often exhibiting decreased accuracy and stability under conditions of drastic lighting changes, partial occlusion, or frequent user posture shifts; traditional multimodal fusion mechanisms often employ fixed weights or simple linear superposition logic, lacking dynamic perception of the current physical environment, device status, and user behavior context, and failing to adjust the contribution of each modality in real time based on external interference; insufficient depth in mining the spatiotemporal correlation of cross-modal data makes it difficult to extract deep collaborative features of physiological and behavioral characteristics in the same spatiotemporal dimension, resulting in limited system identification capabilities against complex spoofing methods; due to the lack of integrated processing of environmental context information, authentication strategies struggle to achieve accurate adaptation to different application sensitivities and operating environments, easily leading to high false alarm rates or recognition delays. These issues collectively result in a difficulty in simultaneously ensuring security and user experience during continuous authentication of smart tablets. To address the shortcomings of traditional authentication technologies, such as poor environmental adaptability, rigid fusion strategies, and low utilization of spatiotemporal features, this invention proposes a multimodal biometric continuous authentication method and system for smart tablets, which is of significant importance. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for continuous authentication of smart tablets using multimodal biometrics, which can solve the problems in the background technology mentioned above, namely, the technical defects of insufficient robustness of single modality in complex environments, fixed weights of traditional multimodal fusion and lack of dynamic adaptability to the situation, and achieve continuous security protection of terminals with high precision, low latency and environmental adaptability.

[0006] To achieve the above objectives, the technical solution adopted by the present invention is as follows: a method for continuous authentication of multimodal biometric features on a smart tablet, comprising the following steps: S1. Synchronously collect multi-source raw data during the use of the smart tablet by integrating a sensor cluster. The multi-source raw data includes physiological characteristic data, behavioral characteristic data and environmental context data. S2. Use a preset feature extraction network to extract heterogeneous features from multi-source raw data to generate corresponding physiological feature vectors, behavioral feature vectors and environmental background vectors. S3. Input the physiological feature vector and the behavioral feature vector into the cross-modal spatiotemporal attention fusion module, and combine them with the dynamic weight factor generated by the environmental background vector to realize the mapping and deep fusion of physiological features and behavioral features in the spatiotemporal dimension. S4. Real-time identity credibility measurement is performed based on the fused cross-modal feature vector, and continuous authentication scores are generated by combining the preset sliding time window. S5. Compare the continuous authentication score with the dynamic threshold, and execute the corresponding device access control logic based on the comparison result.

[0007] Preferably, step S3 specifically includes the following steps: S31. Align the physiological feature vector and the behavioral feature vector in time step and encode them uniformly into a preset low-dimensional latent space to eliminate the dimensional differences and sampling rate deviations between different modalities. S32. Construct a context-aware weight predictor, taking the environmental background vector as input and outputting the initial value of the contribution of each modality feature in the current context; S33. Multi-head self-attention mechanism is used to extract spatiotemporal correlation features across modalities, and the attention mapping matrix is ​​weighted and corrected using the initial contribution value to obtain context-adaptive fusion feature representation.

[0008] Preferably, the physiological feature data acquisition includes: using the front-facing camera module of the smart tablet to acquire an image sequence containing face and iris information, and using a receiver to acquire the user's voiceprint signal.

[0009] Preferably, the behavioral feature data acquisition includes: using a capacitive touchscreen to record the user's fingertip touch trajectory, pressure gradient, and sliding speed; using a gravity sensor, accelerometer, and gyroscope to monitor the three-axis attitude changes, device shaking amplitude, and hand micro-movement characteristics of the smart tablet in real time during the authentication period; and using a virtual keyboard to collect the user's keystroke rhythm, key duration, and switching delay between adjacent keys during the input process.

[0010] Preferably, the environmental context data acquisition includes: detecting the current ambient light intensity gradient using a light sensor; determining the relative distance between the user's face and the device screen using a distance sensor; and obtaining the sensitivity level labels of currently active applications, the device's current geographic location coordinates, and the system's current battery status through the operating system interface.

[0011] Preferably, the process of extracting physiological features by the feature extraction network in step S2 specifically includes: inputting the image sequence into a deep residual convolutional network, extracting facial key point features and iris texture features through multi-layer nonlinear transformation; after framing and windowing the voiceprint signal, extracting acoustic representation features through Mel frequency cepstral coefficient transformation, and capturing the dynamic evolution characteristics of sound using a temporal processing unit.

[0012] Preferably, the process of extracting behavioral features by the feature extraction network in step S2 specifically includes: constructing a time-series analysis model based on a long short-term memory network, encoding the coordinate sequence of the touch trajectory, and extracting path curvature, pressure feedback distribution, and speed envelope features that reflect the user's operating habits; and using a gated loop unit to perform feature transformation on the three-axis signals of the motion sensor to extract micro-vibration frequency features related to the user's physiological rhythm.

[0013] Preferably, the process of generating the environmental background vector in step S2 specifically includes: using a multilayer perceptron to perform nonlinear mapping on numerical environmental parameters such as light intensity, distance, and attitude angle; performing one-hot encoding on categorized data such as application sensitivity level and location area; and concatenating and splicing various environmental features to form a high-dimensional environmental background vector.

[0014] Preferably, the process of eliminating dimensional differences in step S31 specifically includes: performing layer normalization on the physiological feature vector and the behavioral feature vector respectively, and using a linear transformation layer with shared parameters to map features of different dimensions to the same dimensional space.

[0015] Preferably, the working logic of the context-aware weight predictor in step S32 includes: establishing a mapping function between the environmental background vector and the modal weights; when the environmental background vector indicates that the light intensity is lower than a preset brightness threshold or there is facial occlusion, automatically reducing the initial weight of the facial modality in the physiological features, and proportionally increasing the initial weights of the voiceprint modality and the behavioral feature modality; when the environmental background vector indicates that the current application is a highly sensitive financial application, increasing the discrimination criteria of all modalities, and increasing the global weight of the touch trajectory feature in the behavioral features.

[0016] Preferably, the multi-head self-attention mechanism in step S33 specifically includes: obtaining query vector, key vector, and value vector by applying different linear transformation matrices to the aligned physiological feature vector and behavioral feature vector respectively; calculating the inner product of the query vector and key vector to obtain the intermodal relevance score, and introducing the initial contribution value output by the context-aware weight predictor as a bias term into the relevance score; normalizing the processed score by the normalized exponential function to obtain the final attention weight allocation matrix, and using this matrix to perform weighted summation of the value vector to output the fused cross-modal feature vector.

[0017] Preferably, the real-time identity credibility measurement in step S4 specifically includes: calculating the cosine distance or Euclidean distance between the fused cross-modal feature vector and the user template feature vector pre-stored in the secure area; and using a preset nonlinear mapping function to convert the distance value into an instantaneous credibility score between 0 and 1.

[0018] Preferably, the process of generating a continuous authentication score in step S4 specifically includes: setting a sliding time window covering a preset time span, performing a weighted average of all instantaneous credibility scores generated within the window, with scores closer to the current time having higher weights, to form a continuous authentication score that reflects the continuous stability of the user's identity.

[0019] Preferably, the logic for determining the dynamic threshold in step S5 includes: dynamically adjusting the authentication threshold based on the sensitivity level of the application in the current environmental background vector, setting a lower security threshold for office applications and a higher security threshold for payment or privacy applications; adjusting the threshold based on the current posture stability of the device, and appropriately lowering the threshold tolerance when the device is detected to be in a state of violent shaking to prevent abnormal access.

[0020] Preferably, the execution of the corresponding device permission control logic in step S5 specifically includes: if the continuous authentication score is higher than the dynamic threshold, then the current user's login status and operation permissions are maintained; if the continuous authentication score is lower than the dynamic threshold and is within the preset warning range, then implicit secondary verification logic is triggered, such as increasing the voiceprint collection frequency or requesting a specific touch action; if the continuous authentication score continues to be lower than the warning range, then screen locking, account logout, and recording of environmental image information at the time of the abnormality are immediately executed.

[0021] A smart tablet multimodal biometric continuous authentication system for implementing the above-mentioned method includes: a multi-source data acquisition module, whose input end is connected to the sensor cluster of the smart tablet, for acquiring raw physiological, behavioral, and environmental data; a feature extraction and processing module, whose input end is connected to the output end of the multi-source data acquisition module, and which integrates a deep convolutional network, a temporal encoder, and a multilayer perceptron, for generating feature vectors for each modality; a cross-modal spatiotemporal attention fusion module, configured to receive the physiological, behavioral, and environmental feature vectors output by the feature extraction and processing module, and to achieve dynamic adaptive fusion of features from each modality through a context-aware weight predictor and a multi-head self-attention unit; a continuous authentication decision module, whose input end is connected to the output end of the cross-modal spatiotemporal attention fusion module, for performing credibility measurement, time window scoring calculation, and comparison with dynamic thresholds; and a security control execution module, connected to the continuous authentication decision module, for outputting control commands for permission retention, secondary verification, or forced locking based on the judgment result.

[0022] Compared with the prior art, the beneficial effects of the present invention are as follows: 1. This invention solves the problem of single-modality failure in complex environments by constructing a continuous authentication mechanism that deeply couples cross-modal spatiotemporal attention fusion with context-aware weighting, and overcomes the pain point of rigid strategies in traditional multimodal fusion.

[0023] 2. This invention not only comprehensively utilizes physiological features such as facial features, iris scans, and voiceprints, but also deeply mines highly personalized and difficult-to-forge behavioral features such as touch trajectories, keystroke rhythms, and micro-hand movements. By uniformly encoding these heterogeneous features into the same spatiotemporal latent space, the maximum utilization of complementary information between features is achieved. Real-time monitoring of environmental context data is introduced, enabling the system to accurately perceive changes in illumination, distance, posture, and application sensitivity, achieving second-level dynamic adjustment of the weights of each modality.

[0024] 3. This invention achieves a full-cycle, high-strength, and user-friendly proactive defense system for smart tablets by deeply integrating multi-dimensional features and flexibly scheduling context awareness, without interfering with the user's normal operation. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the overall technical solution architecture of the present invention; Figure 2 This is a schematic diagram of the core principle framework of the cross-modal spatiotemporal attention fusion module in this invention; Figure 3 This is a flowchart illustrating the logical process of extracting and encoding heterogeneous features from multi-source raw data in this invention. Figure 4 This is a flowchart illustrating the logical process of continuous authentication score generation and dynamic threshold determination in this invention. Figure 5 This is a schematic diagram of the multi-level interaction relationship and data flow of data acquisition, feature processing, decision-making and secure execution in this invention. Detailed Implementation

[0026] To further illustrate the technical means and effects adopted by the present invention to achieve the intended purpose, the following description is provided in conjunction with the appendix. Figure 1 To be continued Figure 5 The following is a detailed description of the specific implementation methods, structures, features, and effects of the present invention, as well as preferred embodiments.

[0027] Example 1: This example provides a multimodal biometric continuous authentication method for smart tablets in a conventional office environment and indoor mobile office scenario. In this scenario, the ambient lighting is relatively stable and the user is in a relatively stable holding or placing state. The system focuses on constructing an implicit continuous identity verification barrier through high-frequency deep fusion of physiological and behavioral features.

[0028] In step S1, multi-source raw data during the use of the smart tablet is collected synchronously through an integrated sensor cluster. The specific execution process is as follows: In terms of physiological feature data acquisition, the smart tablet utilizes a front-facing dual-camera module with both infrared and visible light capabilities to acquire image sequences containing the user's facial contours, skin texture, and fine iris structure at a frequency of 30 frames per second. The receiver, or microphone, operates in a low-power listening state, capturing voiceprint signals at a sampling rate of 48 kHz and a bit depth of 16 bits when the user performs voice operations or when there is background conversation.

[0029] In terms of behavioral data acquisition, the capacitive touchscreen continuously records the user's fingertip interaction behavior on the screen in the background at a reporting rate of 120 Hz. This includes the real-time displacement trajectory of the touch point in a two-dimensional coordinate system, the downward pressure gradient value per unit area obtained through the pressure sensing layer, and the instantaneous sliding speed obtained by calculating the time interval and displacement vector between adjacent touch sampling points. The inertial measurement unit, composed of a gravity sensor, a three-axis accelerometer, and a gyroscope inside the system, monitors the changes in the pitch, roll, and yaw angles of the smart tablet in three-dimensional space in real time. These sensors can accurately capture the subtle and personalized hand micro-movements generated by the user when holding the device, such as the high-frequency micro-vibrations of the device caused by breathing or muscle tremors. When the user uses the virtual keyboard to input text, the system records the start and release times of each character press, calculates the duration of the keystroke, and analyzes the physical span and time delay between two adjacent different keys to construct the user's keystroke rhythm characteristics.

[0030] For environmental context data acquisition, an ambient light sensor embedded above the screen is used to obtain the real-time light intensity gradient of the current environment. A distance sensor measures the physical distance between the user's face and the device screen by emitting infrared light pulses and receiving the time difference of the reflected echoes. Through the operating system's kernel interface, the system can read the package names of currently running foreground applications in real time and extract their sensitivity level tags according to a preset security mapping table. For example, office document processing applications are marked as low-level, while enterprise internal management systems or encrypted communication tools are marked as medium-high-level.

[0031] In step S2, a preset feature extraction network is used to extract heterogeneous features from the multi-source raw data.

[0032] For physiological features, image sequences are input into a deep residual convolutional neural network. Within this network, the data undergoes sliding window operations across multiple convolutional kernels to extract low-order edge and color speckle features. As the number of network layers increases, through multi-layer nonlinear transformations, the system gradually identifies key facial features, such as the coordinates of the corners of the eyes, the tip of the nose, and the corners of the mouth, as well as the complex texture information of the iris region. After preprocessing, the voiceprint signal is divided into overlapping frames of 25 milliseconds in length and 10 milliseconds in stride, and a Hamming window is applied to reduce spectral leakage. By performing Mel-frequency cepstral coefficient transform, the time-domain signal is converted into a frequency-domain representation, extracting acoustic features that reflect the physical properties of the vocal cords, and using a temporal processing unit to capture the dynamic characteristics of sound evolution over time.

[0033] For behavioral characteristics, a temporal analysis model based on a long short-term memory network is constructed. The coordinate sequence of touch trajectory is input into the network in chronological order. Through internal forgetting, input, and output gate mechanisms, the path curvature reflecting user operating habits is extracted. For example, the arc size formed by different users when swiping the screen shows statistical differences. The distribution of pressure feedback and velocity envelope features are also encoded into high-dimensional vectors. The three-axis signals from the motion sensor are processed by a gated recurrent unit, which can filter out random background noise in the environment and specifically extract micro-vibration frequency features related to the user's biorhythm.

[0034] The generation process of the environmental background vector involves a multilayer perceptron. Numerical parameters such as illumination intensity, distance, and attitude angle are normalized and then fed into the perceptron for nonlinear mapping. Classification data, such as application sensitivity levels, are first converted into mutually orthogonal one-hot encodings. All environmental features are then concatenated and stitched together to form a high-dimensional environmental background vector that can comprehensively characterize the current physical and application environment.

[0035] In step S3, the physiological feature vector and the behavioral feature vector are input into the cross-modal spatiotemporal attention fusion module, and combined with the dynamic weight factor generated by the environmental background vector.

[0036] In step S31, due to the difference in sampling frequencies between physiological features such as facial recognition and behavioral features such as touch trajectories, the system aligns them in time steps using linear interpolation or resampling techniques. Layer normalization is used to eliminate differences in the dimensions of different sensors, ensuring the numerical stability of the data. Through a linear transformation layer with shared parameters, heterogeneous features are mapped to a preset 256-dimensional low-dimensional latent space.

[0037] In step S32, the context-aware weight predictor is driven by the ambient background vector. Its operating logic is as follows: when the ambient background vector indicates that the current light intensity is higher than a preset brightness threshold and the distance between the face and the screen is within the golden recognition range of 30 to 50 centimeters, the predictor outputs a higher initial weight to the face modality in the physiological features. Conversely, if extremely weak ambient light is detected, the predictor automatically lowers the weight of the face modality and proportionally distributes this weight to the voiceprint modality and touch behavior feature modality, which are less affected by the environment.

[0038] In step S33, the multi-head self-attention mechanism comes into play. The aligned feature vectors are converted into query vectors, key vectors, and value vectors, respectively. By calculating the dot product between the query vector and the key vector, a score matrix reflecting the degree of inter-modality correlation at the current time is obtained. The initial contribution values ​​output in step S32 are added to the score matrix as bias terms. After processing with a normalized exponential function, the final attention weight allocation matrix is ​​obtained, and the value vectors are weighted and summed using this matrix to output the fused cross-modal feature vector.

[0039] In step S4, real-time identity credibility measurement is performed. The system calculates the Euclidean distance between the current fused feature vector and the legitimate user template vector stored in the secure isolation zone of the smart tablet. The smaller the distance, the higher the similarity. This distance value is mapped to an instantaneous credibility score between 0 and 1 using a logistic regression function. To eliminate sudden interference that may occur at a single sampling point, the system sets a sliding time window covering 3000 milliseconds. Within this window, all instantaneous scores are weighted according to their time span from the current moment, with scores closer to the current moment having higher weights. The final continuous authentication score is generated through a weighted average calculation.

[0040] In step S5, a decision is made. Based on the sensitivity level of the application in the current environmental background vector, the system dynamically sets a threshold. For general reading applications, the threshold is set to 0.65; for office software involving personal privacy, the threshold is increased to 0.80. If the continuous authentication score is higher than this dynamic threshold, the system maintains the current full access privileges. If the score falls below the threshold but above the warning line of 0.50, the system enters an implicit secondary verification phase, silently increasing the frequency of voiceprint collection in the background or requesting simple touch confirmation in a specific area of ​​the screen without interrupting the user's current operation. If the score remains below 0.50, the system determines the user is an illegitimate holder, immediately enforces screen locking, and activates the front-facing camera to capture the current holder's image information and store it in the security log.

[0041] Example 2: This example focuses on the implementation process of multimodal continuous authentication in outdoor mobile scenarios when the smart tablet is in low-power mode. The substantial difference between this example and Example 1 lies in the dynamic adjustment strategy for environmental awareness and the sampling compression logic adopted to balance power consumption.

[0042] During the data acquisition phase of step S1, the environmental context data module detected that the battery level was below a preset warning threshold of 20%. The system triggered a power consumption control strategy, reducing the face capture frequency in the physiological feature data from 30 frames per second to 5 frames per second. Considering the potential for sensor overload due to extremely high light intensity in outdoor environments, the light sensor will guide the camera module to activate the wide dynamic range mode.

[0043] Since outdoor walking or vehicle bumps can cause frequent and drastic changes in device posture, the feature extraction network in step S2 enhances robustness processing for behavioral features. When processing accelerometer signals, the gated loop unit automatically identifies and filters out external environmental vibrations with frequencies higher than 10 Hz, retaining low-frequency components related to human gait or hand-holding habits.

[0044] The context-aware weight predictor in step S3 plays a crucial role in this embodiment. When the environmental background vector determines that the device is in a state of violent shaking based on gravity sensor data, the predictor automatically determines that the facial modality in the physiological features may have motion blur, reducing the contribution weight of the face vector. The system increases the weight ratio of posture feature vectors with unique user holding inertia caused by device shaking. If the detected environmental background noise exceeds 60 decibels, the weight of voiceprint features will also be automatically suppressed to ensure that the authentication logic is not interfered with by noisy background sounds.

[0045] In the attention mechanism operation of step S33, to meet the low power consumption requirement, the number of heads in the multi-head self-attention is reduced from 8 in Example 1 to 2. This strategy of reducing the processor's computational load by sacrificing a very small portion of feature granularity ensures that the system can still maintain continuous authentication operation without causing the device to shut down in a low power state.

[0046] In step S4, during the score generation process, the span of the sliding time window is extended from 3000 milliseconds to 6000 milliseconds. The purpose of this design difference is to filter out sudden changes in action caused by intense light and shadow flickering or users temporarily avoiding pedestrians in complex and variable outdoor environments, thereby improving the smoothness of the score and avoiding frequent false locking in outdoor scenarios.

[0047] In the execution logic of step S5, considering that the security sensitivity of outdoor scenarios is generally higher than that of indoor fixed office scenarios, the baseline of the dynamic threshold will be raised overall. Even in low-sensitivity applications, the continuous authentication score for determining legitimate identity must reach above 0.72. This security enhancement strategy based on geographical location and mobility status can prevent the risk of data leakage after the smart tablet is snatched or illegally picked up outdoors. When the score triggers the warning interval, the system no longer uses implicit voiceprint verification, but instead sends tactile feedback of a specific frequency through a vibration motor to guide the user to perform a specific screen swipe action to quickly confirm identity.

[0048] Example 3: This example describes a continuous authentication process for performing high-security tasks on a smart tablet, such as making large financial transfers, signing electronic contracts, or accessing confidential databases. The substantial difference between this example and the previous examples lies in the depth of biometric feature mining and the extremely stringent threshold determination.

[0049] In step S1, to ensure the highest level of security, the multi-source data acquisition module activates full-data acquisition mode. Physiological feature acquisition includes not only traditional facial and voiceprint data, but also the extraction of micro-vibration features of the user's eyes and the pulsation patterns of superficial blood vessels under the skin through the near-infrared enhancement mode of the front-facing camera. Regarding behavioral feature acquisition, the capacitive screen's reporting rate is increased to its maximum physical limit, not only recording touch trajectories but also calculating the fingertip's contact angle on the screen by analyzing subtle changes in the fingertip's contact area.

[0050] In the feature extraction stage of step S2, for physiological features, in addition to the static features extracted by the convolutional neural network, a temporal convolutional network is also introduced to capture the synchronization between the user's lip movements and the sound signal when speaking, achieving extremely high-intensity liveness detection. For behavioral features, the feature extraction network extracts the changes in the intensity of the physiological electrical signals sensed by the user when inputting specific sensitive characters. These weak electrical features are highly repeatable for legitimate users.

[0051] In step S3, the context-aware weight predictor enters a full-modal weighted mode when it detects that the currently active application is a highly sensitive financial application. The weight of any modality will not be less than 10%, meaning the system requires all biometric features to exhibit a high degree of compliance. Even in excellent lighting conditions, the system will not rely solely on facial modality but will mandate that touch trajectories and device gesture characteristics conform to historical benchmarks of legitimate user behavior.

[0052] The multi-head self-attention mechanism in step S33 employs a deeper, fully connected structure to uncover higher-order interaction information between different modalities. For example, the system analyzes the impact of subtle synchronous changes in facial expressions or breathing rhythm on the device's tilt angle when a user performs a specific touch operation. This cross-dimensional, deep feature association is extremely difficult to counterfeit.

[0053] Step S4 abandons the simple Euclidean distance for real-time identity credibility measurement, instead employing a nonlinear similarity measure based on probability density estimation. This method can accurately identify extremely subtle shifts in feature vectors within multidimensional space. The sliding time window is shortened to 800 milliseconds, meaning the system reacts extremely quickly to any suspected illegal behavior, making a decision in less than a second to meet the real-time interception requirements of financial security.

[0054] In step S5, the dynamic threshold is fixed at an extremely high level of 0.95. If the continuous authentication score declines for three consecutive sampling periods and falls below this threshold, the security control execution module will execute the highest level of protection response without any warning: disconnecting the network connection, destroying the temporary decryption key currently in memory, and requiring the user to perform an explicit identity reset using a combination of 3D liveness detection and random voice commands. The system locks the current latitude and longitude coordinates through the geolocation interface and silently sends them to the preset security emergency terminal via a background channel.

[0055] To achieve the above method, the present invention provides a smart tablet multimodal biometric continuous authentication system that integrates physical structure and logical hierarchy.

[0056] The multi-source data acquisition module accesses the smart tablet's hardware resource pool through the underlying driver. The camera module driver is responsible for synchronously controlling the visible light and infrared light sensors, ensuring that image frames under different spectra are perfectly aligned in time. The audio acquisition submodule incorporates echo cancellation and environmental noise reduction algorithms, performing preliminary signal-to-noise ratio enhancement before the raw sound data enters the feature extraction module.

[0057] The feature extraction processing module is accelerated by tensor operation units in application-specific integrated circuits (ASICs) or high-performance processors. Internally, it integrates multiple parallel processing branches, including deep convolutional networks and long short-term memory networks. Each branch vectorizes features for a specific modality. To improve efficiency, feature extraction networks for different modalities share weight parameters in their lower-level layers. For example, when processing image sequences and video streams, they share basic convolutional layers to extract common visual representations, reducing memory usage.

[0058] The cross-modal spatiotemporal attention fusion module is the core logic hub of the entire system. It not only includes a weight predictor composed of multiple fully connected layers, but also a dedicated cache queue for storing historical states of each modality. This module dynamically reconstructs the feature fusion model in each authentication loop by performing multi-head self-attention operations. By adjusting the parameters of the attention heads, the system can capture facial layout information in the spatial dimension and action coherence information in the temporal dimension.

[0059] The continuous authentication decision module is responsible for maintaining a dynamic queue of sliding time windows. This module uses a complex set of logic gates to simulate a nonlinear mapping function, converting multidimensional feature distances into confidence percentages. The decision module also possesses self-learning capabilities, slowly updating the user template vector within the safe area based on the user's behavioral evolution over long-term use, to adapt to natural feature drift caused by user aging or changes in usage habits.

[0060] The security control execution module is deeply coupled with the operating system's permission management center. This module can instantly take over control of the screen display and input devices by sending high-priority system interrupt commands. During secondary verification, the module can dynamically generate random authentication commands, such as requiring the user to read a set of numbers displayed on the screen or perform specific gestures, and verify the matching degree between the execution result and the expected characteristics through a closed-loop feedback mechanism.

[0061] This system is also equipped with an anomaly handling loop. When any sensor reports a fault or data anomaly, the system can automatically locate the failure source through the environmental background vector and quickly initiate alternative strategies. For example, if the light sensor is damaged, the system will estimate the ambient light intensity by analyzing the average brightness of the images acquired by the camera module, ensuring the continuous operation of the authentication logic without causing a logic deadlock.

[0062] The entire data flow process in this invention follows a privacy protection mechanism. All raw physiological data is immediately erased from memory after feature extraction, retaining only high-dimensional feature vectors that are not reversibly reproducible. These vectors undergo homomorphic encryption or noise enhancement during cross-modal fusion to ensure that even in extreme system intrusion scenarios, attackers cannot obtain the user's original biological information.

[0063] In summary, this invention achieves all-weather, all-scenario monitoring of smart tablet user identity by constructing a continuous authentication mechanism that deeply couples cross-modal spatiotemporal attention fusion with context-aware weighted authentication. This invention not only solves the problem of single identification methods being susceptible to interference, but also constructs a dynamic security barrier that can automatically adjust its defense strength according to risk levels through keen capture of environmental and behavioral characteristics. This technical solution, which does not rely on active user intervention, has adaptive adjustment capabilities, and dynamically configurable security levels, provides solid engineering support for the security protection of mobile terminals.

[0064] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A method for continuous authentication of multimodal biometrics on a smart tablet, characterized in that, Includes the following steps: Step S1. Synchronously collect multi-source raw data during the use of the smart tablet by integrating a sensor cluster. The multi-source raw data includes physiological feature data, behavioral feature data, and environmental context data. Step S2. Use a preset feature extraction network to extract heterogeneous features from the multi-source raw data to generate corresponding physiological feature vectors, behavioral feature vectors, and environmental background vectors; Step S3. Input the physiological feature vector and the behavioral feature vector into the cross-modal spatiotemporal attention fusion module, and combine them with the dynamic weight factor generated by the environmental background vector. By aligning the physiological feature vector and the behavioral feature vector in time steps and uniformly encoding them into a preset low-dimensional latent space, the dimensional differences and sampling rate deviations between different modalities are eliminated. The context-aware weight predictor uses the environmental background vector as input to output the initial contribution value of each modality feature in the current context. The multi-head self-attention mechanism is used to extract the spatiotemporal correlation features between cross-modalities. The initial contribution value is introduced as a bias term into the attention mapping matrix for weighted correction to obtain the context-adaptive fusion feature representation. The fused cross-modal feature vector is then output. Step S4. Real-time identity credibility measurement is performed based on the fused cross-modal feature vector. The similarity distance between the cross-modal feature vector and the user template feature vector pre-stored in the security area is calculated, and the distance value is converted into an instantaneous credibility score using a nonlinear mapping function. The instantaneous credibility score is then weighted and averaged using a sliding time window covering a preset time span, where the score closer to the current time has a higher weight, thereby generating a continuous authentication score that reflects the continuous stability of the user identity. Step S5. Compare the continuous authentication score with the dynamic threshold, dynamically determine the dynamic threshold based on the sensitivity level of the application in the current environmental background vector and the device posture stability state, and execute the corresponding device permission control logic based on the comparison result.

2. The method for continuous authentication of multimodal biometrics on a smart tablet according to claim 1, characterized in that, In step S1, collecting physiological characteristic data includes: The front-facing infrared and visible light dual-camera module is used to acquire image sequences containing the user's facial contours, skin texture, and fine iris structure; The user's voiceprint signal is captured using a receiver, and the voiceprint signal includes an audio stream when the user is performing a voice operation or when there is background conversation. The collected behavioral feature data includes: using a capacitive touch screen to record the user's fingertip interaction behavior on the screen, the interaction behavior including the real-time displacement trajectory of the touch point in a two-dimensional coordinate system, the downward pressure gradient value per unit area obtained through the pressure sensing layer, and the instantaneous sliding speed obtained by calculating the time interval and displacement vector between adjacent touch sampling points. An inertial measurement unit composed of a gravity sensor, a three-axis accelerometer, and a gyroscope is used to monitor the changes in pitch, roll, and yaw angles of the smart tablet in three-dimensional space in real time, and to capture the micro-motion characteristics of the user's hands when holding the device. The micro-motion characteristics of the hands include high-frequency micro-vibrations of the device. By using a virtual keyboard to record the start and release times of characters pressed during text input, the duration of key presses is calculated, and the physical span and time delay between adjacent key presses are analyzed to construct keystroke rhythm characteristics.

3. The method for continuous authentication of multimodal biometric features on a smart tablet according to claim 2, characterized in that, In step S1, collecting environmental context data includes: The ambient light sensor is used to obtain the current ambient light intensity gradient; the distance sensor is used to determine the physical distance between the user's face and the device screen. The package name of the currently running foreground application is read in real time through the kernel interface of the operating system, and the sensitivity level label of the application is extracted according to the preset security mapping table. The device's current geographic location coordinates and the system's current battery level are obtained through the operating system interface.

4. The method for continuous multimodal biometric authentication of a smart tablet according to claim 3, characterized in that, In step S2, the process of extracting physiological feature vectors using a feature extraction network includes: The image sequence is input into a deep residual convolutional neural network. Edge and color features are extracted through the sliding window operation of the convolution kernel, and the feature coordinates of facial key points and texture information features of the iris region are identified through multi-layer nonlinear transformation. The voiceprint signal is processed by framing and a Hamming window is applied. The time-domain signal is converted into a frequency-domain representation by performing Mel frequency cepstral coefficient transformation. Acoustic features reflecting the physical characteristics of the vocal cords are extracted, and the dynamic characteristics of sound evolution over time are captured by the timing processing unit to form the physiological feature vector.

5. The method for continuous authentication of multimodal biometrics on a smart tablet according to claim 4, characterized in that, In step S2, the process of extracting behavioral feature vectors using a feature extraction network includes: A time-series analysis model based on a long short-term memory network is constructed. The coordinate sequence of the real-time displacement trajectory is input into the time-series analysis model in chronological order. The path curvature features, pressure feedback distribution features, and velocity envelope features reflecting user operation habits are extracted through the internal forget gate, input gate, and output gate mechanism. The three-axis signal output by the inertial measurement unit is processed using a gated loop unit to filter random background noise and extract micro-vibration frequency features that are consistent with the user's biological rhythm. The process of extracting the environmental background vector includes: using a multilayer perceptron to perform nonlinear mapping on the illumination intensity gradient, physical distance, and attitude angle; performing one-hot encoding on the sensitivity level label and geographic location coordinate information; and concatenating and splicing the processed environmental parameters to form a high-dimensional environmental background vector.

6. The method for continuous authentication of multimodal biometrics on a smart tablet according to claim 5, characterized in that, In step S3, the process of eliminating dimensional differences and sampling rate deviations between different modes includes: The physiological feature vector and the behavioral feature vector are aligned at time steps using linear interpolation or resampling techniques; The aligned physiological feature vectors and behavioral feature vectors are subjected to layer normalization, and the features of different dimensions are mapped to the same dimensional space by using a linear transformation layer with shared parameters. The working logic of the context-aware weight predictor includes: establishing a mapping function between the environmental background vector and the modal weights; when the environmental background vector indicates that the current light intensity is lower than a preset brightness threshold or there is facial occlusion, reducing the initial weight of the facial modality in the physiological feature vector, and proportionally increasing the initial weights of the voiceprint modality and the behavioral feature vector. When the environmental background vector indicates that the current application is in a preset high-sensitivity application, the discrimination criteria for all modalities are increased, and the global weight of touch trajectory features in the behavioral feature vector is increased.

7. The method for continuous authentication of multimodal biometric features on a smart tablet according to claim 6, characterized in that, In step S3, the operation of the multi-head self-attention mechanism includes: The aligned physiological feature vector and behavioral feature vector are respectively transformed into query vector, key vector and value vector through different linear transformation matrices; The inner product of the query vector and the key vector is calculated to obtain the relevance score between modalities, and the initial contribution value output by the context-aware weight predictor is introduced into the relevance score as a bias term. The processed relevance scores are normalized using a normalized exponential function to obtain the final attention weight allocation matrix. The value vector is then weighted and summed using the attention weight allocation matrix to output the fused cross-modal feature vector. When the ambient background vector indicates that the device is in a low-power mode, the number of heads in the multi-head self-attention mechanism is reduced.

8. The method for continuous authentication of multimodal biometric features on a smart tablet according to claim 7, characterized in that, In step S4, the process of generating continuous authentication scores further includes: The span of the sliding time window is dynamically adjusted according to the physical environment in which the device is currently located. When the device is detected to be in a state of violent shaking or outdoor movement, the duration of the sliding time window is increased. By increasing the number of observation time steps, noise caused by light flickering and sudden changes in movement is filtered out. The evolution trend of user identity status is predicted based on the rate of change of the continuous authentication score, and the evolution trend is introduced as a correction factor into the credibility measurement of the next period.

9. The method for continuous multimodal biometric authentication of a smart tablet according to claim 8, characterized in that, In step S5, the process of executing the device access control logic includes: If the continuous authentication score is higher than the dynamic threshold, the current user's login status and operation permissions are maintained. If the continuous authentication score is lower than the dynamic threshold and is within the preset warning range, then the implicit secondary verification logic is triggered. The implicit secondary verification logic includes silently increasing the voiceprint collection frequency in the background or sending tactile feedback at a specific frequency through a vibration motor to guide the user to perform screen swiping actions. If the continuous authentication score continues to be lower than the warning range, the screen will be locked and the account will be logged out immediately. The front camera will be activated to capture the image information of the current holder and store it in the security log. When a device is detected to be performing a high-security task, the dynamic threshold is fixed at a preset high security level. When the continuous authentication score is lower than the dynamic threshold, a protection response is executed directly without a warning stage. The protection response includes cutting off the network connection and destroying the temporary decryption key in memory.

10. A smart tablet multimodal biometric continuous authentication system, characterized in that, The continuous authentication method for multimodal biometrics of a smart tablet is implemented using any one of the claims 1-9.

Images