An identity authentication method and system based on dynamic gesture physiological behavior fine-grained spatiotemporal feature analysis

Abstract

The application provides an identity authentication method and system based on dynamic gesture physiological behavior fine-grained spatiotemporal feature analysis. The method comprises: inputting dynamic gesture video data of a user into a gesture fine-grained spatiotemporal feature analysis network to obtain a final identity feature; in a registration link, the identity feature is used as a registered gesture feature; in an authentication link, the identity feature is used as a gesture feature to be authenticated; and based on the gesture feature to be authenticated and the registered gesture feature, an identity authentication result is output. The application can remove redundant information while retaining rich identity features through differentiated data clipping. The physiological analysis branch and the behavior analysis branch are adopted, time and space resolution optimization and cross-frame information integration are used to obtain more complete and stable physiological and behavior representations, then adaptive feature fusion is used to weaken the influence of feature amplitude difference on the fusion result, so that the fusion result can fully consider the reliability of physiological and behavior features, and the distinguishability of the final identity representation is improved.

Description

Technical Field

[0001] This invention belongs to the field of identity authentication, and specifically relates to an identity authentication method and system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior. Background Technology

[0002] Identity authentication technology is widely used in scenarios such as mobile terminal login, access control, financial payment, and network access control. Existing identity authentication methods mainly include password authentication, token authentication, and biometric authentication. Among these, biometric authentication has become an important development direction due to its convenience and individual variability. While static biometric recognition methods such as face, fingerprint, and iris recognition are relatively mature, they suffer from the problems of being easily collected, copied, or forged. In contrast, dynamic gestures, during execution, simultaneously incorporate physiological features such as palm print and hand shape, as well as behavioral features such as movement trajectory and rhythm, reflecting user identity from multiple complementary dimensions. Because it contains more information, authentication is more accurate. Since stealing dynamic gestures requires simultaneously replicating all physiological and behavioral features, gesture authentication is theoretically more secure. Finally, dynamic gestures are a privacy-friendly biometric feature, and the operation is in-flight, making authentication natural and hygienic, thus making them more easily accepted by the public and gradually becoming a promising identity authentication method.

[0003] From the perspective of perception methods, existing dynamic gesture authentication technologies can be broadly divided into two categories: non-visual methods and visual methods.

[0004] Non-visual methods primarily utilize 3D sensors, touchscreens, electromyography (EMG) signals, or electromagnetic / wireless signals to collect user gesture information and then use this information for identity verification. These methods mainly focus on modeling behavioral characteristics, but due to the limited dimensionality of sensor measurements, they often struggle to fully capture physiological information such as hand texture, contours, and appearance details that reflect individual differences, thus resulting in incomplete identity information.

[0005] Visual methods acquire gesture images or videos through cameras and can be further divided into trajectory analysis-based methods and video understanding-based methods. Trajectory analysis-based methods typically first use gesture pose estimation algorithms to extract hand keypoints, and then perform authentication by analyzing the trajectory of these keypoints. While this type of method can describe motion dynamics, it loses a significant amount of original appearance information during keypoint extraction, thus underutilizing physiological features such as palm texture and hand shape. Furthermore, its recognition performance is limited by the accuracy and density of keypoint localization. In contrast, video understanding-based methods can directly model gesture videos, jointly extracting physiological and behavioral features from the original video, thus possessing stronger representational capabilities and becoming an important research direction for current visual gesture authentication.

[0006] Existing dynamic gesture authentication methods based on video understanding mainly revolve around three aspects: network structure design, behavioral pseudo-modal construction, and loss function optimization. Firstly, regarding network structure, existing methods employ 3D convolutional networks, 2D convolutional networks, two-stream networks, and some general video understanding models for dynamic gesture identity modeling. Secondly, in terms of behavioral modeling, existing methods highlight dynamic action information by constructing optical flow, temporal difference maps, energy maps, supergraphs, and various behavioral pseudo-modals. Thirdly, regarding training objectives, existing methods use metric learning loss or classification loss to constrain the distribution of identity features, with classification loss being more widely used due to its simplicity and superior performance.

[0007] However, based on existing technologies, it can be seen that current dynamic gesture authentication schemes still have the following shortcomings: 1. Existing methods generally place more emphasis on temporal resolution while paying insufficient attention to spatial resolution: The physiological and behavioral features in dynamic gestures are essentially fine-grained spatiotemporal information, but existing methods often emphasize preserving frame rate or expanding the temporal receptive field, neglecting the importance of spatial details. This leads to strong spatial downsampling in the network front-end. This weakens or even loses key identity information such as palm texture, hand edges, and local action details in the early stages of feature extraction, which is not conducive to building highly discriminative identity representations.

[0008] 2. Existing physiological feature modeling methods do not adequately utilize complementary information across frames: To reduce redundancy, some methods often employ fixed-interval frame extraction or sparse sampling strategies to process physiological branch inputs. While this reduces repetitive information between adjacent frames to some extent, it also easily overlooks complementary details between adjacent frames. For example, in cases of motion blur or local occlusion, different frames may retain different local texture or shape information. If only simple frame extraction is used, these fine-grained physiological cues may be lost, affecting the accuracy of identity recognition.

[0009] 3. Existing methods for constructing pseudo-modalities of behavior are still unsatisfactory in terms of temporal information transfer, cross-frame information preservation, and feature granularity. Existing methods often employ optical flow, temporal difference, energy maps, or channel stitching to highlight dynamic behavioral information. However, these methods often suffer from insufficient adaptability, loss of spatial detail, inadequate preservation of cross-frame behavioral information, and unreasonable behavioral feature granularity. For example, early pixel-level optical flow or difference methods are prone to losing important identity-related information due to limited expressive power or the need for improved modality extraction algorithm performance. Some pseudo-modalities, such as energy maps, while reducing data volume, suffer from low-quality temporal information transfer and loss of motion sequence information, thus making it difficult to form stable and effective behavioral feature representations.

[0010] 4. Existing methods for fusing physiological and behavioral information cannot accurately reflect the reliability of different branch features. Two-branch models typically require weighted fusion of physiological and behavioral features. However, existing fusion methods are often affected by differences in the amplitude of branch features during weight estimation, making it difficult to truly reflect the reliability of different features under different samples. This leads to unreasonable allocation of fusion weights and affects the final fusion effect of physiological and behavioral features.

[0011] In summary, while existing technologies have achieved dynamic gesture authentication to a certain extent, they still suffer from shortcomings such as insufficient preservation of spatial details, inadequate utilization of physiological cross-frame complementary information, limited ability to express behavioral pseudomodals, and insufficient adaptability of multi-branch fusion. These limitations make it difficult to fully extract the fine-grained spatiotemporal identity information contained in the physiological and behavioral features of dynamic gestures. Therefore, it is necessary to propose a novel identity authentication method and system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior to improve the accuracy and robustness of dynamic gesture identity authentication. Summary of the Invention

[0012] To address at least one of the problems existing in dynamic gesture identity authentication technology, such as insufficient preservation of spatial detail information, inadequate utilization of complementary information across frames of physiological features, limited ability to express behavioral pseudomodalities, and unreasonable weight allocation for physiological behavior information fusion, this invention provides an identity authentication method and system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior, so as to achieve high-precision and robust authentication of user identity.

[0013] In response to the fact that dynamic gestures contain both physiological and behavioral features, a new identity authentication framework is constructed for differential analysis of these two types of features. This framework enables more targeted, fine-grained spatiotemporal modeling of physiological and behavioral features, and on this basis, more effective identity information fusion is achieved, thereby improving authentication performance under various authentication modes such as system-defined gestures, user-defined gestures, and random gestures.

[0014] This invention provides an identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior, comprising the following steps: The dynamic gesture video data of the user to be authenticated is input into the gesture fine-grained spatiotemporal feature analysis network to obtain identity features. This network includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module. The data trimming module performs differentiated data trimming on the input dynamic gesture video to obtain a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. The physiological feature analysis branch performs feature-level temporal analysis on the first video representation to extract physiological features. The behavioral feature analysis branch extracts behavioral pseudomodal highlighting motion information from the second video representation and then obtains behavioral features through pseudomodal analysis. The adaptive physiological-behavioral information fusion module performs reliable adaptive fusion of physiological and behavioral features to generate the final identity features. In the registration process, the final identity features serve as the registration gesture features; in the authentication process, the final identity features serve as the gesture features to be authenticated. The authentication result is output based on the distance or similarity between the gesture features to be authenticated and the gesture features registered.

[0015] This invention provides an identity authentication system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior, comprising the following units: The identity feature authentication unit is used to input the dynamic gesture video data of the user to be authenticated into the gesture fine-grained spatiotemporal feature analysis network to obtain identity features. The gesture fine-grained spatiotemporal feature analysis network includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module. The data trimming module performs differential data trimming on the input dynamic gesture video to obtain a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. The physiological feature analysis branch performs feature-level temporal analysis on the first video representation to extract physiological features. The behavioral feature analysis branch extracts behavioral pseudomodal highlighting motion information from the second video representation and then obtains behavioral features through pseudomodal analysis. The adaptive physiological-behavioral information fusion module performs reliable adaptive fusion of physiological and behavioral features to generate the final identity features. In the registration stage, the final identity features serve as the registration gesture features; in the authentication stage, the final identity features serve as the gesture features to be authenticated. The identity authentication unit is used to output the identity authentication result based on the distance or similarity between the gesture features to be authenticated and the registered gesture features.

[0016] The present invention also provides a computer device.

[0017] The present invention also provides a computer-readable storage medium.

[0018] Compared with the prior art, the beneficial effects that the present invention can achieve include: 1. It can more fully extract fine-grained identity features from dynamic gestures. This invention addresses the problem of excessive spatial downsampling and severe loss of fine-grained spatiotemporal information in existing technologies. It slows down the spatial downsampling rate in both physiological and behavioral branches, preserving higher feature map spatial resolution. This approach more effectively preserves fine-grained identity-related information such as palm texture, hand shape, edge contours, and local motion details, thereby improving the discriminative power of identity features.

[0019] 2. It can fully utilize the temporal complementarity of dynamic gesture physiological characteristics to improve the quality of physiological characteristics. This invention departs from the simple temporal downsampling method that directly discards a large number of adjacent frames. Instead, it introduces an adaptive temporal sampling method during physiological feature analysis, which reduces temporal redundancy while preserving and integrating complementary information from adjacent frames. Thus, even if some frames suffer from motion blur, partial occlusion, or missing details, a more complete physiological identity representation can be obtained through cross-frame information integration, thereby improving the accuracy and stability of physiological feature extraction.

[0020] 3. It can enhance the expressive ability and stability of behavioral characteristics. This invention constructs a behavioral pseudomodal representation more suitable for dynamic gesture authentication by performing grayscale conversion, grouping, intra-group temporal sharpening, and feature-level inter-frame adaptive analysis on the input video. This representation method can more effectively highlight motion information, enhance the ability to retain cross-frame behavioral information, and give behavioral features a more reasonable temporal granularity while reducing data redundancy and computational burden. Therefore, this invention can obtain a more stable and effective representation of behavioral identity features.

[0021] 4. It can improve the quality of the integration of physiological and behavioral characteristics. This invention employs an adaptive fusion mechanism oriented towards feature reliability during the fusion stage. By normalizing and scaling physiological and behavioral features and introducing a reliability-aware surrogate vector, it models the discrimination quality and information complementarity of the two types of features, thereby adaptively determining the fusion weights. Compared to methods that generate weights solely based on ordinary mapping relationships, this invention can also reduce the impact of feature amplitude differences (branch bias) on the fusion result, ensuring that the fusion result fully considers the reliability of physiological and behavioral features, and improving the identifiability of the final identity representation.

[0022] 5. It can improve the accuracy, robustness, and generalization ability of identity authentication. This invention improves the system from data trimming and branch feature extraction to information fusion, forming a complete fine-grained spatiotemporal identity feature analysis scheme. This scheme can more fully mine multi-dimensional identity information in dynamic gestures, thus improving the accuracy of identity authentication results. Furthermore, because this invention considers both physiological and behavioral features, local details and temporal dynamics, and feature quality and fusion reliability, it also has better environmental and scenario adaptability, achieving optimal performance under various dynamic gesture identity authentication modes, including system-defined gestures, user-defined gestures, and random gestures. Attached Figure Description

[0023] Figure 1 This is a schematic diagram illustrating the steps of an identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior in an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of the overall structure of FSHA-Net in an embodiment of the present invention. C1 represents Conv1 in ResNet, and LxBy represents the y-th block of the x-th ResNet layer. TC represents temporal convolution, and TM represents max temporal pooling. Norm represents L2 normalization. L1-L3 correspond to three loss functions.

[0025] Figure 3 This is a schematic diagram of the information fusion module in an embodiment of the present invention. RProxies represents the learnable reliability-aware agent vector. This indicates a shared scaling factor. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] This invention provides an identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gestures and physiological behaviors. This method addresses the characteristic that dynamic gestures simultaneously contain both physiological and behavioral features. It employs a combination of differentiated data processing and dual-branch feature extraction to decouple and model the two types of identity information in the input gesture video. Based on feature reliability, it adaptively fuses physiological and behavioral features to generate the final identity representation. The overall technical solution comprises four parts: data trimming, physiological feature analysis, behavioral feature analysis, and physiological-behavioral information fusion. The model used is abbreviated as Fine-Grained Spatiotemporal Hand Gesture Feature Analysis Network (FSHA-Net), which includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module.

[0028] The “physiological characteristics” mainly refer to static appearance information in dynamic gesture videos that can reflect individual differences in identity, including but not limited to palm texture, hand shape, contour structure and local details; the “behavioral characteristics” mainly refer to dynamic identity information reflected in the execution of dynamic gestures, including but not limited to dynamic characteristics with user characteristics such as movement rhythm, temporal change pattern and exercise habits.

[0029] Unlike methods that directly model dynamic gesture videos as a single input, this invention uses a two-stream convolutional neural network (including a physiological feature analysis branch and a behavioral feature analysis branch). First, based on the differences in information attributes between physiological and behavioral features, targeted data cropping is performed on the input video, converting the original gesture video into a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. Then, the first video representation is input into the physiological feature analysis branch, and the second video representation is input into the behavioral feature analysis branch to extract physiological and behavioral features respectively. Finally, an adaptive physiological-behavioral information fusion module is used to perform weighted fusion of the two types of features to obtain an identity representation for identity determination. The identity authentication result is output by measuring similarity with the registered gesture features.

[0030] In detail, the identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior provided by the embodiments of the present invention includes the following steps: S1. Obtain the dynamic gesture video data of the user to be authenticated.

[0031] Video data of the dynamic gesture process performed by the user to be authenticated is collected and used as the input gesture video. The input gesture video consists of multiple consecutive frames. The input gesture video can be represented as:

[0032] in, This indicates the input gesture video. Indicates the number of video frames. Indicates the number of channels. Indicates frame height, Indicates the frame width.

[0033] S2. Perform differential data cropping on the input gesture video to generate a first video representation for physiological feature analysis and a second video representation for behavioral feature analysis.

[0034] Specifically, regarding the physiological feature analysis path, unlike existing methods that perform fixed-interval temporal downsampling of the input video at the data level, this embodiment of the invention first retains the original temporal sequence information by removing the fixed-interval temporal downsampling at the data level, thereby obtaining the first video representation for subsequent physiological feature analysis. In the subsequent feature extraction stage, feature-level temporal analysis is used to achieve redundancy removal and complementary information integration, thus balancing the analysis quality and efficiency of physiological features.

[0035] For the behavioral feature analysis path, the input gesture video is first converted to grayscale, and then non-overlapping groups are formed according to a preset number of frames. Each group contains Frame image. The grayscale and grouped video can be represented as:

[0036] in, This represents the video after being grayscaled and grouped. Indicates the number of groups. Indicates the number of frames in each group, and satisfies , Represents a grouping function. This indicates the input gesture video. , , , , .

[0037] Intra-group temporal sharpening is performed on each group of images to enhance dynamic change information and preserve spatial details, resulting in:

[0038] The second video representation obtained after processing As input representation for behavioral feature analysis path. In one embodiment, when When =4, the output dimension is .

[0039] S3. Input the first video representation into the physiological feature analysis branch (physiological branch), perform feature-level temporal analysis to achieve redundancy removal and complementary information integration, and on this basis, efficiently extract physiological features.

[0040] The physiological feature analysis branch uses a two-dimensional residual network as its backbone. The original backbone mainly includes Conv1, MaxPool, Layer1 (L1), Layer2 (L2), Layer3 (L3), Layer4 (L4), and a fully connected layer (FC). Each layer contains two convolutional modules, Block1 (B1) and Block2 (B2). L1B1 in the figure is Block1 of Layer1. To preserve richer physiological textures and shape details, this embodiment of the invention proposes to improve the spatial resolution of the physiological feature map. To this end, the spatial max-pooling layer MaxPool after the first convolutional layer Conv1 of the backbone network is removed to slow down the rate of decrease in feature map resolution. Figure 2 The remaining ResNet components in the physiological branch include Layer 2 (L2), Layer 3 (L3), Layer 4 (L4), and fully connected layers (FC).

[0041] Furthermore, considering that high frame rate videos contain both a large amount of redundant information and a small amount of inter-frame complementary information for behavioral feature analysis, this embodiment of the invention sets a first inter-frame adaptive analysis layer after Conv1. This first inter-frame adaptive analysis layer includes a parallel max-temporal pooling (TM) layer and a temporal convolutional (TC) layer. The TM layer is used to extract significant physiological patterns across frames to avoid sampling video frames with unclear physiological features (e.g., gesture self-occlusion and motion blur) at fixed intervals. The TC layer is used to further extract the inter-frame complementary physiological information missed by the TM layer. The outputs of the TM layer and the TC layer are summed to serve as the output of the adaptive temporal sampling layer (i.e., the first inter-frame adaptive analysis layer). The kernel size of both the TM layer and the TC layer is [missing information]. The step size is , Let be the size of the convolution kernel in the temporal dimension. In one embodiment, the kernel size of the adaptive temporal sampling layer is . Step size is .

[0042] Furthermore, a second inter-frame adaptive analysis layer is set within Layer 1, specifically between Block 1 (B1) and Block 2 (B2), to further integrate complementary physiological information across frames. Considering that the main temporal redundancy has been removed after processing by the adaptive temporal sampling layer, further feature compression would lead to the loss of physiological information; therefore, the second inter-frame adaptive analysis layer employs a temporal convolutional layer with residual connections. In one embodiment, the convolutional kernel of the inter-frame adaptive analysis layer... Size is Step length for .

[0043] Figure 2 The remaining ResNet components in the behavior branch include Layer 3 (L3), Layer 4 (L4), and fully connected layers (FC).

[0044] S4. Input the second video representation into the behavior feature analysis branch (behavior branch), extract the behavior pseudomodal to highlight motion information, and further extract behavior features based on this.

[0045] Specifically, the behavioral feature analysis branch also uses a two-dimensional residual network as its backbone network; to preserve richer spatiotemporal motion details, this embodiment of the invention proposes to improve the spatial resolution of the behavioral feature map. To this end, the spatial max-pooling layer after the first convolutional layer Conv1 of the backbone network is removed to slow down the rate of decrease in feature map resolution.

[0046] Furthermore, a third inter-frame adaptive analysis layer is set within Layer 1, specifically within Blocks 1 (B1) and 2 (B2), and a fourth inter-frame adaptive analysis layer is set within Layer 2. Both the third and fourth inter-frame adaptive analysis layers employ temporal convolutional layers with residual connections to integrate the second video representation. The inter-group behavioral information is extracted and the temporal receptive field of the features is expanded. By mitigating the decrease in feature map resolution, the two inter-frame adaptive analysis layers can integrate the behavioral information between adjacent temporal features under conditions of higher spatial resolution and larger spatial receptive field, thereby constructing an enhanced behavioral pseudomodality that can better represent stable local behavioral features: a temporal sharpened behavior map (TSBehvMap). In one embodiment, the temporal convolutional kernel size in the third inter-frame adaptive analysis layer is [size missing]. The temporal convolution kernel size in the fourth inter-frame adaptive analysis layer is Both have a step size of 1.

[0047] S5. The physiological and behavioral features are reliably adaptively fused through the adaptive physiological-behavioral information fusion module to generate the final identity features.

[0048] Specifically, first, regarding physiological characteristics and behavioral characteristics Normalization was performed separately to obtain the normalized physiological characteristics. and normalized behavioral characteristics Then utilize the shared scaling factor Scaling of normalized features.

[0049] Furthermore, four reliability-aware proxy vectors corresponding to the feature dimensions are introduced, namely: , , and .in and Used to assess the reliability of physiological characteristics and Used to assess the reliability of behavioral characteristics. More specifically, Used to assess the identifiability of a physiological characteristic itself. Used to analyze the feature redundancy contained in behavioral characteristics from the perspective of physiological characteristics. Used to assess the identifiability of behavioral characteristics themselves. Used to analyze the feature redundancy contained in physiological characteristics from the perspective of behavioral characteristics.

[0050] Then, the reliability of the two types of features is evaluated using the cosine similarity between physiological and behavioral features and their corresponding reliability-perceived proxy vectors, and the fusion weights are obtained:

[0051]

[0052] in These are the fusion weights assigned to physiological characteristics. These are the fusion weights assigned to behavioral features. This indicates that the vector is being normalized. This represents the transpose of a vector.

[0053] Regarding physiological characteristics, larger Indicates higher quality of physiological characteristics, larger This indicates a high degree of information repetition, that is, it indicates a physiological characteristic From a perspective assessment standpoint, behavioral features can provide limited complementary information. Therefore, when both of the above conditions are met, physiological features need to be assigned a larger fusion weight. For behavioral features, a larger fusion weight is required. Indicates higher quality behavioral characteristics, larger This indicates a high degree of information repetition, meaning it reflects behavioral characteristics. From a perspective assessment standpoint, physiological characteristics can provide limited complementary information. Therefore, when both of the above conditions are met, behavioral characteristics need to be assigned greater fusion weights.

[0054] Finally, the physiological and behavioral features are weighted and concatenated based on the calculated fusion weights to obtain the final identity features. :

[0055] Among them, the larger ones This indicates that the physiological characteristics of the current sample are more reliable and larger. This indicates that the behavioral characteristics of the current sample are more reliable.

[0056] In the registration process, the user's gesture video data is input into the gesture fine-grained spatiotemporal feature analysis network, and the final identity features obtained are used as the registration gesture features and stored. In the authentication process, the final identity features obtained through the gesture fine-grained spatiotemporal feature analysis network are used as the gesture features to be authenticated.

[0057] The gesture fine-grained spatiotemporal feature analysis network FSHA-Net is mainly trained using classification-based loss functions, such as AMSoftmax (Wang, Feng, Jian Cheng, Weiyang Liu, and Haijun Liu. "Additivemargin softmax for face verification." IEEE Signal Processing Letters 25, no.7 (2018): 926-930), QamFace (Zhao, He, Yongjie Shi, Xin Tong, Xianghua Ying, and Hongbin Zha. "Qamface: Quadratic additive angular margin loss for face recognition." In 2020 IEEE international conference on image processing (ICIP), pp. 1901-1905. IEEE, 2020), and SphereFace (Liu, Weiyang, Yandong Wen, Bhiksha Raj, Rita Singh, and Adrian Weller. "Sphereface revived: Unifying hyperperspherical face recognition." IEEE Transactions on Pattern Analysis and Machine Intelligence 45, no. 2 (2022): 2458-2474), ArcFace (Deng, Jiankang,Jia Guo, Niannan Xue, and Stefanos Zafeiriou. "Arcface: Additive angularmargin loss for deep face recognition." In Proceedings of the IEEE / CVFconference on computer vision and pattern recognition, pp. 4690-4699.2019), L3AM (Song, Wenwei, Wenxiong Kang, Adams Wai-Kin Kong, Yufeng Zhang, andYitao Qiao. "L3AM: Linear adaptive additive angular margin loss for video-based hand gesture authentication." International Journal of Computer Vision132, no. 9 (2024): 4073-4090.), etc. loss function. Used for training physiological branches, loss function Loss function used for training behavioral branches Used for training an adaptive physiological-behavioral information fusion module. Determined through experiments. , , The specific type of loss function.

[0058] S6. Output the identity authentication result based on the distance or similarity between the gesture features to be authenticated and the registered gesture features.

[0059] The similarity between the identity feature to be authenticated obtained in step S5 and the registered gesture feature is calculated and compared with a preset threshold to complete the authentication. Specifically, if the similarity is lower than the preset threshold, the authentication fails; otherwise, the authentication succeeds. In one embodiment, cosine similarity is used to evaluate the similarity between the identity feature to be authenticated and the registered gesture feature.

[0060] In one embodiment, the identity authentication result may be identity approved, identity rejected, or a similarity score between the identity and a registered identity feature.

[0061] To facilitate understanding of the technical solutions of the embodiments of the present invention, some terms are explained as follows: (1) Data trimming This refers to performing targeted preprocessing (grayscale conversion, grouping, sharpening, etc.) on the input gesture video according to the needs of different feature analysis tasks, so that the resulting derived video (first video representation and second video representation) is more suitable for subsequent feature extraction branches.

[0062] (2) Physiological characteristics analysis branch This is a feature extraction network branch used to extract static appearance identity information from dynamic gestures. Its main processing object is the first video representation oriented towards physiological feature analysis.

[0063] (3) Behavioral Feature Analysis Branch This is a feature extraction network branch used to extract dynamic behavioral identity information from dynamic gestures. Its main processing object is the second video representation oriented towards behavioral feature analysis.

[0064] (4) Adaptive physiological-behavioral information fusion module This refers to a module that adaptively determines the fusion weights of the two types of features based on the reliability of physiological and behavioral characteristics, and then splices them together to output the final identity features.

[0065] In one embodiment, for ease of comparison and analysis, the method proposed in this embodiment is denoted as FSHA-Net. First, to demonstrate the significant performance improvement of FSHA-Net, this embodiment compares FSHA-Net with nine cutting-edge gesture authentication models based on the SCUT-DHGA dataset and the AMSoftmax loss function. The experiments cover three gesture authentication modes: system-defined gesture authentication, user-defined gesture authentication, and random gesture authentication. The experimental results are as follows: Table 1: Comparison of FSHA-Net with Nine Cutting-Edge Models

[0066] The experimental results are the average error rates (%) for the six test protocols (lower is better). As shown in Table 1, the method proposed in this embodiment of the invention exhibits the best authentication performance across all three authentication modes.

[0067] To verify the generalization ability of the proposed method, this embodiment of the invention also conducted system-defined gesture authentication experiments on two other datasets with complex backgrounds and lighting (SCUT-DHGA-br and SCUT-RealDHGA). The experimental results are as follows: Table 2: Comparison of FSHA-Net with other representative models on two more challenging datasets

[0068] The experimental results are the average of the equal error rates (%) for the six test protocols (lower is better). The experimental results in Table 2 show that FSHA-Net of the present invention can consistently and stably obtain the lowest authentication equal error rate, thus proving that the present invention has better authentication generalization.

[0069] Finally, to verify the training robustness of the method in this embodiment under different loss functions and to explore the upper limit of gesture authentication performance, in one embodiment, five classification-based loss functions—AMSoftmax, QamFace, SphereFace, ArcFace, and L3AM—were used for model training. Under system-defined gesture authentication, the model was compared with PB-Net, which showed the second-best performance in Table 1. The experimental results are as follows: Table 3: Performance Comparison of FSHA-Net and PB-Net under Five Loss Functions

[0070] The experimental results are the average of the equal error rates (%) for the six test protocols (lower is better). The experimental results in Table 3 show that the FSHA-Net embodiment of the present invention can maintain optimal performance under different loss functions, and achieves the lowest authentication equal error rate of 0.159% under L3AM loss, thus proving that the embodiment of the present invention has better training robustness.

[0071] Compared with existing dynamic gesture authentication technologies, in this embodiment of the invention: (1) A differentiated fine-grained spatiotemporal analysis scheme for physiological and behavioral features is proposed: To address the different modeling needs of physiological and behavioral features in dynamic gestures, the data preprocessing paradigm is improved. On the one hand, an adaptive temporal sampling method for physiological feature analysis is introduced; on the other hand, a behavior-enhanced pseudomodal construction method for behavioral feature analysis is proposed. Simultaneously, the spatial resolution of the feature map is improved to provide sufficient spatiotemporal detail information for physiological and behavioral feature analysis. Ultimately, richer fine-grained identity information is retained while significantly reducing redundant information.

[0072] (2) Improve the physiological feature analysis branch, behavioral feature analysis branch, and adaptive physiological-behavioral information fusion module in a targeted manner to enhance the ability of identity feature extraction and fusion. By upgrading the physiological and behavioral branches, they are made more suitable for extracting fine-grained spatiotemporal identity information from dynamic gestures; at the same time, the information fusion module is improved so that the model can allocate fusion weights according to feature reliability, thereby improving the quality of the final identity representation.

[0073] (3) It has good authentication performance, generalization and training robustness. The scheme has shown good results on multiple datasets and with various loss functions. It can be applied to different scenarios such as system-defined gestures, user-defined gestures and random gestures, and has strong promotion and application value.

[0074] In one embodiment, an identity authentication system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior is provided to implement the method described in the foregoing embodiment. The system includes the following units: The data acquisition unit is used to acquire dynamic gesture video data of the user to be authenticated. The identity feature authentication unit is used to input the dynamic gesture video data of the user to be authenticated into the gesture fine-grained spatiotemporal feature analysis network to obtain identity features. The gesture fine-grained spatiotemporal feature analysis network includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module. The data trimming module performs differential data trimming on the input dynamic gesture video to obtain a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. The physiological feature analysis branch performs feature-level temporal analysis on the first video representation to extract physiological features. The behavioral feature analysis branch extracts behavioral pseudomodal highlighting motion information from the second video representation and then obtains behavioral features through pseudomodal analysis. The adaptive physiological-behavioral information fusion module performs reliable adaptive fusion of physiological and behavioral features to generate the final identity features. In the registration stage, the final identity features serve as the registration gesture features; in the authentication stage, the final identity features serve as the gesture features to be authenticated. The identity authentication unit is used to output the identity authentication result based on the distance or similarity between the gesture features to be authenticated and the registered gesture features.

[0075] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the methods described in the foregoing embodiments.

[0076] In one embodiment, a computer-readable storage medium is provided, the computer-readable storage medium storing a computer program that, when executed by a processor, implements the methods described in the foregoing embodiments.

[0077] This invention, based on the modeling requirements of gesture physiological and behavioral features, removes redundant information while retaining rich identity features through differentiated data pruning. Furthermore, it specifically designs physiological and behavioral analysis branches, obtaining more complete and stable physiological and behavioral representations through spatiotemporal resolution optimization and cross-frame information integration. Finally, an adaptive feature fusion module is used to mitigate the impact of feature amplitude differences on the fusion result, ensuring that the fusion result fully considers the reliability of physiological and behavioral features and improves the recognizability of the final identity representation. Experiments show that the proposed method effectively improves the accuracy and robustness of dynamic gesture identity authentication.

[0078] The device and medium described herein have the same technical effects as those achieved by the method described in the foregoing embodiments.

[0079] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior, characterized in that, Includes the following steps: The dynamic gesture video data of the user to be authenticated is input into the gesture fine-grained spatiotemporal feature analysis network to obtain identity features. This network includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module. The data trimming module performs differentiated data trimming on the input dynamic gesture video to obtain a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. The physiological feature analysis branch performs feature-level temporal analysis on the first video representation to extract physiological features. The behavioral feature analysis branch extracts behavioral pseudomodal highlighting motion information from the second video representation and then obtains behavioral features through pseudomodal analysis. The adaptive physiological-behavioral information fusion module performs reliable adaptive fusion of physiological and behavioral features to generate the final identity features. In the registration process, the final identity features serve as the registration gesture features; in the authentication process, the final identity features serve as the gesture features to be authenticated. The authentication result is output based on the distance or similarity between the gesture features to be authenticated and the gesture features registered.

2. The identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to claim 1, characterized in that, The differentiated data cropping includes: By removing fixed-interval temporal downsampling at the data level to preserve the original temporal sequence information, a first video representation for subsequent physiological feature analysis is obtained. The input dynamic gesture video is grayscaled and grouped, and intra-group temporal sharpening is performed on each group of images to enhance dynamic change information and preserve spatial details, thus obtaining a second video representation.

3. The identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to claim 1, characterized in that, The physiological feature analysis branch uses a two-dimensional residual network as the backbone network. The original backbone network includes Conv1, MaxPool, Layer1, Layer2, Layer3, Layer4, and a fully connected layer. In the physiological feature analysis branch, MaxPool is removed, and a first inter-frame adaptive analysis layer is set between Conv1 and Layer1. The first inter-frame adaptive analysis layer includes a parallel max-temporal pooling layer and a temporal convolutional layer. The max-temporal pooling layer is used to extract significant physiological patterns across frames, and the temporal convolutional layer extracts complementary physiological information across frames missed by max pooling. A second inter-frame adaptive analysis layer is set inside Layer1 to integrate complementary physiological information across frames.

4. The identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to claim 1, characterized in that, The behavioral feature analysis branch uses a two-dimensional residual network as the backbone network. The original backbone network includes Conv1, MaxPool, Layer1, Layer2, Layer3, Layer4 and a fully connected layer. In the behavioral feature analysis branch, MaxPool is removed, and inter-frame adaptive analysis layers are set in both Layer1 and Layer2 to integrate the inter-group behavioral information of the second video representation and expand the feature temporal receptive field.

5. The identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to claim 1, characterized in that, The process of adaptively fusing physiological and behavioral characteristics to generate the final identity features includes: The physiological and behavioral characteristics were normalized separately to obtain the normalized physiological and behavioral characteristics. The normalized physiological and behavioral characteristics are scaled using a shared scaling factor. A reliability-aware proxy vector corresponding to the feature dimension is introduced to evaluate the reliability of physiological features and the reliability of behavioral features. The reliability of a feature is evaluated by using the cosine similarity between the feature and its corresponding reliability-aware proxy vector, and the fusion weight is obtained. The physiological and behavioral features are weighted and concatenated according to the fusion weights to obtain the final identity features.

6. An identity authentication method based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to any one of claims 1-5, characterized in that, The similarity between the gesture features to be authenticated and the registered gesture features is calculated and compared with a preset threshold to complete the authentication.

7. An identity authentication system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior, characterized in that, For implementing the method according to any one of claims 1-6, the system comprises the following units: The identity feature authentication unit is used to input the dynamic gesture video data of the user to be authenticated into the gesture fine-grained spatiotemporal feature analysis network to obtain identity features. The gesture fine-grained spatiotemporal feature analysis network includes a data trimming module, a physiological feature analysis branch, a behavioral feature analysis branch, and an adaptive physiological-behavioral information fusion module. The data trimming module performs differential data trimming on the input dynamic gesture video to obtain a first video representation suitable for physiological feature analysis and a second video representation suitable for behavioral feature analysis. The physiological feature analysis branch performs feature-level temporal analysis on the first video representation to extract physiological features. The behavioral feature analysis branch extracts behavioral pseudomodal highlighting motion information from the second video representation and then obtains behavioral features through pseudomodal analysis. The adaptive physiological-behavioral information fusion module performs reliable adaptive fusion of physiological and behavioral features to generate the final identity features. In the registration stage, the final identity features serve as the registration gesture features; in the authentication stage, the final identity features serve as the gesture features to be authenticated. The identity authentication unit is used to output the identity authentication result based on the distance or similarity between the gesture features to be authenticated and the registered gesture features.

8. The identity authentication system based on fine-grained spatiotemporal feature analysis of dynamic gesture physiological behavior according to claim 7, characterized in that, It also includes a data acquisition unit for acquiring dynamic gesture video data of the user to be authenticated.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the method according to any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method described in any one of claims 1-6.

Images