Video-based dynamic gesture authentication method, medium and device

Abstract

The present invention provides a video-based dynamic gesture authentication method, medium, and device. The method comprises the following steps: inputting a dynamic gesture video to be authenticated; generating a multi-scale dynamic gesture hypergraph from the dynamic gesture video; using a dual-stream dynamic gesture authentication network to learn the original RGB image and the multi-scale dynamic gesture hypergraph to obtain an identity feature vector; the dual-stream dynamic gesture authentication network comprises a spatiotemporal feature extraction backbone network, a feature fusion module, and an identity feature aggregation module; the spatiotemporal feature extraction backbone network captures behavioral features and physiological features respectively; the feature fusion module performs feature fusion; and the identity feature aggregation module further processes and integrates the features to generate an identity feature vector; and finally, obtaining a dynamic gesture authentication result. This method utilizes the multi-scale dynamic gesture hypergraph and dual-stream dynamic gesture authentication network technologies to more comprehensively and accurately extract and fuse gesture features, thereby improving the performance and practicality of dynamic gesture authentication.

Description

Technical Field

[0001] The present invention relates to the technical field of dynamic gesture authentication, and more particularly to a video-based dynamic gesture authentication method, medium, and device. Background Art

[0002] Dynamic gestures are a new type of biometric feature that can be used for identity authentication. They contain rich identity information, primarily composed of physiological and behavioral characteristics, and have garnered widespread attention in recent years. Gesture authentication technology significantly enhances the security and convenience of automated identity authentication. It leverages an individual's unique gestures for rapid and accurate identification, improving the security and efficiency of authentication systems and possessing broad value and potential in modern technological applications. The operation of a dynamic gesture authentication system can be summarized into three core steps: model training, user registration, and authentication. In the training phase of the present invention, dynamic gesture images in a database undergo image preprocessing before being input into a convolutional neural network for learning and model building. In the registration phase, newly acquired gesture images undergo image preprocessing. A previously trained dynamic gesture feature extractor is then used to identify and extract key information, forming a user recognition template that is stored in a local database. Finally, in the authentication phase, newly acquired dynamic gesture images undergo image preprocessing and a previously trained feature extractor is used to extract user features. The system then compares the user features with the templates registered in the database. If the gesture features to be authenticated closely match the registered identity features in the database, authentication succeeds; otherwise, authentication fails.

[0003] In the past few years, researchers have conducted extensive explorations on dynamic gesture authentication. Dynamic gesture authentication can be mainly divided into (1) non-vision-based methods and (2) vision-based methods. Non-vision-based methods require users to hold special devices such as inertial measurement units, touch screens, biometric smart pens, or wearable devices to obtain gesture information, which greatly reduces user friendliness. Unlike non-vision-based methods, vision-based methods only require a common single camera to collect user gesture data, which has the advantages of being more convenient, more hygienic, and more efficient, while reducing the risk of contamination by viruses or bacteria.

[0004] Generally speaking, there are two types of vision-based dynamic gesture authentication methods: one based on trajectory analysis and the other based on video understanding. Trajectory analysis-based methods estimate hand key points from videos and then use these motion trajectories to extract identity information. In principle, trajectory-based methods focus primarily on the positional information of hand key points, while ignoring physiological features such as hand shape, skin color, and palm texture. Currently, there is limited work using trajectory analysis for identity authentication. With the development of deep learning methods, research on using convolutional neural networks for video motion analysis has also begun to develop. With the emergence of the largest gesture authentication dataset, the SCUT-DHGA dataset, Liu et al. summarized previous work and proposed a dynamic gesture authentication experimental setup. This is a more comprehensive study of dynamic gesture authentication and lays the foundation for further development in this field.

[0005] Dynamic gestures encompass a rich set of physiological and behavioral characteristics, but current research focuses primarily on effectively extracting and fully utilizing these physiological features. However, mining and utilizing the behavioral features rich in gestures has received insufficient attention. However, effectively extracting behavioral features and integrating them with physiological features would undoubtedly greatly enhance the performance of dynamic gesture authentication. Summary of the Invention

[0006] To overcome the shortcomings and deficiencies in the prior art, the present invention aims to provide a video-based dynamic gesture authentication method, medium, and device. This method utilizes a multi-scale dynamic gesture hypergraph and a dual-stream dynamic gesture authentication network technology to more comprehensively and accurately extract and fuse gesture features, thereby improving the accuracy and efficiency of dynamic gesture authentication.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solution: a video-based dynamic gesture authentication method, comprising the following steps:

[0008] S1. Input the dynamic gesture video to be authenticated;

[0009] S2. Generate a multi-scale dynamic gesture hypergraph from the dynamic gesture video; the multi-scale dynamic gesture hypergraph is composed of multiple dynamic gesture images generated by image frame sequences of different lengths;

[0010] S3. A two-stream dynamic gesture authentication network is used to learn the original RGB images of dynamic gesture videos and the multi-scale dynamic gesture hypergraph to obtain identity feature vectors. The two-stream dynamic gesture authentication network consists of a spatiotemporal feature extraction backbone network, a feature fusion module, and an identity feature aggregation module. The spatiotemporal feature extraction backbone network captures the behavioral features B and physiological features P embedded in the gestures respectively; the feature fusion module fuses the behavioral features B and physiological features P; and the identity feature aggregation module further processes and integrates the features obtained by the spatiotemporal feature extraction backbone network and the feature fusion module to generate a series of highly condensed identity feature vectors.

[0011] S4. Obtain the dynamic gesture authentication result based on the identity feature vector.

[0012] Preferably, in step S2, multiple window sizes T are set; for each window size T, z single-scale dynamic gesture images W are generated respectively. T ; Single-scale dynamic gesture image W T The generation method is: non-repeated sampling of T consecutive frames of original RGB images from the dynamic gesture video sequence, and sequentially generate z single-scale dynamic gesture images W T ;in, N is the number of original RGB image frames of the dynamic gesture video, Indicates floor operation;

[0013] All z single-scale dynamic gesture images W of window size T T Combined, they form a multi-scale dynamic gesture hypergraph.

[0014] Preferably, in step S3, the spatiotemporal feature extraction backbone network is composed of a motion stream and an appearance stream; the motion stream uses a multi-scale dynamic gesture hypergraph as input to capture behavioral clues and obtain behavioral features embedded in the gesture; the appearance stream uses the original RGB image of the dynamic gesture video as input to capture physiological features and obtain physiological features embedded in the gesture;

[0015] The backbone of the motion flow and the appearance flow are both composed of five RepVGG layers; after each RepVGG layer of the motion flow and the appearance flow extracts the behavioral features B and the physiological features P respectively, they are first input into the feature fusion module to fuse the behavioral features B and the physiological features P to obtain the fused behavioral features B' and physiological features P', and then they are respectively input into the next RepVGG layer of the motion flow and the appearance flow for extraction; the execution is repeated until the extraction of the last RepVGG layer of the motion flow and the appearance flow is completed; the behavioral features B and physiological features P extracted by the last RepVGG layer of the motion flow and the appearance flow are input into the feature fusion module for fusion to obtain the final fused behavioral features B' and physiological features P' as the input of the identity feature aggregation module.

[0016] Preferably, in the backbone of the motion flow and the appearance flow, the first RepVGG layer contains one RepVGGBlock block; the second RepVGG layer contains two RepVGGBlock blocks; the third RepVGG layer contains four RepVGGBlock blocks; the fourth RepVGG layer includes fourteen RepVGGBlock blocks; and the fifth RepVGG layer contains one RepVGGBlock block.

[0017] Preferably, during the training phase, the RepVGGBlock block is represented as a multi-branch structure, expressed as y=x+f(x)+g(x); wherein f(x) and g(x) represent convolution branches that perform 1×1 and 3×3 convolution operations, respectively;

[0018] After the training is completed, a structural reparameterization strategy is adopted to convert multiple branch structures into a single linear structure y = h(x); the conversion method of the linear structure y is: by performing a specific linear combination of the trained multi-branch structure parameters, an equivalent single convolution operation h(x) is finally obtained.

[0019] Preferably, the motion flow takes the feature vector d of the multi-scale dynamic gesture hypergraph as input; the feature vector d is calculated as follows:

[0020] For the image frame sequence I1,…,I with a window size of T T , set d to represent the feature vector of the image frame sequence, where p is the length of the eigenvector;

[0021] set up is the image frame sequence I t Extracted features, t = 1, 2, ...; represents the average value of features extracted from the previous t frames of original RGB image;

[0022] Calculate vector d * :

[0023]

[0024] Among them, β t is the image frame sequence I t The fusion coefficient of

[0025] The vector d * Normalize and get the eigenvector d.

[0026] Preferably, in step S3, the feature fusion module sets the weights of the behavioral feature B and the physiological feature P to R1 and R2, respectively, where R1+R2=1; and calculates the fused behavioral feature B' and physiological feature P' respectively:

[0027] B′=BN(B×R1+BN(P)×R2)

[0028] PBN(P)

[0029] Among them, BN is the batch normalization function.

[0030] Preferably, in step S3, the identity feature aggregation module further processes and integrates the features obtained by the spatiotemporal feature extraction backbone network and the feature fusion module through a global average pooling layer and a fully connected layer to generate a series of highly condensed identity feature vectors.

[0031] A readable storage medium stores a computer program, which, when executed by a processor, enables the processor to perform the video-based dynamic gesture authentication method.

[0032] A computer device includes a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the video-based dynamic gesture authentication method is implemented.

[0033] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0034] (1) More comprehensive extraction of gesture features: Unlike previous methods that focus more on the physiological characteristics of dynamic gestures, this invention uses a multi-scale dynamic gesture hypergraph to fully capture the behavioral and physiological characteristics of gestures. This feature extraction method can more accurately authenticate user identities.

[0035] (2) Improved authentication accuracy and real-time performance: Through a dual-stream dynamic gesture authentication network, the present invention effectively extracts and integrates the physiological and behavioral characteristics of dynamic gestures, further improving the accuracy and real-time performance of dynamic gesture authentication. Compared with traditional methods, the method of the present invention has higher recognition accuracy and faster authentication speed.

[0036] (3) Improved ability to cope with complex environments: The generation of a multi-scale dynamic gesture hypergraph takes into account the combination of different gesture image frame sequences, making gesture features richer and less affected by background clutter. This makes the present invention more capable of coping with authentication requirements in complex environments, for example, maintaining high authentication performance in cases of uneven lighting and complex backgrounds.

[0037] (4) Improved user experience: Compared with traditional non-visual methods, this invention adopts a visual method that only requires a common camera to collect gesture data, avoiding the inconvenience of users wearing special equipment. This design improves user friendliness and comfort;

[0038] (5) Promotes the development of biometric technology: The technical solution of the present invention has promoted the development of dynamic gesture authentication. By more comprehensively extracting gesture features and effectively integrating physiological and behavioral characteristics, the present invention provides new ideas and solutions for the application of biometric technology in the field of identity authentication.

[0039] These advantages and positive effects are mainly due to the fact that the present invention adopts advanced multi-scale dynamic gesture hypergraph and dual-stream dynamic gesture authentication network technology to extract and fuse gesture features more comprehensively and accurately, thereby improving the performance and practicality of dynamic gesture authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 Schematic diagram of dynamic gesture image generation in the video-based dynamic gesture authentication method of the present invention;

[0041] Figure 2(a) to Figure 2(f) They are respectively generating six different types of dynamic gesture images according to the video-based dynamic gesture authentication method of the present invention;

[0042] Figure 3 Schematic diagram of the structure of the dual-stream dynamic gesture authentication network of the video-based dynamic gesture authentication method of the present invention;

[0043] Figure 4 Schematic diagram of the motion flow / appearance flow backbone network structure of the video-based dynamic gesture authentication method of the present invention;

[0044] Figure 5 Schematic diagram of a feature fusion module of the video-based dynamic gesture authentication method of the present invention;

[0045] Figure 6 Schematic diagram of the identity feature aggregation module of the video-based dynamic gesture authentication method of the present invention. DETAILED DESCRIPTION

[0046] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0047] Example 1

[0048] This embodiment provides a video-based dynamic gesture authentication method, including the following steps:

[0049] S1. Input the dynamic gesture video to be authenticated.

[0050] S2. Generate a multi-scale dynamic gesture hypergraph from the dynamic gesture video; the multi-scale dynamic gesture hypergraph is composed of multiple dynamic gesture images generated by gesture image frame sequences of different lengths.

[0051] This paper attempts to study the impact of various dynamic gesture image modalities on gesture authentication for the first time. Based on the original RGB images in the SCUT-DHGA dataset, the present invention generates a multi-scale dynamic gesture hypergraph to represent the complex temporal variation information in dynamic gesture videos. This multi-scale dynamic gesture hypergraph is composed of multiple dynamic gesture images generated from gesture image frame sequences of different lengths, containing various local and global gesture motion cues. The pixels in the dynamic gesture hypergraph focus more on the gesture action rather than the background clutter in the image. This helps to solve the problem that existing identity feature extraction methods are easily affected by the image background and ignore the behavioral characteristics contained in the gesture action.

[0052] Specifically, the detailed method for generating a multi-scale dynamic gesture hypergraph is as follows:

[0053] Inspired by the Mixup method, we randomly sample two samples from the training data each time and perform linear interpolation to achieve the purpose of data enhancement. Set multiple window sizes T; for each window size T, generate z dynamic gesture images W respectively. T ; Dynamic gesture image W T The generation method is: non-repeated sampling of T consecutive original RGB images from the dynamic gesture video sequence, and sequentially generate z dynamic gesture images W T ;in, N is the number of original RGB image frames of dynamic gesture video, Indicates floor operation; Figure 1 shown.

[0054] For example, each dynamic gesture video in the SCUT-DHGA dataset contains 64 frames. For each dynamic gesture video, the present invention generates a total of 6 different types of dynamic gesture images: W 64 、W 32 、W 16 , W8, W5 and W3, in order Figure 2(a) to Figure 2(f) As shown, there are 1, 2, 4, 8, 12, and 21 images each. While using more window sizes to generate dynamic gesture images enriches identity behavior information, it also increases computational complexity and may introduce some redundant information. Finally, for each dynamic gesture video, these six different single-scale dynamic gesture images are combined to form a multi-scale dynamic gesture hypergraph containing 48 dynamic gesture images.

[0055] S3. A two-stream dynamic gesture authentication network is used to learn the original RGB image of the dynamic gesture video and the multi-scale dynamic gesture hypergraph to obtain the identity feature vector.

[0056] Inspired by biological knowledge and neuroscience principles, this paper designs a two-stream dynamic gesture authentication (TS-DHGA) network, which consists of a spatiotemporal feature extraction backbone network, a feature fusion module, and an identity feature aggregation module to better extract the discriminative identity features contained in gestures. Figure 3 As shown in Figure 2, where S represents the level of the network, b represents the batch size, and T and k represent the number of image frames and the number of identity feature vectors, respectively. Specifically, this network architecture consists of the following parts: (1) Spatiotemporal feature extraction backbone network. This part is responsible for extracting deep and representative spatiotemporal information from the video sequence, and obtaining gesture features by analyzing the spatial layout of dynamic gestures and their changes over time. (2) Feature fusion module. Its main task is to effectively integrate the motion flow and appearance flow information processed independently in the spatiotemporal feature extraction backbone network, so as to make full use of the appearance features and motion features in the dynamic gesture video and generate a comprehensive identity feature representation for authentication. (3) Identity feature aggregation module. This module further processes and integrates the previously obtained features through the global average pooling (GAP) layer and the fully connected (FC) layer to generate a series of highly condensed identity feature vectors. These three parts together constitute the core of the dual-stream dynamic gesture authentication network. Through the interaction of these three parts, the dual-stream dynamic gesture authentication network can accurately and in real time recognize and authenticate dynamic gestures, successfully simulating the complex visual perception function of humans and improving the accuracy and efficiency of dynamic gesture authentication.

[0057] Among them, the spatiotemporal feature extraction backbone network consists of motion flow and appearance flow, which respectively capture the behavioral features and physiological features embedded in the gesture. The feature fusion module effectively fuses the behavioral features and physiological features. Finally, the identity feature aggregation module aggregates the identity features extracted from each small batch of multi-frame images to form the final identity feature vector. The method of the present invention takes into account the fusion of information between different frames, as well as the local and global gesture motion clues in the entire dynamic gesture video. At the same time, the present invention uses a structural reparameterization strategy to optimize the network structure and improve the inference speed of authentication without reducing performance.

[0058] Specifically, the motion flow and appearance flow in the spatiotemporal feature extraction backbone network capture the behavioral features B and physiological features P embedded in the gesture, respectively, and generate a comprehensive identity feature representation for authentication; the motion flow takes a multi-scale dynamic gesture hypergraph as input, captures behavioral clues such as gesture transformations and their multi-scale temporal relationships, and obtains behavioral features embedded in the gesture.

[0059] Motion flow input acquisition method: For the image frame sequence I1,…,I with a window size of T T , the method of the present invention is to learn a feature vector d to represent the feature information of the image frame sequence, where p is the length of the eigenvector. Let is the image frame sequence I t The extracted features, represents the average value of features extracted from the previous t frames of original RGB image. The process of learning d can be regarded as solving a convex optimization problem using the RankSVM formula:

[0060]

[0061]

[0062] in, is the quadratic regularization term in SVM, S(t|d)= <d,V t > is the score of the t frame in the sorting function; only when S(q|d)>S(t|d)+1, the q>t pair is considered to be sorted correctly; the present invention can use the function ρ(I1,…,I T ; ψ) Extract feature information from T frames to represent vector d * .

[0063] In order to speed up the calculation of dynamic gesture images, for any η>0, the vector d * The first approximate solution of the formula is d * can be written as:

[0064]

[0065] Among them, α t =2(T-t+1)-(T+1)(H T -H t-1 ), is the harmonic number, and H0=0; by directly using t Features extracted from Instead of the average feature V t , we can get α t=2t-T-1; However, in a dynamic gesture image frame sequence, the image frames are temporally correlated, and the current image frame can provide some inspiration for people to understand other gesture image frames. For example, when seeing a frame of a dynamic gesture, one can even try to roughly infer the gesture action before or after the frame. In order to better simulate this complex action relationship, let α′ t =α t +1=2t-T, we can get I t The fusion coefficient β t , Finally we get:

[0066]

[0067] The vector d * Normalize and get the eigenvector d.

[0068] In practice, the feature vector d contains the same number of elements as the video frames, so it can be interpreted as a standard RGB image. This feature vector d can be directly used in our model as input to the motion stream to extract behavioral features. Furthermore, a larger window size n results in more frames being fused, and more pixels being activated to represent the gesture. Dynamic gesture images also tend to focus more on the gesture itself, rather than the background or other information in the gesture video.

[0069] In the field of video behavior understanding, some work uses 3D convolution to obtain temporal information. However, 3D convolution requires a large amount of computational cost, making it difficult to efficiently extract user identity features, which does not meet the real-time requirements of dynamic gesture authentication. To further efficiently extract dynamic gesture features, inspired by the fact that a complete gesture is a combination of multiple finger and palm sub-movements, this paper further creates a multi-scale dynamic gesture hypergraph. This graph contains multiple single-scale dynamic gesture images generated by windows of different sizes. This graph can comprehensively incorporate multi-scale temporal information to better explore user behavior patterns.

[0070] The appearance flow uses the original RGB image of the dynamic gesture video as input to capture physiological features such as hand shape, skin color, and palm texture features, and obtains the physiological features embedded in the gesture.

[0071] The backbone of motion flow and appearance flow is improved by RepVGGA0 network, both of which are composed of five layers of RepVGG layers, such as Figure 4 shown.

[0072] After each RepVGG layer of the motion flow and the appearance flow extracts the behavioral features B and the physiological features P respectively, they are first input into the feature fusion module to fuse the behavioral features B and the physiological features P to obtain the fused behavioral features B' and physiological features P', and then they are respectively input into the next RepVGG layer of the motion flow and the appearance flow for extraction; the execution is repeated until the extraction of the last RepVGG layer of the motion flow and the appearance flow is completed; the behavioral features B and physiological features P extracted from the last RepVGG layer of the motion flow and the appearance flow are input into the feature fusion module for fusion to obtain the final fused behavioral features B' and physiological features P'.

[0073] The RepVGGA0 network architecture is structured into five key layers. These layers are organized in a hierarchical and increasing order of complexity: the first RepVGG layer, containing one RepVGGBlock block, provides the network's foundation for initial feature extraction. The second RepVGG layer, with slightly increased complexity, comprises two RepVGGBlock blocks, further enhancing the model's ability to parse input data. The third RepVGG layer represents a significant jump, comprising four RepVGGBlock blocks, enabling the network to process more complex feature information. The fourth RepVGG layer, the most complex part of the network, comprises fourteen RepVGGBlock blocks, significantly enhancing the network's ability to capture and express details. Finally, the fifth RepVGG layer, containing one RepVGGBlock block, completes the network architecture, ensuring the integrity and efficiency of the overall structure. This layered and modular design not only improves the network's information processing capabilities but also facilitates subsequent optimization and application.

[0074] The RepVGGBlock block adopts a unique structural design. During the training phase, the RepVGGBlock block appears as a complex structure containing multiple branches, which can be specifically expressed as y = x + f(x) + g(x); where f(x) and g(x) represent the convolution branches that perform 1×1 and 3×3 convolution operations, respectively. This design enables the network to learn features of different scales simultaneously during training, thereby capturing richer information and improving the model's expressiveness. In order to optimize the efficiency and speed of the inference process,

[0075] After training is completed, a structural reparametrization strategy is used to convert multiple branch structures into a single straight line structure y = g(x); the conversion method of the straight line structure y is: by performing a specific linear combination of the trained multi-branch structure parameters, an equivalent single convolution operation h(x) is finally obtained. This operation not only contains all the information of the original multi-branch structure, but also significantly reduces the amount of calculation and the number of parameters of the model during inference. This structural transformation between the training and inference stages not only improves the operating efficiency of the network, but also ensures that the performance of the model is not affected. Through structural reparameterization, RepVGGBlock can fully learn and extract features in the training stage, and achieve fast and efficient results through a simplified network structure in the inference stage. This strategy provides a new perspective for the design and optimization of deep learning models, and is particularly suitable for scenarios where it is necessary to optimize model inference speed and efficiency while maintaining high performance.

[0076] In the dual-stream dynamic gesture authentication network, the feature fusion module plays a vital role. Its purpose is to explore and implement an effective two-stream information fusion strategy to significantly improve the gesture authentication performance. To this end, the present invention carefully designs the behavioral and physiological features fusion (BP-Fusion) module, such as Figure 5 As shown in the figure, the goal is to integrate behavioral and physiological features to form richer and more accurate identity authentication information. The importance of behavioral and physiological features is adjusted using weights R1 and R2, where R1 + R2 = 1. The greater the proportion of behavioral features in the identity features, the larger R1 is, indicating a stronger response and greater contribution from the behavioral features. Otherwise, R1 is smaller. This weighting mechanism ensures that behavioral and physiological features are appropriately weighted according to their actual contribution to gesture authentication, thereby achieving more refined feature integration.

[0077] Calculate the fused behavioral features B' and physiological features P' respectively:

[0078] B′=BN(B×R1+BN(P)×R2)

[0079] P′=BN(P)

[0080] Among them, BN is the batch normalization function.

[0081] The Identity Aggregation (IA) module plays an important role in the dual-stream dynamic gesture authentication network. Figure 6As shown in Figure 2, its main task is to extract a feature vector that accurately reflects the user's identity from the comprehensive behavioral and physiological characteristics. The design and implementation of this module aims to improve the accuracy and efficiency of identity authentication through sophisticated feature processing and aggregation strategies.

[0082] The identity feature aggregation module further processes and integrates the features fused by the last feature fusion module through the Global Average Pooling (GAP) layer and the Fully Connected (FC) layer to generate a series of highly condensed identity feature vectors.

[0083] S4. Obtain the dynamic gesture authentication result based on the identity feature vector.

[0084] The operation of the dynamic gesture authentication system can be summarized into three core steps: model training, user registration, and authentication. During the training phase of the present invention, dynamic gesture images in the database undergo image preprocessing and are then input into a convolutional neural network for learning and model building. During the registration phase, newly acquired gesture images undergo image preprocessing and are then used by a previously trained dynamic gesture feature extractor to identify and extract key information, forming a user recognition template that is stored in a local database. Finally, during the authentication phase, newly acquired dynamic gesture images undergo image preprocessing and user features are extracted using a previously trained feature extractor. The system then compares the user features with the templates registered in the database. If the gesture features to be authenticated closely match the identity features registered in the database, authentication is successful; otherwise, authentication fails.

[0085] Dynamic gestures encompass a rich set of physiological and behavioral characteristics, but current research focuses primarily on effectively extracting and fully utilizing these physiological features, while the mining and utilization of the behavioral features rich in gestures has received insufficient attention. However, effectively extracting behavioral features and integrating them with physiological features would undoubtedly significantly enhance the performance of dynamic gesture authentication. Furthermore, existing methods still suffer from poor feature extraction capabilities and low efficiency when extracting behavioral features contained in complex dynamic gestures. To promote the wider application and development of dynamic gesture authentication technology in the field of biometric recognition, this paper conducts in-depth research on these issues and generates a novel visual modality image, called a multi-scale dynamic gesture hypergraph. This image is composed of dynamic images of multiple scales generated from gesture image frame sequences of varying lengths, and is more capable of capturing the key behavioral characteristics of dynamic gestures than existing technologies. Inspired by biological knowledge and neurological principles, this paper further designs a two-stream dynamic gesture authentication network based on the multi-scale dynamic gesture hypergraph. Compared to existing technologies, this network can better extract and integrate the physiological and behavioral features of dynamic gestures, improving the accuracy and real-time performance of dynamic gesture authentication technology, and promoting the development of biometric recognition and artificial intelligence. By implementing the technical solution of the present invention, the electronic identity authentication requirements in many fields such as airport security, banking, access control systems and software login can be met. It is neither as easy to be forgotten and cracked as a password, nor as easy to be stolen as a smart card. It has the advantages of being contactless, reliable and convenient, and has a wide range of important applications in daily life.

[0086] Example 2

[0087] This embodiment provides a readable storage medium, wherein the readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the video-based dynamic gesture authentication method described in the first embodiment.

[0088] Example 3

[0089] This embodiment provides a computer device, including a processor and a memory for storing a program executable by the processor. When the processor executes the program stored in the memory, the video-based dynamic gesture authentication method described in the first embodiment is implemented.

[0090] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A video-based dynamic gesture authentication method, characterized by: The steps include: S1. Input the dynamic gesture video to be authenticated; S2. Generate a multi-scale dynamic gesture hypergraph from the dynamic gesture video; the multi-scale dynamic gesture hypergraph is composed of multiple dynamic gesture images generated by image frame sequences of different lengths; S3, using a two-stream dynamic gesture authentication network to learn the original RGB image of the dynamic gesture video and the multi-scale dynamic gesture hypergraph to obtain the identity feature vector; The dual-stream dynamic gesture authentication network consists of a spatiotemporal feature extraction backbone network, a feature fusion module, and an identity feature aggregation module; the spatiotemporal feature extraction backbone network captures the behavioral features B and physiological features P embedded in the gesture respectively; feature The fusion module fuses the behavioral feature B and the physiological feature P; The identity feature aggregation module further processes and integrates the features obtained by the spatiotemporal feature extraction backbone network and the feature fusion module to generate an identity feature vector; S4. Obtain dynamic gesture authentication results based on the identity feature vector; In step S2, multiple window sizes T are set; for each window size T, z single-scale dynamic gesture images W are generated respectively. T ; Single-scale dynamic gesture image W T The generation method is: non-repeated sampling of T consecutive frames of original RGB images from the dynamic gesture video sequence, and sequentially generate z single-scale dynamic gesture images W T ;in, N is the number of original RGB image frames of dynamic gesture video, Indicates floor operation; All z single-scale dynamic gesture images W of window size T T Combined, to form a multi-scale dynamic gesture hypergraph; In step S3, the spatiotemporal feature extraction backbone network is composed of a motion stream and an appearance stream; the motion stream uses a multi-scale dynamic gesture hypergraph as input to capture behavioral clues and obtain behavioral features embedded in the gesture; the appearance stream uses the original RGB image of the dynamic gesture video as input to capture physiological features and obtain physiological features embedded in the gesture; The backbone of the motion flow and the appearance flow are both composed of five RepVGG layers; after each RepVGG layer of the motion flow and the appearance flow extracts the behavioral features B and the physiological features P respectively, they are first input into the feature fusion module to fuse the behavioral features B and the physiological features P to obtain the fused behavioral features B' and physiological features P', and then they are respectively input into the next RepVGG layer of the motion flow and the appearance flow for extraction; the execution is repeated until the extraction of the last RepVGG layer of the motion flow and the appearance flow is completed; the behavioral features B and physiological features P extracted by the last RepVGG layer of the motion flow and the appearance flow are input into the feature fusion module for fusion to obtain the final fused behavioral features B' and physiological features P' as the input of the identity feature aggregation module.

2. The video-based dynamic gesture authentication method according to claim 1, characterized in that: The backbone of the motion flow and the appearance flow is that the first RepVGG layer contains one RepVGGBlock block; the second RepVGG layer contains two RepVGGBlock blocks; the third RepVGG layer contains four RepVGGBlock blocks; the fourth RepVGG layer includes fourteen RepVGGBlock blocks; and the fifth RepVGG layer contains one RepVGGBlock block.

3. The video-based dynamic gesture authentication method according to claim 2, characterized in that: During the training phase, the RepVGGBlock block is represented as a multi-branch structure, expressed as y1=x+f(x)+g(x); where f(x) and g(x) represent the convolution branches that perform 1×1 and 3×3 convolution operations, respectively. After the training is completed, a structural reparameterization strategy is adopted to convert multiple branch structures into a single straight line structure y2 = h(x); the conversion method of the straight line structure y2 is: by performing a specific linear combination of the trained multi-branch structure parameters, an equivalent single convolution operation h(x) is finally obtained.

4. The video-based dynamic gesture authentication method according to claim 1, characterized in that: The motion flow takes the feature vector d of the multi-scale dynamic gesture hypergraph as input; the feature vector d is calculated as: For the image frame sequence I1,…,I with a window size of T T , set d to represent the feature vector of the image frame sequence, where p is the length of the eigenvector; set up is the image frame sequence I t Extracted features, t = 1, 2, ...; represents the average value of features extracted from the previous t frames of original RGB image; Calculate vector d * : Among them, β t is the image frame sequence I t The fusion coefficient of The vector d * Normalize and get the eigenvector d.

5. The video-based dynamic gesture authentication method according to claim 1, characterized in that: In step S3, the feature fusion module sets the weights of the behavioral feature B and the physiological feature P to R1 and R2, respectively, where R1+R2=1; and calculates the fused behavioral feature B' and physiological feature P' respectively: B′=BN(B×R1+BN(P)×R2) P′=BN(P) Among them, BN is the batch normalization function.

6. The video-based dynamic gesture authentication method according to claim 1, characterized in that: In step S3, the identity feature aggregation module further processes and integrates the features of the spatiotemporal feature extraction backbone network and the feature fusion module through a global average pooling layer and a fully connected layer to generate a series of identity feature vectors.

7. A readable storage medium, characterized in that: The storage medium stores a computer program, which, when executed by a processor, enables the processor to perform the video-based dynamic gesture authentication method according to any one of claims 1 to 6.

8. A computer device comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the video-based dynamic gesture authentication method according to any one of claims 1 to 6 is implemented.

Images