Dynamic gesture recognition method and device, computer equipment and storage medium

Abstract

The embodiment of the invention belongs to the field of computer vision, and relates to a dynamic gesture recognition method, which comprises the following steps: acquiring dynamic gesture data; inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result; and identifying a target gesture action based on the gesture classification result. The invention further provides a dynamic gesture recognition device, computer equipment and a storage medium. According to the method, the problems that feature attention of key information of an existing dynamic gesture recognition model is not accurate enough and the generalization ability of the model is limited are solved, and the dynamic gesture classification accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of computer vision technology, and particularly to a dynamic gesture recognition method, device, computer equipment and storage medium. Background Art

[0002] Vision-based dynamic gesture recognition refers to the recognition of the morphological and motion changes of gestures in time and space for a sequence of continuous gesture images as input, which has a wide range of applications in the field of human-computer interaction, and its application scenarios cover many fields such as interactive games, medical assistance, intelligent driving, etc.

[0003] Existing vision-based dynamic gesture recognition methods mainly adopt deep learning-based algorithms, and the algorithm models often select 3D-CNN (Convolutional Neural Network), recurrent neural networks such as LSTM (Long Short-Term Memory), fusion networks ConvLSTM, attention mechanism Transformer and other models. Under the same hardware configuration, for large-scale data, 3D-CNN has better deep feature extraction ability. Among the 3D forms of models such as ResNet, Inception, GoogleNet, and VGG, the 3D-ResNeXt model has more advantages in terms of training time, deep feature extraction ability, residual connection, and grouped convolution.

[0004] However, the 3D-ResNeXt model is still restricted by limitations such as traditional convolution operations. It may extract a large amount of spatial redundant features, may not be able to accurately focus on features with key information, and there are problems such as limited model generalization ability. Summary of the Invention

[0005] The purpose of the embodiments of this application is to propose a dynamic gesture recognition method, device, computer equipment and storage medium to solve problems such as inaccurate attention to features with key information and limited model generalization ability in existing dynamic gesture recognition models, and improve the accuracy of dynamic gesture classification. To solve the above technical problems, the embodiments of this application provide a dynamic gesture recognition method, which adopts the following technical solutions: Obtain dynamic gesture data; Input the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result; Recognize the target gesture action based on the gesture classification result.

[0006] Further, the step of obtaining dynamic gesture data specifically includes: Collect gesture images, where at least one complete hand is included in the gesture images; Perform image preprocessing on the gesture images to obtain the dynamic gesture data.

[0007] Further, the step of inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result specifically includes: Obtain a historical gesture data set and a dynamic gesture recognition model to be trained; Based on the historical gesture data set, construct a dynamic gesture recognition training set, where the dynamic gesture recognition training set includes sample action data and corresponding sample labels of the sample action data; Based on the dynamic gesture recognition training set, perform gesture classification training on the dynamic gesture recognition model to be trained. After the training is completed, obtain the pre-trained dynamic gesture recognition model; Based on the pre-trained dynamic gesture recognition model, perform gesture classification processing on the dynamic gesture data to obtain the gesture classification result.

[0008] Further, before the step of obtaining the dynamic gesture recognition model to be trained, it specifically includes: Obtain a preset 3D-ResNeXt model, and replace all Bottleneck modules in the preset 3D-ResNeXt model with RefBlock modules; Add a parameter-free attention mechanism to the 3D-ResNeXt model; Add a spatial reconstruction unit to the 3D-ResNeXt model to obtain the dynamic gesture recognition model to be trained.

[0009] Further, the step of performing gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set and obtaining the pre-trained dynamic gesture recognition model after the training is completed specifically includes: Input the sample action data in the dynamic gesture recognition training set into the dynamic gesture recognition model to be trained for gesture classification training to obtain a predicted gesture classification result; Calculate the loss value between the predicted gesture classification result and the corresponding sample label; Taking minimizing the loss value as an optimization goal, iteratively adjust the dynamic gesture recognition model to be trained until the loss value converges at the minimum or reaches a preset number of adjustment times, and then stop the iteration to obtain the pre-trained dynamic gesture recognition model.

[0010] Further, the step of constructing a dynamic gesture recognition training set based on the historical gesture data set specifically includes: Perform cleaning processing on the historical gesture data set to obtain a cleaned historical gesture data set; Perform feature extraction processing on the historical gesture data set after the cleaning process to obtain historical gesture feature data; Perform classification processing on the historical gesture feature data to obtain positive sample gesture data and negative sample gesture data; Assign positive sample labels to the positive sample gesture data and negative sample labels to the negative sample gesture data to obtain the dynamic gesture recognition training set.

[0011] Further, the step of recognizing the target gesture action based on the gesture classification result specifically includes: Determine the gesture type based on the gesture classification result; Recognize the target gesture action based on the gesture type.

[0012] To solve the above technical problems, an embodiment of the present application further provides a dynamic gesture recognition device, which adopts the following technical solutions: An acquisition module for acquiring dynamic gesture data; A classification module for inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result; A recognition module for recognizing a target gesture action based on the gesture classification result.

[0013] To solve the above technical problems, an embodiment of the present application further provides a computer device, which adopts the following technical solutions: A computer device includes a memory and a processor. Computer-readable instructions are stored in the memory, and when the processor executes the computer-readable instructions, the steps of the dynamic gesture recognition method are implemented.

[0014] To solve the above technical problems, an embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solutions: A computer-readable storage medium, characterized in that computer-readable instructions are stored on the computer-readable storage medium, and when the computer-readable instructions are executed by a processor, the steps of the dynamic gesture recognition method are implemented.

[0015] Compared with the prior art, the embodiments of the present application mainly have the following beneficial effects: In the embodiments of the present application, by acquiring dynamic gesture data, inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result, and recognizing the target gesture action based on the gesture classification result, the problems that the feature attention of key information in the existing dynamic gesture recognition model is not accurate enough and the model generalization ability is limited are solved, and the accuracy of dynamic gesture classification is improved. Description of the Drawings

[0016] To more clearly illustrate the solutions in this application, the following will give a brief introduction to the drawings required for the description of the embodiments of this application. Obviously, the drawings in the following description are some embodiments of this application. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0017] Figure 1 is an exemplary system architecture diagram to which this application can be applied; Figure 2 Flowchart of an embodiment of the dynamic gesture recognition method according to this application; Figure 3 is a schematic structural diagram of an embodiment of the dynamic gesture recognition device according to this application; Figure 4 is a schematic structural diagram of an embodiment of the computer device according to this application. Detailed implementation manners

[0018] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those of ordinary skill in the technical field to which this application belongs; the terms used in the description of the application in this specification are only for the purpose of describing specific embodiments, and are not intended to limit this application; the terms "including" and "having" and any variations thereof in the description and claims of this application and the above drawings are intended to cover non-exclusive inclusion. The terms "first", "second", etc. in the description and claims of this application or the above drawings are used to distinguish different objects, not to describe a specific order.

[0019] Referring to "embodiment" herein means that a specific feature, structure, or characteristic described in connection with the embodiment can be included in at least one embodiment of this application. The phrase appears in various places in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that the embodiments described herein can be combined with other embodiments.

[0020] To enable those skilled in the technical field to better understand the solutions of this application, the following will clearly and completely describe the technical solutions in the embodiments of this application in conjunction with the drawings.

[0021] Such as Figure 1As shown, the system architecture 100 may include a terminal device 101, a network 102, and a server 103. The terminal device 101 may be a laptop computer 1011, a tablet computer 1012, or a mobile phone 1013. The network 102 is a medium for providing a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links, or fiber optic cables, etc.

[0022] Users can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications may be installed on the terminal device 101, such as a web browser application, a shopping application, a search application, an instant messaging tool, an email client, a social platform software, etc.

[0023] The terminal device 101 may be various electronic devices with a display screen and supporting web browsing. In addition to the laptop computer 1011, the tablet computer 1012, or the mobile phone 1013, the terminal device 101 may also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV) player, a laptop portable computer, and a desktop computer, etc.

[0024] The server 103 may be a server providing various services, such as a background server supporting the pages displayed on the terminal device 101.

[0025] It should be noted that the dynamic gesture recognition method provided by the embodiments of the present application is generally executed by the server / terminal device. Correspondingly, the dynamic gesture recognition device is generally set in the server / terminal device.

[0026] It should be understood that Figure 1 the numbers of the terminal devices, networks, and servers in

[0027] Continue to refer to Figure 2 , which shows a flowchart of an embodiment of the dynamic gesture recognition method according to the present application. The dynamic gesture recognition method includes the following steps: Step S201, obtain dynamic gesture data.

[0028] In this embodiment, the above dynamic gesture recognition method can be deployed in a dynamic gesture recognition platform, which can be constructed by a server or a server cluster. The server or server cluster can be any electronic device with functions such as image recognition, image processing, data transmission, and data storage. The electronic device (such as Figure 1 the server / terminal device shown) can receive dynamic gesture data through a wired connection or a wireless connection. It should be noted that the above wireless connection methods can include, but are not limited to, 3G / 4G / 5G connections, WiFi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra wideband) connections, and other currently known or future-developed wireless connection methods.

[0029] In this embodiment, the above dynamic gesture data can refer to the hand movement sequence data that continuously changes in the time dimension and can be represented by video frames or a continuous image sequence. Specifically, the above dynamic gesture data can include dynamic features of the gesture in terms of spatial position, shape change, and movement trajectory, and can be obtained by collecting through an RGB camera, a depth camera, or other sensing devices. For example, the dynamic gesture data can include continuous video segments of gestures such as waving, making a fist, and pointing. Each frame of data contains at least the information of one complete hand.

[0030] Specifically, regarding the specific implementation process of obtaining the dynamic gesture data, this application will further describe the details in subsequent specific embodiments and will not elaborate too much here.

[0031] Step S202: Input the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result.

[0032] In this embodiment, the above gesture classification result can be a classification label output after processing the input dynamic gesture data by the above pre-trained dynamic gesture recognition model, and is used to represent the specific gesture type recognized.

[0033] Specifically, the above gesture classification result is usually composed of predefined gesture categories, such as waving, giving a thumbs up, making a fist, etc. Each category corresponds to a unique identifier (such as a name or a number). The classification result can be a single label (such as "Wave") or a multi-label form (when the gesture has composite features), and can include a confidence score, which is used to indicate the credibility of the classification result by the pre-trained dynamic gesture recognition model. The above confidence score can be further used for the execution or feedback of the target gesture action.

[0034] In this embodiment, the above dynamic gesture recognition model can be an algorithm model based on artificial intelligence, aiming to identify and classify the types and meanings of gestures by analyzing continuous gesture action sequence data. Specifically, the above dynamic gesture recognition model can utilize a 3D convolutional neural network (3D-CNN), a recurrent neural network (RNN), or a fusion model (such as ConvLSTM) to capture the temporal and spatial features of gestures in order to extract key action information. The dynamic gesture recognition model can be a pre-trained model (such as 3D-ResNeXt), which is modified by adding modules such as reparameterized convolution (RefConv), attention mechanism (SimAM), and spatial reconstruction unit (SimSRU), thereby improving the classification accuracy of dynamic gestures, the ability to capture key features, and the generalization performance of the model.

[0035] Specifically, regarding the specific implementation process of inputting the dynamic gesture data into the pre-trained dynamic gesture recognition model for classification processing to obtain the gesture classification result, this application will further describe the details in subsequent specific embodiments and will not elaborate too much here.

[0036] Step S203, identify the target gesture action based on the gesture classification result.

[0037] In this embodiment, the above target gesture action can refer to the specific gesture behavior determined according to the gesture classification result during the dynamic gesture recognition process, usually bound to specific interaction instructions or functions.

[0038] Specifically, the above target gesture action can represent the actual meaning and use of the recognized gesture. For example, waving may correspond to starting the device, making a fist may be used to pause the playback, or pointing may be used to select a certain function. The target gesture action can be further mapped to specific operation or response actions through the gesture type in the classification result and is applicable to application scenarios such as intelligent device control, virtual reality interaction, and medical assistance.

[0039] Specifically, regarding the specific implementation process of identifying the target gesture action based on the gesture classification result, this application will further describe the details in subsequent specific embodiments and will not elaborate too much here.

[0040] This application obtains dynamic gesture data, inputs the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result, and identifies the target gesture action based on the gesture classification result, solving problems such as inaccurate feature attention to key information and limited model generalization ability in existing dynamic gesture recognition models, and improving the accuracy of dynamic gesture classification.

[0041] In some optional implementation manners, step S201 includes the following steps: Collect gesture images; Perform image preprocessing on the gesture image to obtain dynamic gesture data.

[0042] In this embodiment, the gesture image can be collected by a camera device (such as an RGB camera, a depth camera, or an infrared sensor), and the hand movements are recorded as a continuous video frame or image sequence (i.e., the above-mentioned gesture image). During the collection process, a fixed or movable camera device can be used to ensure that the gesture is always within the camera's field of view.

[0043] In this embodiment, the above-mentioned image preprocessing may include image resizing (such as scaling to a unified input size), color channel conversion (such as converting from RGB to grayscale or normalizing color values), denoising processing (such as Gaussian filtering), data augmentation (such as rotation, flipping, cropping, etc.), and time synchronization and normalization processing of the frame sequence.

[0044] In a possible embodiment, by performing image preprocessing on the gesture image to obtain dynamic gesture data, the preprocessed image can be input into the model in the form of a tensor to ensure the effectiveness and consistency of feature extraction.

[0045] In some alternative implementation manners, step S202 includes the following steps: Obtain a historical gesture data set and a dynamic gesture recognition model to be trained; Based on the historical gesture data set, construct a dynamic gesture recognition training set; Perform gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set. After the training is completed, obtain a pre-trained dynamic gesture recognition model; Perform gesture classification processing on the dynamic gesture data based on the pre-trained dynamic gesture recognition model to obtain a gesture classification result.

[0046] In this embodiment, the historical gesture data set can be a data set composed of previously collected and labeled gesture images or videos, usually containing multiple categories of gesture samples and their corresponding labels, such as actions like waving, making a fist, and giving a thumbs up.

[0047] Specifically, the historical gesture data set can be sourced from publicly available gesture data sets (such as EgoGesture, NVGesture) or obtained through custom collection. When the user chooses custom collection, diverse gesture actions can be recorded by a camera device, covering different lighting, background, and angle conditions, and the collected gestures can be classified and labeled manually or semi-automatically to form a high-quality training data set for model training and optimization.

[0048] In this embodiment, the above dynamic gesture recognition training set includes sample action data and corresponding sample labels for the sample action data. Specifically, the above dynamic gesture recognition training set can be a high-quality data set for training a recognition model, usually including consecutive frame data of gesture actions and their corresponding classification labels.

[0049] Specifically, regarding the specific implementation process of obtaining the pre-trained dynamic gesture recognition model and constructing the dynamic gesture recognition training set based on the historical gesture data set, this application will further describe the details in subsequent specific embodiments and will not elaborate too much here.

[0050] In some optional implementation manners, before the step of obtaining the dynamic gesture recognition model to be trained, it specifically includes: Obtain a preset 3D-ResNeXt model, and replace all Bottleneck modules in the preset 3D-ResNeXt model with RefBlock modules; Add a parameter-free attention mechanism to the 3D-ResNeXt model; Add a spatial reconstruction unit to the 3D-ResNeXt model to obtain the dynamic gesture recognition model to be trained.

[0051] In this embodiment, the above RefBlock module can be a class inherited from nn.Module. First, calculate the intermediate number of channels mid_planes, and then define three convolutional layers conv1, conv2, conv3 and corresponding batch normalization layers bn1, bn2, bn3, and a ReLU activation function relu, where conv2 uses a modified RefConv; The step of replacing all Bottleneck modules in the preset 3D-ResNeXt model with RefBlock modules can be achieved by importing the architecture and parameter definition file of the 3D-ResNeXt model, locating all Bottleneck modules included in the model structure, and then replacing the key components (such as convolutional layers, batch normalization layers, and activation functions) in the Bottleneck module with the modified RefBlock module, where the second convolutional layer is replaced with a reparameterized refocusing convolution (RefConv). Then, adjust the parameter configuration (such as kernel size, stride, and number of channels) of the RefBlock module to match the network requirements of 3D-ResNeXt.

[0052] In a possible embodiment, since the convolutional module in the 3D-ResNeXt model structure is a Bottleneck module, where the convolutional module includes three convolutional layers, and there is a batch normalization layer and an activation function layer between each convolutional layer. Therefore, by transforming the original two-dimensional reparameterized refocusing convolution (RefConv) into a three-dimensional form and replacing the second convolutional layer in the above convolutional module structure with RefConv, a RefBlock module can be obtained, and then all Bottleneck modules in the 3D-ResNeXt model are replaced with RefBlock modules.

[0053] Specifically, the Bottleneck module in the 3D-ResNeXt model includes three convolutional layers. The convolutional layer conv1 is connected to the batch normalization layer BN and the activation function ReLU, and then connected to the convolutional layer conv2. The convolutional layer conv2 is connected to the batch normalization layer BN and the activation function ReLU, and then connected to the convolutional layer conv3. The convolutional layer conv3 is connected to the batch normalization layer BN, added to the input layer, and then connected to the activation function ReLU; among them, the convolutional layer conv2 is replaced with the transformed RefConv to obtain the RefBlock module.

[0054] In this embodiment, an attention mechanism without parameters, SimAM, can be added before the average pooling layer in the 3D-ResNeXt model structure. Then, the original two-dimensional spatial reconstruction unit (SRU) is transformed into a three-dimensional form. In the transformed spatial reconstruction unit, the separation and reconstruction parts are respectively fused with SimAM to obtain a new spatial reconstruction unit, SimSRU. After the third layer of grouped convolution in the 3D-ResNeXt model structure, the transformed spatial reconstruction unit is added to obtain a dynamic gesture recognition model to be trained.

[0055] Specifically, the 3D-ResNeXt model adopts a 101-layer variant form. The specific structure is that the Input layer is connected to the convolutional layer conv1, the convolutional layer conv1 is connected to the max pooling layer maxpool, the max pooling layer maxpool is connected to the Layer1 layer, which includes 3 Bottleneck modules. The Layer1 layer is connected to the Layer2 layer, which includes 4 Bottleneck modules. The Layer2 layer is connected to the Layer3 layer, which includes 23 Bottleneck modules. The Layer3 layer is connected to the Layer4 layer, which includes 3 Bottleneck modules. The Layer4 layer is connected to the average pooling layer avgpool, and the average pooling layer avgpool is connected to the fully connected layer fc; an attention mechanism without parameters, SimAM, is added before the average pooling layer avgpool.

[0056] In this embodiment, the parameterless attention mechanism can be a lightweight attention mechanism, which aims to assign different weights to the input features through simple mathematical operations, so as to guide the model to pay more attention to key regions or features without introducing additional learnable parameters, and can analyze and process the input data through statistical features (such as mean, variance) or other fixed functions.

[0057] In a possible embodiment, in dynamic gesture recognition, the parameterless attention mechanism (such as SimAM) can generate a weight distribution by calculating the local importance of the input features (such as the importance of each pixel or region), and these weights are used to re-weight the input features, enhancing the model's sensitivity to key features while reducing the attention to irrelevant or redundant information. Adding the parameterless attention mechanism can improve the efficiency and accuracy of feature extraction and classification without increasing the model complexity.

[0058] In some alternative implementation manners, the steps of performing gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set and obtaining the pre-trained dynamic gesture recognition model after training specifically include: Input the sample action data in the dynamic gesture recognition training set into the dynamic gesture recognition model to be trained for gesture classification training to obtain the predicted gesture classification result; Calculate the loss value between the predicted gesture classification result and the corresponding sample label; Taking minimizing the loss value as the optimization goal, iteratively adjust the dynamic gesture recognition model to be trained until the loss value converges at the minimum or reaches the preset number of adjustment times, and then stop the iteration to obtain the pre-trained dynamic gesture recognition model.

[0059] In this embodiment, the labeled dynamic gesture training set is input into the model, and the predicted gesture classification result is generated through forward propagation, and the difference between the predicted gesture classification result and the corresponding sample label, that is, the loss value, is calculated. Then, the loss value is propagated back to each layer of the network through backpropagation, and the parameters of the model are adjusted using gradient descent algorithms (such as Adam or SGD) to gradually optimize the model, so that the model can finally accurately classify the input gesture into the corresponding category.

[0060] Specifically, through the above training method, the dynamic gesture recognition model can learn to extract key features from complex time series data and make correct classifications. By taking minimizing the loss value as the goal, the model can improve its understanding and generalization ability of gesture action features, effectively avoiding overfitting or underfitting. When the loss value tends to converge, it indicates that the model has fully learned the features required for the gesture classification task. This method ensures the robustness and accuracy of the model under different lighting, angle, and action change conditions.

[0061] In some alternative implementations, the steps of constructing a dynamic gesture recognition training set based on a historical gesture data set specifically include: Clean the historical gesture data set to obtain a cleaned historical gesture data set; Extract features from the cleaned historical gesture data set to obtain historical gesture feature data; Classify the historical gesture feature data to obtain positive sample gesture data and negative sample gesture data; Assign positive sample labels to the positive sample gesture data and negative sample labels to the negative sample gesture data to obtain a dynamic gesture recognition training set.

[0062] In this embodiment, the historical gesture data set is cleaned to remove noise data and abnormal samples to ensure data quality and consistency. Then, features are extracted from the cleaned data to extract key information (such as spatio-temporal features or motion trajectories) that can represent gesture features. Next, the extracted feature data is classified, and samples that conform to the definition of the target gesture are marked as positive samples, while those that do not are marked as negative samples. Finally, corresponding labels are assigned to the positive and negative samples respectively to construct a training set containing classification labels and feature data, which serves as the basis for training the dynamic gesture recognition model, ensuring the high quality and clarity of the training data, and helping the model learn gesture features and improve classification performance.

[0063] In some alternative implementations, step S203 includes: Determine the gesture type based on the gesture classification result; Recognize the target gesture action based on the gesture type.

[0064] In this embodiment, the above gesture types can be a predefined set of dynamic gesture categories used to represent specific actions or intentions of the user. For example, gesture types can include common actions such as waving (Wave), making a fist (Fist), giving a thumbs up (Thumbs Up), clapping (Clap), pointing (Point), swiping (Swipe), etc. These types can be further extended according to different application scenarios, such as specific control gestures in interactive games, command gestures in intelligent driving, or gesture feedback in medical device operations. The diversity and clarity of gesture types help the model accurately identify and classify actions during training and inference.

[0065] In a possible embodiment, gesture data can be classified through a dynamic gesture recognition model to determine the type label corresponding to the input gesture (such as "Wave" or "Point"). Then, according to the predefined mapping relationship between the gesture type and the target action, the gesture type is converted into a specific target action. For example, a gesture of the "Wave" type can be mapped to start the device, and a gesture of the "Point" type can be mapped to select a specific screen area or item. Finally, the system triggers the corresponding operation or outputs an action instruction according to the mapping result to realize the linkage from the gesture to the actual function. This process ensures the intuitiveness of gesture input and the consistency of operation feedback.

[0066] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is a theory, method, technology, and application system that uses a digital computer or a machine controlled by a digital computer to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use knowledge to obtain the best results.

[0067] Artificial intelligence basic technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, and mechatronics. Artificial intelligence software technologies mainly include several major directions such as computer vision technology, robotics, biometric technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0068] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, an optical disc, a read-only memory (ROM), or a random access memory (RAM), etc.

[0069] It should be understood that although the steps in the flowchart of the accompanying drawings are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise clearly stated in this document, there is no strict order restriction for the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed alternately or in turn with at least a part of other steps or sub-steps or stages of other steps.

[0070] Further reference Figure 3 , as an implementation of the method shown above Figure 2 , this application provides an embodiment of a dynamic gesture recognition device. This device embodiment corresponds to Figure 2 the method embodiment shown, and this device can be specifically applied to various electronic devices.

[0071] As Figure 3 shown, the dynamic gesture recognition device 300 described in this embodiment includes: an acquisition module 301, a classification module 302, and an identification module 303. Among them: The acquisition module 301 is used to acquire dynamic gesture data; The classification module 302 is used to input the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result; The identification module 303 is used to identify a target gesture action based on the gesture classification result.

[0072] Among them, the acquisition module 301 includes: The acquisition sub-module is used to acquire gesture images; The preprocessing sub-module is used to perform image preprocessing on the gesture images to obtain the dynamic gesture data.

[0073] Among them, the classification module 302 includes: The acquisition sub-module is used to acquire a historical gesture data set and a dynamic gesture recognition model to be trained; The construction sub-module is used to construct a dynamic gesture recognition training set based on the historical gesture data set; The gesture classification training sub-module is used to perform gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set. After the training is completed, the pre-trained dynamic gesture recognition model is obtained; A gesture classification processing sub-module, which is used to perform gesture classification processing on the dynamic gesture data based on the pre-trained dynamic gesture recognition model to obtain the gesture classification result.

[0074] Before the acquisition sub-module, it further includes: A replacement sub-module, which is used to obtain a preset 3D-ResNeXt model and replace all Bottleneck modules in the preset 3D-ResNeXt model with RefBlock modules; A first addition sub-module, which is used to add a parameter-free attention mechanism to the 3D-ResNeXt model; A second addition sub-module, which is used to add a spatial reconstruction unit to the 3D-ResNeXt model to obtain the dynamic gesture recognition model to be trained.

[0075] Among them, the gesture classification training sub-module includes: A training unit, which is used to input the sample action data in the dynamic gesture recognition training set into the dynamic gesture recognition model to be trained for gesture classification training to obtain a predicted gesture classification result; A calculation unit, which is used to calculate the loss value between the predicted gesture classification result and the corresponding sample label; An iteration unit, which is used to iteratively adjust the dynamic gesture recognition model to be trained with the goal of minimizing the loss value until the loss value converges at the minimum or stops iterating when the preset number of adjustments is reached, and obtain the pre-trained dynamic gesture recognition model.

[0076] The construction sub-module includes: A cleaning unit, which is used to clean the historical gesture data set to obtain a cleaned historical gesture data set; An extraction unit, which is used to perform feature extraction processing on the cleaned historical gesture data set to obtain historical gesture feature data; A classification unit, which is used to classify the historical gesture feature data to obtain positive sample gesture data and negative sample gesture data; An assignment unit, which is used to assign positive sample labels to the positive sample gesture data and negative sample labels to the negative sample gesture data to obtain the dynamic gesture recognition training set.

[0077] Among them, the recognition module 303 includes: A determination sub-module, which is used to determine the gesture type based on the gesture classification result; A recognition sub-module, which is used to recognize the target gesture action based on the gesture type.

[0078] In this embodiment, by obtaining dynamic gesture data, inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result, and recognizing a target gesture action based on the gesture classification result, problems such as inaccurate feature attention to key information and limited model generalization ability in existing dynamic gesture recognition models are solved, and the accuracy of dynamic gesture classification is improved.

[0079] In this embodiment, the operations respectively performed by the above units or modules correspond one by one to the steps of the dynamic gesture recognition method in the above embodiment, and will not be elaborated here.

[0080] To solve the above technical problems, an embodiment of the present application also provides a computer device. For details, please refer to Figure 4 , Figure 4 which is the basic structural block diagram of the computer device in this embodiment.

[0081] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that communicate with each other through a system bus. It should be noted that only the computer device 4 with components 41 - 43 is shown in the figure, but it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Among them, those skilled in the art of the present technology can understand that a computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0082] The computer device can be a desktop computer, a notebook, a palm computer, a cloud server, or other computing devices. The computer device can perform human-computer interaction with users through a keyboard, a mouse, a remote control, a touchpad, a voice control device, or other means.

[0083] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (such as SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, FlashCard, etc. equipped on the computer device 4. Of course, the memory 41 may also include both the internal storage unit and the external storage device of the computer device 4. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for the dynamic gesture recognition method. In addition, the memory 41 may also be used to temporarily store various data that have been output or will be output.

[0084] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the computer-readable instructions stored in the memory 41 or process data, such as running the computer-readable instructions for the dynamic gesture recognition method.

[0085] The network interface 43 may include a wireless network interface or a wired network interface, and the network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0086] This embodiment provides a computer device. By acquiring dynamic gesture data, inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result, and recognizing a target gesture action based on the gesture classification result, it solves the problems such as inaccurate feature attention of key information and limited model generalization ability in the existing dynamic gesture recognition model, and improves the accuracy of dynamic gesture classification.

[0087] The present application also provides another implementation manner, that is, to provide a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor, so that the at least one processor executes the steps of the dynamic gesture recognition method as described above.

[0088] This embodiment provides a computer-readable storage medium. By acquiring dynamic gesture data, inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for recognition processing to obtain a gesture classification result, and recognizing a target gesture action based on the gesture classification result, problems such as inaccurate feature attention to key information and limited model generalization ability in the existing dynamic gesture recognition model are solved, and the accuracy of dynamic gesture classification is improved.

[0089] Through the description of the above implementation manners, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation manner. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions for causing a terminal device (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in various embodiments of the present application.

[0090] Obviously, the embodiments described above are only a part of the embodiments of the present application, rather than all of the embodiments. The accompanying drawings show preferred embodiments of the present application, but do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosed content of the present application more thorough and comprehensive. Although the present application has been described in detail with reference to the foregoing embodiments, for those skilled in the art, they can still modify the technical solutions described in the foregoing specific embodiments, or perform equivalent replacements on some of the technical features. Any equivalent structure directly or indirectly using the content of the specification and drawings of the present application in other related technical fields shall be similarly within the scope of the patent protection of the present application.

Claims

1. A dynamic gesture recognition method, characterized in that: The steps include: Get dynamic gesture data; Inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result; A target gesture action is identified based on the gesture classification result.

2. The dynamic gesture recognition method according to claim 1, characterized in that: The step of obtaining dynamic gesture data specifically includes: Collecting a gesture image, wherein the gesture image includes at least one complete hand; Performing image preprocessing on the gesture image to obtain the dynamic gesture data.

3. The dynamic gesture recognition method according to claim 1, characterized in that: The step of inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result specifically includes: Obtain historical gesture datasets and the dynamic gesture recognition model to be trained; Based on the historical gesture data set, a dynamic gesture recognition training set is constructed, wherein the dynamic gesture recognition training set includes sample action data and sample labels corresponding to the sample action data; Performing gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set, and obtaining the pre-trained dynamic gesture recognition model after the training is completed; Perform gesture classification processing on the dynamic gesture data based on the pre-trained dynamic gesture recognition model to obtain the gesture classification result.

4. The dynamic gesture recognition method according to claim 3, characterized in that: Before the step of obtaining the dynamic gesture recognition model to be trained, the following steps are specifically included: Obtain a preset 3D-ResNeXt model, and replace all Bottleneck modules in the preset 3D-ResNeXt model with RefBlock modules; Adding a parameter-free attention mechanism to the 3D-ResNeXt model; A space reconstruction unit is added to the 3D-ResNeXt model to obtain the dynamic gesture recognition model to be trained.

5. The dynamic gesture recognition method according to claim 3, characterized in that: The step of performing gesture classification training on the dynamic gesture recognition model to be trained based on the dynamic gesture recognition training set, and obtaining the pre-trained dynamic gesture recognition model after the training is completed, specifically includes: Inputting the sample action data in the dynamic gesture recognition training set into the dynamic gesture recognition model to be trained to perform gesture classification training to obtain a predicted gesture classification result; Calculating a loss value between the predicted gesture classification result and the corresponding sample label; Taking minimizing the loss value as the optimization goal, the dynamic gesture recognition model to be trained is iteratively adjusted until the loss value converges to the minimum, or when a preset adjustment number is reached, the iteration is stopped to obtain the pre-trained dynamic gesture recognition model.

6. The dynamic gesture recognition method according to claim 3, characterized in that: The step of constructing a dynamic gesture recognition training set based on the historical gesture data set specifically includes: Cleaning the historical gesture data set to obtain a cleaned historical gesture data set; Performing feature extraction processing on the cleaned historical gesture data set to obtain historical gesture feature data; Classify and process the historical gesture feature data to obtain positive sample gesture data and negative sample gesture data; A positive sample label is assigned to the positive sample gesture data, and a negative sample label is assigned to the negative sample gesture data, to obtain the dynamic gesture recognition training set.

7. The dynamic gesture recognition method according to claim 1, characterized in that: The step of identifying the target gesture action based on the gesture classification result specifically includes: determining a gesture type based on the gesture classification result; The target gesture action is identified based on the gesture type.

8. A dynamic gesture recognition device, characterized in that: include: An acquisition module is used to acquire dynamic gesture data; A classification module, used for inputting the dynamic gesture data into a pre-trained dynamic gesture recognition model for classification processing to obtain a gesture classification result; A recognition module is used to recognize a target gesture action based on the gesture classification result.

9. A computer device, characterized in that: The method comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the dynamic gesture recognition method according to any one of claims 1 to 7 when executing the computer-readable instructions.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the dynamic gesture recognition method according to any one of claims 1 to 7 are implemented.

Images