Gesture recognition method, electronic device, storage medium, and program product

Abstract

The application relates to the technical field of computer vision, and provides a gesture recognition method, an electronic device, a storage medium and a program product, the method comprising the following steps: acquiring a gesture image to be recognized; inputting the gesture image into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model; wherein the gesture recognition model comprises a static gesture recognition layer and a dynamic gesture recognition layer, the static gesture recognition layer is used for gesture recognition on a static gesture image, and the dynamic gesture recognition layer is used for gesture recognition on a dynamic gesture image; the static gesture recognition layer is obtained through training based on a sample static gesture image, and the dynamic gesture recognition layer is obtained through transfer learning based on the trained static gesture recognition layer. When static gesture images and dynamic gesture images are recognized at the same time, the application can ensure the recognition accuracy of gesture recognition.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a gesture recognition method, electronic device, storage medium, and program product. Background Technology

[0002] With the rapid development of computer vision technology, gesture recognition is being applied in an increasingly wide range of applications. Gesture recognition is divided into static gesture recognition and dynamic gesture recognition. In static gesture recognition scenarios, the gestures are all still, meaning that the captured gesture images are all static gesture images; while in dynamic gesture recognition scenarios, the gestures are in motion, meaning that the captured gesture images are all dynamic gesture images.

[0003] Currently, the same gesture recognition model is used to recognize both static and dynamic gesture images. Based on this, it is necessary to use mixed sample data of static and dynamic gesture images to train the gesture recognition model. However, using this mixed sample data for model training will reduce the recognition accuracy of the gesture recognition model for static gesture images. Summary of the Invention

[0004] This invention aims to at least solve one of the technical problems existing in the prior art. To this end, this invention proposes a gesture recognition method that can ensure the accuracy of gesture recognition when simultaneously recognizing static and dynamic gesture images.

[0005] The present invention also provides a gesture recognition device, an electronic device, a storage medium, and a program product.

[0006] A gesture recognition method according to a first aspect of the present invention includes:

[0007] Obtain the image of the gesture to be recognized;

[0008] The gesture image is input into the gesture recognition model to obtain the gesture recognition result output by the gesture recognition model;

[0009] The gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. The static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images.

[0010] The static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0011] According to the gesture recognition method of this invention, the gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. This allows for gesture recognition not only on static gesture images but also on dynamic gesture images. The static gesture recognition layer is trained independently on sample static gesture images, thereby improving the accuracy of static gesture recognition. Simultaneously, the dynamic gesture recognition layer is obtained through transfer learning from the trained static gesture recognition layer, reducing the training data requirements of the dynamic gesture recognition layer and improving model training performance, thus enhancing the accuracy of dynamic gesture recognition. In summary, this invention ensures the accuracy of gesture recognition when simultaneously recognizing both static and dynamic gesture images.

[0012] According to one embodiment of the present invention, the step of inputting the gesture image into a gesture recognition model to obtain the gesture recognition result output by the gesture recognition model includes:

[0013] The gesture image is determined to be a static gesture image. The image features of the gesture image are input into the static gesture recognition layer to obtain the gesture recognition result output by the static gesture recognition layer.

[0014] The gesture image is determined to be a dynamic gesture image. The image features of the gesture image are input into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer.

[0015] According to one embodiment of the present invention, the step of inputting the image features of the gesture image into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer includes:

[0016] The image features are input into the static gesture recognition layer to obtain the first recognition result output by the static gesture recognition layer.

[0017] The image features are input into the dynamic gesture recognition layer to obtain the second recognition result output by the dynamic gesture recognition layer;

[0018] Based on the first recognition result and the second recognition result, the gesture recognition result is determined.

[0019] According to an embodiment of the present invention, the image features are determined based on the following steps:

[0020] The gesture image is input into the feature extraction layer of the gesture recognition model to obtain the image features output by the feature extraction layer.

[0021] According to one embodiment of the present invention, the dynamic gesture recognition layer is trained based on the following steps:

[0022] Based on the model parameters of the trained static gesture recognition layer, the model parameters of the dynamic gesture recognition layer are updated;

[0023] The updated dynamic gesture recognition layer is trained based on sample dynamic gesture images.

[0024] According to one embodiment of the present invention, training the updated dynamic gesture recognition layer based on sample dynamic gesture images includes:

[0025] The sample dynamic gesture image is input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample dynamic gesture image is input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0026] Based on the first sample recognition result and the second sample recognition result, the sample gesture recognition result is determined;

[0027] Based on the sample gesture recognition results, the updated dynamic gesture recognition layer is trained.

[0028] According to an embodiment of the present invention, the step of inputting the sample dynamic gesture image into the static gesture recognition layer to obtain a first sample recognition result output by the static gesture recognition layer, and inputting the sample dynamic gesture image into the updated dynamic gesture recognition layer to obtain a second sample recognition result output by the updated dynamic gesture recognition layer, includes:

[0029] The sample dynamic gesture image is input into the feature extraction layer of the gesture recognition model to obtain the sample image features output by the feature extraction layer;

[0030] The sample image features are input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample image features are input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0031] A gesture recognition device according to a second aspect of the present invention includes:

[0032] The acquisition module is used to acquire the image of the gesture to be recognized;

[0033] The recognition module is used to input the gesture image into the gesture recognition model and obtain the gesture recognition result output by the gesture recognition model;

[0034] The gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. The static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images.

[0035] The static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0036] An electronic device according to a third aspect of the present invention includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor, when executing the program, implements the gesture recognition method as described above.

[0037] According to a fourth aspect of the present invention, a non-transitory computer-readable storage medium is provided thereon storing a computer program that, when executed by a processor, implements the gesture recognition method as described above.

[0038] A computer program product according to a fifth aspect of the present invention includes a computer program that, when executed by a processor, implements the gesture recognition method as described above.

[0039] The above-described one or more technical solutions in the embodiments of the present invention have at least one of the following technical effects:

[0040] The gesture recognition model comprises a static gesture recognition layer and a dynamic gesture recognition layer. This allows for gesture recognition not only from static images but also from dynamic images. The static gesture recognition layer is trained independently on sample static gesture images, thus improving the accuracy of static gesture recognition. Simultaneously, the dynamic gesture recognition layer is derived through transfer learning from the trained static gesture recognition layer, reducing the training data requirements of the dynamic gesture recognition layer and improving model training performance, thereby enhancing the accuracy of dynamic gesture recognition. In summary, this invention ensures accurate gesture recognition when simultaneously recognizing both static and dynamic gesture images.

[0041] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is one of the flowcharts illustrating the gesture recognition method provided in this embodiment of the invention;

[0044] Figure 2 This is a second schematic flowchart of the gesture recognition method provided in this embodiment of the invention;

[0045] Figure 3 This is the third flowchart illustrating the gesture recognition method provided in this embodiment of the invention;

[0046] Figure 4 This is a schematic diagram of the structure of the gesture recognition device provided in an embodiment of the present invention;

[0047] Figure 5 This is a schematic diagram of the structure of the electronic device provided in an embodiment of the present invention. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0049] With the rapid development of computer vision technology, gesture recognition is being applied more and more widely. Gesture recognition is divided into static gesture recognition and dynamic gesture recognition. In static gesture recognition scenarios, the gestures are all still, meaning the captured gesture images are all static gesture images, which are relatively clear. In contrast, in dynamic gesture recognition scenarios, the gestures are moving, meaning the captured gesture images are all dynamic gesture images. Due to the low frame rate of the image acquisition device or the excessively fast movement of the person, dynamic gesture images tend to be blurry.

[0050] Currently, a single gesture recognition model is used to simultaneously recognize both static and dynamic gesture images. This requires training the gesture recognition model with a mixture of static and dynamic gesture images as training data. The use of dynamic gesture images in training the model enables it to recognize gestures from dynamic images. However, increasing the training samples with dynamic gesture images can lead to overfitting to static gesture recognition tasks. This is because static gesture recognition scenarios do not involve blurred gestures; therefore, training the model with this mixed sample data reduces the accuracy of the gesture recognition model for static gesture images. Furthermore, for dynamic gesture recognition tasks, using this mixed sample data for model training also leads to overfitting to dynamic gesture recognition tasks, further reducing the accuracy of the gesture recognition model for dynamic gesture images.

[0051] To address the above problems, the present invention proposes the following embodiments. The gesture recognition method of the present invention is described below with reference to the accompanying drawings.

[0052] The execution subject of this gesture recognition method can be a gesture recognition device, a server, a home service robot, or a user's terminal, including but not limited to mobile phones, tablets, PCs, in-vehicle terminals, and smart home appliances.

[0053] Figure 1 This is one of the flowcharts illustrating the gesture recognition method provided in this embodiment of the invention, such as... Figure 1 As shown, the gesture recognition method includes:

[0054] Step 110: Obtain the gesture image to be recognized.

[0055] Here, the gesture image is the image that needs to be recognized. This gesture image can be a static image or a dynamic image. In one embodiment, the static gesture image is a clear image, and the dynamic gesture image is a blurred image.

[0056] Step 120: Input the gesture image into the gesture recognition model to obtain the gesture recognition result output by the gesture recognition model.

[0057] Here, the gesture recognition model is used to extract features from gesture images to obtain image features, and then to perform gesture recognition on the image features to obtain gesture recognition results.

[0058] The gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. The static gesture recognition layer is used to recognize gestures from static gesture images, and the dynamic gesture recognition layer is used to recognize gestures from dynamic gesture images.

[0059] Specifically, if the gesture image is determined to be a static gesture image, the gesture image is input into the static gesture recognition layer to obtain the gesture recognition result output by the static gesture recognition layer; if the gesture image is determined to be a dynamic gesture image, the gesture image is input into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer.

[0060] In one embodiment, the static gesture recognition layer is used to extract features from a gesture image to obtain image features, and then to perform gesture recognition on the image features to obtain a gesture recognition result. In another embodiment, the gesture image is input to the feature extraction layer of the gesture recognition model to obtain image features output by the feature extraction layer, which are then used by the static gesture recognition layer to perform gesture recognition on the image features to obtain a gesture recognition result.

[0061] In one embodiment, the dynamic gesture recognition layer is used to extract features from a gesture image to obtain image features, and then to perform gesture recognition on the image features to obtain a gesture recognition result. In another embodiment, the gesture image is input to the feature extraction layer of the gesture recognition model to obtain image features output by the feature extraction layer, which are then used by the dynamic gesture recognition layer to perform gesture recognition on the image features to obtain a gesture recognition result.

[0062] In one embodiment, the gesture image is determined to be a dynamic gesture image. The gesture image is then input to a static gesture recognition layer to obtain a first recognition result output by the static gesture recognition layer. The gesture image is then input to a dynamic gesture recognition layer to obtain a second recognition result output by the dynamic gesture recognition layer. Based on the first and second recognition results, a gesture recognition result is determined. It should be noted that when the gesture image is a dynamic gesture image, i.e., in a dynamic gesture recognition scenario, both the static and dynamic gesture recognition layers are enabled. Gesture recognition is performed on the gesture image based on both layers, and the recognition results from both are combined to obtain the final gesture recognition result, thereby improving the accuracy of dynamic gesture recognition.

[0063] In one embodiment, the gesture image is determined to be a static gesture image. The gesture image is then input to the static gesture recognition layer to obtain the gesture recognition result output by the static gesture recognition layer. It should be noted that when the gesture image is a static gesture image, i.e. in a static gesture recognition scenario, the static gesture recognition layer is enabled and the dynamic gesture recognition layer is disabled, so that gesture recognition is performed on the gesture image based solely on the static gesture recognition layer.

[0064] The gesture image can be determined as static or dynamic upon acquisition. Alternatively, the gesture recognition model includes an image discrimination layer. The gesture image is input to the image discrimination layer of the gesture recognition model, and the discrimination result output by the image discrimination layer is obtained. Based on the discrimination result, the gesture image is determined to be either static or dynamic.

[0065] The static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0066] In one embodiment, the static gesture recognition layer is trained based on the following steps: inputting sample static gesture images into the static gesture recognition layer to obtain sample gesture recognition results output by the static gesture recognition layer; and training the static gesture recognition layer based on the sample gesture recognition results and the sample label results corresponding to the sample static gesture images. Further, the static gesture recognition layer and the feature extraction layer of the gesture recognition model are trained together based on the sample gesture recognition results and the sample label results corresponding to the sample static gesture images.

[0067] It should be noted that when training the static gesture recognition layer, the dynamic gesture recognition layer is disabled, that is, the forward and backward propagation functions of the dynamic gesture recognition layer are turned off, and only sample data from static gesture images are used to train the static gesture recognition layer. In one embodiment, disabling the backpropagation function can be achieved by freezing the gradient parameters of the dynamic gesture recognition layer, for example, by setting the gradient parameters of the dynamic gesture recognition layer to 0.

[0068] The initial model parameters of the dynamic gesture recognition layer are set with reference to the model parameters of the trained static gesture recognition layer; that is, the dynamic gesture recognition layer is trained using transfer learning. Furthermore, the dynamic gesture recognition layer is trained based on the following steps: inputting sample dynamic gesture images into the dynamic gesture recognition layer to obtain the sample gesture recognition results output by the dynamic gesture recognition layer; training the dynamic gesture recognition layer based on the sample gesture recognition results and the corresponding sample label results of the sample dynamic gesture images. Further, based on the sample gesture recognition results and the corresponding sample label results of the sample dynamic gesture images, the dynamic gesture recognition layer and the feature extraction layer of the gesture recognition model are trained together.

[0069] It should be noted that the sample data used to train the gesture recognition model mainly consists of static gesture images, meaning that the number of static gesture images is greater than the number of dynamic gesture images.

[0070] It is understandable that the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer, thereby reducing the need for training data, improving the model training effect, and thus improving the accuracy of gesture recognition.

[0071] The gesture recognition method provided in this invention includes a static gesture recognition layer and a dynamic gesture recognition layer. This allows for gesture recognition not only from static but also from dynamic gesture images. The static gesture recognition layer is trained independently on sample static gesture images, thus improving the accuracy of static gesture recognition. Simultaneously, the dynamic gesture recognition layer is obtained through transfer learning from the trained static gesture recognition layer, reducing the training data requirements of the dynamic gesture recognition layer and improving model training performance, thereby enhancing the accuracy of dynamic gesture recognition. In summary, this invention ensures accurate gesture recognition when simultaneously recognizing both static and dynamic gesture images.

[0072] Based on the above embodiments, Figure 2 This is a second schematic flowchart of the gesture recognition method provided in this embodiment of the invention, as shown below. Figure 2 As shown, step 120 above includes:

[0073] Step 121: Determine that the gesture image is a static gesture image, input the image features of the gesture image into the static gesture recognition layer, and obtain the gesture recognition result output by the static gesture recognition layer.

[0074] Here, the static gesture recognition layer is used to perform gesture recognition on image features to obtain the gesture recognition result. The specific structure of this static gesture recognition layer can be set according to actual needs, such as a classification layer and a regression layer; this embodiment of the invention does not specifically limit this.

[0075] It should be noted that when the gesture image is a static gesture image, that is, in a static gesture recognition scenario, the static gesture recognition layer is enabled and the dynamic gesture recognition layer is disabled, so that the image features of the gesture image are input into the static gesture recognition layer.

[0076] Whether the gesture image is a static gesture image can be determined at the time of acquisition. Alternatively, the gesture recognition model includes an image discrimination layer. The gesture image is input to the image discrimination layer of the gesture recognition model, and the discrimination result output by the image discrimination layer is obtained. Based on the discrimination result, it is determined whether the gesture image is a static gesture image.

[0077] Step 122: Determine that the gesture image is a dynamic gesture image, input the image features of the gesture image into the dynamic gesture recognition layer, and obtain the gesture recognition result output by the dynamic gesture recognition layer.

[0078] Here, the dynamic gesture recognition layer is used to perform gesture recognition on image features to obtain gesture recognition results. The specific structure of this dynamic gesture recognition layer can be set according to actual needs, such as a classification layer and a regression layer; this embodiment of the invention does not impose specific limitations on this.

[0079] It should be noted that when the gesture image is a dynamic gesture image, that is, in a dynamic gesture recognition scenario, the dynamic gesture recognition layer is enabled so that the image features of the gesture image are input into the dynamic gesture recognition layer.

[0080] Whether the gesture image is a dynamic gesture image can be determined at the time of acquisition. Alternatively, the gesture recognition model includes an image discrimination layer. The gesture image is input to the image discrimination layer of the gesture recognition model, and the discrimination result output by the image discrimination layer is obtained. Based on the discrimination result, it is determined whether the gesture image is a dynamic gesture image.

[0081] The gesture recognition method provided in this embodiment of the invention determines that if the gesture image is a static gesture image, then the image features of the gesture image are input into the static gesture recognition layer to obtain the gesture recognition result output by the static gesture recognition layer; if the gesture image is a dynamic gesture image, then the image features of the gesture image are input into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer. Thus, it can not only perform gesture recognition on static gesture images, but also on dynamic gesture images, improving the applicability of the gesture recognition model.

[0082] Based on any of the above embodiments, in this method, step 122 includes:

[0083] The image features are input into the static gesture recognition layer to obtain the first recognition result output by the static gesture recognition layer.

[0084] The image features are input into the dynamic gesture recognition layer to obtain the second recognition result output by the dynamic gesture recognition layer;

[0085] Based on the first recognition result and the second recognition result, the gesture recognition result is determined.

[0086] Here, the static gesture recognition layer is used to perform gesture recognition on image features to obtain the gesture recognition result, i.e., the first recognition result. The dynamic gesture recognition layer is used to perform gesture recognition on image features to obtain the gesture recognition result, i.e., the second recognition result.

[0087] It should be noted that when the gesture image is a dynamic gesture image, that is, in a dynamic gesture recognition scenario, both the static gesture recognition layer and the dynamic gesture recognition layer are enabled. The image features of the gesture image are input into the static gesture recognition layer and the dynamic gesture recognition layer, and then the recognition results of the two are combined to obtain the gesture recognition result, thereby improving the accuracy of dynamic gesture recognition.

[0088] In some embodiments, if the first recognition result and the second recognition result are determined to be the same, the second recognition result is determined as the gesture recognition result; if the first recognition result and the second recognition result are determined to be different, a first confidence level of the first recognition result is determined, and a second confidence level of the second recognition result is determined, and the gesture recognition result is determined based on the first confidence level and the second confidence level.

[0089] In one embodiment, if the second confidence level is determined to be greater than the first preset threshold, the second recognition result is determined to be a gesture recognition result; if the second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is greater than the second preset threshold, the first recognition result is determined to be a gesture recognition result; if the second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is determined to be less than or equal to the second preset threshold, the second recognition result is determined to be a gesture recognition result.

[0090] In another embodiment, if the second confidence level is determined to be greater than or equal to the first confidence level, the second recognition result is determined as the gesture recognition result; if the second confidence level is determined to be less than the first confidence level, the first recognition result is determined as the gesture recognition result.

[0091] In another embodiment, if a second confidence level is determined to be greater than a first preset threshold, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is greater than the second preset threshold, the first recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is greater than the second preset threshold, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is less than or equal to the second preset threshold, the second recognition result is determined to be a gesture recognition result.

[0092] In another embodiment, if a second confidence level is determined to be greater than a first preset threshold, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is greater than the second preset threshold, the first recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is determined to be less than or equal to the second preset threshold, and the first confidence level is determined to be less than or equal to the second confidence level, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the first confidence level is determined to be less than or equal to the second preset threshold, and the first confidence level is greater than the second confidence level, the first recognition result is determined to be a gesture recognition result.

[0093] In another embodiment, if a second confidence level is determined to be greater than a first preset threshold, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the second confidence level is greater than a third preset threshold, and the first confidence level is greater than the second preset threshold, and the first confidence level is less than or equal to the second confidence level, the second recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the second confidence level is greater than a third preset threshold, and the first confidence level is greater than the second preset threshold, and the first confidence level is greater than the second confidence level, the first recognition result is determined to be a gesture recognition result; if a second confidence level is determined to be less than or equal to the first preset threshold, and the second confidence level is greater than the third preset threshold, and the first confidence level is less than or equal to the second preset threshold... The second recognition result is determined as a gesture recognition result; if the second confidence level is less than or equal to the first preset threshold, and the second confidence level is less than or equal to the third preset threshold, and the first confidence level is greater than the second preset threshold, and the first recognition result is determined as a gesture recognition result; if the second confidence level is less than or equal to the first preset threshold, and the second confidence level is less than or equal to the third preset threshold, and the first confidence level is greater than the second preset threshold, and the first confidence level is less than or equal to the second confidence level, the second recognition result is determined as a gesture recognition result; if the second confidence level is less than or equal to the first preset threshold, and the second confidence level is less than or equal to the third preset threshold, and the first confidence level is less than the second preset threshold, the second recognition result is determined as a gesture recognition result.

[0094] Of course, there are other embodiments that can determine the gesture recognition result based on the first confidence level and the second confidence level, which will not be described in detail in this embodiment of the invention.

[0095] The gesture recognition method provided in this embodiment of the invention, when the gesture image is a dynamic gesture image, inputs the image features of the gesture image to a static gesture recognition layer and a dynamic gesture recognition layer to obtain two recognition results output by the two gesture recognition layers, and then combines the two recognition results to obtain the gesture recognition result, thereby improving the recognition accuracy of dynamic gesture recognition.

[0096] Based on any of the above embodiments, in this method, the image features are determined based on the following steps:

[0097] The gesture image is input into the feature extraction layer of the gesture recognition model to obtain the image features output by the feature extraction layer.

[0098] Here, the feature extraction layer is shared by both the static gesture recognition layer and the dynamic gesture recognition layer. That is, the image features input to the static gesture recognition layer are extracted by this feature extraction layer, and the image features input to the dynamic gesture recognition layer are also extracted by this feature extraction layer. In other words, the two branches of static gesture recognition and dynamic gesture recognition share a common feature extraction layer.

[0099] The gesture recognition method provided in this embodiment of the invention inputs a gesture image into the feature extraction layer of the gesture recognition model to obtain the image features output by the feature extraction layer. Thus, the image features input to the static gesture recognition layer are extracted by the feature extraction layer, and the image features input to the dynamic gesture recognition layer are also extracted by the feature extraction layer. That is, for the two branches of static gesture recognition and dynamic gesture recognition, the feature extraction layer included in the gesture recognition model is shared, thereby reducing the amount of network model, saving memory usage, and reducing the resource usage of the gesture recognition model.

[0100] Based on any of the above embodiments Figure 3 This is the third flowchart illustrating the gesture recognition method provided in this embodiment of the invention, as shown below. Figure 3 As shown, the dynamic gesture recognition layer is trained based on the following steps:

[0101] Step 310: Update the model parameters of the dynamic gesture recognition layer based on the trained model parameters of the static gesture recognition layer.

[0102] Since both the static and dynamic gesture recognition layers need to perform gesture recognition, and although there are differences between static and dynamic gesture recognition, these differences are not significant. Therefore, the initial model parameters of the dynamic gesture recognition layer are set based on the model parameters of the trained static gesture recognition layer. This enables transfer learning based on model parameters, reduces the number of sample dynamic gesture images required by the dynamic gesture recognition layer, ensures the model training effect, and thus improves the recognition accuracy of dynamic gestures.

[0103] Step 320: Train the updated dynamic gesture recognition layer based on the sample dynamic gesture images.

[0104] Specifically, the sample dynamic gesture image is input into the updated dynamic gesture recognition layer to obtain the sample gesture recognition result output by the dynamic gesture recognition layer; based on the sample gesture recognition result and the sample label result corresponding to the sample dynamic gesture image, the updated dynamic gesture recognition layer is trained.

[0105] In one embodiment, the dynamic gesture recognition layer is used to extract features from the sample dynamic gesture image to obtain sample image features, and to perform gesture recognition on the sample image features to obtain sample gesture recognition results.

[0106] In another embodiment, a sample dynamic gesture image is input to the feature extraction layer of the gesture recognition model to obtain sample image features output by the feature extraction layer. These features are then used by the dynamic gesture recognition layer to perform gesture recognition on the sample image features to obtain the sample gesture recognition result. Based on this, the dynamic gesture recognition layer and the feature extraction layer of the gesture recognition model are trained together, using the sample gesture recognition result and the sample label result corresponding to the sample dynamic gesture image.

[0107] It should be noted that, compared with the sample static gesture images, the sample static gesture images are the majority, meaning that the number of sample static gesture images is greater than the number of sample dynamic gesture images.

[0108] The gesture recognition method provided in this embodiment of the invention updates the model parameters of the dynamic gesture recognition layer based on the model parameters of the trained static gesture recognition layer, reducing the number of sample dynamic gesture images required by the dynamic gesture recognition layer, ensuring the model training effect, and thus improving the recognition accuracy of dynamic gestures; and based on the sample dynamic gesture images, the updated dynamic gesture recognition layer is trained to further improve the model training effect, thereby further improving the recognition accuracy of dynamic gestures.

[0109] Based on any of the above embodiments, in this method, step 320 includes:

[0110] The sample dynamic gesture image is input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample dynamic gesture image is input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0111] Based on the first sample recognition result and the second sample recognition result, the sample gesture recognition result is determined;

[0112] Based on the sample gesture recognition results, the updated dynamic gesture recognition layer is trained.

[0113] It should be noted that when the sample dynamic gesture image is a dynamic gesture image, the static gesture recognition layer and the dynamic gesture recognition layer are enabled. The sample dynamic gesture image is input into the static gesture recognition layer and the dynamic gesture recognition layer. Then, the sample gesture recognition results of the two are combined to obtain the sample gesture recognition result, so as to improve the model training effect.

[0114] In some embodiments, if the first sample recognition result and the second sample recognition result are the same, the second sample recognition result is determined as the sample gesture recognition result; if the first sample recognition result and the second sample recognition result are different, the first sample confidence level of the first sample recognition result is determined, and the second sample confidence level of the second sample recognition result is determined. Based on the first sample confidence level and the second sample confidence level, the sample gesture recognition result is determined.

[0115] In one embodiment, if the confidence level of the second sample is determined to be greater than a preset threshold of the first sample, the recognition result of the second sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, the recognition result of the first sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is determined to be less than or equal to the preset threshold of the second sample, the recognition result of the second sample is determined to be the sample gesture recognition result.

[0116] In another embodiment, if the confidence level of the second sample is determined to be greater than or equal to the confidence level of the first sample, the recognition result of the second sample is determined as the sample gesture recognition result; if the confidence level of the second sample is determined to be less than the confidence level of the first sample, the recognition result of the first sample is determined as the sample gesture recognition result.

[0117] In another embodiment, if the confidence level of the second sample is determined to be greater than a preset threshold of the first sample, the recognition result of the second sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, and the confidence level of the first sample is greater than the confidence level of the second sample, the recognition result of the first sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, and the confidence level of the first sample is less than or equal to the confidence level of the second sample, the recognition result of the second sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is less than or equal to the preset threshold of the second sample, the recognition result of the second sample is determined to be the sample gesture recognition result.

[0118] In another embodiment, if the confidence level of the second sample is determined to be greater than a preset threshold of the first sample, the recognition result of the second sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, the recognition result of the first sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is less than or equal to the preset threshold of the second sample, and the confidence level of the first sample is less than or equal to the confidence level of the second sample, the recognition result of the second sample is determined to be the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold of the first sample, and the confidence level of the first sample is less than or equal to the preset threshold of the second sample, and the confidence level of the first sample is greater than the confidence level of the second sample, the recognition result of the first sample is determined to be the sample gesture recognition result.

[0119] In another embodiment, if the confidence level of the second sample is determined to be greater than a preset threshold for the first sample, the recognition result of the second sample is determined as the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold for the first sample, and the confidence level of the second sample is greater than a preset threshold for the third sample, and the confidence level of the first sample is greater than the preset threshold for the second sample, and the confidence level of the first sample is less than or equal to the confidence level of the second sample, the recognition result of the second sample is determined as the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold for the first sample, and the confidence level of the second sample is greater than a preset threshold for the third sample, and the confidence level of the first sample is greater than the preset threshold for the second sample, the recognition result of the first sample is determined as the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold for the first sample, and the confidence level of the second sample is greater than the preset threshold for the third sample, and the confidence level of the first sample is less than or equal to the preset threshold for the second sample, the recognition result of the first sample is determined as the sample gesture recognition result; if the confidence level of the second sample is determined to be less than or equal to the preset threshold for the first sample, and the confidence level of the second sample is greater than the preset threshold for the third sample, and the confidence level of the first sample is less than or equal to the preset threshold for the second sample, the recognition result of the first sample is determined as the sample gesture recognition result. The second sample recognition result is determined as the sample gesture recognition result; if the confidence level of the second sample is less than or equal to the preset threshold of the first sample, and the confidence level of the second sample is less than or equal to the preset threshold of the third sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, and the confidence level of the first sample is greater than the confidence level of the second sample, the first sample recognition result is determined as the sample gesture recognition result; if the confidence level of the second sample is less than or equal to the preset threshold of the first sample, and the confidence level of the second sample is less than or equal to the preset threshold of the third sample, and the confidence level of the first sample is greater than the preset threshold of the second sample, and the confidence level of the first sample is less than or equal to the confidence level of the second sample, the second sample recognition result is determined as the sample gesture recognition result; if the confidence level of the second sample is less than or equal to the preset threshold of the first sample, and the confidence level of the second sample is less than the preset threshold of the third sample, and the confidence level of the first sample is less than the preset threshold of the second sample, the second sample recognition result is determined as the sample gesture recognition result.

[0120] Of course, there are other embodiments that can determine the sample gesture recognition result based on the confidence level of the first sample and the confidence level of the second sample, which will not be described in detail in this embodiment of the invention.

[0121] In one embodiment, the static gesture recognition layer is used to extract features from the sample dynamic gesture image to obtain sample image features, and to perform gesture recognition on the sample image features to obtain gesture recognition results, namely the first sample recognition result.

[0122] In another embodiment, a sample dynamic gesture image is input to the feature extraction layer of the gesture recognition model to obtain sample image features output by the feature extraction layer. These features are then used by the static gesture recognition layer to perform gesture recognition on the sample image features, resulting in a first sample recognition result. Based on this, the dynamic gesture recognition layer and the feature extraction layer of the gesture recognition model are trained together, using the sample gesture recognition result and the sample label result corresponding to the sample dynamic gesture image.

[0123] In one embodiment, the dynamic gesture recognition layer is used to extract features from the sample dynamic gesture image to obtain sample image features, and to perform gesture recognition on the sample image features to obtain gesture recognition results, i.e., the second sample recognition results.

[0124] In another embodiment, a sample dynamic gesture image is input to the feature extraction layer of the gesture recognition model to obtain sample image features output by the feature extraction layer. These features are then used by the dynamic gesture recognition layer to perform gesture recognition on the sample image features, resulting in a second sample recognition result. Based on this, the dynamic gesture recognition layer and the feature extraction layer of the gesture recognition model are trained together using the sample gesture recognition result and the sample label result corresponding to the sample dynamic gesture image.

[0125] It should be noted that, compared with the sample static gesture images, the sample static gesture images are the majority, meaning that the number of sample static gesture images is greater than the number of sample dynamic gesture images.

[0126] It should be noted that when training the dynamic gesture recognition layer, enabling the dynamic gesture recognition layer means enabling both forward and backward propagation functions, enabling the forward propagation function of the static gesture recognition layer, disabling the backward propagation function of the static gesture recognition layer, and training the dynamic gesture recognition layer using only sample dynamic gesture images. In one embodiment, disabling the backward propagation function can be achieved by freezing the gradient parameters of the static gesture recognition layer, for example, by setting the gradient parameters of the static gesture recognition layer to 0.

[0127] The gesture recognition method provided in this embodiment of the invention inputs sample dynamic gesture images into a static gesture recognition layer and a dynamic gesture recognition layer to obtain two sample recognition results output by the two gesture recognition layers. Then, the sample recognition results of the two layers are combined to obtain the sample gesture recognition result, thereby improving the accuracy of the sample gesture recognition result, thereby improving the model training effect, and ultimately further improving the recognition accuracy of dynamic gesture recognition.

[0128] Based on any of the above embodiments, in this method, the step of inputting the sample dynamic gesture image to the static gesture recognition layer to obtain a first sample recognition result output by the static gesture recognition layer, and inputting the sample dynamic gesture image to the updated dynamic gesture recognition layer to obtain a second sample recognition result output by the updated dynamic gesture recognition layer, includes:

[0129] The sample dynamic gesture image is input into the feature extraction layer of the gesture recognition model to obtain the sample image features output by the feature extraction layer;

[0130] The sample image features are input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample image features are input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0131] Here, the feature extraction layer is shared by both the static gesture recognition layer and the dynamic gesture recognition layer. That is, the features of the sample image input to the static gesture recognition layer are extracted by this feature extraction layer, and the features of the sample image input to the dynamic gesture recognition layer are also extracted by this feature extraction layer. In other words, the two branches of static gesture recognition and dynamic gesture recognition share a common feature extraction layer.

[0132] Accordingly, the step of training the updated dynamic gesture recognition layer based on the sample gesture recognition results includes: training the dynamic gesture recognition layer and the feature extraction layer of the gesture recognition model together based on the sample gesture recognition results and the sample label results corresponding to the sample dynamic gesture images.

[0133] The gesture recognition method provided in this embodiment of the invention inputs a sample dynamic gesture image into the feature extraction layer of the gesture recognition model to obtain the sample image features output by the feature extraction layer. Thus, the sample image features input to the static gesture recognition layer are extracted by the feature extraction layer, and the sample image features input to the dynamic gesture recognition layer are also extracted by the feature extraction layer. That is, for the two branches of static gesture recognition and dynamic gesture recognition, the feature extraction layer included in the gesture recognition model is shared, thereby reducing the amount of network model, saving memory usage, and reducing the resource usage of the gesture recognition model.

[0134] The gesture recognition device provided by the present invention is described below. The gesture recognition device described below and the gesture recognition method described above can be referred to in correspondence.

[0135] Figure 4 This is a schematic diagram of the structure of the gesture recognition device provided in an embodiment of the present invention, as shown below. Figure 4 As shown, the gesture recognition device includes:

[0136] The acquisition module 410 is used to acquire the gesture image to be recognized;

[0137] The recognition module 420 is used to input the gesture image into the gesture recognition model and obtain the gesture recognition result output by the gesture recognition model;

[0138] The gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. The static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images.

[0139] The static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0140] The gesture recognition device provided in this invention includes a static gesture recognition layer and a dynamic gesture recognition layer. This allows for gesture recognition not only from static but also from dynamic gesture images. The static gesture recognition layer is trained independently on sample static gesture images, thus improving the accuracy of static gesture recognition. Simultaneously, the dynamic gesture recognition layer is obtained through transfer learning from the trained static gesture recognition layer, reducing the training data requirements of the dynamic gesture recognition layer and improving model training effectiveness, thereby enhancing the accuracy of dynamic gesture recognition. In summary, this invention ensures accurate gesture recognition when simultaneously recognizing both static and dynamic gesture images.

[0141] Based on any of the above embodiments, the identification module 420 includes:

[0142] The first recognition unit is used to determine that the gesture image is a static gesture image, input the image features of the gesture image into the static gesture recognition layer, and obtain the gesture recognition result output by the static gesture recognition layer;

[0143] The second recognition unit is used to determine that the gesture image is a dynamic gesture image, input the image features of the gesture image into the dynamic gesture recognition layer, and obtain the gesture recognition result output by the dynamic gesture recognition layer.

[0144] Based on any of the above embodiments, the second identification unit is further configured to:

[0145] The image features are input into the static gesture recognition layer to obtain the first recognition result output by the static gesture recognition layer.

[0146] The image features are input into the dynamic gesture recognition layer to obtain the second recognition result output by the dynamic gesture recognition layer;

[0147] Based on the first recognition result and the second recognition result, the gesture recognition result is determined.

[0148] Based on any of the above embodiments, the recognition module 420 further includes an image feature determination unit, which is used for:

[0149] The gesture image is input into the feature extraction layer of the gesture recognition model to obtain the image features output by the feature extraction layer.

[0150] Based on any of the above embodiments, the device further includes a model training module, which includes:

[0151] The parameter update unit is used to update the model parameters of the dynamic gesture recognition layer based on the model parameters of the trained static gesture recognition layer.

[0152] The model training unit is used to train the updated dynamic gesture recognition layer based on sample dynamic gesture images.

[0153] Based on any of the above embodiments, the model training unit is further used for:

[0154] The sample dynamic gesture image is input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample dynamic gesture image is input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0155] Based on the first sample recognition result and the second sample recognition result, the sample gesture recognition result is determined;

[0156] Based on the sample gesture recognition results, the updated dynamic gesture recognition layer is trained.

[0157] Based on any of the above embodiments, the model training unit is further used for:

[0158] The sample dynamic gesture image is input into the feature extraction layer of the gesture recognition model to obtain the sample image features output by the feature extraction layer;

[0159] The sample image features are input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample image features are input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

[0160] Figure 5 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 5 As shown, the electronic device may include: a processor 510, a communication interface 520, a memory 530, and a communication bus 540, wherein the processor 510, the communication interface 520, and the memory 530 communicate with each other through the communication bus 540. The processor 510 can call logical instructions in the memory 530 to execute a gesture recognition method, which includes: acquiring a gesture image to be recognized; inputting the gesture image into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model; wherein the gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer, the static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images; the static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0161] Furthermore, the logical instructions in the aforementioned memory 530 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, essentially, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0162] On the other hand, the present invention also provides a computer program product, which includes a computer program that can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the gesture recognition method provided by the above methods. The method includes: acquiring a gesture image to be recognized; inputting the gesture image into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model; wherein the gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer, the static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images; the static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer.

[0163] In another aspect, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the gesture recognition method provided by the above methods. The method includes: acquiring a gesture image to be recognized; inputting the gesture image into a gesture recognition model to obtain a gesture recognition result output by the gesture recognition model; wherein the gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer, the static gesture recognition layer being used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer being used to perform gesture recognition on dynamic gesture images; the static gesture recognition layer is trained based on sample static gesture images, and the dynamic gesture recognition layer is obtained through transfer learning based on the trained static gesture recognition layer.

[0164] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0165] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0166] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

[0167] The above embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Although the invention has been described in detail with reference to the embodiments, those skilled in the art should understand that various combinations, modifications, or equivalent substitutions of the technical solutions of the invention do not depart from the spirit and scope of the invention and should be covered within the scope of the claims of the invention.

Claims

1. A gesture recognition method, characterized in that, include: Obtain the image of the gesture to be recognized; The gesture image is input into the gesture recognition model to obtain the gesture recognition result output by the gesture recognition model; The gesture recognition model includes a static gesture recognition layer and a dynamic gesture recognition layer. The static gesture recognition layer is used to perform gesture recognition on static gesture images, and the dynamic gesture recognition layer is used to perform gesture recognition on dynamic gesture images. The static gesture recognition layer is trained based on sample static gesture images. When training the static gesture recognition layer, the forward and backward propagation functions of the dynamic gesture recognition layer are turned off, and the static gesture recognition layer is trained using only sample data of static gesture images. The dynamic gesture recognition layer is obtained by transfer learning based on the trained static gesture recognition layer. When training the dynamic gesture recognition layer, the backward propagation function of the static gesture recognition layer is turned off, and the dynamic gesture recognition layer is trained using only sample dynamic gesture images.

2. The gesture recognition method according to claim 1, characterized in that, The step of inputting the gesture image into the gesture recognition model and obtaining the gesture recognition result output by the gesture recognition model includes: The gesture image is determined to be a static gesture image. The image features of the gesture image are input into the static gesture recognition layer to obtain the gesture recognition result output by the static gesture recognition layer. The gesture image is determined to be a dynamic gesture image. The image features of the gesture image are input into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer.

3. The gesture recognition method according to claim 2, characterized in that, The step of inputting the image features of the gesture image into the dynamic gesture recognition layer to obtain the gesture recognition result output by the dynamic gesture recognition layer includes: The image features are input into the static gesture recognition layer to obtain the first recognition result output by the static gesture recognition layer. The image features are input into the dynamic gesture recognition layer to obtain the second recognition result output by the dynamic gesture recognition layer; Based on the first recognition result and the second recognition result, the gesture recognition result is determined.

4. The gesture recognition method according to claim 2, characterized in that, The image features are determined based on the following steps: The gesture image is input into the feature extraction layer of the gesture recognition model to obtain the image features output by the feature extraction layer.

5. The gesture recognition method according to claim 1, characterized in that, The dynamic gesture recognition layer is trained based on the following steps: Based on the model parameters of the trained static gesture recognition layer, the model parameters of the dynamic gesture recognition layer are updated; The updated dynamic gesture recognition layer is trained based on sample dynamic gesture images.

6. The gesture recognition method according to claim 5, characterized in that, The training of the updated dynamic gesture recognition layer based on sample dynamic gesture images includes: The sample dynamic gesture image is input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample dynamic gesture image is input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer. Based on the first sample recognition result and the second sample recognition result, the sample gesture recognition result is determined; Based on the sample gesture recognition results, the updated dynamic gesture recognition layer is trained.

7. The gesture recognition method according to claim 6, characterized in that, The step of inputting the sample dynamic gesture image into the static gesture recognition layer to obtain a first sample recognition result output by the static gesture recognition layer, and inputting the sample dynamic gesture image into the updated dynamic gesture recognition layer to obtain a second sample recognition result output by the updated dynamic gesture recognition layer, includes: The sample dynamic gesture image is input into the feature extraction layer of the gesture recognition model to obtain the sample image features output by the feature extraction layer; The sample image features are input into the static gesture recognition layer to obtain the first sample recognition result output by the static gesture recognition layer, and the sample image features are input into the updated dynamic gesture recognition layer to obtain the second sample recognition result output by the updated dynamic gesture recognition layer.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the gesture recognition method as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the gesture recognition method as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the gesture recognition method as described in any one of claims 1 to 7.

Images