Hand Feature Extraction and Gesture Recognition Method, Electronic Device, and Storage Medium

Abstract

The present application discloses a hand feature extraction method, a gesture recognition method, an electronic device, and a computer-readable storage medium. The method includes: obtaining a hand image; respectively performing key point feature extraction and gesture feature extraction on the hand image to obtain hand key point features and gesture features; using one of the gesture features and the hand key point features to enhance the other, so as to update at least one of the hand key point features and the gesture features. In the above manner, the accuracy of the hand key point features and / or the gesture features can be improved.

Description

Technical Field

[0001] This application relates to the field of image processing, and in particular, to a method for extracting hand features, a method for gesture recognition, an electronic device, and a computer-readable storage medium. Background Art

[0002] The features of hand key points, that is, the features of hand key points (such as the joint points of fingers and the hand joint points near the wrist), can be used to determine hand key points, and the hand key points can be used to determine hand gestures, hand detail information, etc. Gesture features can be used to estimate hand gestures, such as 1, 2, 3, 4, 5, fist, like, ok, etc. Gesture features can be applied to fields such as human-computer interaction, virtual reality, and augmented reality.

[0003] However, the accuracy of the hand key point features / gesture features obtained by the current hand feature extraction methods is not high, which affects the application of the hand key point features / gesture features. Summary of the Invention

[0004] This application provides a method for extracting hand features, a method for gesture recognition, an electronic device, and a computer-readable storage medium, which can solve the problem that the accuracy of the hand key point features / gesture features obtained by the current hand feature extraction methods is not high.

[0005] To solve the above technical problems, a technical solution adopted by this application is: to provide a method for extracting hand features. The method includes: obtaining a hand image; respectively performing key point feature extraction and gesture feature extraction on the hand image to obtain hand key point features and gesture features; using one of the gesture features and the hand key point features to enhance the other, so as to update at least one of the hand key point features and the gesture features.

[0006] To solve the above technical problems, a technical solution adopted by this application is: to provide a method for gesture recognition. The method includes: obtaining a hand image; respectively performing key point feature extraction and gesture feature extraction on the hand image to obtain hand key point features and gesture features; using one of the gesture features and the hand key point features to enhance the other, so as to update at least one of the hand key point features and the gesture features; performing recognition based on the hand key point features and / or the gesture features to obtain the gesture of the hand.

[0007] To solve the above technical problems, another technical solution adopted by this application is: to provide an electronic device, which includes a processor and a memory connected to the processor. The memory stores program instructions; the processor is configured to execute the program instructions stored in the memory to implement the above method.

[0008] To solve the above technical problems, another technical solution adopted by this application is: to provide a computer-readable storage medium storing program instructions, which can implement the above method when executed.

[0009] In the above manner, after extracting the hand key point features and gesture features from the hand image, this application will further enhance the gesture features with the hand key point features to update the gesture features, and / or enhance the hand key point features with the gesture features to update the hand key point features. Since the gesture features play an auxiliary role in the process of obtaining the hand key point features, the updated gesture features take into account the hand key point features, and / or the hand key point features play an auxiliary role in the process of obtaining the gesture features, so that the updated hand key point features take into account the gesture features. Therefore, the accuracy is higher and the expression ability for the hand is stronger. Description of the Drawings

[0010] Figure 1 is a schematic flowchart of an embodiment of the hand feature extraction method of this application;

[0011] Figure 2 is a schematic diagram of hand key points;

[0012] Figure 3 is a schematic diagram of a gesture;

[0013] Figure 4 is a schematic diagram of hand key points in the case of the gesture "OK";

[0014] Figure 5 is Figure 1 the specific flowchart of S12 in

[0015] Figure 6 is a schematic flowchart of an embodiment of the training method of the hand feature extraction network of this application;

[0016] Figure 7 is a specific flowchart of an embodiment of the training method of the hand feature extraction network of this application;

[0017] Figure 8 is a schematic flowchart of an embodiment of the gesture recognition method of this application;

[0018] Figure 9 is a schematic structural diagram of an embodiment of the gesture recognition device of this application;

[0019] Figure 10 is a schematic structural diagram of an embodiment of the hand feature extraction device of this application;

[0020] Figure 11 is a schematic structural diagram of an embodiment of the electronic device of this application;

[0021] Figure 12It is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application; Detailed implementation manners

[0022] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0023] The terms "first", "second", and "third" in the present application are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first", "second", and "third" may explicitly or implicitly include at least one of such features. In the description of the present application, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0024] Referring to "embodiments" herein means that the specific features, structures, or characteristics described in conjunction with the embodiments may be included in at least one embodiment of the present application. The phrase appears in various places in the specification and does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment mutually exclusive with other embodiments. Those skilled in the art will explicitly and implicitly understand that, without conflict, the embodiments described herein may be combined with other embodiments.

[0025] Figure 1 It is a schematic flowchart of an embodiment of the hand feature extraction method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 1 the shown flowchart order.

[0026] As Figure 1 shown, this embodiment may include:

[0027] S11: Obtain a hand image.

[0028] The hand image may be an original image obtained by photographing a person's hand using a camera device, or a hand region in the original image. Among them, the hand region is obtained by performing hand detection or hand segmentation on the original image.

[0029] S12: Respectively perform key point feature extraction and gesture feature extraction on the hand image to obtain hand key point features and gesture features.

[0030] The hand key-point features can be used to represent hand key points. The hand key points may include 4 joint points of each finger, and joint points near the wrist, etc. The gesture features can represent the gestures of the hand.

[0031] S13: Use one of the gesture features and the hand key-point features to enhance the other, so as to update at least one of the hand key-point features and the gesture features.

[0032] The gesture features can be used to enhance the hand key-point features to update the hand key-point features. The enhancement method can be to directly fuse the hand key-point features and the gesture features in a multiplicative or weighted manner to obtain the updated hand key-point features. Or, the attention weights of each sub-feature in the hand key-point features can be determined based on the gesture features, and the attention weights of each sub-feature are multiplied by each sub-feature to obtain the updated hand key-point features.

[0033] It can be understood that different gestures have different attentions to different hand key points, and the attentions to obvious (visible) hand key points under each gesture are higher. Combine Figures 2 to 4 to illustrate:

[0034] Figure 2 is a schematic diagram of hand key points, which includes hand key points 1 to 20, where hand key point 0 is the joint point of the wrist, hand key points 1 to 4 are the joint points of the thumb, hand key points 5 to 8 are the joint points of the index finger, hand key points 9 to 12 are the joint points of the middle finger, hand key points 13 to 16 are the joint points of the ring finger, and hand key points 17 to 20 are the joint points of the little finger.

[0035] Figure 3 is a schematic diagram of gestures, which includes gestures "1", "2", "3", "OK", "like" and "clench fist". Gesture "1" pays more attention to hand key points 0, 1 to 4, 5 to 8, 10 to 11, 14 to 15, 17 to 20; gesture "2" pays more attention to hand key points 0, 1 to 4, 5 to 8, 9 to 12, 14 to 15, 17 to 20; gesture "3" pays more attention to 0, 1 to 4, 5 to 8, 9 to 12, 13 to 16, 17, 19 to 20; gesture "OK" pays more attention to 0, 1 to 3, 5 to 7, 9 to 12, 13 to 16, 17 to 20. For the schematic diagram of hand key points under gesture "OK", see Figure 4 ; gesture "like" pays more attention to hand key points 1 to 4, 5, 9, 13, 17. Gesture "clench fist" pays more attention to hand key points 1, 2, 5 to 6, 9 to 10, 13, 17.

[0036] Therefore, the gesture features imply obvious information of hand key points. By enhancing the hand key point features with the gesture features, it can play a guiding role, increase the attention to the sub-features corresponding to the obvious hand key points, thereby increasing the expression ability of the hand key point features for the hand key points. Subsequently, the hand key point features can be used to more accurately locate the hand key points.

[0037] The hand key point features can be used to enhance the gesture features to update the gesture features. Among them, the hand region feature and the background region feature in the gesture features can be determined based on the hand key point features; the background region feature is removed to obtain the updated gesture features. It can be understood that the hand features include the background region features other than the hand region features. For example, in a hand image, if the hand is holding a black cup and the background color of the hand image is green, the corresponding parts of the visible black cup region and the green region in the gesture features are the background region features. The background region features may interfere with the subsequent application of the hand features. Using the hand key point features to enhance the gesture features enables the gesture features to combine the structured information of the hand key points and effectively eliminates the interference of the background region features.

[0038] Alternatively, in a multiplicative or weighted manner, the hand key point features can act on the gesture features to obtain the updated gesture features. It can be understood that the hand key point features can represent the hand region. Therefore, by acting the hand key point features on the gesture features, the updated gesture features can implicitly contain potential hand region information and effectively reduce the interference of the background region features.

[0039] When using the hand key point features to enhance the gesture features, the hand key point features used can be obtained from S12 or the updated hand key point features. When using the gesture features to enhance the hand key point features, the gesture features used can be obtained from S12 or the updated gesture features.

[0040] Through the implementation of this embodiment, after the hand key point features and the gesture features are extracted from the hand image in this application, the hand key point features will also be used to enhance the gesture features to update the gesture features, and / or the gesture features will be used to enhance the hand key point features to update the hand key point features. Since the gesture features play an auxiliary role in the process of obtaining the hand key point features, the updated gesture features consider the hand key point features, and / or the hand key point features play an auxiliary role in the process of obtaining the gesture features, so that the updated hand key point features consider the gesture features. Therefore, the accuracy is higher and the expression ability for the hand is stronger.

[0041] Further, the hand feature extraction method provided by this application can be implemented based on a hand feature extraction network. The architecture of the hand feature extraction network can be Resnet, Hrnet, Transformer, etc.

[0042] The hand feature extraction network can include a key point feature extraction branch and a gesture feature extraction branch. The key point feature extraction branch is used for key point feature extraction tasks, and the gesture feature extraction branch is used for gesture feature extraction tasks. The key point feature extraction branch and the gesture feature extraction branch can be completely independent or share some network layers. For example, the key point feature extraction branch and the gesture feature extraction branch share a common feature layer. The key point feature extraction branch includes a shared feature extraction layer and a key point feature extraction layer connected in sequence. The gesture feature extraction branch includes a shared feature extraction layer and a gesture feature extraction layer connected in sequence. The shared feature extraction layer is used for common feature extraction. The key point feature extraction layer is used for key point feature extraction based on the common feature extraction result and updating the key point feature using the gesture feature. The gesture feature extraction layer is used for gesture feature extraction based on the common feature extraction result and updating the gesture feature using the hand key point feature.

[0043] Based on this, referring to Figure 5 , the above S12 can include the following sub-steps:

[0044] S121: Perform common feature extraction on the hand image to obtain common hand features.

[0045] Among them, the common hand features can include basic information in the hand image, including but not limited to color information and texture information.

[0046] S122: Perform key point feature extraction and gesture feature extraction on the common hand features respectively to obtain hand key point features and gesture features respectively.

[0047] Compared with the common hand features, the gesture features contain deeper semantic information and can express the gestures of the hand.

[0048] It can be understood that based on the hand feature extraction network to implement hand feature extraction, the hand image only needs to be sent into the hand feature extraction network once, and the updated hand key point features and updated gesture features of the hand image can be obtained, so the feature extraction efficiency is high.

[0049] Further, before applying the hand feature extraction network to the hand feature extraction method, the hand feature extraction network can be trained.

[0050] The hand feature extraction network can be trained based on sample hand images, and the sample hand images are labeled with the true results of sample hand key point features and the true results of sample gesture features. The training mode is adapted to the structures of the gesture feature extraction branch and the key point feature extraction branch.

[0051] In some embodiments, if the gesture feature extraction branch and the key point feature extraction branch are completely independent, the gesture feature extraction branch and the key point feature extraction branch can be trained separately. For example, after training the gesture feature extraction branch to the expectation first, then train the key point feature extraction branch to the expectation.

[0052] In some embodiments, if the gesture feature extraction branch and the key point feature extraction branch share some network layers, the gesture feature extraction branch and the key point feature extraction branch can be jointly trained.

[0053] The following takes the joint training as an example for illustration:

[0054] Figure 6 is a schematic flowchart of an embodiment of the training method of the hand feature extraction network of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 6 the shown flow order. As Figure 6 shown, this embodiment may include:

[0055] S21: Respectively perform key point feature extraction and gesture feature extraction on the sample hand images to obtain the predicted results of the sample hand key point features and the predicted results of the sample gesture features.

[0056] S22: Use one of the predicted results of the sample gesture features and the predicted results of the sample hand key point features to enhance the other, so as to update at least one of the predicted results of the hand key point features and the gesture features.

[0057] S23: Based on the difference between the predicted result and the true result of the gesture feature, obtain the first loss, and based on the difference between the predicted result and the true result of the hand key point feature, obtain the second loss.

[0058] The first loss and the second loss include but are not limited to mean square error loss, L1 loss, and so on.

[0059] S24: Adjust the parameters of the hand feature extraction network based on the first loss and the second loss.

[0060] The first loss and the second loss can be combined to obtain the final loss; adjust the parameters of the hand feature extraction network based on the final loss.

[0061] The weights of the first loss and the second loss in the final loss can be fixed or dynamic. In the dynamic case, the weight of the first loss in the final loss is positively correlated with the first difference, and the weight of the second loss in the final loss is positively correlated with the second difference. The first difference is the difference between the first loss and the second loss, and the second difference is the difference between the second loss and the first loss. The sum of the weight of the first loss and the weight of the second loss is fixed.

[0062] For example, a difference threshold can be set. If the first difference between the first loss and the second loss is greater than the difference threshold, the weight of the first loss is increased by a certain proportion, and correspondingly, the weight of the second loss is decreased according to the adjusted weight of the first loss. If the second difference between the second loss and the first loss is greater than the difference threshold, the weight of the second loss is increased by a certain proportion; and correspondingly, the weight of the first loss is decreased according to the adjusted weight of the second loss.

[0063] It can be understood that in the dynamic case, when the hand feature extraction network focuses on the gesture feature extraction task, that is, when the first loss is greater than the second loss, the weight of the first loss is positively correlated with the first difference, that is, when the first difference is large, the weight of the first loss is automatically increased, and vice versa. The same is true when focusing on the key point feature extraction task. Thus, during the end-to-end training of the hand feature extraction network, it does not favor any one feature extraction task, enabling better training of the feature extraction branches of the two feature extraction tasks.

[0064] In this embodiment, the sample hand image only needs to be input into the hand feature extraction network once, and the prediction results of the updated hand key point features and the updated gesture features of the sample hand image can be obtained, with high training efficiency.

[0065] For other detailed descriptions of this embodiment, please refer to the previous embodiments and will not be elaborated here.

[0066] Through the implementation of this embodiment, during the training of the hand feature extraction network, the gesture feature task and the hand key point feature extraction task are coupled and promoted, which can improve the training effect.

[0067] The following combines Figure 7 , and illustrates the training of the hand feature extraction network in the form of an example:

[0068] 1) Input the sample hand image into the hand feature extraction network.

[0069] 2) Extract the sample common hand features based on the sample hand image.

[0070] 3) Extract the sample hand key point features (key point feature head) and sample gesture features (gesture feature head) based on the sample common hand features.

[0071] 4) Obtain the attention weights of each sub - feature in the sample hand key - point features using the sample gesture features, and apply the attention weights to the sample hand key - point features (sample hand key - point feature extraction based on sample gesture attention) to update the sample hand key - point features.

[0072] 5) Determine the sample background region features in the sample gesture features using the sample hand key - point features, and remove the sample background region features (sample gesture feature extraction based on sample hand key - point structure) to update the sample gesture features.

[0073] 6) Calculate the first loss (KPLOSS) based on the sample gesture features, and calculate the second loss (OCLOSS) based on the sample hand key - point features.

[0074] 7) Dynamically update the weights of the first loss and the second loss based on the first loss and the second loss.

[0075] 8) Combine the first loss and the second loss based on the updated weights to obtain the final loss.

[0076] 9) Adjust the parameters of the hand feature extraction network based on the final loss.

[0077] Furthermore, the foregoing feature extraction method can have various application scenarios, such as judging hand gestures, hand detail information, etc. The following introduces the hand gesture recognition method, which can be specifically as follows:

[0078] Figure 8 It is a schematic flowchart of an embodiment of the gesture recognition method of the present application. It should be noted that if there are substantially the same results, this embodiment is not limited to Figure 8 the shown process sequence. As Figure 8 shown, this embodiment may include:

[0079] S31: Obtain a hand image.

[0080] S32: Respectively perform key - point feature extraction and gesture feature extraction on the hand image to obtain hand key - point features and gesture features.

[0081] S33: Use one of the gesture features and the hand key - point features to enhance the other, so as to update at least one of the hand key - point features and the gesture features.

[0082] S34: Perform recognition based on the hand key - point features and / or gesture features to obtain the hand gesture.

[0083] In some embodiments, hand key - points can be determined based on the hand key - point features; the hand gesture can be obtained based on the hand key - points.

[0084] In some embodiments, hand key points may be determined based on hand key point features; and a hand gesture may be determined based on the hand key points and gesture features.

[0085] In some embodiments, a hand gesture may be determined based on gesture features.

[0086] For a detailed description of this embodiment, please refer to the previous embodiments and will not be elaborated here.

[0087] Through the implementation of this embodiment, in the present application, instead of directly extracting key point features and gesture features and applying the obtained hand key point features and gesture features to hand gesture recognition, one of the gesture features and hand key point features is used to enhance the other to update at least one of the hand key point features and gesture features, and then the updated result is applied to hand gesture recognition. Since the updated result has a stronger gesture expression ability for the hand, the accuracy of the finally determined hand gesture can be improved.

[0088] Further, Figure 9 a gesture recognition device for implementing the gesture recognition method will be described. As Figure 9 shown, the gesture recognition device may include a first camera module and a gesture recognition module. The first camera module may be a camera device including several cameras. The first camera module is used to obtain hand images, and the gesture recognition module is used to process the hand images to obtain hand gestures.

[0089] A specific hand gesture recognition scenario is introduced as follows:

[0090] A mobile phone includes a first camera module, a gesture recognition module, and a control module. The mobile phone APP is operated using gestures. Different gesture-to-operation mappings in the APP are pre-stored in the APP. For example, the gesture "like" corresponds to opening the APP, and the gestures "1", "2", "3", and "4" respectively correspond to different functions of the APP, and the gesture "OK" corresponds to turning the page. During the user's use of the mobile phone, the hand image of the user is detected in real time using the camera module to obtain the hand image, the gesture recognition module is used to recognize the hand gesture of the user based on the hand image, and the control module executes corresponding operations based on the recognized gesture.

[0091] Figure 10 is a schematic structural diagram of a hand feature extraction device of the present application. As Figure 10 shown, the hand feature extraction device includes a second camera module 11, a first feature extraction module 12, and a second feature extraction module 13.

[0092] Among them, the second imaging module 11 can be used to acquire a hand image; the first feature extraction module 12 can respectively perform key point feature extraction and gesture feature extraction on the hand image to obtain hand key point features and gesture features; the second feature extraction module 13 can use one of the gesture features and the hand key point features to enhance the other, so as to update at least one of the hand key point features and the gesture features.

[0093] By implementing this embodiment of the present application, after the present application extracts hand key point features and gesture features from a hand image by using the first feature extraction module, the first feature extraction module is further used to enhance the gesture features based on the hand key point features to update the gesture features, and / or enhance the hand key point features based on the gesture features to update the hand key point features. Since the gesture features play an auxiliary role in the process of obtaining the hand key point features, the updated gesture features take into account the hand key point features, and / or the hand key point features play an auxiliary role in the process of obtaining the gesture features, so that the updated hand key point features take into account the gesture features. Therefore, the accuracy is higher and the expression ability for the hand is stronger.

[0094] Figure 11 It is a schematic structural diagram of an embodiment of an electronic device of the present application. As Figure 11 shown, the electronic device includes a processor 21 and a memory 22 coupled to the processor 21.

[0095] Among them, the memory 22 stores program instructions for implementing the method of any of the above embodiments; the processor 21 is used to execute the program instructions stored in the memory 22 to implement the steps of the above method embodiments. Among them, the processor 21 can also be called a CPU (Central Processing Unit, central processing unit). The processor 21 may be an integrated circuit chip with signal processing capabilities. The processor 21 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0096] Figure 12 It is a schematic structural diagram of an embodiment of a computer-readable storage medium of the present application. As Figure 12As shown in the figure, the computer-readable storage medium 30 of the embodiment of the present application stores program instructions 31, and when the program instructions 31 are executed, the methods provided in the above embodiments of the present application are implemented. Among them, the program instructions 31 can form a program file and be stored in the above computer-readable storage medium 30 in the form of a software product, so that a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor can execute all or part of the steps of the methods in various embodiments of the present application. The aforementioned computer-readable storage medium 30 includes: various media that can store program codes such as USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, or terminal devices such as computers, servers, mobile phones, and tablets.

[0097] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical, or other forms.

[0098] In addition, each functional unit in various embodiments of the present application can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit. The above is only the embodiment of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the specification and drawings of the present application, or directly or indirectly applied in other related technical fields, is equally included in the patent protection scope of the present application.

Claims

1. A hand feature extraction method, characterized in that, Including: Obtain a hand image; Respectively perform key-point feature extraction and gesture feature extraction on the hand image to obtain hand key-point features and gesture features; Use one of the gesture features and the hand key-point features to enhance the other, so as to update at least one of the hand key-point features and the gesture features; The using one of the gesture features and the hand key-point features to enhance the other includes: determining the attention weights of each sub-feature in the hand key-point features based on the gesture features; Multiply the attention weights of each sub-feature by each sub-feature to obtain the updated hand key-point features.

2. The method according to claim 1, wherein The using one of the gesture features and the hand key-point features to enhance the other includes: Determine the hand region feature and the background region feature in the gesture features based on the hand key-point features; Remove the background region feature to obtain the updated gesture features.

3. The method according to claim 1, wherein The respectively performing key-point feature extraction and gesture feature extraction on the hand image includes: Perform common feature extraction on the hand image to obtain common hand features; Respectively perform key-point feature extraction and gesture feature extraction on the common hand features to respectively obtain the hand key-point features and the gesture features.

4. The method according to claim 1, characterized in that, The hand feature extraction method is implemented based on a hand feature extraction network, and the hand feature extraction network is trained based on sample hand images, and the sample hand images are labeled with the true results of sample hand key-point features and the true results of sample gesture features.

5. The method according to claim 4, wherein The training steps of the hand feature extraction network include: Respectively perform key-point feature extraction and gesture feature extraction on the sample hand images to obtain the predicted results of sample hand key-point features and the predicted results of sample gesture features; Use one of the predicted results of the sample gesture features and the predicted results of the sample hand key-point features to enhance the other, so as to update at least one of the predicted results of the hand key-point features and the predicted results of the gesture features; Obtain a first loss based on the difference between the predicted result and the true result of the gesture features, and obtain a second loss based on the difference between the predicted result and the true result of the hand key-point features; Adjust the parameters of the hand feature extraction network based on the first loss and the second loss.

6. The method according to claim 5, characterized in that, The adjusting the parameters of the hand feature extraction network based on the first loss and the second loss includes: Combine the first loss and the second loss to obtain a final loss, the weight of the first loss in the final loss is positively correlated with a first difference, the weight of the second loss in the final loss is positively correlated with a second difference, the first difference is the difference between the first loss and the second loss, and the second difference is the difference between the second loss and the first loss; Adjust the parameters of the hand feature extraction network based on the final loss.

7. A gesture recognition method, characterized in that Including: Obtain a hand image; Respectively perform key-point feature extraction and gesture feature extraction on the hand image to obtain hand key-point features and gesture features; Enhance one of the gesture features and the hand key point features with the other to update at least one of the hand key point features and the gesture features; Based on the hand key point features and / or the gesture features, perform recognition to obtain the gesture of the hand; The enhancing one of the gesture features and the hand key point features with the other includes determining the attention weights of the sub-features in the hand key point features based on the gesture features; multiplying the attention weights of the sub-features by the sub-features to obtain the updated hand key point features.

8. An electronic device, characterized in that, Comprising a processor and a memory connected to the processor, wherein, The memory stores program instructions; The processor is configured to execute the program instructions stored in the memory to implement the method according to any one of claims 1-7.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that can be executed by a processor and, when executed, implement the method according to any one of claims 1-7.

Images