Gesture recognition method and device, electronic device, and storage medium

Abstract

Embodiments of the present disclosure disclose a gesture recognition method and device, electronic equipment and a storage medium. The gesture recognition method comprises: obtaining image information based on an image of a gesture action collected; obtaining electromyographic information sent by a second device, wherein the electromyographic information is determined by the second device based on an electromyographic signal corresponding to the gesture action collected; and obtaining a gesture recognition result of the gesture action based on the image information and the electromyographic information. The gesture recognition disclosed in the embodiments of the present disclosure can accurately improve the accuracy of the gesture action.

Description

Technical Field

[0001] This disclosure relates to, but is not limited to, the field of gesture recognition or control technology, and in particular to a gesture recognition method, device, electronic device, and storage medium. Background Technology

[0002] In related technologies, human-computer interaction can be achieved through contact interaction or contactless interaction. Contact interaction can be performed using interactive devices such as keyboards, mice, joysticks, touchscreens, and microphones. However, existing contact interaction requires additional specific hardware, occupies the user's hands, provides a poor user experience, and cannot accurately detect interactive actions. Summary of the Invention

[0003] This disclosure provides a gesture recognition method, apparatus, user equipment, and storage medium.

[0004] According to a first aspect of this disclosure, a gesture recognition method is provided, executed by a first device, the method comprising:

[0005] Image information is obtained based on the captured images of hand gestures;

[0006] Acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the gesture;

[0007] Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained.

[0008] In some embodiments, obtaining the gesture recognition result of the gesture action based on the image information and the electromyographic information includes:

[0009] Based on the image information, obtain the first gesture information of the first type of gesture in the gesture action;

[0010] Based on the electromyographic information, the second gesture information of the second type of gesture in the gesture action is obtained;

[0011] Based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained;

[0012] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0013] In some embodiments, the first type of action includes at least one of the following: arm movement, wrist movement, and finger movement;

[0014] The second category of movements includes at least one of the following: wrist movements and finger movements.

[0015] In some embodiments, obtaining second gesture information of the second type of gesture in the gesture based on the electromyographic information includes:

[0016] Based on the electromyographic information, the second type of movement in the gesture is determined;

[0017] The second gesture information is determined based on the second type of action.

[0018] In some embodiments, determining the second type of movement in the gesture based on the electromyographic information includes one of the following:

[0019] If the electromyographic information indicates that the electromyographic signal in at least one first detection area is greater than a predetermined threshold, then the finger movement of at least one finger corresponding to the first detection area is determined.

[0020] If the electromyographic information indicates that the electromyographic signal in at least one of the first and second detection areas is greater than the predetermined threshold, wrist rotation is determined.

[0021] The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

[0022] In some embodiments, the method further includes:

[0023] Obtain motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action;

[0024] The determination of the second type of movement in the gesture based on the electromyographic information includes:

[0025] If the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold, the second type of movement is determined based on the motion information.

[0026] In some embodiments, obtaining the gesture recognition result of the gesture action based on the image information and the electromyographic information includes:

[0027] Based on the image information and the electromyographic information, obtain the first gesture information of the first type of gesture in the gesture action;

[0028] Based on the image information and the electromyographic information, the second gesture information of the second type of gesture in the gesture action is obtained;

[0029] Based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained;

[0030] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0031] In some embodiments, obtaining the first gesture information of the first type of gesture in the gesture based on the image information and the electromyographic information includes:

[0032] Based on the image information, obtain the first sub-gesture information of the first type of gesture in the gesture action;

[0033] Based on the electromyographic information, the second sub-gesture information of the first type of gesture in the gesture action is obtained;

[0034] Based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, the first gesture information is obtained; wherein the first weight is greater than the second weight.

[0035] And / or,

[0036] The step of obtaining second gesture information for the second type of gesture in the gesture based on the image information and the electromyographic information includes:

[0037] Based on the image information, obtain the third sub-gesture information of the second type of gesture in the gesture action;

[0038] Based on the electromyographic information, the fourth sub-gesture information of the second type of gesture in the gesture action is obtained;

[0039] Based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, the second gesture information is obtained; wherein the third weight is less than the fourth weight.

[0040] According to a second aspect of the present disclosure, a gesture recognition method is provided, executed by a second device, comprising:

[0041] Acquire image information sent by a first device, wherein the image information is determined by the first device based on images of collected hand gestures;

[0042] Electromyographic signals corresponding to the gestures are collected to obtain electromyographic information;

[0043] Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained.

[0044] According to a third aspect of the present disclosure, a gesture recognition method is provided, executed by a third device, comprising:

[0045] Acquire image information sent by a first device, wherein the image information is determined by the first device based on images of collected hand gestures;

[0046] Acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the gesture;

[0047] Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained.

[0048] According to a fourth aspect of the present disclosure, a gesture recognition device is provided, applied to a first device, comprising:

[0049] The first acquisition module is used to obtain image information based on the acquired gesture images;

[0050] The first acquisition module is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action.

[0051] The first processing module is used to obtain the gesture recognition result of the gesture based on the image information and the electromyographic information.

[0052] In some embodiments, the first processing module is configured to obtain first gesture information of a first type of gesture in the gesture based on the image information; and to obtain second gesture information of a second type of gesture in the gesture based on the electromyographic information.

[0053] The first processing module is used to obtain the gesture recognition result of the gesture action based on the first gesture information and the second gesture information;

[0054] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0055] In some embodiments, the first type of action includes at least one of the following: arm movement, wrist movement, and finger movement;

[0056] The second category of movements includes at least one of the following: wrist movements and finger movements.

[0057] In some embodiments, the first acquisition module is configured to determine a second type of action in the gesture based on the electromyographic information;

[0058] The first acquisition module is used to determine the second gesture information based on the second type of action.

[0059] In some embodiments, the first acquisition module is configured to determine the finger movement of at least one finger corresponding to the first detection area if the electromyographic information indicates that the electromyographic signal of at least one first detection area is greater than a predetermined threshold.

[0060] or,

[0061] The first acquisition module is configured to determine wrist rotation if the electromyographic information indicates that the electromyographic signal in at least one of the first and second detection areas is greater than the predetermined threshold.

[0062] The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

[0063] In some embodiments, the apparatus further includes:

[0064] The first acquisition module is used to acquire motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action;

[0065] The first acquisition module is used to determine the second type of action based on the motion information if the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold.

[0066] In some embodiments, the first processing module is configured to obtain first gesture information of a first type of gesture in the gesture based on the image information and the electromyographic information; and to obtain second gesture information of a second type of gesture in the gesture based on the image information and the electromyographic information.

[0067] The first processing module is used to obtain the gesture recognition result of the gesture action based on the first gesture information and the second gesture information;

[0068] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0069] In some embodiments, the first processing module is configured to obtain first sub-gesture information of a first type of gesture in the gesture based on the image information; and to obtain second sub-gesture information of the first type of gesture in the gesture based on the electromyographic information;

[0070] The first processing module is configured to obtain the first gesture information based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight; wherein the first weight is greater than the second weight;

[0071] And / or,

[0072] The first processing module is configured to obtain, based on the image information, the third sub-gesture information of the second type of gesture in the gesture action; and based on the electromyographic information, obtain the fourth sub-gesture information of the second type of gesture in the gesture action;

[0073] The first processing module is used to obtain the second gesture information based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight; wherein the third weight is less than the fourth weight.

[0074] According to a fifth aspect of the present disclosure, a gesture recognition device is provided, executed by a second device, comprising:

[0075] The second acquisition module is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0076] The second acquisition module is used to acquire the electromyographic signals corresponding to the gesture movements to obtain electromyographic information.

[0077] The second processing module is used to obtain the gesture recognition result of the gesture based on the image information and the electromyographic information.

[0078] According to a sixth aspect of the present disclosure, a gesture recognition device is provided, executed by a third device, comprising:

[0079] The third acquisition module is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0080] The third acquisition module is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action.

[0081] The third processing module is used to obtain the gesture recognition result of the gesture based on the image information and the electromyographic information.

[0082] According to a seventh aspect of the present disclosure, an electronic device is provided, comprising:

[0083] processor;

[0084] Memory used to store processor-executable instructions;

[0085] The processor is configured to implement the gesture recognition method described in any embodiment of this disclosure when running the executable instructions.

[0086] According to an eighth aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing an executable program, wherein, when the executable program is executed by a processor, it implements the gesture recognition method described in any embodiment of the present disclosure.

[0087] The technical solutions provided by the embodiments of this disclosure may include the following beneficial effects:

[0088] In this embodiment, a first device can acquire image information of a captured gesture, and a second device can acquire electromyographic (EMG) information corresponding to the gesture. Based on the image information and EMG information, a gesture recognition result is obtained. Thus, in this embodiment, both the image information acquired by the first device and the EMG information acquired by the second device can be synchronized to the first device; the gesture can be recognized by combining the image information and EMG information. This achieves collaborative processing of gesture recognition between the first and second devices and improves the accuracy of gesture recognition.

[0089] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0090] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0091] Figure 1 This is a schematic diagram illustrating a gesture recognition method according to an exemplary embodiment.

[0092] Figure 2 This is a block diagram illustrating a second device for electromyography detection according to an exemplary embodiment.

[0093] Figure 3 This is a block diagram illustrating an image acquisition method using a first device according to an exemplary embodiment.

[0094] Figure 4 This is a schematic diagram illustrating a gesture recognition method according to an exemplary embodiment.

[0095] Figure 5 This is a schematic diagram illustrating a gesture recognition method according to an exemplary embodiment.

[0096] Figure 6 This is a schematic diagram illustrating a gesture recognition method according to an exemplary embodiment.

[0097] Figure 7 This is a block diagram illustrating a gesture recognition device according to an exemplary embodiment.

[0098] Figure 8 This is a block diagram illustrating a gesture recognition device according to an exemplary embodiment.

[0099] Figure 9This is a block diagram illustrating a gesture recognition device according to an exemplary embodiment.

[0100] Figure 10 This is a block diagram illustrating an electronic device according to an exemplary embodiment. Detailed Implementation

[0101] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0102] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0103] like Figure 1 This is an exemplary embodiment of a gesture recognition method; such as Figure 1 As shown, the gesture recognition method is executed by a first device and includes the following steps:

[0104] Step S11: Obtain image information based on the captured images of hand gestures;

[0105] Step S12: Obtain electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the hand gesture;

[0106] Step S13: Based on the image information and the electromyographic information, obtain the gesture recognition result of the gesture action.

[0107] In one embodiment, the first device may be a mobile device or a fixed device; for example, the first device may be a mobile phone, computer, server, tablet computer, etc.

[0108] In one embodiment, the first device includes an image acquisition module; this image acquisition module can be any device, component, or apparatus capable of acquiring images. For example, the image acquisition module of the first device can be a camera. The image acquisition module here includes an image recognition sensor.

[0109] In one embodiment, the second device may be, but is not limited to, a wearable device; for example, the second device may be a watch or a bracelet, etc.

[0110] In one embodiment, acquiring electromyographic information transmitted by the second device in step 12 includes acquiring electromyographic information transmitted by one or more second devices. For example, the second device is worn on the left hand and / or the right hand.

[0111] In one embodiment, the second device includes an electromyography (EMG) sensor. For example, such as... Figure 2 As shown, the second device is worn on the left arm. The inner side of the second device is in close contact with the skin of the arm, and the major muscles and / or tendons of the fingers and wrist are close to the electromyography (EMG) sensor of the second device; for example, the flexor digitorum superficialis and tendons, flexor pollicis longus and tendons, and / or flexor retinaculum are close to the EMG sensor of the second device.

[0112] The electromyography (EMG) sensor here can be a single electrode that covers the major muscles and tendons of the fingers and wrist. Alternatively, it can consist of multiple electrodes, with one electrode covering the major muscles and tendons of a finger, or another electrode covering the major muscles and tendons of the wrist. In this way, the EMG sensor can detect the electromyographic signals of the superficial flexor muscles of each finger during contraction, as well as the electromyographic signals of the wrist flexor retinaculum.

[0113] In one embodiment, the major muscles and / or tendons of the fingers correspond to the electrode areas of the electromyography (EMG) sensor as a first detection area; the major muscles and / or tendons of the wrist correspond to the electrode areas of the EMG sensor as a second detection area. For example, after the wristband is worn, the flexor digitorum superficialis muscles of the fingers correspond to the electrode areas of the EMG sensor of the second device as a first detection area; the flexor pollicis longus muscles of the thumb correspond to the electrode areas of the EMG sensor of the second device as a first detection area; and the flexor retinaculum of the wrist corresponds to the electrode areas of the EMG sensor of the second device as a second detection area.

[0114] In one embodiment, prior to step S11, the method includes: acquiring at least one frame of an image of the gesture. For example, a camera of a first device acquires at least one frame of an image when a specific object performs a gesture. This specific object includes a person. For example, such as... Figure 3 As shown, the first device is a mobile phone, which uses its camera to capture images of hand gestures.

[0115] In one embodiment, step S11 includes: obtaining image information based on the captured image of the gesture, including: obtaining an image containing a hand from the captured image of the gesture; and determining the image information based on the image of the hand. Thus, embodiments of this disclosure can perform image recognition only based on the image portion containing the hand, thereby saving the cost of the first device.

[0116] In one embodiment, the gesture includes: a first type of gesture and / or a second type of gesture; wherein the amplitude of the first type of gesture is greater than the amplitude of the second type of gesture. For example, the first type of gesture may be an arm swing, and the second type of gesture may be a wrist rotation or finger bending. Alternatively, the first type of gesture may be a wrist rotation angle of 120 degrees; and the second type of gesture may be a wrist rotation angle of 90 degrees.

[0117] In other embodiments, the motion frequency of the first type of action is greater than the motion frequency of the second type of action.

[0118] In one embodiment, the gesture includes, but is not limited to, at least one of the following: arm movements, wrist movements, and finger movements. Arm movements include, but are not limited to, arm swinging and / or arm movement. Wrist movements include, but are not limited to, wrist rotation. Finger movements include, but are not limited to, bending one or more fingers.

[0119] In one embodiment, step S13 includes: performing image recognition of the gesture based on the image information, and performing electromyographic signal analysis of the gesture based on the electromyographic information, to obtain a gesture recognition result for recognizing the gesture.

[0120] The image recognition of the gesture action in the image can be performed by comparing the gesture action contained in the image information with a preset gesture action, and determining the preset gesture action that matches the gesture action contained in the image information as the gesture recognition result.

[0121] The electromyographic signal analysis of the gesture based on the electromyographic information can be as follows: determine the gesture recognition result of the gesture based on the intensity of the electromyographic signal and / or the detection area corresponding to the electromyographic signal.

[0122] The gesture recognition result here can be, but is not limited to: the gesture is a gesture to control the first device, the gesture is a sign language gesture, or the gesture is an AR or VR virtual reality interaction gesture. If the gesture is a gesture to control the first device, then the gesture can be, but is not limited to: turning the first device on or off, entering a predetermined application (APP), or performing a predetermined operation in the predetermined APP. Entering the predetermined APP here can be entering a shopping APP or a browser APP, etc.; performing a predetermined operation in the predetermined APP can be selecting goods in a shopping APP or reading articles in a browser APP, etc.

[0123] In this embodiment, a first device can acquire image information of a captured gesture, and a second device can acquire electromyographic (EMG) information corresponding to the gesture. Based on the image information and EMG information, a gesture recognition result is obtained. Thus, in this embodiment, both the image information acquired by the first device and the EMG information acquired by the second device can be synchronized to the first device; the gesture can be recognized by combining the image information and EMG information. This achieves collaborative processing of gesture recognition between the first and second devices and improves the accuracy of gesture recognition.

[0124] like Figure 4 As shown, in some embodiments, step S13 includes:

[0125] Step S131: Based on the image information, obtain the first gesture information of the first type of gesture in the gesture action;

[0126] Step S132: Based on the electromyographic information, obtain the second gesture information of the second type of gesture in the gesture action;

[0127] Step S133: Based on the first gesture information and the second gesture information, obtain the gesture recognition result of the gesture action;

[0128] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0129] In one embodiment, the first type of action includes at least one of the following: arm movement, wrist movement, and finger movement; the second type of action includes at least one of the following: wrist movement and finger movement.

[0130] In one embodiment, the amplitude of motion refers to the distance or angle by which a part of the body moves relative to a reference position. For example, the amplitude of motion of the first type of movement being greater than that of the second type of movement can mean that: the distance the first type of movement moves relative to a reference position is greater than the distance the second type of movement moves relative to a reference position; and / or, the angle of rotation of the first type of movement relative to a reference position is greater than the angle of rotation of the second type of movement relative to a reference position. For example, consider the first type of movement of swinging the right arm and the second type of movement of rotating the right wrist; if the vertical direction of the arm is used as the reference position, the first type of movement typically moves about 20 centimeters relative to that reference position, while the second type of movement moves about 5 centimeters; and / or if the parallel direction to the vertical direction of the arm is used as the axis of rotation (where this axis of rotation is the reference position), the first type of movement typically rotates 120 degrees relative to that axis of rotation, while the second type of movement typically rotates 60 degrees relative to that axis of rotation; therefore, the amplitude of motion of the first type of movement is greater than that of the second type of movement.

[0131] Arm movements here include, but are not limited to, at least one of the following: arm swinging or arm movement. Wrist movements here include, but are not limited to, wrist rotation. Finger movements here include, but are not limited to, finger bending.

[0132] The arm swing here can be in any direction; for example, swinging the arm upwards, downwards, etc. The wrist rotation here can also be in any direction; for example... Figure 1 The rotation shown is from top to bottom; it can also be a counter-clockwise or clockwise rotation of the wrist, etc. The fingers here can also be bent at any angle.

[0133] For example, the first device can acquire a first type of arm swinging motion through image information; the first device can acquire electromyographic information and determine a second type of wrist rotation motion based on the electromyographic information. Then the first device can recognize the gesture by combining the arm swinging and wrist rotation motions.

[0134] For example, the first device can acquire a first type of wrist rotation movement through image information; the first device can acquire electromyographic information and determine a second type of finger bending movement based on the electromyographic information. Then the first device can recognize the gesture by combining the wrist rotation and finger bending movements.

[0135] The first gesture information here is information used to describe a first type of action. For example, if the first type of action is an arm swinging action, then the first gesture information is information describing the arm swinging action; for example, the first gesture information describes that the arm swings from left to right at an angle of 60 degrees, etc.

[0136] The second gesture information here is information used to describe a second type of action. For example, if the second type of action is a finger bending action, then the second gesture information is information describing the finger bending action; for example, the second gesture information describes the bending of the right index finger, whether the index finger is bent completely from top to bottom, and the degree of bending of the index finger, etc.

[0137] In this embodiment of the present disclosure, image information can be used to determine a first type of movement, such as the arm, wrist, and fingers, with a relatively large range of motion, and electromyography information can be used to determine a second type of movement, such as the wrist and fingers, with a relatively small range of motion. Thus, this embodiment of the present disclosure can identify more detailed gestures, thereby further improving the accuracy of gesture recognition.

[0138] In some embodiments, step S131 includes:

[0139] Based on the image information, the first type of gesture is determined;

[0140] Based on the first type of action, the first gesture information is determined.

[0141] In some embodiments, step S132 includes:

[0142] Based on the electromyographic information, the second type of movement in the gesture is determined;

[0143] The second gesture information is determined based on the second type of action.

[0144] In some embodiments, determining the second type of movement in the gesture based on the electromyographic information includes one of the following:

[0145] If the electromyographic information indicates that the electromyographic signal in at least one first detection area is greater than a predetermined threshold, then the finger movement of at least one finger corresponding to the first detection area is determined.

[0146] If the electromyographic information indicates that the electromyographic signal in at least one of the first and second detection areas is greater than the predetermined threshold, wrist rotation is determined.

[0147] The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

[0148] The gesture information here describes the gesture actions. The first gesture information describes the first type of action; the second gesture information describes the second type of action.

[0149] The first detection area here can be the detection area of ​​one electrode of the electromyography sensor; the second detection area can also be the detection area of ​​one electrode of the electromyography sensor.

[0150] The flexor muscles of the fingers mentioned here include the flexor digitorum superficialis and / or the flexor pollicis longus. For example, the flexor muscles of the fingers could refer to the flexor digitorum superficialis of the index finger or the flexor pollicis longus of the thumb.

[0151] If the electromyographic (EMG) signal in the first detection area corresponding to one finger is greater than a predetermined threshold, it indicates that the finger is bent; if the EMG signals in the first detection areas corresponding to multiple fingers are greater than the predetermined threshold, it indicates that the multiple fingers are bent. If the EMG signals in the second detection area corresponding to the wrist and the first detection areas corresponding to at least one finger are both greater than the predetermined threshold, it indicates that the wrist is rotated.

[0152] The magnitude of the electromyographic signal here is directly proportional to the degree of finger flexion; the magnitude of the electromyographic signal here is directly proportional to the amplitude of wrist rotation.

[0153] In this embodiment of the disclosure, based on the signal intensity of the electromyographic signal and / or the location of the detection area corresponding to the electromyographic signal, it can be accurately determined whether it is the bending of one or more fingers, or the rotation of the wrist. Thus, this embodiment of the disclosure can accurately determine hand gestures of the second type of movement, such as fingers and wrists, with relatively small amplitudes of motion.

[0154] In some embodiments, the method further includes:

[0155] Obtain motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action;

[0156] The determination of the second type of movement in the gesture based on the electromyographic information includes:

[0157] If the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold, the second type of movement is determined based on the motion information.

[0158] In one embodiment, the second device further includes a motion sensor. This motion sensor includes, but is not limited to, at least one of the following: an accelerometer and an angular velocity sensor. The motion sensor here is used to detect the speed and / or direction of movement of the arm, wrist, etc.; for example, detecting hand gestures of arm movement, hand gestures of wrist movement, etc.

[0159] In this embodiment of the disclosure, if the electromyographic signals in the first detection area and the second detection area are less than or equal to a predetermined threshold, a second type of action is also determined based on motion information. For example, if the electromyographic information determines that the wrist is not rotating, but the motion information determines that the wrist is moving, then the hand gesture of wrist movement can be determined based on the motion information. Thus, the received motion information can assist in the detection of the second type of action, thereby reducing the probability of erroneous detection when the electromyographic sensor fails to detect electromyographic signals due to external environmental factors.

[0160] Of course, in other embodiments, the motion information can also be used to determine the first type of action. For example, based on the motion data of arm movement in the running information, the first type of arm movement can be determined. In this way, motion information can also assist in the detection of the first type of action to further improve the accuracy of recognizing gestures.

[0161] In some embodiments, step S13 includes:

[0162] Based on the image information and the electromyographic information, obtain the first gesture information of the first type of gesture in the gesture action;

[0163] Based on the image information and the electromyographic information, the second gesture information of the second type of gesture in the gesture action is obtained;

[0164] Based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained;

[0165] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0166] In the embodiments of this disclosure, the image information and electromyographic information typically include first gesture information of a first type of action and second gesture information of a second type of action. Thus, the components of the first type of action and the second type of action can be determined based on the image and electromyographic information, respectively. Then, based on the components of each first type of action and each second type of action in the image and electromyographic information, the recognition result of the gesture action can be accurately determined.

[0167] The first gesture information here includes: first sub-gesture information and / or second sub-gesture information; the second gesture information here includes: third sub-gesture information and / or fourth sub-gesture information. The first sub-gesture information here is the information describing a first type of movement in the image information; the second sub-gesture information here is the information describing a first type of movement in the electromyographic information. The third sub-gesture information here is the information describing a second type of movement in the image information; the fourth sub-gesture information here is the information describing a second type of movement in the image information.

[0168] In some embodiments, obtaining the first gesture information of the first type of gesture in the gesture based on the image information and the electromyographic information includes:

[0169] Based on the image information, obtain the first sub-gesture information of the first type of gesture in the gesture action;

[0170] Based on the electromyographic information, the second sub-gesture information of the first type of gesture in the gesture action is obtained;

[0171] Based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, the first gesture information is obtained; wherein the first weight is greater than the second weight.

[0172] The step of obtaining second gesture information for the second type of gesture in the gesture based on the image information and the electromyographic information includes:

[0173] Based on the image information, obtain the third sub-gesture information of the second type of gesture in the gesture action;

[0174] Based on the electromyographic information, the fourth sub-gesture information of the second type of gesture in the gesture action is obtained;

[0175] Based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, the second gesture information is obtained; wherein the third weight is less than the fourth weight.

[0176] Here, the weight represents the proportion of image information or electromyographic information within the entire image information; the weight can be used to indicate the importance of a first-type or second-type action. For example, in an application scenario where it is necessary to determine a first-type action with a relatively large amplitude of motion, the first weight corresponding to the first sub-gesture information in the image information can be determined to be greater than the second weight corresponding to the second sub-gesture information in the electromyographic information; for example, the first weight can be 0.8, 0.9, or 1, and the corresponding second weight is 0.2, 0.1, or 0. As another example, in an application scenario where it is necessary to determine a second-type action with a relatively small amplitude of motion, the third weight corresponding to the third sub-gesture information in the image information can be determined to be less than the fourth weight corresponding to the fourth sub-gesture information in the electromyographic information; for example, the third weight can be 0.2, 0.1, or 0, and the corresponding fourth weight is 0.8, 0.9, or 1.

[0177] The weighting here can be determined based on historical information. For example, if the electronic device stores historical cases of determining the first type of action, and the probability of determining it from image information is 80, then the first weighting corresponding to the first sub-gesture information in the image information can be determined to be 0.8, and the second weighting corresponding to the second sub-gesture information in the electromyographic information can be determined to be 0.2.

[0178] The weights here can also be preset. For example, for the first type of action, the first weight corresponding to the first sub-gesture information in the image information is set to 0.9, and the second weight corresponding to the second sub-gesture information in the electromyographic information is set to 0.1.

[0179] In one embodiment, obtaining the first gesture information based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, includes:

[0180] The first value is determined based on the product of the first sub-gesture information and the first proportion;

[0181] The second value is determined based on the product of the second sub-gesture information and the second proportion;

[0182] The first gesture information is determined based on the sum of the first value and the second value.

[0183] In one embodiment, the first weight can be 1, and the second weight can be 0. In this embodiment, image information is used to determine the first type of action, and electromyographic information is not used to determine the first type of action.

[0184] For example, if a first sub-gesture information of wrist rotation is determined based on image information, but a second sub-gesture information of wrist non-rotation is determined based on electromyographic information, then the gesture of wrist non-rotation is determined based on the first sub-gesture information and a first weight, and the second sub-gesture information and a second weight.

[0185] Of course, in other embodiments, it is sufficient to satisfy that the first proportion is greater than the second proportion. In this way, the image information can more accurately determine the first type of movement compared to the electromyographic information.

[0186] In one embodiment, obtaining the second gesture information based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, includes:

[0187] The third value is determined based on the product of the third sub-gesture information and the third proportion;

[0188] Based on the product of the fourth sub-weight information and the fourth weight, the fourth value is determined;

[0189] The second gesture information is determined based on the sum of the third and fourth values.

[0190] In some embodiments, the third weight can be 0, and the fourth weight can be 1. In this embodiment, image information is not used to determine the second type of action, and electromyographic information is used for the second type of action.

[0191] For example, if a third sub-gesture information of index finger bending is determined based on image information, but a fourth sub-gesture information of index finger not bending and middle finger bending is determined based on electromyographic information, then a gesture action of index finger not bending and middle finger bending is determined based on the third sub-gesture information and the third weight, and the fourth sub-gesture information and the fourth weight.

[0192] Of course, in other embodiments, it is only necessary to satisfy that the third weight is less than the fourth weight. In this way, electromyographic information can more accurately determine the second type of movement compared to image information.

[0193] In this embodiment, the components of a first type of action and a second type of action in the image information, as well as the components of a first type of action and a second type of action in the electromyographic (EMG) information, can be determined first. Then, different weights are determined based on the importance of the first and second types of actions in the image information and EMG information, respectively. Thus, this embodiment can extract the components of the first and second types of actions from both the image information and EMG information and match appropriate weights for gesture recognition, thereby accurately determining the recognition result of the gesture.

[0194] Furthermore, considering that image information can more accurately determine the recognition of the first type of movement compared to electromyographic (EMG) information, the first proportion of the first type of movement in image information is determined to be greater than the second proportion in EMG information. Similarly, considering that EMG information can more accurately determine the recognition of the second type of movement compared to image information, the fourth proportion of the second type of movement in EMG information is determined to be greater than the third proportion. In this way, appropriate proportions can be assigned to various movements in both image and EMG information, resulting in more accurate gesture recognition results.

[0195] like Figure 5 As shown, this disclosure provides a gesture recognition method, executed by a second device, including:

[0196] Step S21: Obtain image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0197] Step S22: Collect the electromyographic signals corresponding to the gesture to obtain electromyographic information;

[0198] Step S23: Based on the image information and the electromyographic information, obtain the gesture recognition result of the gesture action.

[0199] In some embodiments of this disclosure, any of the embodiments included in step S13 performed by the first device can be performed by the second device.

[0200] In this embodiment, the second device can acquire image information of the hand gesture captured by the first device, and obtain electromyographic (EMG) information from the corresponding EMG signal of the hand gesture; and based on the image information and EMG information, obtain the hand gesture recognition result. Thus, in this embodiment, the image information captured by the first device and the EMG information captured by the second device can be synchronized to the second device; the hand gesture can be recognized by combining the image information and EMG information. This achieves collaborative processing of hand gesture recognition between the first and second devices, and improves the accuracy of hand gesture recognition.

[0201] like Figure 6 As shown, this disclosure provides a gesture recognition method, executed by a third device, including:

[0202] Step S31: Obtain image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0203] Step S32: Obtain electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the hand gesture;

[0204] Step S33: Based on the image information and the electromyographic information, obtain the gesture recognition result of the gesture action.

[0205] In some embodiments of this disclosure, any of the embodiments included in step S13 performed by the first device can be performed by the third device.

[0206] In one embodiment, the third device can be any mobile or fixed device other than the first and second devices; for example, the third device can be a mobile phone, computer, server, tablet computer, etc.

[0207] In this embodiment, the second device can acquire image information of the gesture captured by the first device, and the first device can acquire electromyographic (EMG) information corresponding to the gesture captured by the second device. Based on the image information and EMG information, a gesture recognition result is obtained. Thus, in this embodiment, both the image information captured by the first device and the EMG information captured by the second device can be synchronized to the second device; the gesture can be recognized by combining the image information and EMG information. This achieves collaborative processing of gesture recognition between the first and second devices, and improves the accuracy of gesture recognition.

[0208] Figure 7 An exemplary embodiment of a gesture recognition device is provided, applied to a first device, comprising:

[0209] The first acquisition module 41 is used to obtain image information based on the acquired gesture images;

[0210] The first acquisition module 42 is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action.

[0211] The first processing module 43 is used to obtain the gesture recognition result of the gesture action based on the image information and the electromyographic information.

[0212] In some embodiments, the first processing module 43 is configured to obtain first gesture information of a first type of gesture in the gesture based on the image information; and to obtain second gesture information of a second type of gesture in the gesture based on the electromyographic information.

[0213] The first processing module 43 is used to obtain the gesture recognition result of the gesture action based on the first gesture information and the second gesture information;

[0214] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0215] In some embodiments, the first type of action includes at least one of the following: arm movement, wrist movement, and finger movement;

[0216] The second category of movements includes at least one of the following: wrist movements and finger movements.

[0217] In some embodiments, the first acquisition module 42 is used to determine a second type of action in the gesture based on the electromyographic information;

[0218] The first acquisition module 42 is used to determine the second gesture information based on the second type of action.

[0219] In some embodiments, the first acquisition module 42 is configured to determine the finger movement of at least one finger corresponding to the first detection area if the electromyographic information indicates that the electromyographic signal of at least one first detection area is greater than a predetermined threshold.

[0220] or,

[0221] The first acquisition module 42 is used to determine wrist rotation if the electromyographic information indicates that the electromyographic signal in at least one of the first detection areas and the second detection areas is greater than the predetermined threshold.

[0222] The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

[0223] In some embodiments, the apparatus further includes:

[0224] The first acquisition module 42 is used to acquire motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action;

[0225] The first acquisition module 42 is used to determine the second type of action based on the motion information if the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold.

[0226] In some embodiments, the first processing module 43 is configured to obtain first gesture information of a first type of gesture in the gesture based on the image information and the electromyographic information; and to obtain second gesture information of a second type of gesture in the gesture based on the image information and the electromyographic information.

[0227] The first processing module 43 is used to obtain the gesture recognition result of the gesture action based on the first gesture information and the second gesture information;

[0228] The range of motion of the first type of action is greater than the range of motion of the second type of action.

[0229] In some embodiments, the first processing module 43 is configured to obtain first sub-gesture information of a first type of gesture in the gesture based on the image information; and to obtain second sub-gesture information of the first type of gesture in the gesture based on the electromyographic information.

[0230] The first processing module 43 is used to obtain the first gesture information based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight; wherein the first weight is greater than the second weight.

[0231] In some embodiments, the first processing module 43 is configured to obtain third sub-gesture information of the second type of gesture in the gesture based on the image information; and obtain fourth sub-gesture information of the second type of gesture in the gesture based on the electromyographic information;

[0232] The first processing module 43 is used to obtain the second gesture information based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight; wherein the third weight is less than the fourth weight.

[0233] Figure 8 An exemplary embodiment of a gesture recognition device is provided, applied to a second device, comprising:

[0234] The second acquisition module 51 is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0235] The second acquisition module 52 is used to acquire the electromyographic signals corresponding to the gesture movements to obtain electromyographic information.

[0236] The second processing module 53 is used to obtain the gesture recognition result of the gesture action based on the image information and the electromyographic information.

[0237] In some embodiments of this disclosure, the second processing module can execute all the methods that the first processing module can execute; for example, the second processing module 53 is used to obtain first gesture information of a first type of gesture in the gesture based on the image information; and to obtain second gesture information of a second type of gesture in the gesture based on the electromyographic information; the second processing module 53 is used to obtain the gesture recognition result of the gesture based on the first gesture information and the second gesture information.

[0238] Figure 9 An exemplary embodiment of a gesture recognition device is provided, applied to a third device, comprising:

[0239] The third acquisition module 61 is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action;

[0240] The third acquisition module 61 is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action.

[0241] The third processing module 62 is used to obtain the gesture recognition result of the gesture action based on the image information and the electromyographic information.

[0242] In some embodiments of this disclosure, the third processing module is capable of executing all methods that the first processing module is capable of executing; for example, the third processing module 62 is used to obtain first gesture information of a first type of gesture in the gesture based on the image information; and to obtain second gesture information of a second type of gesture in the gesture based on the electromyographic information; the third processing module 63 is used to obtain the gesture recognition result of the gesture based on the first gesture information and the second gesture information.

[0243] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0244] Embodiments of this disclosure also provide an electronic device, characterized in that it includes:

[0245] processor;

[0246] Memory used to store processor-executable instructions;

[0247] The processor is configured to implement the gesture recognition method described in any embodiment of this disclosure when running the executable instructions.

[0248] In one embodiment, the electronic device is the first device, the second device, or the third device described in the above embodiments.

[0249] The memory may include various types of storage media, which are non-temporary computer storage media that can continue to store information after the communication device loses power.

[0250] The processor can be connected to the memory via a bus or similar means to read executable programs stored in the memory, for example, to implement... Figure 1 , 4 At least one of the methods shown in 6.

[0251] Embodiments of this disclosure also provide a computer-readable storage medium storing an executable program, wherein the executable program, when executed by a processor, implements the gesture recognition method described in any embodiment of this disclosure. For example, implementing as... Figure 1 , 4 At least one of the methods shown in 6.

[0252] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0253] Figure 10 This is a block diagram illustrating an electronic device 800 according to an exemplary embodiment. For example, the electronic device 800 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0254] Reference Figure 10The electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power supply component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.

[0255] Processing component 802 typically controls the overall operation of electronic device 800, such as operations associated with display, telephone calls, data communication, camera operation, and recording operations. Processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the methods described above. Furthermore, processing component 802 may include one or more modules to facilitate interaction between processing component 802 and other components. For example, processing component 802 may include a multimedia module to facilitate interaction between multimedia component 808 and processing component 802.

[0256] Memory 804 is configured to store various types of data to support the operation of device 800. Examples of this data include instructions for any application or method operating on electronic device 800, contact data, phonebook data, messages, pictures, videos, etc. Memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0257] Power supply component 806 provides power to various components of electronic device 800. Power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to electronic device 800.

[0258] Multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 808 includes a front-facing camera and / or a rear-facing camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0259] Audio component 810 is configured to output and / or input audio signals. For example, audio component 810 includes a microphone (MIC) configured to receive external audio signals when electronic device 800 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 804 or transmitted via communication component 816. In some embodiments, audio component 810 also includes a speaker for outputting audio signals.

[0260] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0261] Sensor assembly 814 includes one or more sensors for providing state assessments of various aspects of electronic device 800. For example, sensor assembly 814 may detect the on / off state of device 800, the relative positioning of components such as the display and keypad of electronic device 800, changes in position of electronic device 800 or a component of electronic device 800, the presence or absence of user contact with electronic device 800, orientation or acceleration / deceleration of electronic device 800, and temperature changes of electronic device 800. Sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 814 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 814 may also include an accelerometer, gyroscope, magnetometer, pressure sensor, or temperature sensor.

[0262] Communication component 816 is configured to facilitate wired or wireless communication between electronic device 800 and other devices. Electronic device 800 can access wireless networks based on communication standards, such as WiFi, 2G, or 3G, or combinations thereof. In one exemplary embodiment, communication component 816 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 816 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0263] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0264] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 804 including instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0265] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0266] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A gesture recognition method, characterized in that, Performed by the first device, including: Image information is obtained based on the captured images of hand gestures; Acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the gesture; Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained; The step of obtaining the gesture recognition result of the gesture action based on the image information and the electromyographic information includes: Based on the image information, obtain the first sub-gesture information of the first type of gesture in the gesture action; Based on the electromyographic information, the second sub-gesture information of the first type of gesture in the gesture action is obtained; Based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, the first gesture information is obtained; wherein the first weight is greater than the second weight. Based on the image information, obtain the third sub-gesture information of the second type of gesture in the gesture action; Based on the electromyographic information, the fourth sub-gesture information of the second type of gesture in the gesture action is obtained; Based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, second gesture information is obtained; wherein the third weight is less than the fourth weight. The weight is used to indicate the importance of the first type of action or the second type of action; based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained; The range of motion of the first type of action is greater than the range of motion of the second type of action.

2. The method according to claim 1, characterized in that, The first category of movements includes at least one of the following: arm movements, wrist movements, and finger movements; The second category of movements includes at least one of the following: wrist movements and finger movements.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Based on the electromyographic information, the second type of action in the gesture is determined.

4. The method according to claim 3, characterized in that, Based on the electromyographic information, the second type of gesture is determined, including one of the following: If the electromyographic information indicates that the electromyographic signal in at least one first detection area is greater than a predetermined threshold, then the finger movement of at least one finger corresponding to the first detection area is determined. If the electromyographic information indicates that the electromyographic signal in at least one of the first and second detection areas is greater than the predetermined threshold, wrist rotation is determined. The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

5. The method according to claim 4, characterized in that, The method further includes: Obtain motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action; The determination of the second type of movement in the gesture based on the electromyographic information includes: If the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold, the second type of movement is determined based on the motion information.

6. A gesture recognition method, characterized in that, Performed by a second device, including: Acquire image information sent by a first device, wherein the image information is determined by the first device based on images of collected hand gestures; Electromyographic signals corresponding to the gestures are collected to obtain electromyographic information; Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained; The step of obtaining the gesture recognition result of the gesture action based on the image information and the electromyographic information includes: Based on the image information, obtain the first sub-gesture information of the first type of gesture in the gesture action; Based on the electromyographic information, the second sub-gesture information of the first type of gesture in the gesture action is obtained; Based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, the first gesture information is obtained; wherein the first weight is greater than the second weight. Based on the image information, obtain the third sub-gesture information of the second type of gesture in the gesture action; Based on the electromyographic information, the fourth sub-gesture information of the second type of gesture in the gesture action is obtained; Based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, second gesture information is obtained; wherein the third weight is less than the fourth weight. The weight is used to indicate the importance of the first type of action or the second type of action; based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained; The range of motion of the first type of action is greater than the range of motion of the second type of action.

7. A gesture recognition method, characterized in that, Performed by a third device, including: Acquire image information sent by a first device, wherein the image information is determined by the first device based on images of collected hand gestures; Acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signals corresponding to the gesture; Based on the image information and the electromyographic information, the gesture recognition result of the gesture action is obtained; The step of obtaining the gesture recognition result of the gesture action based on the image information and the electromyographic information includes: Based on the image information, obtain the first sub-gesture information of the first type of gesture in the gesture action; Based on the electromyographic information, the second sub-gesture information of the first type of gesture in the gesture action is obtained; Based on the first sub-gesture information and the first weight, and the second sub-gesture information and the second weight, the first gesture information is obtained; wherein the first weight is greater than the second weight. Based on the image information, obtain the third sub-gesture information of the second type of gesture in the gesture action; Based on the electromyographic information, the fourth sub-gesture information of the second type of gesture in the gesture action is obtained; Based on the third sub-gesture information and the third weight, and the fourth sub-gesture action and the fourth weight, second gesture information is obtained; wherein the third weight is less than the fourth weight. The weight is used to indicate the importance of the first type of action or the second type of action; based on the first gesture information and the second gesture information, the gesture recognition result of the gesture action is obtained; The range of motion of the first type of action is greater than the range of motion of the second type of action.

8. A gesture recognition device, characterized in that, Applied to the first device, including: The first acquisition module is used to obtain image information based on the acquired gesture images; The first acquisition module is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action. A first processing module is configured to obtain a gesture recognition result of the gesture based on the image information and the electromyographic information. Obtaining the gesture recognition result based on the image information and the electromyographic information includes: obtaining first sub-gesture information of a first type of gesture in the gesture based on the image information; obtaining second sub-gesture information of the first type of gesture based on the electromyographic information; obtaining first gesture information based on the first sub-gesture information and a first weight, and the second sub-gesture information and a second weight, wherein the first weight is greater than the second weight; obtaining third sub-gesture information of a second type of gesture in the gesture based on the image information; obtaining fourth sub-gesture information of the second type of gesture based on the electromyographic information; obtaining second gesture information based on the third sub-gesture information and a third weight, and the fourth sub-gesture information and a fourth weight, wherein the third weight is less than the fourth weight; the weight is used to indicate the importance of the first type of gesture or the second type of gesture; obtaining a gesture recognition result of the gesture based on the first gesture information and the second gesture information; wherein the amplitude of movement of the first type of gesture is greater than the amplitude of movement of the second type of gesture.

9. The apparatus according to claim 8, characterized in that, The first category of movements includes at least one of the following: arm movements, wrist movements, and finger movements; The second category of movements includes at least one of the following: wrist movements and finger movements.

10. The apparatus according to claim 8 or 9, characterized in that, The first acquisition module is used to determine the second type of action in the gesture based on the electromyographic information.

11. The apparatus according to claim 10, characterized in that, The first acquisition module is used to determine the finger movement of at least one finger corresponding to the first detection area if the electromyographic information indicates that the electromyographic signal of at least one first detection area is greater than a predetermined threshold. or, The first acquisition module is configured to determine wrist rotation if the electromyographic information indicates that the electromyographic signal in at least one of the first and second detection areas is greater than the predetermined threshold. The first detection area is the area corresponding to the flexor muscles of the fingers; the second detection area is the area corresponding to the flexor retinaculum of the wrist.

12. The apparatus according to claim 11, characterized in that, The device further includes: The first acquisition module is used to acquire motion information sent by the second device, wherein the motion information is determined by the second device collecting motion signals of the gesture action; The first acquisition module is used to determine the second type of action based on the motion information if the electromyographic signals of the first detection area and the second detection area in the electromyographic information are less than or equal to the predetermined threshold.

13. A gesture recognition device, characterized in that, Applied to a second device, including: The second acquisition module is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action; The second acquisition module is used to acquire the electromyographic signals corresponding to the gesture movements to obtain electromyographic information. The second processing module is used to obtain a gesture recognition result of the gesture based on the image information and the electromyography (EMG) information. Obtaining the gesture recognition result based on the image information and the EMG information includes: obtaining first sub-gesture information of a first type of gesture in the gesture based on the image information; obtaining second sub-gesture information of the first type of gesture based on the EMG information; obtaining first gesture information based on the first sub-gesture information and a first weight, and the second sub-gesture information and a second weight, wherein the first weight is greater than the second weight; obtaining third sub-gesture information of a second type of gesture in the gesture based on the image information; obtaining fourth sub-gesture information of a second type of gesture in the gesture based on the EMG information; obtaining second gesture information based on the third sub-gesture information and a third weight, and the fourth sub-gesture information and a fourth weight, wherein the third weight is less than the fourth weight; the weight is used to indicate the importance of the first type of gesture or the second type of gesture; obtaining a gesture recognition result of the gesture based on the first gesture information and the second gesture information; wherein the amplitude of movement of the first type of gesture is greater than the amplitude of movement of the second type of gesture.

14. A gesture recognition device, characterized in that, Applied to third-party devices, including: The third acquisition module is used to acquire image information sent by the first device, wherein the image information is determined by the first device based on the image of the collected gesture action; The third acquisition module is used to acquire electromyographic information sent by the second device, wherein the electromyographic information is determined by the second device based on the electromyographic signal corresponding to the gesture action. The third processing module is used to obtain a gesture recognition result of the gesture based on the image information and the electromyographic information. Obtaining the gesture recognition result based on the image information and the electromyographic information includes: obtaining first sub-gesture information of a first type of gesture in the gesture based on the image information; obtaining second sub-gesture information of the first type of gesture based on the electromyographic information; obtaining first gesture information based on the first sub-gesture information and a first weight, and the second sub-gesture information and a second weight, wherein the first weight is greater than the second weight; obtaining third sub-gesture information of a second type of gesture in the gesture based on the image information; obtaining fourth sub-gesture information of a second type of gesture in the gesture based on the electromyographic information; obtaining second gesture information based on the third sub-gesture information and a third weight, and the fourth sub-gesture information and a fourth weight, wherein the third weight is less than the fourth weight; the weight is used to indicate the importance of the first type of gesture or the second type of gesture; obtaining a gesture recognition result of the gesture based on the first gesture information and the second gesture information; wherein the amplitude of movement of the first type of gesture is greater than the amplitude of movement of the second type of gesture.

15. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the gesture recognition method according to any one of claims 1-7 when running the executable instructions.

16. A computer-readable storage medium, characterized in that, The readable storage medium stores an executable program, wherein the executable program, when executed by a processor, implements the gesture recognition method according to any one of claims 1-7.

Images