Hand action recognition device and intelligent wearable equipment

Abstract

The embodiment of the invention discloses a hand action recognition device and intelligent wearable equipment, and belongs to the technical field of augmented reality. The hand action recognition device comprises an acquisition module which comprises a camera and an electromyogram sensor, and the camera is used for shooting hand actions and outputting digital image signals; the processor is electrically connected with the camera and the electromyogram sensor, the processor is set to respond to the action recognition scene switching instruction and determine the action classification of the hand actions according to the digital image signal and a preset action set, and the action classification comprises large actions, small actions and tiny actions; determining a starting scheme corresponding to the camera and the electromyogram sensor according to the action classification; and collecting hand action data according to the starting scheme, identifying the hand action data through a preset algorithm, and outputting an identification result. According to the embodiment of the invention, all-around hand action recognition can be realized.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the technical field of extended reality, in particular to a hand action recognition device and a smart wearable device. BACKGROUND

[0002] With the rapid development of the field of VR (Virtual Reality, virtual reality) / AR (Augmented Reality, augmented reality), the research and development of the surrounding supporting devices are also increasing, which are committed to improving the convenience and comfort of the use of VR / AR devices. When the related supporting devices realize gesture recognition, either a camera is used or an EMG (electromyogram, electromyogram) sensor is used. However, the camera has a blind area and is affected by light, and cannot capture small actions and subtle finger actions. The EMG sensor has a high requirement on the algorithm, and the low recognition accuracy leads to a small number of recognized action types. SUMMARY

[0003] The main purpose of the embodiment of the present application is to provide a hand action recognition device and a smart wearable device, which aims to solve the technical problem of how to realize all-around action gesture recognition.

[0004] To achieve the above purpose, the embodiment of the present application provides a hand action recognition device, which comprises: A collection module, the collection module comprises a camera and an electromyogram sensor, the camera is used for shooting hand action and outputting a digital image signal; A processor, the processor is electrically connected with the camera and the electromyogram sensor respectively, the processor is set to determine the action classification of the hand action according to the digital image signal and a preset action set in response to an action recognition scene switching instruction, the action classification comprises a large action, a small action and a micro action; determine the corresponding start scheme of the camera and the electromyogram sensor according to the action classification; collect hand action data according to the start scheme, and identify the hand action data through a preset algorithm to output a recognition result.

[0005] In an embodiment, the determination of the corresponding start scheme of the camera and the electromyogram sensor according to the action classification comprises at least one of the following: In response to the action classification being a large action, the start scheme is determined to enable the camera; In response to the action classification being a small action, the start scheme is determined to enable the camera and the electromyogram sensor; In response to the action classification being a micro action, the start scheme is determined to enable the electromyogram sensor.

[0006] In one embodiment, determining the action classification of the hand movement based on the digital image signal and a preset action set includes: The finger displacement distance in the digital image signal is compared with the first preset interval and the second preset interval in the preset action set, wherein the minimum value of the first preset interval is greater than the maximum value of the second preset interval; If the finger displacement distance is within the first preset range, the hand movement is classified as a large movement. If the finger displacement distance is within the second preset range, the hand movement is classified as a small movement. If the finger displacement distance is not within the first preset range and the second preset range, the hand movement is classified as a minor movement.

[0007] In one embodiment, the camera includes a camera module, which is electrically connected to the processor via a data interface, a command interface, a master clock interface, and a pixel clock interface. The digital power interface of the camera module is electrically connected to a digital power supply, a first filter capacitor, and a second filter capacitor, respectively. The input / output power interface of the camera module is electrically connected to an input / output power supply, a third filter capacitor, and a fourth filter capacitor, respectively. The analog power interface of the camera module is electrically connected to an analog power supply, a fifth filter capacitor, and a sixth filter capacitor, respectively.

[0008] In one embodiment, the camera module includes: a lens, an image sensor, a first analog-to-digital converter, and a digital signal processing chip connected in sequence, wherein the digital signal processing chip is electrically connected to the processor; The lens is used to project an optical image generated by the external scene onto the image sensor to generate an electrical signal. The first analog-to-digital converter is used to convert the electrical signal into a digital image signal. The digital signal processing chip is used to process the digital image signal to obtain the hand movement data and transmit the hand movement data to the processor.

[0009] In one embodiment, the four pairs of differential electromyography signal input terminals of the electromyography sensor are electrically connected to four sets of metal electrodes and four electrostatic protection devices. The electromyography sensor has four pairs of capacitor terminals electrically connected to four capacitors to simulate the input of a high-pass filter. The analog power input terminal of the electromyography sensor is connected to the analog power supply through an inductor and is electrically connected to the first low-frequency filter capacitor and the first high-frequency filter capacitor. The digital power input terminal of the electromyography sensor is electrically connected to the digital power supply, the second low-frequency filter capacitor, and the second high-frequency filter capacitor, respectively. The general-purpose input / output power supply terminal of the electromyography sensor is electrically connected to the general-purpose input / output power supply and the third high-frequency filter capacitor, respectively. The signal transceiver of the electromyography sensor is electrically connected to the processor. The reference output terminal of the electromyography sensor is grounded through a reference output filter capacitor; The reference voltage terminal of the electromyography sensor's buffer is grounded through a buffer filter capacitor.

[0010] In one embodiment, the electromyography sensor includes a built-in instrumentation amplifier, a power amplifier, and a second analog-to-digital converter; The input terminal of the instrumentation amplifier is used to receive differential electromyographic signals characterizing the hand movement data collected by the four sets of metal electrodes. The output terminal of the instrumentation amplifier is electrically connected to the input terminal of the power amplifier. The output terminal of the power amplifier is electrically connected to the input terminal of the second analog-to-digital converter. The output terminal of the second analog-to-digital converter is electrically connected to the processor. The instrumentation amplifier, the power amplifier, and the second analog-to-digital converter are used to convert the differential electromyography (EMG) signal into a digital EMG signal and transmit the digital EMG signal to the processor.

[0011] In one embodiment, the acquisition module further includes an inertial measurement unit, the signal transceiver terminal of which is electrically connected to the processor, and the power supply input / output terminal of which is electrically connected to a power supply and a filter capacitor.

[0012] In one embodiment, the inertial measurement unit includes an accelerometer and a gyroscope, the accelerometer and the gyroscope being electrically connected to the processor, respectively. The accelerometer and the gyroscope are used to convert the monitored hand movement trajectory into hand motion data and transmit the hand motion data to the processor.

[0013] In addition, to achieve the above objectives, this application also provides a smart wearable device, which includes: a wearable structure and a hand motion recognition device as described above, wherein the hand motion recognition device is integrated into the wearable structure.

[0014] This application proposes a hand gesture recognition device and a smart wearable device, overcoming the problem in related technologies that cannot achieve comprehensive hand gesture recognition by using only one technology. By responding to the action recognition scene switching command, the processor can determine whether the current application scenario requires action interaction, thereby saving power when action recognition is not required. When action recognition is required, the camera first captures the hand gesture and compares it with a preset action set to determine whether the hand gesture is classified as a large, small, or minute movement. Then, based on the action classification, an appropriate activation scheme is selected to collect hand gesture data. This allows for the collection of hand gesture data of various amplitudes. Finally, a preset recognition algorithm is used to recognize the hand gesture data and output the recognition result, thus completing comprehensive gesture recognition. Attached Figure Description

[0015] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on the structures shown in these drawings without creative effort.

[0016] Figure 1 A schematic diagram of the structure of a hand motion recognition device provided in an embodiment of this application; Figure 2 This is a schematic diagram illustrating an application scenario of a hand motion recognition device provided in an embodiment of this application; Figure 3 A circuit connection diagram illustrating an application example of a camera in a hand motion recognition device provided in this application embodiment; Figure 4 for Figure 3 A schematic diagram of the internal structure of the camera module; Figure 5 A circuit connection diagram illustrating an application example of an electromyography sensor in a hand motion recognition device provided in this application embodiment; Figure 6 for Figure 5 A schematic diagram of the internal structure of a medium-sized electromyography (EMG) sensor; Figure 7 A schematic diagram of an extended structure of a hand motion recognition device provided in an embodiment of this application; Figure 8 for Figure 7 A schematic diagram of the internal structure of the inertial measurement unit; Figure 9 This is a partial planar layout diagram of a smart wearable device provided in an embodiment of this application.

[0017] The realization of the objectives, functional features and advantages of the embodiments of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings.

[0018] Explanation of icon numbers: 10. Acquisition module; 11. Camera; 12. Electromyography sensor; 20. Processor; J1. Camera module; 111. Lens; 112. Image sensor; 113. First analog-to-digital converter; 114. Digital signal processing chip; 121. Instrumentation amplifier; 122. Power amplifier; 123. Second analog-to-digital converter; 13. Inertial measurement unit; 131. Accelerometer; 132. Gyroscope. Detailed Implementation

[0019] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the embodiments of this application.

[0020] With the rapid development of the VR (Virtual Reality) / AR (Augmented Reality) field, the research and development of related supporting equipment is also increasing, all dedicated to improving the convenience and comfort of using VR / AR devices. When related supporting equipment realizes gesture recognition, it either uses a camera or an EMG (electromyogram) sensor. However, cameras have blind spots and are affected by light, making it impossible to capture small movements and subtle finger movements. On the other hand, EMG sensors have high requirements for algorithms, and low recognition accuracy will result in a limited number of recognized movements.

[0021] Based on this, embodiments of this application provide a hand gesture recognition device and a smart wearable device. By responding to the action recognition scene switching command, the processor can know whether the current application scenario requires action interaction. This saves power when action recognition is not required. When action recognition is required, the camera first captures the hand gesture and compares it with a preset action set to determine whether the hand gesture is a large, small, or minute movement. Then, based on the action classification, an appropriate activation scheme is selected to collect hand gesture data. This enables the collection of hand gesture data with various amplitudes. Afterward, the hand gesture data is recognized by a preset recognition algorithm, and the recognition result is output, thus completing comprehensive gesture recognition.

[0022] The hand motion recognition device and smart wearable device provided in this application are specifically described through the following embodiments. First, the hand motion recognition device in this application is described.

[0023] This application provides a hand gesture recognition device, referring to... Figure 1 , Figure 1 This is a schematic diagram of the structure of a hand gesture recognition device according to an embodiment of this application. The hand gesture recognition device includes: The acquisition module 10 includes a camera 11 and an electromyography sensor 12. The camera 11 is used to capture hand movements and output digital image signals. The processor 20 is electrically connected to the camera 11 and the electromyography sensor 12, respectively. The processor 20 is configured to, in response to the action recognition scene switching command, determine the action classification of the hand action based on the digital image signal and the preset action set. The action classification includes large actions, small actions and micro actions. Based on the action classification, determine the corresponding activation scheme of the camera 11 and the electromyography sensor 12. Based on the activation scheme, collect hand action data, identify the hand action data through the preset algorithm, and output the recognition result.

[0024] In this embodiment, the hand motion recognition device can be applied to a smart wearable device. The motion recognition scene switching command can be generated by the user operating the smart wearable device, or it can be generated by the smart wearable device itself detecting that the current running scene matches the preset motion recognition scene. When the motion recognition scene switching command is detected, the smart wearable device will first use a camera to capture the hand motion to determine the magnitude of the hand motion, and then select an appropriate data acquisition unit to complete the data acquisition of the hand motion.

[0025] In some feasible embodiments, the processor 20 may also be configured to monitor the wrist displacement; when the wrist displacement is greater than a preset distance, capture the surrounding environment of the smart wearable device through the camera; and when the surrounding environment matches a preset motion recognition scenario, generate a motion recognition scenario switching instruction. In this embodiment, the smart wearable device can also be equipped with an IMU (Inertial Measurement Unit) sensor. The smart wearable device can also dynamically detect the user's hand movements and the surrounding environment even without receiving a motion recognition scene switching command generated based on user operation. If the surrounding environment falls within a pre-set scene requiring motion recognition, it will execute the step of collecting hand data. In other words, the processor 20 can switch its operating mode to motion recognition mode either due to a motion recognition scene switching command or after real-time detection by the inertial measurement unit and camera 11 confirms that preset conditions are met.

[0026] As an example, a large-amplitude wrist movement, such as a wrist flip, can be detected using an inertial measurement unit. For instance, if a wrist displacement exceeding a certain distance is detected, a camera can be activated to capture the surrounding environment to determine if it is in a specific scenario requiring motion recognition. If it is in everyday scenarios such as driving, drinking water, using a mobile phone, or using a computer, it is determined that motion recognition is not required. Of course, the specific scenarios in which motion recognition is enabled and disabled can be flexibly adjusted according to actual needs, thereby pre-configuring the preset motion recognition scenarios. This embodiment does not impose any restrictions on this.

[0027] As an example, users can enter specific scenarios through the screen on a smartwatch or wristband, or other compatible electronic devices, such as switching to game mode, remote control mode, or other scenarios that require action interaction.

[0028] Understandably, after the hand movement is captured by the camera 11, it needs to be compared with a preset set of movements to determine the movement category of the current hand movement, so as to select the appropriate data acquisition unit to collect the complete data of the current hand movement and improve the final recognition accuracy.

[0029] In this embodiment, the smart wearable device pre-stores preset action sets corresponding to different action categories, including: large movements with relatively obvious amplitudes that can be captured by the camera 11 (such as flipping up, flipping down, clenching a fist, tapping with fingers, etc.); small movements that can be captured by the camera 11 but with less obvious amplitudes (such as pinching fingers, rubbing fingers, etc.); and minute movements that are in the blind spot of the camera 11 and cannot be captured by the camera 11 (such as finger clicking in air mouse mode). It is understood that these three action categories included in the preset action set are only one feasible example and can be flexibly adjusted according to actual application needs. This embodiment does not limit this.

[0030] In some feasible embodiments, the action classification of the hand action is determined based on the digital image signal and a preset action set. Specifically, this may include: comparing the finger displacement distance in the digital image signal with a first preset interval and a second preset interval in the preset action set, wherein the minimum value of the first preset interval is greater than the maximum value of the second preset interval; if the finger displacement distance is within the first preset interval, the hand action is classified as a large action; if the finger displacement distance is within the second preset interval, the hand action is classified as a small action; if the finger displacement distance is not within the first or second preset interval, the hand action is classified as a minute action.

[0031] In this embodiment, considering that the palm and back of the hand movements are not obvious, the finger movements are mainly used as the basis for determining which movement category in the preset movement set the hand movement belongs to. Specifically, the judgment can be made by comparing the finger displacement distance. Generally, large finger movements such as flipping up, flipping down, clenching a fist, and tapping with fingers involve significant finger displacement. Therefore, a first preset interval is set to correspond to large movements. The first preset interval can have only a minimum value set, without a maximum value. That is, as long as the finger displacement distance is greater than or equal to the minimum value of the first preset interval, the hand movement is considered a large movement. Similarly, small finger movements such as pinching or rubbing fingers involve significantly smaller finger displacement distances compared to large movements. Therefore, a second preset interval is set to correspond to small movements, and the maximum value of the second preset interval is equal to the minimum value of the first preset interval. To avoid confusion between small and micro movements, a minimum value is also set for the second preset interval. That is, if the finger displacement distance is less than the maximum value of the second preset interval but greater than or equal to the minimum value of the second preset interval, the hand movement is considered a small movement. Furthermore, if the finger displacement distance is less than the minimum value of the second preset interval, i.e., it is neither within the first nor the second preset interval, the hand movement is considered a micro movement.

[0032] After comparing the current hand movement captured by camera 11 with a preset set of movements, the movement category of the current hand movement can be determined. Then, at least one of the two data acquisition units, camera 11 and electromyography sensor 12, can be selected to collect hand movement data. Whether to activate camera 11, electromyography sensor 12, or both depends on the specific movement category.

[0033] In some feasible embodiments, the activation scheme for determining the corresponding camera 11 and electromyography sensor 12 based on the action classification may include at least one of the following: in response to the action being classified as a large action, the activation scheme is determined to enable the camera 11; in response to the action being classified as a small action, the activation scheme is determined to enable the camera 11 and electromyography sensor 12; in response to the action being classified as a minute action, the activation scheme is determined to enable the electromyography sensor 12.

[0034] In this embodiment, if the current hand movement is classified as a large movement, it means that the camera 11 can completely collect the hand movement data. Therefore, the activation scheme corresponding to large movements is set to activate the camera 11. If the current hand movement is classified as a small movement, it means that the camera 11 alone may not be able to completely collect the hand movement data. In order to ensure that the accuracy of the final recognition is not affected, it is necessary to combine the electromyography sensor 12 with the camera 11 to complete the data collection. Therefore, the activation scheme corresponding to small movements is set to activate both the camera 11 and the electromyography sensor 12. If the current hand movement is classified as a minute movement, it means that the hand movement is almost in the blind spot of the camera 11. At this time, activating the camera 11 will hardly contribute to the data collection. Therefore, the activation scheme corresponding to minute movements is set to activate the electromyography sensor 12.

[0035] In this embodiment, the hand movement data collected by the camera 11 and the electromyography (EMG) sensor 12 are transmitted to the processor 20 for further processing to complete the movement recognition, thereby outputting the recognition result, such as what operation command the current gesture represents. As an example, for image data in the hand movement data, the processor 20 can recognize it using a preset image recognition algorithm, and for EMG data in the hand movement data, the processor 20 can recognize it using a preset EMG recognition algorithm.

[0036] In addition, there may be a special case when the camera 11 captures hand movements, that is, the camera 11 detects that the ambient light is less than the preset threshold, which means that the light is very dark or completely dark. In this case, the camera 11 cannot capture hand movements and classify them. Therefore, it is necessary to use the electromyography sensor 12 to collect hand movement data. At the same time, in special cases, the inertial measurement unit can be combined to perform some auxiliary movement position judgments, thereby further improving the accuracy of movement recognition.

[0037] As an example, the smart wearable device in this embodiment can be a smartwatch or smart bracelet worn on the wrist, such as... Figure 2As shown, with the IMU and EMG set in the meter head, and camera 1 and camera 2 set on the back of the hand and the inner side of the wrist respectively, the specific implementation of this embodiment may include: (1) For large-amplitude hand movements, the two cameras on the back of the hand and the inner side of the wrist can be used to complete the task. At this time, the hand movements that can be clearly captured by the camera are collected, such as clenching a fist, flipping fingers up / down, and tapping fingers, which are clearly within the field of view of the camera. After the camera captures such finger movements, it can be transmitted to the processor, which performs image recognition, identifies the corresponding hand movements, and then outputs the recognition results to the upper-level device, such as the display screen, for display output; (2) For the recognition of small-amplitude hand movements, since the amplitude is small, or the change is not significant, or only a part of the fingers are in the field of view of the camera, there is a possibility that the camera may miss or misidentify them, such as rubbing fingers, pinching fingers, and some movements that need to show the strength of the finger movements. Relying solely on the camera to collect these movements will reduce the success rate. Therefore, it is also necessary to collect the movements by the camera. Combine with EMG electromyography sensor to collect data, for example, the camera is the main one and the EMG electromyography sensor is the auxiliary one. The specific data acquisition algorithm can be flexibly planned according to the actual situation. The collected results are also transmitted to the processor, which performs action recognition, identifies the corresponding hand movements, and then outputs the recognition results to the upper-level device such as the display screen for display output; (3) For micro-action recognition, since this part of the action is completely in the camera blind zone, such as some finger clicks in the air mouse mode, it can only be collected by the EMG electromyography sensor. The results collected by the EMG electromyography sensor are also transmitted to the processor, which performs action recognition, identifies the corresponding hand movements, and then outputs the recognition results to the upper-level device such as the display screen for display output; (4) For special cases, such as in a very dark or completely dark environment, the camera cannot capture the image clearly or at all, similar to the situation where the hand movements are in the camera blind zone, so it can only be collected by the EMG electromyography sensor.

[0038] As an example, the processor 20 in this embodiment can be implemented using an MCU (Microcontroller Unit), a CPU (Central Processing Unit), or a single-chip microcomputer. It can also be flexibly adjusted according to the actual situation. This embodiment does not impose any restrictions on this.

[0039] Reference Figure 3In some feasible embodiments, the camera 11 may include a camera module J1. The camera module J1 is electrically connected to the processor 20 through a data interface, a command interface, a master clock interface, and a pixel clock interface. The digital power interface of the camera module J1 is electrically connected to the digital power supply, the first filter capacitor C1, and the second filter capacitor C2, respectively. The input and output power interfaces of the camera module J1 are electrically connected to the input and output power supply, the third filter capacitor C3, and the fourth filter capacitor C4, respectively. The analog power interface of the camera module J1 is electrically connected to the analog power supply, the fifth filter capacitor C5, and the sixth filter capacitor C6, respectively.

[0040] As an example, in this embodiment, the camera module J1 can be implemented using the ultra-low power, ultra-small packaged HM01B0 camera module, with a resolution of 324*324, a maximum power consumption of 4mW, and a package size of only 4.5mm*5mm. Figure 3In the example circuit shown, the digital power supply can provide a power supply voltage of 1.5V. The first filter capacitor C1 and the second filter capacitor C2 are used to filter out high-frequency and low-frequency noise from the digital power supply, respectively. The capacitance values ​​can be adjusted flexibly according to actual needs. The I / O (Input / Output) power supply can provide a power supply voltage of 1.8V. The third filter capacitor C3 and the fourth filter capacitor C4 are used to filter out high-frequency and low-frequency noise from the input / output power supply, respectively. The capacitance values ​​can be adjusted flexibly according to actual needs. The analog power supply can provide a power supply voltage of 2.8V. The fifth filter capacitor C5 and the sixth filter capacitor C6 are used to filter out high-frequency and low-frequency noise from the analog power supply, respectively. The capacitance values ​​can be adjusted flexibly according to actual needs. The data interface can be a parallel port, electrically connected to the processor 20 through MCU_SLMOSI_CAMERA, MCU_CAM_DATA1, MCU_CAM_DATA2, MCU_CAM_DATA3, MCU_CAM_DATA4, MCU_CAM_DATA5, MCU_CAM_DATA6, and MCU_CAM_DATA7. The command interface can be I2C (Inter-Integrated Circuit). The circuit (built-in integrated circuit) interface is electrically connected to the processor 20 via MCU_CAM_INT, MCU_I2C6_SDA, MCU_I2C6_SCL, MCU_CAM_LVLD, MCU_SLCS_CAMERA, and MCU_CAM_TRIG. The MCLK (Master clock) frequency can be 3 to 36 MHz, and it receives the MCU_CAM_MCLK master clock signal from the processor 20 through the master clock interface and resistor R1'. The PCLK (Pixel clock) frequency can also be 3 to 36 MHz, and it outputs the MCU_SLCSK_CAMERA pixel clock signal to the processor 20 through the pixel clock interface. It should be noted that the filter capacitors connected to each power input terminal of the camera module J1 can be increased according to actual needs; this embodiment does not impose any restrictions on this.

[0041] Reference Figure 4 In some feasible embodiments, the camera module J1 may specifically include: a lens 111, an image sensor 112, a first analog-to-digital converter 113 and a digital signal processing chip 114 connected in sequence, wherein the digital signal processing chip 114 is electrically connected to the processor 20. Lens 111 is used to project an optical image generated from the external scene onto image sensor 112 to generate an electrical signal. First analog-to-digital converter 113 is used to convert the electrical signal into a digital image signal. Digital signal processing chip 114 is used to process the digital image signal to obtain hand movement data and transmit the hand movement data to processor 20.

[0042] It should be understood that the internal structure of the camera module J1 provided in this embodiment is only a feasible example. The specific selection of the camera module J1 can also be flexibly adjusted according to the actual situation, as long as it can achieve similar functions. This embodiment does not limit this.

[0043] Reference Figure 5 In some feasible embodiments, the four pairs of differential electromyography signal input terminals of the electromyography sensor 12 described above are electrically connected to four sets of metal electrodes and four electrostatic protection devices. The four pairs of capacitive terminals of the electromyography sensor 12 are electrically connected to four capacitors to simulate the input of a high-pass filter; The analog power input terminal AVDD of the electromyography sensor 12 is connected to the analog power supply through the inductor FB1 and is electrically connected to the first low-frequency filter capacitor C11 and the first high-frequency filter capacitor C10. The digital power input terminal DVDD of the electromyography sensor 12 is electrically connected to the digital power supply, the second low-frequency filter capacitor C9, and the second high-frequency filter capacitor C8, respectively. The general-purpose input / output power supply terminal IOVDD of the electromyography sensor 12 is electrically connected to the general-purpose input / output power supply and the third high-frequency filter capacitor C7, respectively. The signal transceiver of the electromyography sensor 12 is electrically connected to the processor 20; The reference output terminal VMID_EMG of the electromyography sensor 12 is grounded through the reference output filter capacitor C6; The buffer reference voltage terminal VREF_EMG of the electromyography sensor 12 is grounded through the buffer filter capacitor C5.

[0044] As an example, in this embodiment, the electromyography (EMG) sensor 12 can be implemented using a highly integrated human biosensor (EMG). The human biosensor (EMG) can be used for human electromyography signal processing and can simultaneously receive four sets of differential electromyography signal inputs, such as... Figure 5As shown, the four pairs of differential electromyography (EMG) signal input terminals EL1 / EL2, EL3 / EL4, EL5 / EL6, and EL7 / EL8 of the EMG sensor 12 are connected to four sets of metal electrodes, namely electrode group 1, electrode group 2, electrode group 3, and electrode group 4. These electrode groups are essential components for acquiring EMG signals. Four optional electrostatic discharge (ESD) protection devices, ESD1, ESD2, ESD3, and ESD4, can prevent damage to the device due to static electricity generated near the skin. R1, R2, R3, R4, R5, R6, R7, and R8 are optional protective resistors that can limit current. They can be flexibly configured when current limiting adjustment is needed and removed when no current limiting adjustment is required. CAP1P / CAP1N, CAP2P / CAP2N, CAP3P / CAP3N, and CAP4P are also included. The four capacitor pairs on the / CAP4N are connected to capacitors C1', C2', C3', and C4' respectively, to simulate the input of a high-pass filter, thereby adjusting the frequency range of the acquired electromyography (EMG) signal. Their capacitance values ​​can be flexibly adjusted according to the required high-frequency signal band. For the components at the power input terminals, inductor FB1 is used to filter out noise from the analog power supply AVDD; the first low-frequency filter capacitor C11 is used to filter out low-frequency noise; the first high-frequency filter capacitor C10 is used to filter out high-frequency noise; the second high-frequency filter capacitor C8 and the second low-frequency filter capacitor C9 are used to filter the digital power supply DVDD; the second high-frequency filter capacitor C8 is used to filter out high-frequency noise; the second low-frequency filter capacitor C9 is used to filter out low-frequency noise; and the third high-frequency filter capacitor C7 is used to filter the GPIO (General Purpose) input. The PurposeInput / Output power supply terminal IOVDD is filtered to remove high-frequency noise from the general-purpose input / output power supply. The reference output filter capacitor C6' is the internal reference output VMID_EMG filter capacitor, and the buffer filter capacitor C5' is the internal buffer reference voltage VREF_EMG filter capacitor. The electromyography sensor 12 can communicate with the processor 20 by transmitting SPI signals through the signal transceiver terminal. The electrical parameters of each device can be flexibly adjusted according to actual needs. The filter capacitors connected to each power input terminal can also be increased according to actual needs. This embodiment does not impose any restrictions on this.

[0045] Reference Figure 6 In some feasible embodiments, the electromyography sensor 12 may also include a built-in instrumentation amplifier 121, a power amplifier 122, and a second analog-to-digital converter 123. The input terminal of the instrumentation amplifier 121 is used to receive differential electromyographic signals representing hand movement data collected by four sets of metal electrodes. The output terminal of the instrumentation amplifier 121 is electrically connected to the input terminal of the power amplifier 122. The output terminal of the power amplifier 122 is electrically connected to the input terminal of the second analog-to-digital converter 123. The output terminal of the second analog-to-digital converter 123 is electrically connected to the processor 20. The instrumentation amplifier 121, the power amplifier 122, and the second analog-to-digital converter 123 are used to convert differential electromyography signals into digital electromyography signals and transmit the digital electromyography signals to the processor 20.

[0046] As an example, in this embodiment, the electromyography (EMG) sensor 12 can convert the differential EMG signals representing hand movement data collected by four sets of metal electrodes into digital EMG signals that can be processed by the processor 20 through its built-in INA (Instrumentation Amplifier), PGA (Programmable Gain Amplifier), and 20-bit ADC (Analog-to-Digital Converter). It should be understood that the internal structure of the EMG sensor 12 provided in this embodiment is only a feasible example, and the specific selection of the built-in components of the EMG sensor 12 can be flexibly adjusted according to actual conditions, as long as similar functions can be achieved. This embodiment does not impose any limitations on this.

[0047] Reference Figure 7 In some feasible embodiments, the acquisition module 10 may further include an inertial measurement unit 13, the signal transceiver terminal of the inertial measurement unit 13 being electrically connected to the processor 20, and the power supply input / output terminal of the inertial measurement unit 13 being electrically connected to the power supply and the filter capacitor.

[0048] As an example, in this embodiment, the inertial measurement unit 13 can be implemented by any type of IMU sensor chip. When the IMU sensor chip communicates with the processor 20 via SPI signal, its signal transceiver can include MCU_M1CS_IMU, MCU_M1SCK_IMU, MCU_M1MOSI_IMU, MCU_M1MISO_IMU, etc. Its power supply can be a 1.8V power supply. Both the GPIO power supply VDDIO and the main power supply VDD can be connected to filter capacitors. When placing the filter capacitors, they need to be placed close to the corresponding pins.

[0049] Reference Figure 8In some feasible embodiments, the inertial measurement unit 13 may further include an accelerometer 131 and a gyroscope 132, which are electrically connected to the processor 20 respectively. The accelerometer 131 and the gyroscope 132 are used to convert the monitored hand movement trajectory into hand motion data and transmit the hand motion data to the processor 20.

[0050] As an example, in this embodiment, the inertial measurement unit 13 can integrate a 3-axis accelerometer and a 3-axis gyroscope, thereby enabling it to monitor the motion trajectory of hand movements and convert it into digital signals that the processor 20 can process. It should be understood that the internal structure of the inertial measurement unit 13 provided in this embodiment is only a feasible example, and the specific selection of the built-in components of the inertial measurement unit 13 can be flexibly adjusted according to actual conditions, as long as similar functions can be achieved; this embodiment does not impose any limitations on this.

[0051] In addition, this application also provides a smart wearable device, which includes a wearable structure and the hand motion recognition device provided in the above embodiments, wherein the hand motion recognition device is integrated into the wearable structure.

[0052] As an example, refer to Figure 9 In this embodiment, the wearable structure can be a ring-shaped structure that can be worn on the hand. Two cameras can be set on the wearable structure, and the lenses of the cameras are both facing the direction of the fingers. If it is necessary to further improve the acquisition accuracy, the number of cameras can be further increased. The electrodes of the electromyography sensor are set on the side of the wearable structure that is close to the skin when worn. In order to ensure the accuracy of data acquisition, the diameter of each electrode is not less than 4 mm, and the center distance between the two closest electrodes is not less than 7 mm.

[0053] In some feasible embodiments, the hand motion recognition device may also include an inertial measurement unit, which may also be integrated into the wearable structure.

[0054] Those skilled in the art will understand that Figure 9 The structure shown does not constitute a limitation on smart wearable devices and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0055] The smart wearable device proposed in this embodiment belongs to the same technical concept as the hand motion recognition device proposed in the above embodiments. Technical details not described in detail in this embodiment can be found in any of the above embodiments. Furthermore, this embodiment has the same beneficial effects as the above embodiments of the hand motion recognition device.

[0056] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of this application are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0057] Furthermore, in the embodiments of this application, descriptions involving "first," "second," etc., are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of the embodiments of this application, "multiple" means at least two, such as two, three, etc., unless otherwise explicitly specified. Additionally, the meaning of "and / or" throughout the text includes three parallel solutions; for example, "A and / or B" includes solution A, solution B, or a solution that simultaneously satisfies A and B.

[0058] In the embodiments of this application, unless otherwise expressly specified and limited, the terms "connection" and "fixed" should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two components or the interaction between two components, unless otherwise expressly limited. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this application according to the specific circumstances.

[0059] It should also be understood that references to "one embodiment" or "some embodiments" in the specification of embodiments of this application mean that one or more embodiments of this application include the specific features, structures, or characteristics described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0060] It should be noted that the technical solutions of the various embodiments of this application can be combined with each other, but only if they are implemented by those skilled in the art. When the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the embodiments of this application.

[0061] The above are merely optional embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the description and drawings of this application, or direct or indirect applications in other related technical fields, are similarly included within the patent protection scope of this application.

Claims

1. A hand gesture recognition device, characterized in that, The hand gesture recognition device includes: The acquisition module includes a camera and an electromyography sensor. The camera is used to capture hand movements and output digital image signals. The processor is electrically connected to both the camera and the electromyography sensor. The processor is configured to, in response to a motion recognition scene switching command, determine the motion classification of the hand motion based on the digital image signal and a preset motion set, wherein the motion classification includes large movements, small movements, and minute movements; determine the corresponding activation scheme for the camera and the electromyography sensor based on the motion classification; collect hand motion data according to the activation scheme; recognize the hand motion data using a preset algorithm; and output the recognition result.

2. The hand gesture recognition device as described in claim 1, characterized in that, The step of determining the activation scheme corresponding to the camera and the electromyography sensor based on the action classification includes at least one of the following: In response to the action being classified as a major action, the activation scheme is determined to enable the camera; In response to the action being classified as a minor action, the activation scheme is determined to enable the camera and the electromyography sensor; In response to the action being classified as a minor action, the activation scheme is determined to enable the electromyography sensor.

3. The hand gesture recognition device as described in claim 1, characterized in that, The step of determining the action classification of the hand action based on the digital image signal and a preset action set includes: The finger displacement distance in the digital image signal is compared with the first preset interval and the second preset interval in the preset action set, wherein the minimum value of the first preset interval is greater than the maximum value of the second preset interval; If the finger displacement distance is within the first preset range, the hand movement is classified as a large movement. If the finger displacement distance is within the second preset range, the hand movement is classified as a small movement. If the finger displacement distance is not within the first preset range and the second preset range, the hand movement is classified as a minor movement.

4. The hand gesture recognition device as described in claim 1, characterized in that, The camera includes a camera module, which is electrically connected to the processor via a data interface, a command interface, a master clock interface, and a pixel clock interface. The digital power interface of the camera module is electrically connected to a digital power supply, a first filter capacitor, and a second filter capacitor. The input / output power interface of the camera module is electrically connected to an input / output power supply, a third filter capacitor, and a fourth filter capacitor. The analog power interface of the camera module is electrically connected to an analog power supply, a fifth filter capacitor, and a sixth filter capacitor.

5. The hand gesture recognition device as described in claim 4, characterized in that, The camera module includes: a lens, an image sensor, a first analog-to-digital converter, and a digital signal processing chip connected in sequence, wherein the digital signal processing chip is electrically connected to the processor; The lens is used to project an optical image generated by the external scene onto the image sensor to generate an electrical signal. The first analog-to-digital converter is used to convert the electrical signal into a digital image signal. The digital signal processing chip is used to process the digital image signal to obtain the hand movement data and transmit the hand movement data to the processor.

6. The hand gesture recognition device as described in claim 1, characterized in that, The electromyography sensor has four pairs of differential electromyography signal input terminals that are electrically connected to four sets of metal electrodes and four electrostatic protection devices. The electromyography sensor has four pairs of capacitor terminals electrically connected to four capacitors to simulate the input of a high-pass filter. The analog power input terminal of the electromyography sensor is connected to the analog power supply through an inductor and is electrically connected to the first low-frequency filter capacitor and the first high-frequency filter capacitor. The digital power input terminal of the electromyography sensor is electrically connected to the digital power supply, the second low-frequency filter capacitor, and the second high-frequency filter capacitor, respectively. The general-purpose input / output power supply terminal of the electromyography sensor is electrically connected to the general-purpose input / output power supply and the third high-frequency filter capacitor, respectively. The signal transceiver of the electromyography sensor is electrically connected to the processor. The reference output terminal of the electromyography sensor is grounded through a reference output filter capacitor; The reference voltage terminal of the electromyography sensor's buffer is grounded through a buffer filter capacitor.

7. The hand gesture recognition device as described in claim 6, characterized in that, The electromyography sensor includes a built-in instrumentation amplifier, power amplifier, and second analog-to-digital converter. The input terminal of the instrumentation amplifier is used to receive differential electromyographic signals characterizing the hand movement data collected by the four sets of metal electrodes. The output terminal of the instrumentation amplifier is electrically connected to the input terminal of the power amplifier. The output terminal of the power amplifier is electrically connected to the input terminal of the second analog-to-digital converter. The output terminal of the second analog-to-digital converter is electrically connected to the processor. The instrumentation amplifier, the power amplifier, and the second analog-to-digital converter are used to convert the differential electromyography (EMG) signal into a digital EMG signal and transmit the digital EMG signal to the processor.

8. The hand gesture recognition device as described in claim 1, characterized in that, The acquisition module also includes an inertial measurement unit, whose signal transceiver terminal is electrically connected to the processor, and whose power supply input / output terminal is electrically connected to a power supply and a filter capacitor.

9. The hand gesture recognition device as described in claim 8, characterized in that, The inertial measurement unit includes an accelerometer and a gyroscope, and the accelerometer and the gyroscope are electrically connected to the processor, respectively. The accelerometer and the gyroscope are used to convert the monitored hand movement trajectory into hand motion data and transmit the hand motion data to the processor.

10. A smart wearable device, characterized in that, The smart wearable device includes: a wearable structure and a hand motion recognition device as described in any one of claims 1 to 9, wherein the hand motion recognition device is integrated into the wearable structure.

Images