Three-dimensional gesture recognition device and method based on elbow joint positioning

Abstract

The invention provides a three-dimensional gesture recognition device and method based on elbow joint positioning. The device comprises a finger end wearing device, an elbow joint end wearing device and a dimension reduction processing module. The finger end wearing device adopts a ring form, and a built-in UWB (Ultra Wideband) signal emission source emits 3.1-10.6 GHz ultra-wideband signals; the elbow joint end wearing device receives positioning signals through four antennas arranged in a rhombus shape, and millimeter-level positioning precision is achieved through a ToF algorithm. The dimension reduction processing module establishes a polar coordinate system with the elbow joint as the circle center, and converts three-dimensional position information into two-dimensional coordinates through an angle variation direct mapping algorithm. The system adopts a low-power-consumption accelerometer to realize on-demand activation, the power consumption is controlled at a microampere level when the system is static, and 50Hz high-frequency sampling is adopted when the system moves. According to the invention, the problem that traditional gesture recognition lacks a stable space reference system is effectively solved, and high-precision and low-power-consumption real-time gesture recognition is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer input devices, in particular to a three-dimensional gesture recognition device and method based on elbow joint positioning. BACKGROUND

[0002] With the rapid development of artificial intelligence and Internet of Things technology, human-computer interaction technology is evolving towards a more natural and intuitive direction. Gesture recognition, as an important human-computer interaction method, has shown great application potential in the fields of intelligent device control, virtual reality, augmented reality, etc. Traditional gesture recognition technology mainly relies on computer vision, inertial sensors or electromagnetic induction principles, but these technologies face many challenges in practical applications.

[0003] Although the gesture recognition system based on computer vision can provide a more intuitive interaction experience, its recognition accuracy and stability are easily affected by external factors such as lighting conditions, background environment, and occlusion. In complex use environments, the camera often has difficulty accurately capturing the subtle movements of the hand, resulting in large recognition errors. In addition, visual recognition systems usually require large computing resources and storage space, which is not conducive to the application of mobile devices and wearable devices.

[0004] The gesture recognition scheme based on inertial sensors detects hand acceleration, angular velocity and other motion parameters to recognize gestures, with the advantages of small size and low power consumption. However, inertial sensors have cumulative error problems, and the positioning accuracy will decrease significantly after long-term use. At the same time, inertial sensors cannot provide absolute position information, and cannot establish a stable spatial coordinate system, which limits their application in precise positioning scenarios.

[0005] Although electromagnetic induction technology can provide relatively stable positioning accuracy, its working distance is limited and it is easily affected by metal objects and electromagnetic interference. In a complex electromagnetic environment, the reliability and stability of the system are difficult to guarantee.

[0006] A key problem exists in existing gesture recognition systems: the lack of a stable spatial reference coordinate system. When the user's arm moves as a whole, the position of the hand relative to the external environment changes, causing the gesture recognition system to be unable to accurately distinguish between effective operation actions of the fingers and irrelevant movements of the arm. This instability of the spatial reference system seriously affects the accuracy of gesture recognition and user experience.

[0007] Another technical difficulty lies in the mapping and conversion of three-dimensional space motion to two-dimensional screen control. The motion trajectory of the fingers in three-dimensional space needs to be accurately converted into the movement of the cursor on the two-dimensional screen. This dimension reduction mapping process involves complex coordinate transformation and algorithm processing. Existing technologies often use simple linear mapping or projection transformation, which is difficult to balance mapping accuracy and computational efficiency.

[0008] In addition, wearable gesture recognition devices also face the challenge of power consumption management. In order to ensure the real-time and accuracy of gesture recognition, the system needs to continuously collect high-frequency signals and process data, which will significantly increase the power consumption of the device. How to realize low-power operation under the premise of ensuring system performance is an important factor restricting the practicality of wearable gesture recognition devices.

[0009] Ultra-wideband (UWB) technology provides a new way to solve the above technical problems due to its high time resolution, strong anti-interference ability and millimeter-level positioning accuracy. However, existing UWB-based positioning systems are mainly used for large-scale positioning scenarios such as indoor navigation and asset tracking. For the application scenario of gesture recognition with small range, high precision and real-time requirement, how to optimize the architecture design, signal processing algorithm and power consumption management strategy of the UWB system still needs further technical breakthroughs. SUMMARY

[0010] The purpose of the present application is to provide a three-dimensional gesture recognition device and method based on elbow joint positioning, which can establish a stable spatial reference system, realize high-precision three-dimensional positioning, have an efficient dimension reduction mapping algorithm and controllable power consumption, to meet the growing demand for human-computer interaction applications.

[0011] To achieve the above purpose, the present application realizes the following technical solutions: A three-dimensional gesture recognition device based on elbow joint positioning, comprising: A finger end wearing device worn on the finger and comprising a signal emission source for emitting a positioning signal; An elbow joint end wearing device worn on the upper end of the elbow joint and comprising a signal processor for receiving the positioning signal and calculating three-dimensional position information of the finger end wearing device; A dimension reduction processing module for establishing a polar coordinate system with the elbow joint end wearing device as the center and converting the three-dimensional position information into two-dimensional plane coordinate information.

[0012] Further, the signal emission source is a UWB signal emission source, the positioning signal is an ultra-wideband signal, and the frequency band of the ultra-wideband signal is 3.1GHz-10.6GHz.

[0013] Further, the elbow joint end wearing device comprises a plurality of antennas for receiving the positioning signal, and the signal processor calculates distance and angle information of the finger end wearing device relative to the elbow joint end wearing device based on the signals received by the plurality of antennas.

[0014] Further, the plurality of antennas are four antennas arranged in a rhombus on the elbow joint end wearing device.

[0015] Further: the dimension reduction processing module realizes the conversion from three dimensions to two dimensions by the following steps: S1: obtaining the azimuth of the finger end wearing device in the polar coordinate system and the elevation angle ; S2: calculating the angle change amount of the current frame and the previous frame and ; S3: converting the angle change amount into two-dimensional plane coordinate change amount based on the preset proportion coefficient and ; S4: updating the cursor position according to the two-dimensional plane coordinate change amount.

[0016] Further: in the step S3, the calculation formula of the two-dimensional plane coordinate change amount is: , wherein and are adjustable proportion coefficients.

[0017] Further: the finger end wearing device further comprises a motion detection sensor, which is used to detect the finger motion state, and activates the signal emission source when detecting the finger motion, and makes the signal emission source enter the low-power consumption mode when detecting the finger static.

[0018] Further: the motion detection sensor is a low-power consumption accelerometer, and the power consumption of the low-power consumption accelerometer is less than 5μA.

[0019] Further: the finger end wearing device further comprises a wireless charging device, the wireless charging device comprises a receiving coil and a charging management circuit, the elbow joint end wearing device comprises a transmitting coil, and the finger end wearing device is charged by electromagnetic induction.

[0020] Further: it further comprises an error compensation module, and the error compensation module adopts Kalman filtering algorithm to filter the positioning signal, so as to eliminate the influence of noise interference and multipath effect.

[0021] The application also provides a three-dimensional gesture recognition method based on elbow joint positioning, comprising the following steps: S1: emitting a positioning signal through a finger end wearing device worn on a finger; S2: receiving the positioning signal through an elbow joint end wearing device worn on the upper end of the elbow joint, and calculating the three-dimensional position information of the finger end wearing device; S3: establishing a polar coordinate system with the elbow joint end wearing device as the center. S4: Convert the three-dimensional position information into azimuth and elevation information in the polar coordinate system; S5: Calculate the change in two-dimensional plane coordinates based on the changes in azimuth and elevation angles; S6: Control cursor movement based on the change in the two-dimensional plane coordinates.

[0022] Further: In step S2, the distance between the fingertip wearing device and the elbow joint wearing device is calculated by measuring the time of flight of the positioning signal. The calculation formula is as follows: ,in d For distance, c At the speed of light, t This refers to the signal transmission time.

[0023] Compared with the prior art, the present invention has the following advantages: First, by establishing a polar coordinate system with the elbow joint as the center, the positional relationship of the fingers relative to the elbow joint remains stable when the user's arm moves as a whole, effectively solving the problem of inaccurate recognition caused by the lack of a fixed spatial reference system in traditional gesture recognition systems.

[0024] II. An innovative algorithm for directly mapping angle changes is adopted, through... , The simple formula achieves dimensionality reduction transformation, avoiding the complex 3D reconstruction and projection transformation process in existing technologies, and significantly improving computational efficiency and response speed.

[0025] Third, the device detects motion status through a low-power accelerometer to achieve on-demand activation. When stationary, the power consumption is controlled at the microampere level, and when moving, it uses 50Hz high-frequency sampling. This ensures millisecond-level real-time response while achieving ultra-low power operation, thus resolving the technical contradiction between battery life and performance in wearable devices. Fourth, UWB technology is used to achieve millimeter-level positioning accuracy. Combined with Kalman filtering algorithm and multi-antenna array design, it effectively overcomes the technical defects of traditional visual recognition, such as susceptibility to light effects, large cumulative error of inertial sensors, and susceptibility to electromagnetic induction interference. Attached Figure Description

[0026] Figure 1 This is a schematic diagram illustrating the application state of the three-dimensional gesture recognition device based on elbow joint positioning according to one embodiment of the present invention. Figure 2 This is a flowchart illustrating another embodiment of the three-dimensional gesture recognition method based on elbow joint localization according to the present invention; In the picture: 1. Finger-end wearing device; 2. Elbow joint-end wearing device; 3. Dimension reduction processing module; 4. Arm. Detailed Implementation

[0027] The technical solutions of the present application will be described clearly and completely in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0028] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship based on the orientation or positional relationship shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0029] The three-dimensional gesture recognition device based on elbow joint positioning provided by the present application mainly consists of a finger end wearing device 1, an elbow joint end wearing device 2 and a dimension reduction processing module 3. The finger end wearing device 1 adopts a ring shape and is worn on the user's finger, mainly responsible for transmitting positioning signals; the elbow joint end wearing device 2 is fixed on the upper end of the user's elbow joint of the same arm 4 through a flexible band, responsible for receiving positioning signals and processing signals; the dimension reduction processing module 3 can be integrated in the elbow joint end wearing device 2, or can be an independent processing unit, responsible for converting three-dimensional position information into two-dimensional coordinate information.

[0030] The finger end wearing device 1 includes a UWB signal transmitting source, a motion detection sensor, a power supply system and a wireless charging device. The UWB signal transmitting source uses an ultra-wideband chip with a frequency band of 3.1GHz-10.6GHz, the actual power consumption is less than 10mW, and the antenna type uses a flexible PCB antenna or a ceramic antenna. The motion detection sensor is preferably an ADXL362 or LIS2DW12 type low-power three-axis accelerometer, with a typical power consumption of 1-3μA. When a preset acceleration threshold is exceeded, it is considered that the finger starts to move and activates the UWB tag to send signals. The power supply system uses a small lithium battery or a button cell with a capacity of 20mAh-50mAh, which can meet the long-term use requirement.

[0031] The wireless charging device is realized based on the principle of magnetic induction. The receiving coil adopts a high-efficiency and small-size planar spiral coil, which can be a PCB integrated coil or a micro copper wire winding. The outer diameter of the coil is recommended to be 8 mm-15 mm, the thickness is controlled to be 1 mm-2 mm, the number of turns is 10-30 turns, and the working frequency is 100 kHz-500 kHz. A layer of ferrite sheet is added to the back of the coil as a magnetic material for guiding the magnetic field, improving the charging efficiency and reducing the electromagnetic radiation to the fingers. The charging circuit includes a receiving coil, a resonance capacitor, a full-bridge rectifier, an LDO stabilizer and a charging management chip. The charging management chip can adopt TP4056 or BQ51050B, which provides overcharge, overheat and short circuit protection functions, and outputs a stable 3.7V or 3.3V voltage.

[0032] The shell structure of the finger end wearing device 1 refers to the design concept of Samsung Galaxy ring, which is made of lightweight materials to ensure wearing comfort. All electronic components are integrated through miniaturized PCB, and the PCB can be designed flexibly to fit the curved shape of the inner cavity of the ring.

[0033] The elbow joint end wearing device 2 includes a UWB signal processor, a multi-antenna array, a data processing module, a wireless charging transmitting device and a shell fixing device. The UWB signal processor includes a UWB base station module for receiving and processing positioning signals emitted by the finger end. The module needs more than 2 high-frequency antennas for collecting multi-path information to support triangulation positioning. The preferred solution is to arrange 4 antennas in a diamond shape, respectively placed in the upper, lower, left and right directions to achieve omnidirectional signal coverage.

[0034] The data processing module adopts STM32F407 or STM32F405 type MCU based on ARM Cortex-M4 architecture with a main frequency of 120 MHz to 168 MHz, which has sufficient computing power to process real-time data from multiple base stations. The MCU is equipped with multiple high-speed SPI interfaces with a SPI clock rate of more than 20 MHz for data exchange with each UWB chip. Data is transmitted from the SPI interface to the memory buffer at high speed in DMA mode to reduce CPU load. The system deploys a lightweight real-time operating system such as FreeRTOS to arrange task priorities reasonably and separate data acquisition, signal processing, error compensation and data transmission tasks to ensure efficient and real-time data processing flow.

[0035] The signal positioning system can calculate the position of the finger end in real time, has high-precision time measurement capability, and the time resolution reaches 10 ps, which can realize millimeter-level positioning accuracy. The system calculates the distance between the ring and each base station using ToF flight time or TDoA time difference algorithm, and the calculation formula is where d is the distance from the ring to the base station, c is the speed of light,t The signal propagation time is used as a reference.

[0036] The wireless charging transmitting device is located in the elbow joint end wearing device 2, including a transmitting coil and a driving circuit. The transmitting coil is a planar spiral coil, and the inner diameter is slightly larger than that of the receiving coil to improve the alignment efficiency. The working frequency is also 100 kHz-500 kHz. The driving circuit uses a wireless charging control chip such as BQ500212A of TI, cooperates with an H-bridge inverter circuit to generate high-frequency alternating current, and optimizes the magnetic field of the transmitting coil through a resonance capacitor. In order to ensure good alignment during charging, a magnet is arranged between the transmitting end and the receiving end to guide the alignment of the receiving coil and the transmitting coil.

[0037] The shell structure and the fixing device are fixed by using a flexible bandage. The outer material is durable plastic, the inner layer is in contact with the skin, and the material is leather or silicone. The middle structure layer uses high-strength plastic to support and accommodate electronic components and other internal structures.

[0038] The dimension reduction processing module 3 is built-in with a dimension reduction program to establish a three-dimensional coordinate system with the elbow joint as the center. The specific implementation is to establish a polar coordinate system with the elbow joint as the center, and to determine the initial coordinate axis with the distance calculated by the signal flight time as the radius. The mapping conversion from three-dimensional to two-dimensional is calculated by the change amount of the elevation angle θ and the azimuth angle .

[0039] When the system is working, the initialization stage is carried out first. After the UWB tag built-in the ring end is initialized, it starts to periodically send ultra-wideband signals, and the system enters a low-power mode and is activated when motion is detected. The working process of the low-power mode is: when the system enters deep sleep, the UWB tag and the main MCU are turned off, only the accelerometer is kept running, and the power consumption is controlled at microampere level; when the accelerometer detects a motion signal, it triggers an interrupt to wake up the MCU to enter an active state; the MCU activates the UWB tag to start sending signals; after gesture recognition is completed or a finger is detected to be stationary, the system reenters the low-power mode.

[0040] The UWB base station module at the elbow joint end starts to enter the receiving mode, and the built-in MCU initializes to prepare to process signal data. The system needs to be calibrated, including position calibration, clock synchronization and parameter configuration. The position calibration sets calibration points at different plane heights and different horizontal positions near the elbow joint device, selects the uppermost calibration point as the base station origin, and determines the positions of other base stations according to the relative angle of gravity measurement. Since the distance between the base stations is determined before the hardware product is shipped, the positions of other base stations can be directly determined according to the relative angle of gravity. Clock synchronization ensures that the master-slave clock synchronization between the base stations is completed, reducing the measurement error caused by clock offset. Parameter configuration includes setting the sensitivity, error compensation threshold and other parameters of the system.

[0041] In the signal acquisition phase, the ring end periodically sends UWB signals containing a unique ID and timestamp. The transmission frequency is dynamically adjusted according to the gesture action: low frequency when stationary, such as 1-2 times per second; high frequency when moving, such as 50 times per second. The base station antenna array on the elbow joint simultaneously receives the UWB signals from the ring end, and each base station records the reception time, signal strength RSSI, etc. The system filters out invalid signals caused by multipath effects or noise: uses a time threshold to exclude reflected paths, and uses an RSSI threshold to eliminate weak signals. The data from multiple base stations is uploaded to the MCU at the elbow end for synchronous data transmission.

[0042] In the position calculation phase, the system calculates the distance between the ring and each base station based on ToF or TDoA, and directly calculates the azimuth and elevation based on the distance data received by the base station. To improve accuracy, the system uses an error compensation mechanism, using filtering and correction algorithms to optimize the calculation results: maximum likelihood estimation is used to select the direct path for multipath effect correction; Kalman filter is used to process angle change data for noise smoothing; the error parameters obtained in the calibration phase are used to correct the azimuth and elevation for base station offset compensation.

[0043] The specific implementation of Kalman filtering uses finger motion angle values as system states, assuming that the angle changes with time at a constant speed. The state vector contains angle and angular velocity information, and the state transition model considers the sampling time interval and process noise. The measurement model directly observes the angle value and considers the measurement noise. The filtering process includes initialization, prediction step and update step, which continuously optimizes the angle estimation value through recursive calculation.

[0044] In the dimensionality reduction mapping phase, the system first calculates the angle change quantity, compares the azimuth and elevation of the current frame and the previous frame, and calculates the change quantity , . Then use the preset scale factor to map the angle change quantity to the plane movement quantity of the cursor: , , where , is the sensitivity scale factor, which is related to user settings, and the initial value is set to 0.2. Users can adjust these parameters through the software interface to adapt to different screen sizes and user habits. Finally, update the cursor position according to the increment: , .

[0045] Based on the above device, in another embodiment, the present application also provides a three-dimensional gesture recognition method based on elbow joint positioning, comprising the following steps: S1: Emit a positioning signal through the finger end wearing device 1 worn on the finger; S2: receiving the positioning signal through the elbow joint end wearing device 2 worn on the upper end of the elbow joint, and calculating the three-dimensional position information of the finger end wearing device 1; S3: establishing a polar coordinate system with the elbow joint end wearing device 2 as the center; S4: converting the three-dimensional position information into azimuth and elevation information in the polar coordinate system; S5: calculating the two-dimensional plane coordinate change amount based on the change amount of the azimuth and the elevation; S6: controlling the cursor to move according to the two-dimensional plane coordinate change amount.

[0046] Further, in the step S2, the distance between the finger end wearing device 1 and the elbow joint end wearing device 2 is calculated by measuring the time of flight of the positioning signal, and the calculation formula is: Wherein d is the distance, c is the speed of light, and t is the signal transmission time.

[0047] The working principle of the present application ensures high precision, low power consumption and strong stability of the system. By taking the elbow joint as a fixed reference point, the interference of the whole arm 4 movement on gesture recognition is avoided; the UWB technology is adopted to realize millimeter-level positioning accuracy; the intelligent power consumption management strategy ensures long-time use; the Kalman filtering error compensation algorithm improves the stability of the system in complex environment. The whole system design considers various requirements in actual use, has good practicability and user experience.

[0048] The above embodiments only serve to illustrate the technical concept and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the content of the present application and implement it, and cannot limit the protection scope of the present application. Any equivalent transformation or modification made according to the spirit and essence of the present application should be covered within the protection scope of the present application.

Claims

1. A three-dimensional gesture recognition device based on elbow joint positioning, characterized by, The application relates to a finger-end wearing device, a wrist-joint-end wearing device, a dimension reduction processing module, an error compensation module and a method for controlling a cursor. The finger-end wearing device is worn on a finger and comprises a signal emission source for emitting a positioning signal. The wrist-joint-end wearing device is worn on the upper end of the elbow joint of the same arm as the finger-end wearing device and comprises a signal processor for receiving the positioning signal and calculating three-dimensional position information of the finger-end wearing device. The dimension reduction processing module is used for establishing a polar coordinate system with the wrist-joint-end wearing device as the center and converting the three-dimensional position information into two-dimensional plane coordinate information.

2. The three-dimensional gesture recognition apparatus of claim 1, wherein, The signal emission source is a UWB signal emission source, the positioning signal is an ultra-wideband signal, and the frequency band of the ultra-wideband signal is 3.1 GHz-10.6 GHz.

3. The three-dimensional gesture recognition apparatus of claim 1, wherein, The wrist-joint-end wearing device comprises a plurality of antennas for receiving the positioning signal, and the signal processor calculates distance and angle information of the finger-end wearing device relative to the wrist-joint-end wearing device based on the signals received by the plurality of antennas.

4. The three-dimensional gesture recognition apparatus of claim 3, wherein, The plurality of antennas are four antennas arranged in a diamond shape on the wrist-joint-end wearing device.

5. The three-dimensional gesture recognition apparatus of claim 1, wherein, The dimension reduction processing module realizes the conversion from three-dimensional to two-dimensional through the following steps: S1: obtaining an azimuth angle of the finger tip wearing device in a polar coordinate system and an elevation angle ; S2: calculate the angle change amount of the current frame and the previous frame and ; S3: convert the angle change amount into a two-dimensional plane coordinate change amount based on a preset proportion coefficient and ; S4: updating the cursor position according to the two-dimensional plane coordinate change amount.

6. The three-dimensional gesture recognition apparatus of claim 5, wherein, In the step S3, the calculation formula of the two-dimensional plane coordinate variation is: , wherein and are adjustable proportional coefficients.

7. The three-dimensional gesture recognition apparatus of claim 1, wherein, The finger-end wearing device further comprises a motion detection sensor for detecting the motion state of the finger, activating the signal emission source when the finger motion is detected, and making the signal emission source enter a low-power consumption mode when the finger is static.

8. The three-dimensional gesture recognition device of claim 7, wherein, The motion detection sensor is a low-power consumption accelerometer, and the power consumption of the low-power consumption accelerometer is less than 5 mu A.

9. The three-dimensional gesture recognition apparatus of claim 1, wherein, The finger-end wearing device further comprises a wireless charging device, the wireless charging device comprises a receiving coil and a charging management circuit, the wrist-joint-end wearing device comprises a transmitting coil, and the finger-end wearing device is charged through electromagnetic induction.

10. The three-dimensional gesture recognition apparatus of claim 1, wherein, The error compensation module adopts a Kalman filtering algorithm to filter the positioning signal, so as to eliminate the influence of noise interference and multipath effect.

11. A three-dimensional gesture recognition method based on elbow joint positioning, characterized in that, The method comprises the following steps: S1: emitting a positioning signal through a finger-end wearing device worn on a finger; S2: receiving the positioning signal through a wrist-joint-end wearing device worn on the upper end of the elbow joint and calculating three-dimensional position information of the finger-end wearing device; S3: establishing a polar coordinate system with the wrist-joint-end wearing device as the center; S4: converting the three-dimensional position information into azimuth and elevation information in the polar coordinate system; S5: calculating a two-dimensional plane coordinate change amount based on the change amount of the azimuth and the elevation; S6: controlling the cursor to move according to the two-dimensional plane coordinate change amount.

12. The three-dimensional gesture recognition method of claim 11, wherein, In the step S2, the distance between the finger-end wearable device and the elbow-joint-end wearable device is calculated by measuring the time of flight of the positioning signal, and the calculation formula is: wherein d is the distance, c is the speed of light, t is the signal transmission time.

Images