Gesture recognition method, gesture recognition controller and storage medium

Abstract

This invention provides a gesture recognition method, a gesture recognition controller, and a storage medium, relating to the field of intelligent human-computer interaction technology. The method includes: in response to a gesture recognition trigger signal, acquiring current sensor data from an inertial measurement unit (IMU) and determining a reference attitude angle based on the current sensor data; acquiring real-time sensor data from the IMU and determining a real-time attitude angle based on the real-time sensor data; calculating a relative angle change based on the real-time attitude angle and the reference attitude angle; and determining a gesture recognition result based on the relative angle change. This invention enables real-time and accurate gesture recognition on low-cost local hardware.

Description

Technical Field

[0001] This invention relates to the field of intelligent human-computer interaction technology, and in particular to a gesture recognition method, a gesture recognition controller, and a storage medium. Background Technology

[0002] With the increasing prevalence of robots and various intelligent devices in daily production and life, the naturalization and intuitiveness of human-computer interaction have become important development trends. Breaking free from the constraints of traditional physical contact controllers (such as mice, keyboards, and remote controls) and directly "translating" human hand gestures into machine commands is one of the key directions in the development of control methods. Gesture-based interaction not only has extremely low learning costs but also provides an intuitive and immersive user experience, significantly improving operational efficiency and user satisfaction, especially demonstrating unique advantages in spatial interaction, dynamic control, and wearable scenarios.

[0003] However, current mainstream gesture recognition methods, especially those aiming for high accuracy and complex pattern recognition, typically rely on one or more cameras to capture hand images, while simultaneously running complex image processing and gesture recognition models on a backend server or high-performance computing terminal to interpret the gestures. While these methods can achieve high recognition accuracy, they usually require expensive hardware such as cameras and image processors, and the inference process is computationally complex and resource-intensive, often exhibiting latency, making them unsuitable for interactive scenarios with extremely high real-time requirements. Furthermore, gesture recognition performance is easily affected by ambient lighting conditions, background interference, occlusion, and other factors, thus impacting the accuracy of the gesture recognition results.

[0004] Therefore, how to achieve real-time and accurate gesture recognition at low cost is an urgent problem to be solved. Summary of the Invention

[0005] This invention provides a gesture recognition method, a gesture recognition controller, and a storage medium to achieve real-time and accurate gesture recognition on low-cost local hardware.

[0006] This invention provides a gesture recognition method, applied to a gesture recognition controller, comprising: In response to a gesture recognition trigger signal, the current sensor data of the inertial measurement unit is acquired, and a reference attitude angle is determined based on the current sensor data; Acquire real-time sensor data from the inertial measurement unit and determine the real-time attitude angle based on the real-time sensor data; The relative angle change is calculated based on the real-time attitude angle and the reference attitude angle. The gesture recognition result is determined based on the relative angle change.

[0007] According to a gesture recognition method provided by the present invention, the step of calculating the relative angle change based on the real-time attitude angle and the reference attitude angle includes: Calculate the attitude angle difference between the real-time attitude angle and the reference attitude angle; If the attitude angle difference exceeds the preset angle range, the attitude angle difference is corrected to obtain the relative angle change, which is within the preset angle range.

[0008] According to a gesture recognition method provided by the present invention, determining the gesture recognition result based on the relative angle change includes: The rotation matrix is ​​determined based on the reference pitch angle in the reference attitude angle; The relative angle change is mapped using the rotation matrix to obtain the mapped angle change. The gesture recognition result is determined based on the angle change after mapping.

[0009] According to a gesture recognition method provided by the present invention, the step of acquiring current sensor data of an inertial measurement unit and determining a reference attitude angle based on the current sensor data includes: Obtain current gyroscope data, current magnetometer data, and current accelerometer data; Based on the current gyroscope data and the state calculation matrix of the previous moment, a state prediction matrix for the current moment is predicted; wherein, the state prediction matrix for the current moment includes at least the estimated value of the reference attitude angle and the estimated value of the gyroscope zero bias. Based on the current magnetometer data and the current accelerometer data, the state prediction matrix at the current moment is updated, and the reference attitude angle is extracted.

[0010] According to a gesture recognition method provided by the present invention, the current state prediction matrix includes a first state prediction matrix, a second state prediction matrix, and a third state prediction matrix for the current time, and the current accelerometer data includes the current... x Axis acceleration, current y Axis acceleration and current z Axial acceleration, the process of updating the state prediction matrix at the current moment based on the current magnetometer data and the current accelerometer data, and extracting the reference attitude angle, includes: Based on the current magnetometer data, the first state prediction matrix at the current moment is updated to obtain the first state calculation matrix at the current moment; The current yaw angle is extracted from the first state calculation matrix at the current moment; Based on the current situation xAxial acceleration, the current y Axial acceleration and the current z The axis acceleration is used to update the second state prediction matrix at the current moment, thus obtaining the second state calculation matrix at the current moment; The current pitch angle is extracted from the second state calculation matrix at the current moment; Based on the current situation x Axial acceleration, the current y Axial acceleration and the current z The axis acceleration is used to update the third state prediction matrix at the current time, thus obtaining the third state calculation matrix at the current time. The current roll angle is extracted from the third state calculation matrix at the current moment; The reference attitude angles include the current yaw angle, the current pitch angle, and the current roll angle.

[0011] According to a gesture recognition method provided by the present invention, the gesture recognition method further includes: Establish a communication connection with the host computer; Based on the communication connection, the gesture recognition result is sent to the host computer.

[0012] The present invention also provides a gesture recognition controller, comprising: a control module, an inertial measurement unit, and a manipulation module; The control module is connected to both the inertial measurement unit and the manipulation module. The control module is used to send a gesture recognition trigger signal to the control module; The control module is used to implement the gesture recognition method as described in any of the above.

[0013] According to a gesture recognition controller provided by the present invention, the inertial measurement unit includes a gyroscope, the gyroscope being connected to the control module; and / or, The inertial measurement unit further includes a magnetometer, which is connected to the control module; and / or, The inertial measurement unit also includes an accelerometer, which is connected to the control module.

[0014] According to a gesture recognition controller provided by the present invention, the control module includes a recognition switch button, the recognition switch button being connected to the control module; and / or, The gesture recognition controller further includes a control feedback module, which is connected to the control module; and / or, The gesture recognition controller further includes a communication module connected to the control module, and the communication module is also used for communication with a host computer; and / or, The gesture recognition controller also includes an energy storage power supply module, which is connected to the control module, the inertial measurement unit, and the operation module.

[0015] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the gesture recognition method as described in any of the preceding claims.

[0016] The gesture recognition method, gesture recognition controller, and storage medium provided by this invention, in response to a gesture recognition trigger signal, acquire current sensor data from an inertial measurement unit (IMU) and determine a current attitude angle based on the current sensor data as a reference attitude angle; then, acquire real-time sensor data from the IMU and determine a real-time attitude angle based on the real-time sensor data; calculate the relative angle change based on the real-time attitude angle and the reference attitude angle; and determine the gesture recognition result based on the relative angle change. This invention does not rely on an external camera but instead acquires sensor data through a low-cost IMU unaffected by ambient light. Simultaneously, this invention determines the reference attitude angle based on the sensor data at the moment corresponding to the gesture recognition trigger signal (i.e., the current sensor data) and determines the real-time attitude angle based on subsequently acquired real-time sensor data, and then determines the gesture recognition result by calculating the relative angle change. Through the above methods, this invention transforms the core of gesture recognition into a simple relative angle change. This process involves only simple subtraction calculations and threshold range comparisons, enabling the entire algorithm to be deployed in resource-constrained portable or embedded devices without increasing hardware costs. For example, it can run locally in real-time on a low-cost, low-power SOC (System on a Chip) single control chip. Furthermore, the gesture recognition logic of this invention is stable and reliable, significantly reducing the development threshold and system power consumption. In summary, this invention can achieve real-time and accurate gesture recognition on low-cost local hardware, making it particularly suitable for portable and embedded applications. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0018] Figure 1 This is one of the flowcharts illustrating the gesture recognition method provided by the present invention.

[0019] Figure 2 This is the second flowchart of the gesture recognition method provided by the present invention.

[0020] Figure 3 This is one of the full-body side views of the gesture recognition origin provided by the present invention.

[0021] Figure 4 This is one of the front views of the gesture recognition origin provided by the present invention.

[0022] Figure 5 This is one of the hand gesture diagrams provided by the present invention.

[0023] Figure 6 This is the third flowchart of the gesture recognition method provided by the present invention.

[0024] Figure 7 This is the second full-body side view of the gesture recognition origin provided by the present invention.

[0025] Figure 8 This is the second front view of the gesture recognition origin provided by the present invention.

[0026] Figure 9 This is the second illustration of a gesture action provided by the present invention.

[0027] Figure 10 This is the fourth flowchart of the gesture recognition method provided by the present invention.

[0028] Figure 11 This is a schematic diagram of the gesture recognition controller provided by the present invention. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0030] This invention proposes a gesture recognition method, a gesture recognition controller, and a storage medium, which are described below in conjunction with... Figures 1-11 Describe it.

[0031] Figure 1 This is one of the flowcharts illustrating the gesture recognition method provided by the present invention, such as... Figure 1 As shown, the gesture recognition method includes steps S110, S120, S130 and S140.

[0032] Step S110: In response to the gesture recognition trigger signal, the current sensor data of the inertial measurement unit is acquired, and a reference attitude angle is determined based on the current sensor data.

[0033] In this embodiment, the gesture recognition method is applied to a gesture recognition controller. This gesture recognition controller is worn on the user's hand, such as... Figures 3-5 or Figures 7-9 As shown.

[0034] In one embodiment, the gesture recognition controller includes a control module, an inertial measurement unit (IMU), and a manipulation module. The IMU is used to collect current sensor data and real-time sensor data. The control module is connected to both the IMU and the manipulation module. Specifically, the control module and the IMU can communicate using the I2C (Inter-Integrated Circuit) protocol or the SPI (Serial Peripheral Interface) protocol to acquire the current sensor data and real-time sensor data collected by the IMU, thereby executing the gesture recognition method of this embodiment.

[0035] In gesture recognition applications, the Inertial Measurement Unit (IMU) 120 has natural advantages. First, its working environment is unaffected by light and background noise, enabling stable operation in complex environments and compensating for the shortcomings of existing vision solutions. Second, it does not require an external camera, avoiding privacy risks, and the device is easy to integrate into wearable devices, enabling natural interaction in mobile scenarios. In addition, it can track the rotation and acceleration of the hand in three-dimensional space with high frequency and low latency, making it extremely adept at capturing fast, subtle, and continuous movements.

[0036] Furthermore, the processor is a SOC control chip, specifically an ESP32 (Espressif Systems 32-bit) chip. Since the ESP32 allows for software-configurable hardware pins, pins such as I2C, buttons, and LEDs can be adjusted via software. Of course, the processor can also be other main control chips with communication capabilities, including but not limited to ARM, RISC-V, and x86 chips. In this embodiment, the processor is described as an example of a SOC control chip.

[0037] This gesture recognition method is applicable to scenarios such as robot control and smart home control.

[0038] When a user wants to initiate gesture recognition, they can trigger a gesture recognition signal via the control module. In response to this signal, the SOC control chip acquires the current sensor data from the inertial measurement unit (referred to as current sensor data) and determines the current attitude angle based on this data, using it as a reference attitude angle. This reference attitude angle represents the starting point of the gesture.

[0039] Furthermore, the control module can use physical or virtual recognition switch buttons, and the triggering method for gesture recognition trigger signals can be: (1) Physical button trigger: The user presses the physical recognition switch button on the gesture recognition controller to generate a trigger signal. This method is simple, reliable, and provides clear feedback, which can effectively reduce the possibility of misoperation.

[0040] (2) Virtual button trigger: The user clicks the virtual recognition switch button "Start Recognition" on the display screen of the gesture recognition controller.

[0041] Of course, it should be understood that in specific embodiments, a specific gesture trigger can also be used: when a user makes a preset specific gesture, such as shaking the device twice quickly, a gesture recognition trigger signal is automatically generated.

[0042] An inertial measurement unit (IMU) includes a gyroscope, a magnetometer, and an accelerometer, which are used to collect gyroscope data, magnetometer data, and accelerometer data, respectively.

[0043] Attitude angle refers to the orientation of an object in three-dimensional space, usually expressed as yaw angle. z (axis rotation), pitch angle (around) x (axis rotation) and roll angle (around) y (axis rotation) is used to represent this.

[0044] The calculation process for the reference attitude angle is as follows: Obtain current gyroscope data, current magnetometer data, and current accelerometer data; based on the current gyroscope data and the state calculation matrix from the previous moment, predict the state prediction matrix for the current moment; wherein, the state prediction matrix for the current moment includes at least the predicted value of the reference attitude angle and the predicted value of the gyroscope zero bias; based on the current magnetometer data and the current accelerometer data, update the state prediction matrix for the current moment and extract the reference attitude angle. Assume the current moment is... k The time is based on the current gyroscope data and k- Calculate the state matrix at time 1 and predict. k The state prediction matrix at time t; where, k The state prediction matrix at time t should include at least the following: kThe estimated value of the attitude angle at time and k The gyroscope zero-bias prediction at a given moment; based on current magnetometer and accelerometer data, the following is performed: k The state prediction matrix at time step is updated and extracted. k The attitude angle at a given time is used as the reference attitude angle. The specific execution process can be found in the following embodiment, and will not be elaborated upon here.

[0045] Step S120: Obtain the real-time sensor data of the inertial measurement unit, and determine the real-time attitude angle based on the real-time sensor data.

[0046] After the user triggers the gesture recognition signal, the sensor data of the inertial measurement unit is continuously acquired (referred to as real-time sensor data), and the real-time attitude angle is determined based on the real-time sensor data.

[0047] The real-time sensor data includes real-time gyroscope data, real-time magnetometer data, and real-time accelerometer data. Assume the acquisition time of the real-time sensor data is... k+n At any given moment, the real-time attitude angle calculation process is as follows: acquire real-time gyroscope data, real-time magnetometer data, and real-time accelerometer data; based on the real-time gyroscope data and... k+n- Calculate the state matrix at time 1 and predict. k+n The state prediction matrix at time t; where, k+n The state prediction matrix at time t should include at least the following: k+n The estimated value of the attitude angle at time and k+n The gyroscope zero-bias prediction at any given time; based on real-time magnetometer and real-time accelerometer data, the following is performed: k+n The state prediction matrix at time step is updated and extracted. k+n The attitude angle at a given time is used as the real-time attitude angle. The specific execution process is similar to that of obtaining the reference attitude angle, and will not be elaborated here.

[0048] Step S130: Calculate the relative angle change based on the real-time attitude angle and the reference attitude angle.

[0049] After obtaining the real-time attitude angle and the reference attitude angle, the relative angle change is calculated.

[0050] The reference attitude angles include the yaw angle, pitch angle, and roll angle at the current moment, which are denoted as the current yaw angle, current pitch angle, and current roll angle, respectively.

[0051] Real-time attitude angles include real-time yaw angle, pitch angle, and roll angle, denoted as real-time yaw angle, real-time pitch angle, and real-time roll angle, respectively.

[0052] The difference between the real-time yaw angle and the current yaw angle is calculated to obtain the relative angle change of the yaw angle; simultaneously, the difference between the real-time pitch angle and the current pitch angle is calculated to obtain the relative angle change of the pitch angle; and the difference between the real-time roll angle and the current roll angle is calculated to obtain the relative angle change of the roll angle. The relative angle changes include the relative angle changes of the yaw angle, pitch angle, and roll angle.

[0053] Furthermore, to ensure the correct physical meaning of the angle difference, if the calculated attitude angle difference is not within the preset angle range, it needs to be corrected. The specific correction process can be found in the following embodiments, which will not be elaborated here.

[0054] Step S140: Determine the gesture recognition result based on the relative angle change.

[0055] The relative angle change is matched against a pre-stored gesture feature library, which includes different relative angle change threshold ranges and their corresponding gestures. If a predefined gesture is matched, it is determined that the user has performed that gesture, and the gesture recognition result is obtained.

[0056] The gesture recognition method provided in this invention, in response to a gesture recognition trigger signal, acquires current sensor data from an inertial measurement unit (IMU) and determines a current attitude angle based on this data as a reference attitude angle. Then, it acquires real-time sensor data from the IMU and determines a real-time attitude angle based on this data. Based on the real-time attitude angle and the reference attitude angle, it calculates the relative angle change. Finally, it determines the gesture recognition result based on the relative angle change. This invention does not rely on an external camera but instead uses a low-cost IMU unaffected by ambient light to acquire sensor data. Furthermore, it determines the reference attitude angle based on the sensor data at the moment corresponding to the gesture recognition trigger signal (i.e., the current sensor data) and the real-time attitude angle based on subsequently acquired real-time sensor data. Finally, it determines the gesture recognition result by calculating the relative angle change. Through the above methods, this invention transforms the core of gesture recognition into a simple relative angle change. This process involves only simple subtraction calculations and threshold range comparisons, enabling the entire algorithm to be deployed in resource-constrained portable or embedded devices without increasing hardware costs. For example, it can run locally in real time on a low-cost, low-power SOC single-control chip. Furthermore, the gesture recognition logic of this invention is stable and reliable, significantly reducing development barriers and system power consumption. In summary, this invention enables real-time and accurate gesture recognition on low-cost local hardware, making it particularly suitable for portable and embedded applications.

[0057] Based on any of the above embodiments Figure 2This is the second flowchart illustrating the gesture recognition method provided by the present invention, as shown below. Figure 2 As shown, step S130 includes step S131 and step S132.

[0058] Step S131: Calculate the attitude angle difference between the real-time attitude angle and the reference attitude angle.

[0059] Step S132: If the attitude angle difference exceeds the preset angle range, the attitude angle difference is corrected to obtain the relative angle change, and the relative angle change is within the preset angle range.

[0060] For ease of description, in this embodiment of the invention, the origin of gesture recognition is set to be the same as the origin of the IMU (where yaw, pitch, and roll angles are all zero). For example... Figure 3 and Figure 4 In addition, in this embodiment of the invention, besides the origin, six gestures are defined, as illustrated in the diagram below. Figure 5 As shown.

[0061] Because everyone's perception of angle is different, the yaw, pitch, and roll angles of the gesture recognition controller are generally non-zero when the user raises their hand. Furthermore, the direction of the user's raised hand is usually at an angle to the IMU's origin. Therefore, the differences in yaw, pitch, and roll angles need to be corrected during gesture recognition.

[0062] First, calculate the attitude angle difference between the real-time attitude angle and the reference attitude angle. Then, determine whether the attitude angle difference exceeds the preset angle range. If the attitude angle difference exceeds the preset angle range, correct the attitude angle difference to obtain the relative angle change, so that the relative angle change is within the preset angle range. If the attitude angle difference does not exceed the preset angle range, there is no need to correct the attitude angle difference, and the attitude angle difference is directly used as the relative angle change.

[0063] Furthermore, considering the angle output range of the IMU, the preset angle range is set to (-180°, 180°).

[0064] The reference attitude angles include the yaw angle, pitch angle, and roll angle at the current moment, which are denoted as the current yaw angle, current pitch angle, and current roll angle, respectively.

[0065] Real-time attitude angles include real-time yaw angle, pitch angle, and roll angle, denoted as real-time yaw angle, real-time pitch angle, and real-time roll angle, respectively.

[0066] Correspondingly, the relative angle changes include the relative angle changes of yaw angle, pitch angle, and roll angle.

[0067] Assume the current time is k Real-time response k+n At any given time, the correction method for the relative change in yaw angle is as follows: in, This represents the change in yaw angle relative to the angle. express k+n Yaw angle at any moment express k Yaw angle at any moment express k+n Yaw angle at time and k The difference in yaw angle at any given time.

[0068] For example, suppose k The yaw angle is obtained by constant fusion. θ Yaw_k The angle is 150°. After the user triggers the gesture recognition signal, they will shift their hand 41° in the positive yaw direction. k + n The yaw angle is obtained by constant fusion. θ Yaw_k+n The value is -169°. Correspondingly, when the actual hand gesture's yaw angle is corrected according to the aforementioned correction method for relative angle changes, since -169° - 150° = -319° < -180°, the correction result is... The value is -169° - 150° + 360° = 41°. After correction, this value matches the user's actual gesture offset. If this correction were not performed and -319° were directly used as the relative angle change of the yaw angle, incorrect gesture recognition results would be obtained.

[0069] Furthermore, the correction method for the relative angle change of the pitch angle is as follows: in, This represents the relative change in pitch angle. express k+n The angle of elevation at any moment, express k The angle of elevation at any moment, express k+n The pitch angle at any moment and k The difference in pitch angle at any given moment.

[0070] Furthermore, the correction method for the relative change in roll angle is as follows: in, This represents the change in the roll angle relative to the angle. express k+nThe roll angle at any moment express k The roll angle at any moment express k+n The roll angle at any moment and k The difference in roll angle at any given moment.

[0071] Furthermore, after correction, the relative angle change is matched with the pre-stored gesture feature library in Table 1 below to recognize the gesture.

[0072] Furthermore, if the matching result falls within the relative angle change threshold range corresponding to a certain gesture, then it can be determined that the gesture was performed. If the matching result falls within the relative angle change threshold range corresponding to any gesture, then it can be determined that no valid gesture was performed.

[0073] Table 1

[0074] Note: ACT1~ACT6 represent gesture 1~gesture 6 respectively, corresponding to... Figure 5 (A)~(F) in the text.

[0075] It should be understood that the 30° in Table 1 above represents the gesture recognition range. In practical applications, the recognition range can be modified according to the user's characteristics.

[0076] Furthermore, θ* Pitch1 ~ θ* Pitch6 , θ* Roll1 ~ θ* Roll6 , θ* Yaw1 ~ θ* Yaw6 The preset angle corresponding to the origin of the gesture, when... Figure 3 and Figure 4 When the gesture is the origin, its initial value is set as shown in Table 2.

[0077] Table 2

[0078] For example, users k A gesture recognition trigger signal is constantly activated, and the gesture recognition controller enters the recognition state to read relevant angles and perform gesture recognition. Assuming... k The attitude angles at time points are as follows: θ Pitch_k+ =-3°, θ Roll_k =1°, θ Yaw_k =2°, then, the user at k+n The gestures are made at various times, with the following posture angles: θ Pitch_k+n =-63°, θ Roll_k+n =13°, θ Yaw_k+n =14°, calculate the relative angle change used for gesture judgment as follows: Δ θ Pitch_k+n =-60°, Δ θ Roll_k+n =12°, Δ θ Yaw_k+n =12°. After matching, it falls within the relative angle change threshold range of ACT2; therefore, it is determined that the user performed gesture 2 (ACT2). If... k+n Time Δ θ Pitch_k+n =-13°, Δ θ Roll_k+n =10° and Δ θ Yaw_k+n If the angle is 11°, it does not fall within the relative angle change threshold range of ACT1~ACT6, and therefore it is determined that no valid gesture was executed.

[0079] The gesture recognition method provided in this invention corrects the posture angle difference to obtain the relative angle change, ensuring that the relative angle change is within a preset angle range with clear physical meaning. This solves the problem of angle jumps at the ±180° boundary, guarantees that the calculated relative angle change truly reflects the physical rotation angle and direction of the user's gesture, avoids ambiguity and errors caused by the periodicity of the angle representation, and thus improves the accuracy of gesture recognition results.

[0080] Based on any of the above embodiments Figure 6 This is the third flowchart of the gesture recognition method provided by the present invention, as shown below. Figure 6 As shown, step S140 includes: step S141, step S142 and step S143.

[0081] Step S141: Determine the rotation matrix based on the reference pitch angle in the reference attitude angle.

[0082] Considering that if it continues to be like this during use Figure 3 Raising the arm to a horizontal position can easily tire the user, especially after 5-6 minutes. The feel for the hand deteriorates compared to the initial use, the angle perception worsens, and the posture deviation increases, thus affecting the final gesture recognition performance. Therefore, in this embodiment of the invention, an angle mapping method is used to achieve gesture recognition when the user's arm is in a vertical position, such as... Figure 7 and Figure 8 As shown. Furthermore, in the vertical arm position, the states of the six movements are as follows: Figure 9 As shown.

[0083] Analysis revealed that the pitch angle differs between the horizontal arm-raised state and the vertical arm-holding state, which affects Δ θ Pitch_k+n Δ θ Roll_k+n With Δ θ Yaw_k+n Therefore, angle transformation is used to correct the angle. First, the rotation matrix is ​​determined based on the reference pitch angle in the reference attitude angle, as follows: ; Step S142: Using the rotation matrix, perform coordinate system mapping on the relative angle change to obtain the mapped angle change.

[0084] Coordinate system mapping transformations are achieved through the following matrix multiplication: ; in, This represents the change in pitch angle after mapping. This represents the change in roll angle after mapping. This represents the change in yaw angle after mapping.

[0085] The changes in angles after mapping include the changes in pitch angle, roll angle, and yaw angle after mapping.

[0086] Step S143: Determine the gesture recognition result based on the mapped angle change.

[0087] Finally, the gesture recognition result is determined based on the mapped angle change. This involves matching the mapped angle change with the pre-stored gesture feature library shown in Table 1 above to recognize the gesture.

[0088] The gesture recognition method provided in this invention performs coordinate system mapping transformation on the relative angle change through the aforementioned rotation matrix mapping based on the reference pitch angle, thereby decoupling the gesture recognition algorithm from the initial angle of the user's hand holding the gesture recognition controller. Users no longer need to exert effort to keep their arms horizontal, allowing them to operate in a more natural and comfortable posture (such as with arms hanging naturally), greatly improving the user experience, reducing fatigue, and expanding the product's applicable scenarios.

[0089] Based on any of the above embodiments Figure 10 This is the fourth flowchart of the gesture recognition method provided by the present invention, as shown below. Figure 10As shown, the step "acquiring the current sensor data of the inertial measurement unit and determining the reference attitude angle based on the current sensor data" includes steps S111, S112 and S113.

[0090] Step S111: Obtain the current gyroscope data, current magnetometer data, and current accelerometer data.

[0091] Acquire the sensor data of the inertial measurement unit at the current moment (denoted as current sensor data), specifically including the gyroscope data at the current moment (denoted as current gyroscope data), the magnetometer data at the current moment (denoted as current magnetometer data), and the accelerometer data at the current moment (denoted as current accelerometer data).

[0092] Among them, the current gyroscope data can be the gyroscope angle value at the current moment, the current magnetometer data can be the magnetometer angle value at the current moment, and the current accelerometer data can be the current... x Axial acceleration, y Axial acceleration and z Axial acceleration.

[0093] Step S112: Based on the current gyroscope data and the state calculation matrix of the previous moment, predict the state prediction matrix for the current moment; wherein, the state prediction matrix for the current moment includes at least the estimated value of the reference attitude angle and the estimated value of the gyroscope zero bias.

[0094] After acquiring the current gyroscope data, current magnetometer data, and current accelerometer data, a state estimation algorithm is used to fuse these data to obtain the reference attitude angle. Preferably, a Kalman filter algorithm is used for data fusion.

[0095] It should be understood that for each reference attitude angle, including the reference yaw angle, reference pitch angle, and reference roll angle, a Kalman filter is used for parallel processing.

[0096] Specifically, for each reference attitude angle, the state prediction matrix for the current moment is first calculated based on the current gyroscope data and the state at the previous moment. This matrix is ​​then used to predict the state at the current moment. The state prediction matrix for the current moment is also called the prior state estimate for the current moment, and it includes at least the predicted value of the reference attitude angle and the predicted value of the gyroscope zero bias.

[0097] Assume the current time is k At time 10, the state prediction matrix for the current time is as follows: ; in, for k The state prediction matrix at time 1, also known ask Prior state estimation at time 10:00 for k The state calculation matrix at time -1 is also called... k Posterior state estimation at time -1 for k The gyroscope angle value at that moment. F The state transition matrix is ​​as follows: ; To determine the time interval for reading the gyroscope and magnetometer readings, or the calculation interval for data fusion, in this embodiment of the invention, the two time intervals for reading the gyroscope and magnetometer can be the same, in which case that time interval is directly used; the two time intervals for reading the gyroscope and magnetometer can also be different, in which case the calculation interval for data fusion can be selected. B is the control input matrix, as follows: .

[0098] Furthermore, At least include the estimated reference attitude angle. gyroscope zero bias prediction .Right now, for , The initial value is 0. The initial value is also 0.

[0099] When the reference attitude angle is the reference yaw angle, then for At this point, the state prediction matrix for the yaw angle... for When the reference attitude angle is the reference pitch angle, then for At this point, the state prediction matrix for the pitch angle... for When the reference attitude angle is the reference roll angle, then for At this point, the roll angle state prediction matrix for .

[0100] Meanwhile, the prediction error covariance matrix, denoted as the error covariance prediction matrix, is also known as the prior error covariance matrix. k Error covariance prediction matrix at time step The calculation method is as follows: ; The initial value is a diagonal matrix. P 0 = diag([0.12 0.001 2 ]), 0.1 is the initial angle error, and 0.001 is the initial drift error. for k The error covariance matrix at time -1 is also called... k The posterior error covariance matrix at time -1 F T for F transpose, Q The noise covariance is related to the noise level of the sensor and needs to be adjusted in actual measurements. It can be set as follows: .

[0101] Step S113: Based on the current magnetometer data and the current accelerometer data, update the state prediction matrix at the current moment and extract the reference attitude angle.

[0102] Then, based on the current magnetometer data and the current accelerometer data, the state prediction matrix at the current moment is updated, and the reference attitude angle is extracted.

[0103] Specifically, first calculate the Kalman gain. S for: ; in, R To observe the noise covariance, in this embodiment of the invention, the noise value of the sensor is set to 0.1. H for H T transpose, H The observation matrix is ​​as follows: ; Next, the Kalman gain is updated to obtain the updated Kalman gain. K for: ; Finally, update k The state prediction matrix at time t is obtained. k State calculation matrix at time step , (also known as k The posterior state estimation at time t(t) is as follows: ; ; in, for k The magnetometer angle value or accelerometer angle value at time t, where y represents the measurement residual.

[0104] At the same time, updatek The error covariance prediction matrix at time t is obtained. k The error covariance matrix at time t is also called the error covariance matrix at time t. k The posterior error covariance matrix at time t is as follows: ; Finally, the extraction was obtained k At any given moment, refer to the attitude angle. The calculated value is related to the gyroscope's zero bias. b gyro_k The calculated values ​​are as follows: ; .

[0105] Specifically, according to k Magnetometer data at any given time, for k The state prediction matrix of the yaw angle at time step (denoted as the first state prediction matrix) is updated, and the reference yaw angle is extracted; based on k Moment x Axial acceleration, y Axial acceleration and z Axial acceleration, for k The state prediction matrix for the elevation angle at time step (denoted as the second state prediction matrix) is updated, and the reference elevation angle is extracted; based on k Moment x Axial acceleration, y Axial acceleration and z Axial acceleration, for k The state prediction matrix (denoted as the third state prediction matrix) of the roll angle at time t is updated, and the reference roll angle is extracted. The specific update and extraction process can be found in the following embodiment, which will not be elaborated here.

[0106] The gesture recognition method provided in this invention uses Kalman filtering to dynamically fuse the short-term high precision of the gyroscope with the long-term stability of the magnetometer / accelerometer. This not only outputs a high-precision reference attitude angle in real time, but also simultaneously estimates and compensates for the zero bias of the gyroscope. This fundamentally suppresses the angle drift problem caused by pure gyroscope integration, laying the foundation for long-term stable gesture recognition.

[0107] Based on any of the above embodiments, the current state prediction matrix includes a first state prediction matrix, a second state prediction matrix, and a third state prediction matrix for the current time, and the current accelerometer data includes the current... x Axis acceleration, current y Axis acceleration and current zShaft acceleration, step S113 includes: step S1131, step S1132, step S1133, step S1134, step S1135 and step S1136.

[0108] It should be noted that the execution order of steps S1131-S1132, S1133-S1134, and S1135-S1136 is not important and they can be executed in parallel.

[0109] Step S1131: Based on the current magnetometer data, update the first state prediction matrix at the current moment to obtain the first state calculation matrix at the current moment.

[0110] Step S1132: Extract the current yaw angle from the first state calculation matrix at the current moment.

[0111] The current yaw angle is obtained by fusing current gyroscope data and current magnetometer data. The current magnetometer data can be the magnetometer angle value at the current moment.

[0112] During the update extraction, the state prediction matrix of the yaw angle at the current moment (denoted as the first state prediction matrix) is updated based on the magnetometer angle value at the current moment, resulting in the state calculation matrix of the yaw angle at the current moment (denoted as the first state calculation matrix). Then, the current yaw angle is extracted from the first state calculation matrix at the current moment.

[0113] Assume the current time is k time, k State prediction matrix of yaw angle at time ( k The first state prediction matrix at time t is denoted as When updating it, Specifically k Magnetic angle value at time Correspondingly, the update yields... k The first state calculation matrix at time t is as follows: ; Furthermore, the extraction yielded k The yaw angle at that moment is: .

[0114] It should be understood that the specific calculation formula can be found in the corresponding calculation formula in the previous embodiment.

[0115] Step S1133, based on the current x Axial acceleration, the current y Axial acceleration and the current zThe axis acceleration is used to update the second state prediction matrix at the current moment, thus obtaining the second state calculation matrix at the current moment.

[0116] Step S1134: Extract the current pitch angle from the second state calculation matrix at the current moment.

[0117] The current pitch angle is obtained by fusing current gyroscope and accelerometer data. The current acceleration data can be the current moment's... x Axial acceleration (denoted as current) x (axis acceleration), current moment y Axial acceleration (denoted as current) y (axial acceleration) and the current moment z Axial acceleration (denoted as current) z Axial acceleration).

[0118] When updating the extraction, based on the current... x Axis acceleration, current y Axis acceleration and current z The axis acceleration is used to update the state prediction matrix of the pitch angle at the current moment (denoted as the second state prediction matrix), resulting in the state calculation matrix of the pitch angle at the current moment (denoted as the second state calculation matrix). Then, the current pitch angle is extracted from the second state calculation matrix at the current moment.

[0119] Assume the current time is k time, k State prediction matrix of elevation angle at time ( k The second state prediction matrix at time (time) is denoted as When updating it, Specifically, according to k Moment x Axial acceleration (denoted as) ), k Moment y Axial acceleration (denoted as) )and k Moment z Axial acceleration (denoted as) The calculated pitch angle value The calculation method is as follows: ; Where arctan2 is the arctangent function in the four quadrants, and the specific calculation return value is shown in Table 3 below: Table 3 Correspondingly, the update is obtained k The second state calculation matrix at time t is as follows: ; Furthermore, the extraction yielded k The pitch angle at that moment is: .

[0120] Step S1135, based on the current x Axial acceleration, the y Axial acceleration and the current z The axis acceleration is used to update the third state prediction matrix at the current time, thus obtaining the third state calculation matrix at the current time.

[0121] Step S1136: Extract the current roll angle from the third state calculation matrix at the current moment.

[0122] The reference attitude angles include the current yaw angle, the current pitch angle, and the current roll angle.

[0123] The current roll angle is obtained by fusing current gyroscope and accelerometer data. The current acceleration data can be the current moment's... x Axial acceleration (denoted as current) x (axis acceleration), current moment y Axial acceleration (denoted as current) y (axial acceleration) and the current moment z Axial acceleration (denoted as current) z Axial acceleration).

[0124] When updating the extraction, based on the current... x Axis acceleration, current y Axis acceleration and current z The axis acceleration is used to update the state prediction matrix of the roll angle at the current moment (denoted as the third state prediction matrix), resulting in the state calculation matrix of the roll angle at the current moment (denoted as the third state calculation matrix). Then, the current roll angle is extracted from the third state calculation matrix at the current moment.

[0125] Assume the current time is k time, k State prediction matrix of roll angle at time ( k The third state prediction matrix at time (time) is denoted as When updating it, Specifically, according to k Moment x Axial acceleration (denoted as) ), k Moment y Axial acceleration (denoted as) )and kMoment z Axial acceleration (denoted as) The calculated roll angle value The calculation method is as follows: ; Here, arctan2 is the arctangent function in the four quadrants, and the specific calculation return value is shown in Table 3 above.

[0126] Correspondingly, the update is obtained k The third state calculation matrix at time t is as follows: ; Furthermore, the extraction yielded k The roll angle at that moment is: .

[0127] Based on the above data fusion methods k The three reference angle values ​​at time are respectively θ Pitch_k , θ Roll_k and θ Yaw_k .

[0128] The gesture recognition method provided in this invention utilizes low-cost magnetometers and accelerometers to provide absolute angle references, and combines them with a Kalman filter algorithm to achieve stable calculations of reference yaw angle, reference pitch angle, and reference roll angle on a resource-constrained SOC control chip. All calculations are performed locally, without the need for a network or cloud, ensuring real-time performance, privacy, and offline availability, and the hardware cost is significantly lower than solutions using high-performance processors or AI accelerators.

[0129] Based on any of the above embodiments, the gesture recognition method further includes steps S150 and S160.

[0130] It should be noted that step S150 can be executed before step S140, and step S160 can be executed after step S140.

[0131] Step S150: Establish a communication connection with the host computer.

[0132] When the gesture recognition controller is powered on and initialized, the communication module in its SOC control chip (such as ESP32) will start and enter the connection-ready state.

[0133] The communication module first enters broadcast or discoverable mode. For example, the Bluetooth module broadcasts a signal containing the device name and services. A host computer (such as a laptop, smartphone, robot, or smart home device running control software) enables Bluetooth scanning, discovers the device from the list, and initiates a pairing connection request. After establishing a physical link layer connection, both parties can perform an application layer protocol handshake.

[0134] Furthermore, to ensure security, the host computer can send a pre-agreed verification message (such as a token). Only after the gesture recognition controller verifies the message can it confirm a successful connection and enter the data communication ready state.

[0135] Furthermore, at this point, the LED indicator on the gesture recognition controller can change from flashing to solid light, indicating a successful connection.

[0136] Step S160: Based on the communication connection, send the gesture recognition result to the host computer.

[0137] Once gesture recognition is complete and a valid gesture is obtained, the data transmission process is triggered. Based on the aforementioned communication connection, the gesture recognition result is sent to the host computer.

[0138] Of course, it is understandable that if no gesture is recognized or no valid gesture is recognized, the gesture recognition controller can send "0" or other information representing invalid gestures to the host computer at equal time intervals.

[0139] The above identification process repeats continuously until the conditions for sleep and power-off are met, at which point information transmission stops. The conditions for sleep and power-off can be set according to actual needs. For example, when any angle of the gyroscope is detected to be less than a preset angle (e.g., 1°) and the duration exceeds a preset time (e.g., 1 minute), sleep mode can be activated.

[0140] Furthermore, the gesture recognition controller can receive gesture parameter setting instructions sent by the host computer based on the above communication connection, so as to adjust the parameters in Tables 1 and 2 above.

[0141] Furthermore, the gesture recognition controller can also receive gesture setting instructions sent by the host computer based on the aforementioned communication connection, in order to... Figure 5 and Figure 9 The gestures can be reset.

[0142] The gesture recognition method provided in this invention establishes a wireless communication link and sends the gesture recognition results locally recognized by the gesture recognition controller to the host computer, thus realizing a complete closed loop from gesture perception to device control. Simultaneously, by embedding the gesture recognition algorithm locally in the gesture recognition controller, while placing the complex control logic on the host computer, the gesture recognition controller only needs to complete the core gesture recognition and low-power wireless transmission of the gesture recognition results. This ensures the system's real-time responsiveness and low latency, constructing a low-power, highly reliable, loosely coupled, and easily scalable human-computer interaction system architecture.

[0143] The gesture recognition device provided in the embodiments of the present invention is described below. The gesture recognition device described below and the gesture recognition method described above can be referred to each other.

[0144] Figure 11 This is a schematic diagram of the gesture recognition controller provided by the present invention, as shown below. Figure 11 As shown, the gesture recognition controller includes: a control module 110, an inertial measurement unit 120, and a control module 130.

[0145] The control module 110 is connected to the inertial measurement unit 120 and the control module 130 respectively; The control module 130 is used to send a gesture recognition trigger signal to the control module 110; The control module 110 is configured to, in response to a gesture recognition trigger signal, acquire current sensor data of the inertial measurement unit and determine a reference attitude angle based on the current sensor data; acquire real-time sensor data of the inertial measurement unit and determine a real-time attitude angle based on the real-time sensor data; calculate a relative angle change based on the real-time attitude angle and the reference attitude angle; and determine a gesture recognition result based on the relative angle change.

[0146] The control module 110 and the inertial measurement unit 120 can communicate using the I2C or SPI protocol.

[0147] The gesture recognition controller is equipped with a control module 130. Users can actively operate the control module 130 to interact with the gesture recognition process according to their needs. This allows users to flexibly enable the gesture recognition function, which helps to avoid accidental gesture recognition and miscontrol, and improves the convenience and reliability of use. At the same time, by activating gesture recognition on demand, unnecessary power consumption and processing resource usage can be reduced.

[0148] When a user wants to initiate gesture recognition, they can perform a corresponding operation through the control module 130 to trigger a gesture recognition signal. The control module 130, in response to the user's operation, sends a gesture recognition trigger signal to the control module 110.

[0149] Of course, it should be understood that users can also actively operate the control unit 120 to interact with the gesture recognition process according to their needs.

[0150] The control module 110 is used to lock the current sensor data input by the inertial measurement unit 120 as reference attitude data when it receives a gesture recognition trigger signal, so as to determine the reference attitude angle; it is also used to receive the real-time sensor data input by the inertial measurement unit 120 as real-time attitude data to determine the real-time attitude angle, and calculate the relative angle change by comparing the reference attitude angle and the real-time attitude angle, and then determine and output the gesture recognition result based on the relative angle change.

[0151] It should be noted that, in one embodiment, the inertial measurement unit 120 continuously transmits sensor data to the control module 110. When the control module 110 receives a gesture recognition trigger signal, it locks the currently input sensor data of the inertial measurement unit 120 (denoted as current sensor data) as reference attitude data, and uses the subsequently input sensor data (denoted as real-time sensor data) as real-time attitude data. In another embodiment, when the control module 110 receives a gesture recognition trigger signal, it sends a data acquisition command to the inertial measurement unit 120. At this time, the inertial measurement unit 120 begins to acquire data and continuously transmits sensor data to the control module 110. Correspondingly, the control module 110 locks the currently input sensor data of the inertial measurement unit 120 as reference attitude data, and uses the subsequently input real-time sensor data as real-time attitude data.

[0152] This invention provides a gesture recognition controller, comprising: a control module, an inertial measurement unit (IMU), and a manipulation module. The control module is connected to both the IMU and the manipulation module. The manipulation module sends a gesture recognition trigger signal to the control module. Upon receiving the gesture recognition trigger signal, the control module locks the current sensor data input by the IMU as reference attitude data. It also receives real-time sensor data input by the IMU as real-time attitude data and outputs a gesture recognition result based on the reference attitude data and the real-time attitude data. This invention constructs a physically triggered gesture recognition hardware architecture through electrical connections established between the control module and the IMU and the manipulation module. This architecture utilizes the manipulation module as the hardware trigger for the reference attitude, combined with the signal acquisition path of the control module, to lock the reference attitude data, ensuring the accuracy of the reference attitude data from the data source. Furthermore, by combining the real-time attitude data (i.e., real-time sensor data) with the output gesture recognition result, the accuracy and reliability of the gesture recognition result can be improved. Meanwhile, this invention does not rely on external cameras, complex image processing units, or high-performance computing units. It only requires a simple control module to send trigger signals, a low-cost inertial measurement unit unaffected by ambient light to collect data, and a control module to perform efficient coordination and data comparison processing. This reduces the complexity and cost of the gesture recognition controller at the hardware level while ensuring real-time data processing. In summary, this invention achieves real-time and accurate gesture recognition through the aforementioned low-cost hardware architecture.

[0153] According to a gesture recognition controller provided by the present invention, the control module 110 is further specifically used for: Calculate the attitude angle difference between the real-time attitude angle and the reference attitude angle; If the attitude angle difference exceeds the preset angle range, the attitude angle difference is corrected to obtain the relative angle change, which is within the preset angle range.

[0154] According to a gesture recognition controller provided by the present invention, the control module 110 is further specifically used for: The rotation matrix is ​​determined based on the reference pitch angle in the reference attitude angle; The relative angle change is mapped using the rotation matrix to obtain the mapped angle change. The gesture recognition result is determined based on the angle change after mapping.

[0155] According to a gesture recognition controller provided by the present invention, the control module 110 is further specifically used for: Obtain current gyroscope data, current magnetometer data, and current accelerometer data; Based on the current gyroscope data and the state calculation matrix of the previous moment, a state prediction matrix for the current moment is predicted; wherein, the state prediction matrix for the current moment includes at least the estimated value of the reference attitude angle and the estimated value of the gyroscope zero bias. Based on the current magnetometer data and the current accelerometer data, the state prediction matrix at the current moment is updated, and the reference attitude angle is extracted.

[0156] According to a gesture recognition controller provided by the present invention, the current state prediction matrix includes a first state prediction matrix, a second state prediction matrix, and a third state prediction matrix for the current time, and the current accelerometer data includes the current... x Axis acceleration, current y Axis acceleration and current z The axis acceleration, the control module 110, is also specifically used for: Based on the current magnetometer data, the first state prediction matrix at the current moment is updated to obtain the first state calculation matrix at the current moment; The current yaw angle is extracted from the first state calculation matrix at the current moment; Based on the current situation x Axial acceleration, the current y Axial acceleration and the current z The axis acceleration is used to update the second state prediction matrix at the current moment, thus obtaining the second state calculation matrix at the current moment; The current pitch angle is extracted from the second state calculation matrix at the current moment; Based on the current situation x Axial acceleration, the current y Axial acceleration and the current z The axis acceleration is used to update the third state prediction matrix at the current time, thus obtaining the third state calculation matrix at the current time. The current roll angle is extracted from the third state calculation matrix at the current moment; The reference attitude angles include the current yaw angle, the current pitch angle, and the current roll angle.

[0157] According to a gesture recognition controller provided by the present invention, the gesture recognition controller further includes: A communication module 150 is connected to the control module 110 and is used to establish a communication connection with the host computer. The control module 110 is also used to send the gesture recognition result to the host computer based on the communication connection.

[0158] The gesture recognition controller provided in this embodiment of the invention can communicate with a host computer by setting a communication module 150 on the gesture recognition controller, thereby sending the gesture recognition results to the host computer. Of course, it can also receive gesture parameter setting instructions sent by the host computer to adjust the relevant parameters.

[0159] Furthermore, the communication module 150 can use devices such as Bluetooth chips and WIFI chips.

[0160] It should be noted that the gesture recognition controller provided in this embodiment of the invention can implement all the method steps implemented in the above gesture recognition method embodiment and can achieve the same technical effect. Here, the parts that are the same as those in the method embodiment and the beneficial effects will not be described in detail.

[0161] Based on any of the above embodiments, such as Figure 11 As shown, the inertial measurement unit 120 includes a gyroscope 121, which is connected to the control module 110.

[0162] A gyroscope is a sensor that measures angular velocity and can detect the attitude of an object. It is used to collect gyroscope data, specifically gyroscope angle values. Gyroscope 121 can acquire gyroscope data at the moment corresponding to the gesture recognition trigger signal (denoted as current gyroscope data) and subsequent gyroscope data (denoted as real-time gyroscope data), and transmit them to control module 110. This allows control module 110 to obtain reference attitude angle and real-time attitude angle based on the current gyroscope data and real-time gyroscope data, respectively, providing detection information data to support subsequent calculations of relative angle changes and determination of gesture recognition results.

[0163] Understandably, in scenarios where high recognition accuracy is not required, detection can be performed using only a gyroscope.

[0164] Based on any of the above embodiments, such as Figure 1 As shown, the inertial measurement unit 120 also includes a magnetometer 122, which is connected to the control module 110.

[0165] A magnetometer can measure the strength and direction of the Earth's magnetic field, calculate the deflection angle of an object, and collect magnetometer data, specifically magnetometer angle values.

[0166] Based on any of the above embodiments, such as Figure 11 As shown, the inertial measurement unit 120 also includes an accelerometer 123, which is connected to the control module 110.

[0167] Accelerometers can measure linear acceleration and the gravitational vector, and can calculate the tilt angle of an object in space. They are used to collect accelerometer data, which can specifically include… x Axial acceleration, y Axial acceleration and z Axial acceleration.

[0168] In one embodiment, the inertial measurement unit 120 may be an inertial measurement module chip, which integrates a gyroscope, a magnetometer, and an accelerometer.

[0169] In another embodiment, the inertial measurement unit 120 may employ a gyroscope and a magnetometer, or a gyroscope and an accelerometer, or only a gyroscope.

[0170] It should be noted that the changes in hand posture are determined by reference posture angles and real-time posture angles, which can be described based on pitch angle, yaw angle, and roll angle to quantify the changes in posture.

[0171] The gesture recognition controller provided in this invention, based on an inertial measurement unit (IMU) including a gyroscope, incorporates a magnetometer and an accelerometer. This allows the IMU to more comprehensively and accurately detect the object's posture. The accelerometer provides a gravity reference, which can correct for gyroscope drift in pitch and roll angles. The magnetometer provides a reference to the Earth's magnetic field, which can correct for gyroscope drift in deflection angles. Therefore, the drift problem of the gyroscope can be improved, hand posture can be detected more accurately, and the accuracy of gesture recognition can be enhanced.

[0172] Based on any of the above embodiments, such as Figure 11 As shown, the control module 130 includes an identification switch button, which is connected to the control module 110.

[0173] The gesture recognition controller provided in this embodiment of the invention features a recognition switch button, allowing users to easily activate or deactivate the gesture recognition function. This simple structure enables convenient user operation.

[0174] Furthermore, the control module 130 can employ devices such as buttons, contactless sensing buttons, and touch screens.

[0175] Based on any of the above embodiments, such as Figure 1 As shown, the gesture recognition controller also includes a control feedback module 140, which is connected to the control module 110.

[0176] The gesture recognition controller provided in this embodiment of the invention, by setting a control feedback module 140 on the gesture recognition controller, can provide feedback prompts when the user performs operations, so that the user can know whether the operation is successful, whether the communication pairing is successful, and whether the gesture recognition function is enabled, etc. In this way, it helps the user to confirm the validity of the operation and know the current operating status, thereby improving the user experience.

[0177] Furthermore, the control feedback module 140 can employ devices such as LEDs (Light-Emitting Diodes), vibration motors, speakers, and touchscreens. It is understood that when using devices such as touchscreens, the control module and the control feedback module can be the same device.

[0178] Furthermore, the touchscreen can be a TFT (Thin-Film Transistor) or OLED (Organic Light-Emitting Diode) display.

[0179] Based on any of the above embodiments, such as Figure 11 As shown, the gesture recognition controller also includes an energy storage power supply module 160, which is connected to the control module 110, the inertial measurement unit 120 and the control module 130 respectively.

[0180] The gesture recognition controller provided in this embodiment of the invention, by incorporating an energy storage power supply module 160, enables the entire gesture recognition controller to be used portablely, eliminating dependence on external power sources and improving user convenience during exercise. Thus, the built-in energy storage power supply unit eliminates the need for external battery power and avoids cable tangling issues, thereby enhancing the user experience.

[0181] Based on any of the above embodiments, such as Figure 11 As shown, the gesture recognition controller also includes an interface module 170, which is connected to the control module 110 and the energy storage power supply module 160 respectively.

[0182] The gesture recognition controller provided in this embodiment of the invention includes an interface module 170, which provides a structural foundation for charging, data transmission, and maintenance. Users can charge the built-in energy storage power supply module 160 through the interface module 170 to ensure continuous device operation. Simultaneously, the interface module 170 can also connect to external devices such as computers to enable functions such as firmware upgrades, parameter configuration, or system debugging.

[0183] Furthermore, the interface module 170 can adopt device structures such as USB interface and contact interface.

[0184] Furthermore, the interface module 170 includes a magnetic interface. By adopting a magnetic interface, the interface can be automatically aligned and connected by magnetic force, which simplifies the interface plugging operation. In addition, the magnetic interface is usually a closed design, which has excellent waterproof and dustproof performance, which helps to improve durability and reliability in complex environments such as humid and dusty environments, and is suitable for use in outdoor or sports scenarios.

[0185] Based on any of the above embodiments, such as Figure 11 As shown, the energy storage power supply module 160 includes a battery, a battery protection circuit, and a battery charging circuit. The battery is connected to the control module 110, the inertial measurement unit 120, and the control module 130, respectively. The battery protection circuit is connected to the battery, and the battery charging circuit is connected to the interface module 170 and the battery, respectively.

[0186] The gesture recognition controller provided in this embodiment of the invention integrates a battery, a battery protection circuit, and a battery charging circuit in its energy storage power supply module 160, forming a complete and reliable power management structure. The battery protection circuit prevents excessive battery output current, avoiding overheating or overcurrent, thus extending battery life and improving safety. The battery charging circuit ensures an efficient and safe charging process, preventing overcharging and battery damage.

[0187] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the gesture recognition method provided by the above methods. The method includes: in response to a gesture recognition trigger signal, acquiring current sensor data of an inertial measurement unit and determining a reference attitude angle based on the current sensor data; acquiring real-time sensor data of the inertial measurement unit and determining a real-time attitude angle based on the real-time sensor data; calculating a relative angle change based on the real-time attitude angle and the reference attitude angle; and determining a gesture recognition result based on the relative angle change.

[0188] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0189] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0190] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gesture recognition method, characterized in that, include: In response to a gesture recognition trigger signal, the current sensor data of the inertial measurement unit is acquired, and a reference attitude angle is determined based on the current sensor data; Acquire real-time sensor data from the inertial measurement unit and determine the real-time attitude angle based on the real-time sensor data; Calculate the relative angle change based on the real-time attitude angle and the reference attitude angle; The gesture recognition result is determined based on the relative angle change.

2. The gesture recognition method according to claim 1, characterized in that, The calculation of the relative angle change based on the real-time attitude angle and the reference attitude angle includes: Calculate the attitude angle difference between the real-time attitude angle and the reference attitude angle; If the attitude angle difference exceeds the preset angle range, the attitude angle difference is corrected to obtain the relative angle change, which is within the preset angle range.

3. The gesture recognition method according to claim 1, characterized in that, The determination of the gesture recognition result based on the relative angle change includes: The rotation matrix is ​​determined based on the reference pitch angle in the reference attitude angle; The relative angle change is mapped using the rotation matrix to obtain the mapped angle change. The gesture recognition result is determined based on the angle change after mapping.

4. The gesture recognition method according to claim 1, characterized in that, The step of acquiring the current sensor data of the inertial measurement unit and determining the reference attitude angle based on the current sensor data includes: Obtain current gyroscope data, current magnetometer data, and current accelerometer data; Based on the current gyroscope data and the state calculation matrix of the previous moment, a state prediction matrix for the current moment is predicted; wherein, the state prediction matrix for the current moment includes at least the estimated value of the reference attitude angle and the estimated value of the gyroscope zero bias. Based on the current magnetometer data and the current accelerometer data, the state prediction matrix at the current moment is updated, and the reference attitude angle is extracted.

5. The gesture recognition method according to claim 4, characterized in that, The current state prediction matrix includes a first state prediction matrix, a second state prediction matrix, and a third state prediction matrix. The current accelerometer data includes the current... x Axis acceleration, current y Axis acceleration and current z Axial acceleration, the process of updating the state prediction matrix at the current moment based on the current magnetometer data and the current accelerometer data, and extracting the reference attitude angle, includes: Based on the current magnetometer data, the first state prediction matrix at the current moment is updated to obtain the first state calculation matrix at the current moment; The current yaw angle is extracted from the first state calculation matrix at the current moment; Based on the current situation x Axial acceleration, the current y Axis acceleration and the current z The axis acceleration is used to update the second state prediction matrix at the current moment, thus obtaining the second state calculation matrix at the current moment; The current pitch angle is extracted from the second state calculation matrix at the current moment; Based on the current situation x Axial acceleration, the current y Axis acceleration and the current z The axis acceleration is used to update the third state prediction matrix at the current time, thus obtaining the third state calculation matrix at the current time. The current roll angle is extracted from the third state calculation matrix at the current moment; The reference attitude angles include the current yaw angle, the current pitch angle, and the current roll angle.

6. The gesture recognition method according to any one of claims 1 to 5, characterized in that, The gesture recognition method further includes: Establish a communication connection with the host computer; Based on the communication connection, the gesture recognition result is sent to the host computer.

7. A gesture recognition controller, characterized in that, include: Control module, inertial measurement unit, and manipulation module; The control module is connected to both the inertial measurement unit and the manipulation module. The control module is used to send a gesture recognition trigger signal to the control module; The control module is used to implement the gesture recognition method as described in any one of claims 1 to 6.

8. The gesture recognition controller according to claim 7, characterized in that, The inertial measurement unit includes a gyroscope, which is connected to the control module; and / or, The inertial measurement unit further includes a magnetometer, which is connected to the control module; and / or, The inertial measurement unit also includes an accelerometer, which is connected to the control module.

9. The gesture recognition controller according to claim 7, characterized in that, The control module includes an identification switch button, which is connected to the control module; and / or, The gesture recognition controller further includes a control feedback module, which is connected to the control module; and / or, The gesture recognition controller further includes a communication module connected to the control module, and the communication module is also used for communication with a host computer; and / or, The gesture recognition controller also includes an energy storage power supply module, which is connected to the control module, the inertial measurement unit, and the operation module.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the gesture recognition method as described in any one of claims 1 to 6.

Images