A gesture-controlled robot arm system

Abstract

A gesture-controlled robot arm system, consisting of: a portable controller that includes the following: a glove structure configured to be worn on a user's hand; at least one inertial measurement sensor attached to the glove and configured to capture data on hand orientation and movement along multiple axes; a microcontroller unit that is functionally connected to the sensor and configured to process gesture data and transmit control signals; a robot arm assembly comprising the following: a multitude of servomotors functionally connected to one or more mechanical linkages configured to replicate arm movements, including base rotation, shoulder tilt, elbow extension, and wrist movement; a second microcontroller configured to receive control signals derived from gestures and convert these signals into servo drive commands using pulse width modulation; a wireless communication protocol implemented between the portable controller and the robot arm, the protocol enabling real-time peer-to-peer data transmission without external network infrastructure; the system is configured so that the movement of the user's hand, detected by the wearable sensor, is translated in real time into a corresponding movement of the robot arm.

Description

[0001] The present invention relates to the field of human-machine interaction and robot control systems. In particular, it relates to a gesture-controlled robot arm system that uses wearable sensors and wireless real-time communication to translate human hand gestures into precise robot movements.

[0002] Robotic arms are widely used in various fields, including industrial automation, remote control, medical prosthetics, and education. Conventional robotic arm systems are typically operated via predefined programming, wired remote controls, or joystick-based controllers. While functional, these methods lack intuitive human-machine interaction and often require extensive technical training. Furthermore, many of these systems are limited by wired connections or high-latency wireless communication protocols, impacting their mobility, responsiveness, and real-time performance. Recent developments in gesture-based control systems aim to improve usability through hands-free and more natural control interfaces.Many of these systems, however, rely heavily on visually based gesture recognition using cameras and complex image processing algorithms. These approaches are expensive, computationally intensive, and perform poorly in low-light or cluttered environments. Furthermore, image-based systems often introduce significant latency and are not ideal for portable or field-deployable applications. Bluetooth- or Wi-Fi-based gesture systems have also been developed, but these have drawbacks such as limited range, high power consumption, device pairing issues, and network dependency. Moreover, existing commercial solutions either offer limited degrees of freedom (DoF) or are too expensive for widespread use in education, assistance, or hobbies.There remains an urgent need for a portable, cost-effective, low-latency and user-friendly gesture control system that does not rely on external infrastructure.

[0003] To solve this problem, the present invention offers a gesture-controlled robot arm system.

[0004] The system enables intuitive and real-time control of robot movements using human hand gestures.

[0005] The system also features a modular and scalable architecture that supports multiple degrees of freedom (DoF) in robot arms, allowing for the addition of more servos and gesture inputs as needed.

[0006] The system is cost-effective and portable, making it suitable for a wide range of applications, including assistive technologies, prosthetic control, educational robotics, industrial automation and remote control.

[0007] The system can optionally integrate a machine learning-based gesture classification for predefined gesture commands, thereby increasing its accuracy, adaptability, and scalability for future improvements.

[0008] In one embodiment, the present invention provides a novel, flexible, and self-powered glove system designed for real-time recognition of hand gestures and wireless motion control of remotely controlled devices, such as motorized carts or robotic platforms. The system integrates thin-film piezoelectric nanocomposite sensors, consisting of a mixture of polyvinylidene fluoride (PVDF) and piezoelectric nanoparticles, positioned at key joint points of a wearable glove. When a user performs a hand or wrist gesture, the embedded sensors are mechanically deformed, generating characteristic electrical signals through the piezoelectric effect. These signals are captured and processed by a low-power microcontroller unit mounted on the glove. A wireless communication module, such as a smartphone, is also integrated.The ESP32 or ESP8266 is used to transmit the recognized gesture commands to a remote receiver module attached to a mobile device, such as a cart. The invention supports several intuitive gestures, each associated with specific directional commands—such as forward, stop, left, and right—and enables real-time, contactless control of the cart with a latency of less than 200 milliseconds. The system features self-powered sensors, eliminating the need for external batteries for gesture input. The sensor's high flexibility and sensitivity make it ideal for curved or dynamic surfaces and enhance usability in real-world portable applications. Furthermore, the sensor's manufacturing process is scalable and cost-effective, involving the casting of nanocomposite films, ultrasonic treatment, tempering, and encapsulation in silicone rubber.The system's modular design enables bidirectional communication, remote feedback and seamless integration into IoT platforms, making it suitable for industrial automation, assistive technologies and intelligent robotics.

[0009] The invention will be explained again below.

[0010] The present invention discloses a wearable, glove-based system for real-time hand gesture recognition and wireless motion control using flexible piezoelectric sensors. The glove integrates thin-film nanocomposite sensors made of polyvinylidene fluoride (PVDF) combined with piezoelectric materials, enabling high sensitivity to mechanical deformations caused by wrist or hand movements. These sensors are embedded at specific joint locations, such as the wrist or fingers, and encased in flexible silicone rubber to ensure durability and comfort during extended use. When a user performs a gesture—for example, bending the wrist upwards or downwards—the sensors generate electrical signals due to the piezoelectric effect.These analog signals, typically in the range of 2.1 to 2.2 volts, are captured by a low-power microcontroller unit (e.g., ESP32 or ESP8266) positioned near the wrist of the glove. The microcontroller digitizes and processes the signals, filters out noise, and identifies the gesture pattern by comparing it to stored templates. Once a valid gesture is recognized, the corresponding command signal is wirelessly transmitted to a remote-controlled receiver module integrated into a motorized cart. Wireless communication is enabled by protocols such as Wi-Fi or Bluetooth Low Energy (BLE), which ensure fast and reliable data transmission. The receiver decodes the signal and activates the motor to perform a specific movement—such as moving forward, stopping, or turning—based on the user's gesture.For example, an upward wrist movement generates a signal interpreted as a "MOUNT FORWARD" command, while a downward wrist movement is interpreted as a "STOP" command. These interactions enable real-time, contactless control of the cart with minimal latency (typically less than 200 milliseconds), making the system (100) ideal for fast-paced environments such as warehouses, healthcare facilities, or robotics labs. The sensor manufacturing process involves dissolving PVDF and piezoelectric particles in DMF, followed by ultrasonic treatment and annealing at 150 °C to promote β-phase crystallinity. The resulting nanocomposite film is cast, cured, cut into sensor strips, and encapsulated in silicone.These strips are then attached to the glove with adhesive or stitching to ensure flexibility and a close fit with the user's skin and joints.

[0011] This glove-based system offers several advantages over conventional gesture recognition platforms. It is self-contained and uses only piezoelectric power for its sensors, eliminating the need for external batteries in the sensor layer. Furthermore, it is lightweight, scalable, and ergonomic, making it ideal for assistive applications, industrial automation, and human-machine interaction scenarios. The system's modular design also allows for future upgrades such as bidirectional communication, haptic feedback, and integration with IoT platforms for data logging and remote monitoring. In real-world applications, the glove enables users to intuitively control machines or devices without physical contact. For example, in a warehouse environment, an operator can navigate a cart through narrow aisles using only wrist gestures, increasing efficiency and safety.The combination of flexible sensor technology, gesture-based command recognition and wireless control represents a significant advancement in wearable human-machine interface systems.

Claims

[1] A gesture-controlled robot arm system consisting of: a portable controller that includes the following: a glove structure configured to be worn on a user's hand; at least one inertial measurement sensor attached to the glove and configured to capture data on hand orientation and movement along multiple axes; a microcontroller unit that is functionally connected to the sensor and configured to process gesture data and transmit control signals; a robot arm assembly comprising the following: a multitude of servomotors functionally connected to one or more mechanical linkages configured to replicate arm movements, including base rotation, shoulder tilt, elbow extension, and wrist movement; a second microcontroller configured to receive control signals derived from gestures and convert these signals into servo drive commands using pulse width modulation; a wireless communication protocol implemented between the portable controller and the robot arm, the protocol enabling real-time peer-to-peer data transmission without external network infrastructure; the system is configured so that the movement of the user's hand, detected by the wearable sensor, is translated in real time into a corresponding movement of the robot arm. [2] System according to claim 1, wherein the piezoelectric nanocomposite sensors are made from a mixture of polyvinylidene fluoride (PVDF) and piezoelectric nanoparticles, which are processed by ultrasonic treatment and annealing to improve the formation of the β-phase. [3] System according to claim 1, wherein the sensors are configured to generate electrical signals in the range of 2.1 to 2.2 volts during bending of the wrist or movement of the fingers. [4] System according to claim 1, wherein the microcontroller unit is selected from ESP32 or ESP8266 and is configured to digitize and classify voltage signals using predefined gesture templates. [5] System according to claim 1, wherein the wireless communication module operates via Wi-Fi or Bluetooth Low Energy (BLE) to transmit gesture signals to the receiver module. [6] System according to claim 1, wherein the movable device comprises a motorized carriage and the receiver module is configured to activate the motor in response to received control signals. [7] System according to claim 1, wherein the latency between gesture recognition and the execution of the corresponding movement command is less than 200 milliseconds. [8] System according to claim 1, wherein the glove further comprises a sensor encapsulation layer made of flexible silicone rubber to protect the sensors and ensure durability.