Reinforcement Learning Gesture Control for UAV Hand Tracking
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for remotely controlling unmanned aerial vehicles (UAVs) are difficult for beginners to operate, especially in low light conditions, and require complex calculations and additional sensors for gesture recognition and trajectory control.
Innovation Solution
A device and method using reinforcement learning with neural networks to determine hand pose and movement, allowing intuitive control of UAV direction, speed, and trajectory through a wearable device with sensors, enabling direct and figural trajectory control modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If camera-based gesture recognition is used to control UAV, then ease of operation is improved, but measurement precision deteriorates when light intensity is insufficient
Solution Approach 1:
The patent replaces camera-based optical gesture recognition with an inertial sensor-based mechanical sensing system. The inertial sensor detects hand movements through acceleration and orientation data, eliminating dependence on light conditions while maintaining ease of operation through intuitive gesture-based control.
2Ease of operation
If camera-based gesture recognition is used to control UAV, then ease of operation is improved, but device complexity increases due to additional sensors
Solution Approach 1:
The patent extracts and removes the camera component from the control system, retaining only the essential inertial sensing functionality. This simplifies the device by eliminating the camera hardware and associated complex image processing algorithms while preserving the core gesture-based control capability.
3Adaptability or versatility
If figural trajectory control is implemented with camera-based recognition, then adaptability is improved, but device complexity increases due to depth perception camera or additional sensor requirements
Solution Approach 1:
The patent makes the inertial sensor serve multiple functions: it detects both simple directional gestures for basic control and complex figural trajectories for advanced control modes. This multi-functionality eliminates the need for separate depth perception cameras or additional sensors, as the inertial sensor captures all necessary movement data for both control types.
4Device complexity
If traditional radio controller is used to control UAV, then device complexity is reduced, but ease of operation deteriorates for beginners
Solution Approach 1:
The patent replaces the traditional mechanical radio controller with a gesture-based control interface detected by inertial sensors. This substitution maintains relatively simple device architecture while dramatically improving ease of operation, as users can naturally control the UAV with hand gestures rather than learning complex controller manipulations.
Data Source
AI summary
A device and method for remotely controlling an unmanned aerial vehicle based on reinforcement learning are disclosed. An embodiment provides a device for remotely controlling an unmanned aerial vehicle based on reinforcement learning, where the device includes a processor and a memory connected to the processor, and the memory includes program instructions that can be executed by the processor to determine an inclination direction corresponding to the hand pose of a user, the movement direction of the hand, and the angle in the inclination direction based on sensing data associated with the pose of the hand or the movement of the hand acquired by way of at least one sensor, and determine one of a movement direction, a movement speed, a mode change, a figural trajectory, and a scale of the figural trajectory of the unmanned aerial vehicle according to the determined inclination direction, movement direction, and angle.