Flying Digital Assistant Gesture Control for Autonomous UAV Imaging
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current UAV systems for image and video capture require piloting skills similar to fixed-wing or rotorcraft, leading to crashes due to pilot error, and lack intuitive and user-friendly control paradigms for indirect control of autonomous UAVs.
Innovation Solution
The implementation of a Flying Digital Assistant (FDA) system that uses a portable multifunction device (PMD) for user interaction, employing localization and navigation systems, including GPS, cellular, Wi-Fi, and computer vision, to enable autonomous control of UAVs through virtual camera, drawn paths, touch-to-focus, and preset patterns, allowing users to control image and video capture without direct piloting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If direct control of UAV is used (similar to fixed-wing or rotorcraft piloting), then control precision is improved, but ease of operation deteriorates and reliability worsens due to pilot error and crashes
Solution Approach 1:
The patent introduces an intermediary control system that translates simple user gestures (touch inputs on a display) into complex UAV control commands. This intermediary layer handles the complexity of pitch, roll, yaw, and thrust control automatically, allowing users to control the UAV through intuitive gestures rather than direct piloting inputs, thereby improving ease of operation while maintaining control precision through the intermediary's automated processing
Solution Approach 2:
The patent replaces the mechanical direct-control system (joysticks, throttles, and manual piloting interfaces) with a gesture-based control system using touch inputs on a display device. This substitution eliminates the need for users to physically manipulate complex control mechanisms, replacing them with simple touchscreen gestures that are automatically translated into UAV control commands, significantly improving ease of operation
2Adaptability or versatility
If direct control of UAV is used, then adaptability is improved for experienced pilots, but reliability deteriorates due to crashes caused by pilot error
Solution Approach 1:
The patent implements a self-service control system where the UAV and control device work together to automatically interpret and execute control commands. The system autonomously processes gesture inputs, calculates appropriate flight adjustments, and executes commands without requiring user expertise in piloting techniques. This self-service approach eliminates pilot error while maintaining adaptability through flexible gesture recognition and automated flight control algorithms
Solution Approach 2:
The patent incorporates feedback mechanisms where the control device receives real-time data from the UAV (position, orientation, velocity) and continuously adjusts control commands based on current flight state. This closed-loop feedback system automatically compensates for disturbances and maintains stable flight, improving reliability by preventing crashes that would result from lack of situational awareness or improper control inputs in manual piloting
3Ease of operation
If autonomous control is implemented, then ease of operation is improved, but device complexity increases due to localization and navigation systems
Solution Approach 1:
The patent employs a portable multifunction device (such as a smartphone or tablet) that serves multiple roles: it acts as the control interface for gesture inputs, runs the localization and navigation algorithms, processes image and video data, and communicates with the UAV. By consolidating these functions into a single universal device, the system achieves autonomous control capabilities without proportionally increasing overall complexity, as the multifunction device already possesses the necessary processing power and sensors
4Ease of operation
If intuitive control paradigms are implemented, then ease of operation is improved, but productivity may deteriorate due to additional processing requirements
Solution Approach 1:
The patent implements preliminary action by pre-processing and analyzing visual data from the environment to identify actionable objects and surfaces before user interaction. The system continuously builds a understanding of the scene, pre-calculates possible control interpretations, and prepares response options in advance. When a user gesture is detected, the system can quickly select from pre-prepared control commands rather than processing everything from scratch, thereby maintaining intuitive control while improving productivity through advance preparation
Data Source
AI summary
Methods and systems are described for new paradigms for user interaction with an unmanned aerial vehicle (referred to as a flying digital assistant or FDA) using a portable multifunction device (PMD) such as smart phone. In some embodiments, a user may control image capture from an FDA by adjusting the position and orientation of a PMD. In other embodiments, a user may input a touch gesture via a touch display of a PMD that corresponds with a flight path to be autonomously flown by the FDA.