3D Camera Object Selection for Manipulator Commissioning
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The commissioning and programming of manipulator systems, such as industrial robots, are often cumbersome due to complex menu structures, requiring users to navigate laboriously to select and control specific tasks or objects, which can be confusing and time-consuming.
Innovation Solution
A method using a portable terminal device with a 3D camera to capture images of the manipulator system, determine features, and identify controllable objects, allowing for intuitive selection and control without navigating menu trees, by providing reference data sets and using image recognition to automatically identify objects, and displaying optical markings for direct feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If a tree or menu structure is used to structure tasks and controllable objects, then the interface can be organized systematically, but the user has to navigate laboriously through the structure to find the desired element
Solution Approach 1:
The patent replaces the mechanical navigation through menu trees with an optical recognition system. The camera captures images of the manipulator system, and image recognition algorithms automatically identify controllable objects and tasks, substituting the manual navigation process with automated visual detection and pattern recognition.
Solution Approach 2:
The system performs self-identification of controllable objects and tasks through automated image recognition. Instead of requiring the user to manually search and select items from menus, the system automatically detects and presents relevant controllable elements based on the captured image, allowing the interface to serve itself in identifying target objects.
2Adaptability or versatility
If a large number of devices, objects and tasks are mapped in the menu structure, then comprehensive control is achieved, but the search for the right menu item becomes tedious
Solution Approach 1:
The patent extracts the identification function from the menu navigation process. Instead of requiring users to search through comprehensive menu structures, the system extracts and automatically identifies the specific controllable object or task from the image, presenting only the relevant options to the user rather than the entire menu hierarchy.
Solution Approach 2:
The camera and image recognition system serve as an intermediary between the user and the comprehensive menu structure. This intermediary automatically processes the visual information, identifies relevant controllable elements, and translates them into selectable options, mediating between the user's intent and the system's comprehensive control capabilities.
3Productivity
If traditional operator panels are used for commissioning and programming, then control functionality is provided, but the process is confusing and time-consuming
Solution Approach 1:
The patent replaces the mechanical interaction with traditional operator panels (buttons, switches, manual navigation) with an optical interface system. The camera-based image recognition technology substitutes manual panel operation with automated visual identification and selection, fundamentally changing the interaction paradigm from mechanical to optical-cognitive.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to a method for controlling a manipulator system using a portable terminal device, a corresponding portable terminal device and a robot system. The portable terminal preferably includes a 3D camera, which is also preferably a plenoptic camera. The method according to the invention includes, among other things, capturing a three-dimensional image using the 3D camera of the portable terminal device of at least part of the manipulator system to be controlled, and identifying at least one controllable object of the manipulator system.