Agricultural Machine Voice Control with Gaze and Gesture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Agricultural working machines with voice control systems face delays and incorrect command recognition due to the time lag in interpreting and evaluating voice commands, leading to reduced usability and precision in operating multiple control elements.
Innovation Solution
Incorporating a sensor device that recognizes gestures and detects the operator's viewing direction to assign functions to operating elements, allowing for more precise and intuitive control through voice commands, eliminating the need for direct physical actuation and enhancing the flexibility of voice command allocation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If voice control is used to operate control elements, then the operator can control devices without looking at displays or physically operating controls, but there is a time lag between voice command input and execution
Solution Approach 1:
The system performs preliminary actions by detecting the operator's gaze direction and gesture in advance to pre-select the control element before the voice command is fully processed. This preliminary selection reduces the overall processing time by having the target control element ready before the voice command execution begins.
Solution Approach 2:
The patent replaces traditional mechanical control operations (physical button pressing) and direct voice-to-action systems with an integrated system combining optical sensors (gaze detection), gesture recognition, and voice command processing. This substitution creates a more efficient control pathway that reduces time lag.
2Adaptability or versatility
If multiple control elements are available for assignment, then the system offers flexibility in function allocation, but it becomes more difficult to precisely select the intended control element
Solution Approach 1:
The system introduces intermediate selection mechanisms (gaze detection and gesture recognition) that mediate between the operator's intent and the final control element selection. These intermediaries provide a clear, unambiguous indication of which control element the operator intends to assign a function to, even when multiple elements are available.
Solution Approach 2:
The patent adds spatial dimensions to the selection process by using gaze direction (angular position) and gesture location (spatial coordinates) to identify the intended control element. This dimensional approach allows precise selection among multiple control elements by determining which element is in the operator's line of sight or being gestured toward.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
This solution reduces the time lag and improves the accuracy of command execution, enabling operators to select control elements more intuitively and precisely, even when multiple elements are involved, by using sensor devices for gesture recognition and line-of-sight detection, thereby enhancing the overall usability of the machine.
Implementation Method 1
at least one sensor device is provided, which is configured to determine the control element to be assigned by gesture recognition and/or detection of the operator's gaze direction
Implementation Method 2
the control device is equipped by a voice control system for receiving and evaluating voice commands from an operator of the machine
Implementation Method 3
at least one sensor device is provided, which is configured to determine the control element to be assigned by gesture recognition and/or detection of the operator's gaze direction
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The present invention relates to an agricultural machine (1) with at least one device (17) to be controlled and/or regulated, comprising at least one control device (10) for controlling the at least one device (17) and at least one user interface (20) having at least one operating element (26) and connected to the control device (10) by means of a signal, by means of which control commands for controlling the at least one device (17) can be transmitted to the control device (10), wherein the at least one user interface (20) is configured to assign a function comprising at least one control command to the at least one operating element (16), and wherein the control device (10) is equipped by a voice control system (30) for receiving and evaluating voice commands from an operator (15) of the machine (1), wherein the at least one control device (10) is configured toto assign at least one control element (26) of the at least one user interface (20) to at least one function and/or at least one function set via voice control, wherein at least one sensor device (29) is provided for determining the control element (26) to be assigned, which is configured to determine the control element (26) to be assigned by means of gesture recognition and/or detection of the direction of gaze of the operator (15).