Human-Robot Interaction Activity Classification via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods lack the capability to comprehensively analyze the interactions and work activities between human workers and robotic systems, failing to effectively record and evaluate their movements for improved workflow processes.
Innovation Solution
A computer-assisted method and device that utilize sensor systems to record and process movement data from both workers and robotic systems, employing pattern recognition and machine learning to classify activities and extract work processes through data-driven models, enabling comprehensive analysis and feedback.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If sensor systems are used to record movements of worker and robotic system, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The sensor systems are designed to perform multiple functions: they record positions of body parts, track movement of robotic components, and provide data for both safety monitoring and work activity classification. This multi-functionality reduces the need for separate specialized sensors for each measurement task.
Solution Approach 2:
A computer system acts as an intermediary that receives data from multiple sensor systems, processes the information, and generates classifications of work activities. This central processing unit simplifies the overall system architecture by consolidating the complexity of data fusion and analysis in a single dedicated component.
2Loss of information
If pattern recognition and machine learning are applied to classify work activities, then information completeness is improved, but computational requirements increase
Solution Approach 1:
The system pre-processes sensor data by extracting relevant features and patterns before applying machine learning classification. Motion patterns are identified and segmented from raw position data, reducing the dimensionality and complexity of the data that requires intensive computational processing.
Solution Approach 2:
The work activities are classified by segmenting the continuous sensor data into discrete motion patterns and activity types. This segmentation approach breaks down complex continuous data into manageable discrete units that can be processed more efficiently by classification algorithms.
3Productivity
If comprehensive analysis of worker and robot movements is performed, then productivity optimization is improved, but analysis time increases
Solution Approach 1:
The system provides real-time feedback by continuously monitoring sensor data, classifying current work activities, and comparing them against optimal workflow patterns. This immediate feedback loop enables rapid identification of optimization opportunities without requiring lengthy batch processing of historical data.
Solution Approach 2:
The system dynamically adjusts analysis parameters and classification thresholds based on the specific work context and activity type. By adapting the analysis depth and complexity to the current operational needs, the system maintains high productivity insights while minimizing unnecessary computational overhead.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a method for the computer-aided acquisition and evaluation of a work process in which a human worker (1) and a robotic system (3) interact. Within the framework of the method according to the invention, digital motion data (BD) of the worker (1) are determined based on acquisition by means of a first sensor system (2, 4, 5). Furthermore, digital motion data (BD') of the robotic system (3) are determined based on acquisition by means of a second sensor system (12) and/or based on kinematic control data (KD) of the robotic system (3). From the digital motion data (BD, BD') of the worker (1) and the robotic system (3), motion patterns (BM) contained in the digital motion data (BD, BD') are subsequently determined, wherein the extraction of the motion patterns (BM) is based on pattern recognition using previously known motion patterns (BM').The determined movement patterns (BM) are fed as input data to a data-driven model (MO) that is learned via machine learning based on training data (TD), wherein the data-driven model (MO) determines activities (AK) of the worker (1) and the robotic system (3) as output data, wherein each activity (AK) comprises one or more movement patterns (BM) and corresponds to a work activity of the worker (1) or the robotic system (3) from a plurality of predefined work activities.