Spatial Process Mapping for Readable AR Work Trajectory Mining
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing process mining techniques struggle to effectively utilize augmented reality (AR) systems and wearable sensors to map and analyze the fine-grained physical operations of workers in industrial environments, as raw sensor data is too granular and lacks direct mapping to recognizable activities, leading to unreadable visualizations and incomplete process insights.
Innovation Solution
A method and device for building a spatial process map using AR-obtained 3D trajectories, involving wearable devices to capture position and orientation data, segmenting these into macro-activities, labeling and clustering micro-activities, and integrating them with high-level process information to create a balanced view of work processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If raw sensor data from wearables is used directly for process mining, then detailed view of manual work processes is obtained, but data is too granular to be useful and leads to unreadable visualizations
Solution Approach 1:
The patent segments raw sensor data into hierarchical levels: individual sensor events are grouped into micro-activities, which are then grouped into macro-activities, and finally into process steps. This segmentation allows the system to maintain detailed measurement precision while presenting aggregated, readable visualizations at appropriate levels of abstraction.
Solution Approach 2:
The patent introduces a temporal dimension to organize granular sensor data by grouping events within time windows into micro-activities, then grouping micro-activities into macro-activities. This temporal aggregation transforms the data from a flat stream of granular events into a hierarchical temporal structure that is both detailed and visually manageable.
2Measurement precision
If fine-grained trajectory data is visualized using traditional Spaghetti diagrams, then every movement is recorded, but small or irrelevant differences lead to unreadable visualizations
Solution Approach 1:
The patent merges multiple granular trajectory points that belong to the same macro-activity into single aggregated representations. By combining trajectory data with activity labels and grouping them into macro-activities, the system preserves the underlying detailed movements while presenting a simplified visual representation that highlights process patterns rather than individual movements.
Solution Approach 2:
The patent segments the continuous trajectory data into discrete macro-activities based on activity recognition. Each macro-activity represents a meaningful work step that groups multiple trajectory points, allowing the visualization to show process structure and variations without being overwhelmed by the granularity of individual movement points.
3Extent of automation
If discrete steps are recognized from sensor data, then activity recognition is achieved, but steps are too abstract for domain experts to make use of
Solution Approach 1:
The patent creates a dynamic hierarchical structure where activity recognition operates at multiple levels. The system automatically recognizes discrete steps (micro-activities) from sensor data, then dynamically groups these into higher-level macro-activities that align with domain-specific process knowledge. This dynamic multi-level recognition allows both automated activity detection and domain-relevant process detail to coexist.
Solution Approach 2:
The patent introduces macro-activities as an intermediary layer between raw sensor data and high-level process steps. This intermediary level preserves the automated recognition capability while adding domain-specific context and detail that makes the data useful for domain experts. The macro-activities serve as a bridge that translates automated detections into domain-relevant process information.
4Loss of information
If AR systems record contextual data on work process progression, then connection between fine-grained trajectory data and higher-level process information is enabled, but technology to fully exploit this connection is missing
Solution Approach 1:
The patent creates a universal framework that integrates multiple data sources (sensor data, AR contextual data, trajectory data) into a unified process map. The spatial process mining approach serves as a multi-functional technology that simultaneously handles trajectory analysis, activity recognition, and process modeling, exploiting the connection between fine-grained and high-level data without requiring separate specialized systems for each function.
Data Source
AI summary
The present disclosure relates to a method and device for building a process map of operator physical operations in a work environment, also described as spatial process mining, in particular for an augmented reality work environment. In at least one implementation a method includes receiving acquired operation trajectories which have been previously segmented into macro-activities which include micro-activities labelled by computer-implemented detecting of previously-known micro-activity patterns; labelling the unlabelled micro-activities with a label that denotes the micro-activity was originally unlabelled; clustering, for each macro-activity, the labelled and unlabelled micro-activities based on the a respective spatial-temporal sub-trajectory of each micro-activity and the associated activity label; and outputting the clustered micro-activities for each macro-activity.