Process Mining Visual Logs Video Streams
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional process mining techniques face challenges in capturing comprehensive insights into operation processes due to limitations in event logs, which only record completed states and fail to capture interactions between workers and devices, making it difficult to identify bottlenecks and improve processes efficiently.
Innovation Solution
A process mining system generates visual logs from video streams of worker devices, extracting activity labels from images and assigning case identifiers to provide a holistic view of operation processes, including interactions and handovers, thereby enhancing the analysis of operation processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If conventional event logs are used to record worker actions, then the logging system remains simple and easy to implement, but the logs only capture completed states and miss valuable interaction details between workers and devices
Solution Approach 1:
The patent introduces video streams as an intermediary medium between the worker-device interaction and the logging system. Instead of directly capturing all interactions through complex sensors, the system uses video recording as a mediator that captures comprehensive interaction details, which are then processed through image recognition algorithms to extract meaningful events and interactions.
Solution Approach 2:
The patent replaces traditional mechanical/event-driven logging mechanisms with vision-based detection. Instead of relying on device-generated events and manual monitoring, the system uses computer vision technology to automatically detect and log worker actions, device interactions, and process activities from video streams, thereby capturing comprehensive interaction details without proportionally increasing system complexity.
2Measurement precision
If human operators directly observe workers to determine actions, then comprehensive process insights can be obtained, but this becomes time-consuming and infeasible for large numbers of worker devices
Solution Approach 1:
The patent enables the logging system to be self-sufficient by automatically analyzing video streams through image recognition algorithms. The system extracts activity labels, detects worker actions, identifies device interactions, and generates structured logs without requiring human operators to manually observe and record each worker's activities, thereby maintaining high analysis accuracy while achieving scalable throughput for large numbers of worker devices.
Solution Approach 2:
The patent replaces manual human observation with automated computer vision systems. Image recognition algorithms process video streams to detect and classify worker actions, device interactions, and process events, substituting human operators with an automated system that can analyze multiple worker devices simultaneously, thereby increasing analysis throughput while maintaining measurement precision.
3Loss of information
If video streams are used to capture comprehensive worker-device interactions, then detailed process insights are obtained, but the data processing complexity and computational requirements increase
Solution Approach 1:
The patent extracts only the essential and relevant information from video streams through image recognition algorithms. Instead of processing and storing all video data, the system identifies and extracts key elements such as worker actions, device interactions, activity labels, and process events, converting them into structured log entries. This extraction approach captures comprehensive process interaction details while reducing data processing complexity by focusing only on meaningful information.
Solution Approach 2:
The patent creates simplified representations (copies) of the complex video data in the form of structured event logs with activity labels and interaction details. Instead of working with raw video streams, the system generates condensed log entries that capture the essential process information, thereby reducing computational requirements while preserving the necessary process interaction details for analysis.
4Loss of information
If traditional event logs are used, then the logging infrastructure remains simple, but variations such as task handovers between workers cannot be effectively tracked
Solution Approach 1:
The patent uses video streams as an intermediary to capture task handover information that traditional event logs miss. The image recognition system analyzes video frames to detect worker presence, device usage patterns, and contextual cues that indicate task handovers, thereby capturing this information without requiring complex additional monitoring infrastructure.
Solution Approach 2:
The patent makes the video-based logging system multi-functional by enabling it to capture not only task handovers but also worker actions, device interactions, activity labels, and other process details using the same infrastructure. This universal approach tracks task handovers and other process variations comprehensively without requiring separate specialized monitoring systems for each function.
Data Source
AI summary
A process mining system performs process mining using visual logs generated from video streams of worker devices. Specifically, for a given worker device, the process mining system obtains a series of images capturing a screen of a worker device while the worker device processes one or more tasks related to an operation process. The process mining system determines activity labels for a plurality of images. An activity label for an image may indicate an activity performed on the worker device when the image was captured. The activity label is determined by extracting information from pixels of the image and inferring the activity of the worker device from the extracted information. The process mining system generates event logs from the visual logs of worker devices and uses the event logs for process mining.