Action Recognition Analytics for Real-Time Process Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methodologies in manufacturing, such as Toyota's Production System and Six-Sigma, rely on manual data collection, which results in incomplete and biased data sets due to the Hawthorne and Heisenberg effects, and IIoT devices only cover a small portion of manufacturing tasks, limiting the effectiveness of quality improvement and process optimization.
Innovation Solution
An action recognition and analytics system utilizing sensors like video, thermal, and depth sensors to collect and analyze data on cycles, processes, actions, and objects in real-time, providing anomaly detection and quality assurance, and integrating machine learning techniques like deep learning to compare current data sets with representative data sets for optimization.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual techniques are used to gather data on human activity, then data collection is simple and low-cost, but the data set is small, incomplete, and biased due to the Hawthorne and Heisenberg effects
Solution Approach 1:
The patent replaces manual data collection methods with automated computer vision and machine learning systems. Sensors capture video and sensor data streams that are automatically processed by deep learning models to identify actions, objects, and processes, eliminating the Hawthorne and Heisenberg effects inherent in manual observation while dramatically increasing data completeness and volume
Solution Approach 2:
The patent introduces an intermediary automated analysis system between the manufacturing process and data collection. This system uses sensors and computer vision algorithms to objectively measure human activity without direct human observation, thereby preventing the bias and behavioral changes caused by manual data gathering methods
2Extent of automation
If IIoT devices are used to collect data, then automation and data exchange are improved, but data coverage is limited to only a small portion of manufacturing tasks
Solution Approach 1:
The patent implements a universal data collection system using multi-functional sensors and computer vision algorithms that can detect and analyze diverse manufacturing tasks across different workstations. The system handles various action types (manipulation, transport, assembly, inspection) and object types (parts, tools, products) within a single unified framework, dramatically expanding data coverage beyond what specialized IIoT devices can achieve
Solution Approach 2:
The patent adds a visual and spatial dimension to data collection by using video sensors and depth sensors. This enables the system to capture and analyze human actions, object manipulations, and process sequences that traditional IIoT devices mounted on machines cannot detect, thereby expanding the scope of automated data collection to include the full range of manufacturing tasks
3Productivity
If the number of actions per station increases due to increased complexity or decreased time, then productivity is improved, but cognitive load on operators increases resulting in higher deviation rates
Solution Approach 1:
The patent implements real-time feedback systems where the automated analysis system continuously monitors operator actions and provides immediate feedback on deviations from standard procedures. This allows operators to correct errors promptly while maintaining high productivity, and enables managers to identify training needs and process improvements without increasing cognitive load on individual operators
Data Source
AI summary
The systems and methods provide an action recognition and analytics tool for use in manufacturing, health care services, shipping, retailing and other similar contexts. Machine learning action recognition can be utilized to determine cycles, processes, actions, sequences, objects and or the like in one or more sensor streams. The sensor streams can include, but are not limited to, one or more video sensor frames, thermal sensor frames, infrared sensor frames, and or three-dimensional depth frames. The analytics tool can provide for process validation, anomaly detection and in-process quality assurance.