Holistic IoT Sensing With Acoustic Heatmaps for Shop Floor Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Industrial IoT systems face challenges in increasing productivity on the shop floor due to inadequate camera image information for applications like production line failure monitoring and product quality checks, particularly in large factory environments where manual methods for capturing acoustic sensor data become costly and inefficient.
Innovation Solution
The integration of camera and acoustic sensor data fusion, using acoustic sensors to complement camera data and automatically generate heatmaps, which reduces manual operation costs by employing a novel sensor device independent of GPS systems, allowing for detailed investigation of abnormal situations and machine conditions through sensor data overlay.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If camera image information is used for production line monitoring and quality checks, then visual detection capability is provided, but the information is inadequate for detailed analysis of abnormal situations and machine conditions
Solution Approach 1:
The patent combines camera imaging sensors with acoustic sensors into an integrated sensing system. The camera captures visual information while acoustic sensors simultaneously capture sound information from the same scene, merging multiple sensing modalities to provide comprehensive information for production monitoring and quality checks without requiring separate independent systems.
Solution Approach 2:
The patent adds acoustic sensing as another dimension to the traditional visual-only monitoring system. By incorporating sound field information alongside image data, the system transitions from two-dimensional visual inspection to multi-dimensional sensing that includes both spatial visual data and acoustic characteristics, enabling more thorough analysis of machine conditions and abnormal situations.
2Measurement precision
If manual methods are used for capturing acoustic sensor data, then acoustic information can be obtained, but the process becomes costly and inefficient
Solution Approach 1:
The sensing system is designed to automatically capture and process acoustic sensor data without requiring manual intervention. The integrated system autonomously collects acoustic information, synchronizes it with camera data, and performs analysis, eliminating the need for manual data capture methods while maintaining high measurement precision and significantly improving productivity.
Solution Approach 2:
The system performs preliminary automatic capture and processing of acoustic sensor data as part of its standard operation. By pre-configuring the acoustic sensors and establishing automatic data capture protocols, the system prepares and executes measurements without manual involvement, thereby achieving both high precision and efficient productivity in acoustic data acquisition.
3Area of stationary object
If holistic sensing with camera is used for shop floor monitoring, then broad area detection is achieved, but detailed investigation of essential causality is limited
Solution Approach 1:
The patent merges holistic camera-based monitoring with localized acoustic sensing to bridge the gap between broad coverage and detailed detection. While the camera provides wide-area monitoring capability, acoustic sensors positioned at specific locations offer precise detection of machine conditions and abnormal sounds, combining both approaches to achieve comprehensive monitoring with detailed investigative capability.
Solution Approach 2:
The sensing system is segmented into holistic camera monitoring components for broad area detection and localized acoustic sensor components for detailed investigation. This segmentation allows each component to excel at its specific function - the camera covers large areas while acoustic sensors provide precise localized measurements - and the results are integrated to achieve both broad coverage and detailed precision simultaneously.
Data Source
Figure 1
Figure 2~3
Figure 4
AI summary
Systems and methods described herein are directed to a specialized Internet of Things (IoT) device deploying both acoustic and radio wave signals. In example implementations described herein, camera data and acoustic sensor data is integrated to generate an acoustic sensor heatmap for the holistic sensing systems in an IoT area.