Neural Network Training via Synchronized Visual and Non-Visual Sensor Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current object classification systems using deep learning neural networks rely heavily on labeled imagery data and require both visual and non-visual data for accurate object detection and movement identification in security contexts, limiting their effectiveness when only visual data is available.
Innovation Solution
A method is developed to train a neural network model using synchronized non-visual and visual data, where non-visual data from sensors is paired and synchronized with visual data from cameras to label and train the model, allowing it to detect objects and movements without relying on sensors for real-time visual data analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If non-visual sensors are used to detect objects and movements, then detection accuracy is improved, but device complexity and dependency on multiple sensor inputs increase
Solution Approach 1:
The system performs preliminary training by collecting and synchronizing non-visual sensor data with visual data during a training phase. This preliminary action creates a trained neural network model that encapsulates the detection patterns, allowing the system to later perform accurate detections using only visual data without requiring the original sensors during operation.
Solution Approach 2:
The invention extracts the essential detection patterns from the combined sensor and visual data during training, then separates the sensor dependency from the operational phase. The trained model contains only the necessary detection knowledge, allowing visual-only operation while maintaining the accuracy benefits of multi-sensor training.
2Measurement precision
If both visual and non-visual data are used for training, then object detection accuracy is improved, but data processing complexity increases
Solution Approach 1:
The patent introduces a neural network model as an intermediary that learns to map visual data patterns to object detection outcomes. During training, this intermediary processes both visual and non-visual data together, learning their relationships. During operation, it serves as a self-contained processor that applies the learned patterns to visual data alone, simplifying operational complexity while maintaining training-phase accuracy benefits.
3Reliability
If sensor data is continuously collected for real-time detection, then detection reliability is improved, but energy consumption and data storage requirements increase
Solution Approach 1:
The system extracts the essential detection capabilities during a training phase using sensor data, then removes the need for continuous sensor data collection during operation. The trained neural network model contains the extracted detection knowledge, allowing the system to maintain reliability without the ongoing energy cost of sensor operation and data processing.
Solution Approach 2:
The patent performs the energy-intensive data collection and processing actions preliminarily during the training phase. Once the model is trained, these resource-consuming operations are no longer needed during real-time detection, as the model has already learned the detection patterns from the preliminary sensor data collection.
Data Source
AI summary
A computer-implemented method includes obtaining data from one or more non-visual sensors and a camera from a first monitoring system. The data includes non-visual data from the non-visual sensors and visual data obtained from the camera. The non-visual data from the non-visual sensors are paired with corresponding visual data from the camera. Data points of the non-visual data are synchronized with frames of the visual data based on a likelihood of an event indicated in the non-visual data. The synchronized data points of the non-visual data with the frames of the visual data are provided as labeled input to a neural network to train the neural network to detect the event. The trained neural network is provided to one or more cameras corresponding to one or more additional monitoring systems to detect the event in the visual data obtained by the one or more cameras.