User-in-the-Loop Object Detection for Communication Disruptions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems for object detection and classification in image processing are limited by resource constraints, such as memory and processing power, which restrict their ability to run robust neural networks and adapt to various end-use scenarios, leading to inaccurate detection results, especially in environments with communication disruptions.
Innovation Solution
A user-in-the-loop system that employs unmanned vehicles equipped with sensors and trained neural networks to detect and classify objects, allowing for real-time communication and data storage for user approval and correction when communication is lost, and enabling the generation of training data samples for improving the object classifier.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a trained neural network is loaded on a server system to receive and classify images from imaging devices on a network, then classification accuracy is improved, but system complexity and communication requirements increase
Solution Approach 1:
The system divides the image classification task into two segments: local processing (unmanned vehicle performs object detection and generates sensor data) and remote processing (control system receives data and provides user feedback). This segmentation allows the neural network to be distributed rather than requiring all processing power on a single server, reducing overall system complexity while maintaining accuracy.
Solution Approach 2:
The unmanned vehicle acts as an intermediary between the imaging device and the control system. It performs preliminary object detection and classification locally, then transmits only relevant data to the control system for user review and feedback. This intermediary approach reduces communication bandwidth requirements and system complexity while maintaining high classification accuracy through the trained neural network.
2Extent of automation
If the unmanned vehicle processes object detection locally without continuous communication, then operational autonomy is improved, but detection accuracy may deteriorate due to lack of real-time feedback
Solution Approach 1:
The unmanned vehicle performs preliminary object detection and classification using a trained neural network before transmitting data to the control system. This preliminary action enables the vehicle to operate autonomously in real-time, making immediate detections without waiting for user feedback, while still maintaining the ability to improve accuracy through subsequent user review and retraining.
Solution Approach 2:
The system implements a feedback loop where the control system receives object detection data from the unmanned vehicle, allows user review and correction, and uses this feedback to retrain the neural network. This feedback mechanism enables the system to maintain high detection accuracy through continuous improvement while preserving operational autonomy during real-time operations.
3Adaptability or versatility
If training data is collected and processed in real-time during operations, then system adaptability is improved, but processing time and computational resources increase
Solution Approach 1:
The system performs preliminary data collection and preliminary classification during operational phases, storing data for later processing. This allows the system to maintain adaptability by continuously collecting real-world data while avoiding the time-consuming retraining process during active operations. The trained neural network provides immediate classification results, and retraining occurs in batches using collected data.
Solution Approach 2:
The system maintains continuous operation by separating data collection from processing. The unmanned vehicle continuously detects and classifies objects in real-time, while the control system processes and retrains the neural network in batches using collected data. This continuity ensures the system remains adaptable and responsive without the processing time delay of real-time retraining.
4Reliability
If the system stores object analysis information for later user review, then communication reliability is improved, but data storage requirements and system complexity increase
Solution Approach 1:
The system extracts and stores only the essential object analysis information (detection results, classification, confidence factors) rather than storing all raw sensor data. This extraction approach maintains communication reliability by ensuring critical data is preserved during communication disruptions, while minimizing storage requirements and system complexity by storing only necessary information.
Solution Approach 2:
The unmanned vehicle performs preliminary data processing and stores condensed object analysis information locally before transmission to the control system. This preliminary action ensures that even if communication fails, the vehicle maintains operational autonomy with stored analysis data, while reducing the amount of data that needs to be transmitted and stored, thereby lowering system complexity.
Data Source
AI summary
A detection device is adapted to traverse a search area and generate sensor data associated with an object that may be present in the search area, the detection device comprising a first logic device configured to detect and classify the object in the sensor data, communicate object detection information to a control system when the detection device is within a range of communications of the control system, and generate and store object analysis information for a user of the control system when the detection device is not in communication with the control system. A control system facilitates user monitoring and/or control of the detection device during operation and to access the stored object analysis information. The object analysis information is provided in an interactive display to facilitate user detection and classification of the detected object by the user to update the detection information, trained object classifier, and training dataset.