AR Manipulation Assistance Using Edge-Based Action Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing augmented reality systems struggle with accurate and efficient object manipulation assistance due to limited processing power in mobile devices, small field of view, and sensitivity to environmental factors like lighting conditions, which affects the accuracy of object and action recognition.
Innovation Solution
A method and system utilizing a monitoring unit that receives user and object position data via a cellular network, matches actual positions to predefined data, and determines correct manipulation through object and human action recognition, providing feedback via a viewing apparatus.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If object recognition tasks are offloaded to cloud servers, then recognition accuracy improves, but end-to-end latency increases
Solution Approach 1:
The system segments the object recognition pipeline by separating data capture (performed by mobile device cameras) from complex computation (performed by edge servers). This division allows the mobile device to remain lightweight while offloading only the computationally intensive recognition tasks to edge infrastructure, balancing accuracy and latency requirements.
Solution Approach 2:
Edge servers act as intermediaries between mobile devices and cloud infrastructure. They receive image data from mobile devices, perform object recognition using pre-trained models, and return results to the devices. This intermediary layer reduces latency compared to direct cloud communication while maintaining high recognition accuracy through access to powerful computing resources.
2Measurement precision
If frame rate and video frame size are increased, then object detection accuracy improves, but network bandwidth consumption increases
Solution Approach 1:
The system implements adaptive frame sampling where not every captured frame is transmitted to the edge server. Instead, frames are selected based on detection confidence thresholds and motion detection, transmitting only those frames that are most likely to contain relevant objects. This partial transmission approach maintains detection accuracy while significantly reducing network bandwidth consumption.
3Measurement precision
If computational model complexity is increased, then object recognition accuracy improves, but processing time increases
Solution Approach 1:
The system performs preliminary actions by pre-training computational models on edge servers before deployment. These pre-trained models are then deployed to edge servers where they can process incoming frames efficiently. The heavy lifting of model training is done in advance, allowing for faster inference times during actual object recognition tasks while maintaining high accuracy through sophisticated model architectures.
Data Source
AI summary
A method for operating a monitoring unit (100), configured to monitor a manipulation of at least one object (40, 45) by a user, the method comprising: —receiving, via a cellular network (60), actual user position data comprising an actual position of at least one portion of user used to manipulate the at least one object, and actual object position data comprising an actual position of the at least one object, —matching the actual user position data to predefined user position data provided at the monitoring unit (100), the predefined user position data indicating a correct position of the at least one portion of the user for manipulating the at least one object, and matching the actual object position data to predefined object position data provided at the monitoring unit, the predefined object position data indicating a correct position of the at least one object, —determining, based on the matching, whether the manipulation of the at least one object by the at least one portion of the user is correct or not.