Augmented Reality Hand Recognition with Split AI Models
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Augmented reality devices face challenges in real-time hand recognition due to low hardware specifications and low-performance AI models, leading to prolonged processing times and reduced accuracy in hand recognition, limiting their usability as input means.
Innovation Solution
The augmented reality device offloads hand recognition tasks to a high-level AI model on an external device, such as a smartphone, and uses a low-level AI model on-device to adjust and recognize specific hand poses, enhancing processing speed and accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a high-performance AI model is embedded in the augmented reality device for on-device hand recognition, then recognition accuracy improves, but processing time increases and real-time recognition becomes difficult
Solution Approach 1:
The patent divides the AI model into two segments: a high-performance first AI model embedded in the external device for accurate hand recognition, and a low-performance second AI model in the augmented reality device for real-time processing. This segmentation allows each model to operate at its optimal performance level without compromising the other.
Solution Approach 2:
The patent introduces an external device (smartphone or server) as an intermediary that hosts the high-performance first AI model. This intermediary processes complex recognition tasks and communicates results back to the augmented reality device, enabling accurate recognition without burdening the device's limited hardware.
2Productivity
If the augmented reality device uses its own embedded AI model for hand recognition, then processing speed improves, but recognition accuracy decreases due to low model performance
Solution Approach 1:
The patent segments the AI processing functions by placing the high-accuracy first AI model in the external device and the real-time second AI model in the augmented reality device. This allows the device to maintain fast processing while the external device provides accurate recognition results.
Solution Approach 2:
The patent creates a simplified copy (second AI model) of the high-performance first AI model that can run on the augmented reality device's limited hardware. This copy performs real-time processing and can be refined using feedback from the more accurate first model.
3Loss of time
If the augmented reality device uses a low-performance AI model for on-device recognition, then real-time processing is achieved, but recognition accuracy remains low
Solution Approach 1:
The patent implements a feedback mechanism where the second AI model's recognition results are communicated back to the first AI model in the external device. The first model uses this feedback to further refine and improve recognition accuracy, creating a continuous improvement loop that maintains real-time performance while enhancing precision.
Solution Approach 2:
The patent creates a dynamic system where the two AI models work in coordination, with the second model handling real-time processing and the first model providing periodic accuracy improvements. The system adapts by using feedback to continuously refine recognition performance.
Data Source
AI summary
An augmented reality device is provided. The augmented reality device includes a camera, an inertial measurement unit (IMU) sensor including an accelerometer, a communication interface configured to perform data communication with a mobile device, memory, comprising one or more storage media, storing one or more instructions, and at least one processor communicatively coupled to the memory, wherein the instructions, when executed by the at least one processor individually or collectively, cause the augmented reality device to obtain an image captured by using the camera and sensor data measured by using the accelerometer, control the communication interface to transmit the obtained image and the obtained sensor data to a mobile device, receive, from the mobile device, information about a direction, a type, and an object detection area of an object obtained from the image according to a result of object recognition performed by the mobile device by using a first artificial intelligence model included in the mobile device, adjust a location of the object detection area, based on a value of the sensor data changed at a time point when the result of the object recognition is received from the mobile device, determine a second artificial intelligence model for recognizing the object, based on the direction and the type of the object included in the object detection area of which the location has been adjusted, and recognize the object from the object detection area in the image by using the determined second artificial intelligence model.