Smart Glasses Eye-Tracking Image Offload for Faster AI Recognition
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Smart glasses with integrated AI functions face limitations due to limited computing power, leading to long image processing times and poor user experience, especially when capturing and processing high-definition images for AI tasks.
Innovation Solution
Incorporating an eye tracking apparatus and a first camera in smart glasses to capture fixation points and perform downsampling on images, sending target area images to a mobile terminal for processing, thereby reducing data volume and enhancing transmission efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high-definition cameras are integrated into smart glasses to capture detailed images for AI tasks, then image quality and AI task accuracy are improved, but computing power requirements increase and processing time extends
Solution Approach 1:
The patent divides the image processing task into two parts: the smart glasses perform initial processing (segmentation based on eye fixation points, downsampling) to extract and compress key information, then transmit only essential data to the mobile terminal for final AI processing. This segmentation of processing tasks between devices reduces the computational burden and time on the smart glasses while maintaining image quality where needed.
Solution Approach 2:
The patent extracts only the most critical image data for transmission to the mobile terminal. By using eye tracking to identify fixation points and segmenting images around these points, the system extracts and transmits only the portions of images that contain meaningful information for AI tasks, rather than transmitting entire high-definition images, thus reducing transmission time and processing load.
2Measurement precision
If high-definition images are transmitted to mobile terminals for processing, then AI task accuracy is improved, but communication data volume increases and transmission efficiency decreases
Solution Approach 1:
The patent changes the parameters of image data before transmission by performing downsampling processing that reduces resolution while preserving key features. The system transforms full-resolution images into compressed representations that retain essential information for AI tasks but occupy significantly less data space, thereby improving transmission efficiency without substantially compromising AI task accuracy.
Solution Approach 2:
The patent segments images into multiple regions based on eye fixation points and transmits only the relevant segments (target area images) along with downsampled versions. This segmentation approach ensures that only necessary image portions are transmitted at high detail, while other areas are transmitted in compressed form, optimizing the balance between AI task accuracy and transmission efficiency.
3Measurement precision
If the smart glasses perform extensive image processing locally, then AI task accuracy is improved, but device complexity and power consumption increase
Solution Approach 1:
The patent introduces the mobile terminal as an intermediary processing device between the smart glasses and the AI processing pipeline. The smart glasses perform preliminary processing (segmentation, downsampling, feature extraction) and transmit processed data to the mobile terminal, which then performs more computationally intensive AI tasks. This intermediary approach allows the smart glasses to maintain simpler hardware while still achieving accurate AI results through distributed processing.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
The solution reduces communication data between smart glasses and mobile terminals, increases transmission rate, and improves AI task accuracy by focusing on the area of interest, thus enhancing user experience and efficiency.
Implementation Method 1
the light source component is configured to emit an infrared light source toward the user eye, the infrared camera is configured to capture a spot image of the user eye
Implementation Method 2
the spot image includes a spot formed on the user eye by the infrared light source emitted by the light source component
Data Source
AI summary
This specification provides smart glasses and an image processing method therefor. The smart glasses include an eye tracking apparatus, a first camera, and a processor. When detecting that an eye movement type is non-saccade, the processor determines a target area image corresponding to a fixation point based on eye tracking, performs downsampling on a first image captured by the first camera to obtain a second image, and sends the second image and the target area image to a mobile terminal. Through interaction between the smart glasses and the mobile terminal and with reference to eye tracking, AI recognition and an intelligent suggestion are implemented for an object on which a user fixates. Furthermore, a manner of combining a high-definition local image with a low-definition panoramic image is implemented. In addition, full references are made to context background information of the recognized object.