Human-Object Action Recognition Using Mixed-Resolution Vision Regions
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision systems struggle to efficiently identify human actions with respect to small objects in near real-time due to the need for high resolution images, which increases processing time and hinders applications in domains like augmented reality, robotics, and security.
Innovation Solution
A system combines lower resolution images of a body with higher resolution images of targeted body parts using a single vision model to process both, allowing for efficient identification of human actions by resizing and dividing images into non-overlapping patches with different resolutions, and using positional embeddings to capture detailed object interactions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high resolution images are used to identify small objects, then measurement precision is improved, but loss of time increases
Solution Approach 1:
The patent divides the image processing task into two segments: a first image processing operation on a first image (e.g., full scene at lower resolution) and a second image processing operation on a second image (e.g., cropped region of interest at higher resolution). This segmentation allows the system to process only relevant portions at high resolution, reducing overall processing time while maintaining identification accuracy for small objects.
Solution Approach 2:
The patent applies different processing qualities to different regions of the image. The first image is processed at a baseline resolution, while a specific region of interest (containing the small object) is extracted and processed at higher resolution. This local quality approach ensures high measurement precision where needed without incurring the time penalty of processing the entire image at high resolution.
2Device complexity
If a single vision model is used to process both body and target images, then device complexity is reduced, but measurement precision deteriorates
Solution Approach 1:
The patent employs a single vision model that dynamically adapts its processing based on the input image characteristics. The model receives both the first image (full scene) and the second image (cropped region of interest) and dynamically adjusts its attention and processing focus. This dynamic approach allows one model to perform the work of multiple specialized models while maintaining high measurement precision through adaptive processing of different image resolutions and regions.
Data Source
AI summary
A system uses a single vision model to combine lower resolution images of a body and higher resolution images of a targeted body part to more efficiently identify a human action with respect to an object. The system receives images of a scene that include a body. For instance, the images may be sequential frames in a video captured by a camera. The system generates a body image by extracting a region from an image that includes a body. The system generates a target image by extracting a region from the image that includes a targeted body part interacting with an object. The system is configured to perform similar operations on the body image and the target image to ensure that a single vision model can process the target image at a more granular level compared to the body image.