Human Action Recognition Using Body and Target Region Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Computer vision systems struggle to identify human actions with respect to small objects in near real-time due to the need for high resolution images, which increases processing time and is infeasible for applications like augmented reality, robotics, and security.
Innovation Solution
A system combines lower resolution images of the body with higher resolution images of targeted body parts using a single vision model to process both, allowing for efficient identification of human actions by resizing and dividing the images into non-overlapping patches with different resolutions, and using positional embeddings to capture detailed information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If high resolution images are used to identify small objects, then identification accuracy is improved, but processing time increases
Solution Approach 1:
The patent divides the image processing task into two segments: a first vision model processes the entire image at lower resolution to identify potential regions, while a second vision model processes only the identified region at higher resolution for detailed object identification. This segmentation allows the system to maintain high identification accuracy for small objects while reducing overall processing time by avoiding full high-resolution image processing.
2Device complexity
If a single vision model is used to process both body and target images, then device complexity is reduced, but processing efficiency decreases
Solution Approach 1:
The patent segments the vision processing task into two distinct models with specialized functions: a first vision model for scanning the entire image at lower resolution to locate regions of interest, and a second vision model for detailed analysis of the target region at higher resolution. This functional segmentation improves processing efficiency by allowing each model to be optimized for its specific task and by enabling parallel processing of different image regions.
Solution Approach 2:
The patent introduces a dimensional distinction in processing by applying different resolution levels and processing depths to different parts of the image. The first vision model operates on the full image dimension at lower resolution, while the second vision model focuses on a subset dimension (the identified region) at higher resolution, creating a multi-dimensional processing approach that balances complexity and efficiency.
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
A system uses a single vision model to combine lower resolution images of a body and higher resolution images of a targeted body part to more efficiently identify a human action with respect to an object. The system receives images of a scene that include a body. For instance, the images may be sequential frames in a video captured by a camera. The system generates a body image by extracting a region from an image that includes a body. The system generates a target image by extracting a region from the image that includes a targeted body part interacting with an object. The system is configured to perform similar operations on the body image and the target image to ensure that a single vision model can process the target image at a more granular level compared to the body image.