Video Analysis for Occluded Person-Object Interaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods struggle to accurately determine whether a series of actions performed by a person on an object is appropriate, particularly when the person turns their back or is obstructed from view, leading to difficulties in identifying errors in work content.
Innovation Solution
An information processing program that analyzes video data captured by a camera to identify interactions between a person and an object, using machine learning models to specify actions even when direct views are obstructed, by compensating for missing frames through analysis of preceding and succeeding frames.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If video analysis is used to determine work content appropriateness, then monitoring capability is improved, but accuracy deteriorates when the person turns their back or is obstructed
Solution Approach 1:
The system performs preliminary action by specifying actions during visible periods (first and second time points) before the occlusion occurs. By analyzing video data when the person is visible and pre-specifying actions, the system prepares action specifications in advance. When occlusion occurs at the third time point, these pre-specified actions are utilized to maintain continuous monitoring without gaps, thereby resolving the contradiction between maintaining reliability and preventing information loss during occlusion events.
2Productivity
If continuous video monitoring is implemented, then complete action tracking is improved, but gaps occur when the person is out of view
Solution Approach 1:
The system implements continuity of useful action by processing video data across multiple time points (first, second, and third time points) to continuously specify actions. When occlusion occurs at the third time point, the system maintains continuous action specification by utilizing actions specified at the first and second time points. This approach ensures uninterrupted monitoring productivity while managing the inherent limitation of measurement precision during occlusion periods through temporal extrapolation.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An information processing program including instructions for causing a computer to execute processing including: acquiring a video data including frames each of which is associated with a time point and includes the video captured at the time point, by capturing a video at a site in which a person and an object exist; identifying a first relation by analyzing a first frame associated with a first time point, the first relation identifying an interaction between the object and the person in the first frame; identifying a second relation by analyzing a second frame associated with a second time point, the second relation identifying an interaction between the object and the person in the second frame; and specifying, based on the first and second relations, an action of the person on the object in a third frame associated with a third time point between the first and second time points.