Video Relationship Graph Scoring for Real-Time Interaction Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Evaluating the strength of relationships between individuals, particularly for those with impairments affecting verbal and non-verbal communication, is difficult and often not done in real-time.
Innovation Solution
A system and method that utilizes digital video data to identify interactions, create a relationship graph, and determine a relationship score using a computer-implemented machine learning model, incorporating features like image, audio, and semantic text data to analyze and output relationship dynamics.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If video data analysis is used to evaluate relationship strength, then measurement precision is improved, but device complexity increases
Solution Approach 1:
The system segments the relationship evaluation process into distinct components: video data acquisition, feature extraction (visual, audio, semantic), relationship graph construction, and machine learning-based scoring. This segmentation allows each component to be optimized independently while maintaining overall system precision for evaluating relationship strength.
2Productivity
If real-time relationship evaluation is implemented, then productivity is improved, but use of energy increases
Solution Approach 1:
The system performs preliminary actions by pre-processing video data to extract key features (visual, audio, and semantic) before the actual relationship evaluation. Relationship graphs are constructed in advance from these features, allowing the machine learning model to operate on pre-processed data structures, thereby reducing real-time computational energy requirements while maintaining fast evaluation speeds.
3Measurement precision
If multiple video features are analyzed to create relationship graph, then measurement precision is improved, but difficulty of detecting and measuring increases
Solution Approach 1:
The system introduces relationship graphs as an intermediary data structure that bridges raw video features and final relationship strength evaluation. Multiple video features (visual, audio, semantic) are first transformed into structured relationship graphs that capture interaction patterns, making the subsequent measurement of relationship strength more straightforward and precise while reducing the direct complexity of analyzing all features simultaneously.
Data Source
AI summary
A method includes acquiring digital video data that portrays an interacting event, identifying a plurality of video features in the digital video data, analyzing the plurality of video features to create a relationship graph, determining a relationship score based on the relationship graph using a first computer-implemented machine learning model, and outputting the relationship score with a user interface. The interacting event comprises a plurality of interactions between a first individual and a second individual and each video feature of the plurality of video features corresponds to an interaction of the plurality of interactions. The relationship graph comprises a first node, a second node, and a first edge extending from the first node to the second node. The first node represents the first individual, the second node represents the second individual, and a weight of the first edge represents a relationship strength between the first individual and the second individual.