Active Interaction System Using Spatiotemporal Feature Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional active interaction methods, such as human body detection and face detection, struggle with accuracy and intelligence in complex scenarios, particularly in multi-person environments, leading to potential misidentification and disturbance of pedestrians due to their reliance on rule-driven approaches.
Innovation Solution
An active interaction method that involves real-time video acquisition, extraction of visual targets, generation of feature vectors, and fusion of spatial and temporal information to determine interaction intentions, using neural networks and self-attention mechanisms to improve accuracy and select appropriate interaction modes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If rule-driven methods such as human body detection or face detection are used to trigger simple interaction logic, then the system can implement active interaction with basic functionality, but the accuracy and intelligence of interaction are insufficient in complex scenarios
Solution Approach 1:
The patent segments the interaction analysis into multiple independent modules: human body detection module, human face detection module, gait recognition module, and interaction intention recognition module. Each module processes specific features independently and their results are fused to achieve comprehensive interaction analysis, thereby improving both accuracy and adaptability
Solution Approach 2:
The patent introduces temporal dimension by analyzing video sequences rather than single images. It extracts features across multiple frames and uses temporal fusion to recognize interaction intentions, transforming the problem from 2D spatial analysis to 3D spatiotemporal analysis, which significantly improves interaction accuracy in complex scenarios
2Reliability
If simple detection methods are used, then the system complexity remains low, but the system cannot handle multi-person scenarios and may mistakenly disturb pedestrians
Solution Approach 1:
The patent merges multiple detection results (human body detection, face detection, gait recognition) and fuses their feature vectors to make comprehensive interaction decisions. This fusion approach improves reliability by cross-validating multiple sources while maintaining a unified system architecture that manages complexity
Solution Approach 2:
The system implements feedback mechanisms where detection results from previous frames inform current frame analysis. The temporal feature fusion uses historical interaction states to refine current predictions, creating a closed-loop system that improves reliability through continuous validation while managing complexity through iterative refinement
Data Source
AI summary
An active interaction method, an electronic device and a readable storage medium, relating to the field of deep learning and image processing technologies, are disclosed. According to an embodiment, the active interaction method includes: acquiring a video shot in real time; extracting a visual target from each image frame of the video, and generating a first feature vector of each visual target; for each image frame of the video, fusing the first feature vector of each visual target and identification information of the image frame to which the visual target belongs to generate a second feature vector of each visual target; aggregating the second feature vectors with the same identification information respectively to generate a third feature vector corresponding to each image frame; and initiating active interaction in response to determining that the active interaction is to be performed according to the third feature vector of a preset image frame.