Multimodal Finger Trajectory Fusion for Touch Interaction Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing man-machine interaction technologies fail to comprehensively collect and analyze finger trajectory data during touch screen interactions, leading to incomplete understanding of user interaction rules and habits.
Innovation Solution
A multimodal trajectory information processing and analysis system that includes a collection device and an information processing device, capable of simultaneously collecting and analyzing finger operation trajectory information, including a bracket, lifting device, and touch or optical sensor units to gather interaction data and display interface information.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If only finger trajectory data is collected, then the collection system is simple, but the data completeness is insufficient for accurate interaction analysis
Solution Approach 1:
The patent merges trajectory collection and interaction data collection into a unified multimodal collection system. The collection device integrates multiple sensors (trajectory sensors, touch sensors, force sensors) that simultaneously capture both finger movement trajectories and interaction information (touch location, pressure, duration), resolving the contradiction by combining multiple data sources into one coordinated system rather than separate systems.
Solution Approach 2:
The collection device is designed with multi-functionality to perform both trajectory collection and interaction data collection. The sensor array can detect multiple types of information (position, pressure, touch duration, swipe gestures) using the same hardware platform, allowing the system to gather comprehensive data without requiring separate specialized devices for each function.
2Measurement precision
If interaction data is collected separately from trajectory data, then each collection method is simple, but the integrated analysis capability is insufficient
Solution Approach 1:
The information processing device merges trajectory data and interaction data into a unified data structure for simultaneous processing. The system correlates trajectory points with interaction events (touches, clicks, swipes) by matching spatial and temporal parameters, enabling precise interaction analysis that considers both movement patterns and contact information together rather than separately.
Solution Approach 2:
The patent introduces an intermediary processing layer that correlates trajectory data with interaction data. The system uses timestamp synchronization and spatial coordinate mapping as intermediary mechanisms to match trajectory points with interaction events, serving as a bridge between the two data streams and enabling accurate interaction analysis without direct complex integration of the raw data sources.
3Loss of information
If multiple sensors are integrated for simultaneous data collection, then data completeness improves, but the device structure becomes more complex
Solution Approach 1:
The collection device is segmented into functional modules: trajectory sensors for movement tracking, touch sensors for contact detection, and force sensors for pressure measurement. Each sensor type is independently positioned and calibrated, allowing the system to collect multiple data types simultaneously while managing complexity through modular architecture rather than a monolithic sensor array.
Data Source
AI summary
Provided is a multimodal trajectory information processing and analysis system based on a man-machine interaction terminal. The system includes a collection device and an information processing device. The collection device is configured to collect, analyze and process finger operation trajectory information and send it to the information processing device, and the information processing device is configured to receive a display interface of a device under test in real time, and is configured to fuse and display the collected finger operation trajectory information corresponding to the display interface of the device under test, and obtain an evaluation system of man-machine interaction experience by analyzing a finger operation trajectory. The system achieves the simultaneous collection of the movement trajectory data and interaction data of the target object, which is convenient for accurately analyzing the rules and habits of the interactions between the users and the information systems.