Multi-screen Mouse Switching via Visual Attention Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional multi-screen display systems face challenges in user experience due to difficulties in determining which screen the mouse is currently on, especially with varying screen positions and user habits, leading to inconvenient mouse switching control.
Innovation Solution
A method that uses cameras on each screen to collect user images, input them into a neural network model to predict which screen the user is paying attention to, and automatically switch the mouse to that screen, enhancing user experience with robustness and high recognition accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional mouse switching control is used in multi-screen display systems, then the system structure remains simple, but the user experience deteriorates due to inability to determine current screen location
Solution Approach 1:
The patent replaces manual mouse sliding operations with an automated vision-based system. Cameras capture user gaze position, neural networks process image data to determine attention focus, and the system automatically switches mouse cursor to the corresponding screen, eliminating the need for users to manually track and slide the mouse across multiple displays.
Solution Approach 2:
The system performs self-service by automatically detecting which screen the user is attending to and switching the mouse cursor without requiring user intervention. The neural network model processes camera inputs and autonomously determines the current screen, eliminating the need for users to manually determine screen location.
2Measurement precision
If cameras and neural network models are added to determine user attention, then mouse switching accuracy improves, but device complexity increases
Solution Approach 1:
The patent introduces an intermediary neural network model that processes camera captures and translates visual input into screen identification results. This intermediary component bridges the gap between raw camera data and useful screen detection information, enabling accurate measurement without direct complex hardware modifications.
3Speed
If automatic mouse switching based on visual attention is implemented, then response speed to user operations improves, but energy consumption increases
Solution Approach 1:
The system employs periodic action by capturing images at specific intervals rather than continuously. The camera captures images periodically, the neural network processes only these periodic inputs, and mouse switching occurs only when attention detection triggers a state change, reducing energy consumption while maintaining responsive performance.
Data Source
AI summary
A multi-screen display system and a mouse switching control method are disclosed. The mouse switching control method is applied to a multi-screen display system comprising a main display screen and at least one extended display screen, and comprises: obtaining user images collected by cameras installed on the main display screen and the extended display screen respectively; inputting the user images into a neural network model, and predicting a screen that a user is currently paying attention to using the neural network model to obtain a prediction result; and controlling to switch a mouse to the screen that a user is currently paying attention to according to the prediction result. The system and mouse switching control method are based on self-learning of visual attention, predict the current screen operated by the user, automatically switch the mouse to the corresponding screen position, and improve the user experience.