Mobile Device Visual Hacking Prevention via Sensor Fusion
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The increasing use of mobile devices for sensitive operations has led to a rise in visual hacking through shoulder surfing, where unauthorized individuals observe sensitive information displayed on these devices, and existing solutions provide little to no protection against such threats.
Innovation Solution
A mobile device application that utilizes a gyroscope, proximity sensor, and camera to detect the viewing angles and proximity of the user, obscuring sensitive information if the environment is deemed unsafe, ensuring that only the user can view the information by classifying the environment based on pitch and roll angles, distance, and the presence of multiple faces.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If mobile devices are used for sensitive operations in public places, then productivity and convenience are improved, but security against visual hacking deteriorates
Solution Approach 1:
The system proactively detects potential visual hacking attempts before sensitive information can be compromised by monitoring viewing angles, distance, and environmental conditions. When unsafe conditions are detected (such as improper device orientation or proximity to other persons), the system automatically obscures sensitive fields on the display, preventing unauthorized observation before data exposure occurs.
Solution Approach 2:
The invention introduces an intermediary security layer between the sensitive information and potential observers. This intermediary mechanism uses sensors (gyroscope, accelerometer, proximity sensor) and image processing to detect environmental conditions and mediate the display of sensitive information, selectively obscuring it based on detected viewing conditions rather than allowing direct exposure.
2Ease of operation
If sensitive information is displayed clearly on the mobile device screen, then ease of operation is improved, but protection against unauthorized access deteriorates
Solution Approach 1:
The display of sensitive information is made dynamic rather than static. The system continuously monitors device orientation, distance, and environmental conditions, and automatically adjusts the display state of sensitive fields in real-time. When safe conditions are detected, sensitive fields are displayed clearly for user interaction. When unsafe conditions occur, the fields are automatically obscured, providing adaptive security that responds to environmental changes.
Solution Approach 2:
The security protection is applied locally to specific elements of the display rather than the entire screen. Sensitive fields (such as password entries, account numbers) are individually identified and obscured only when unauthorized viewing is detected, while non-sensitive content remains visible. This allows the system to maintain ease of operation for legitimate users while providing targeted protection against unauthorized access.
3Object-affected harmful factors
If environmental monitoring sensors are added to detect viewing conditions, then security against visual hacking is improved, but device complexity increases
Solution Approach 1:
The system leverages existing multi-functional sensors already present in mobile devices for security purposes. The gyroscope and accelerometer, originally used for motion detection and orientation, are also employed to detect device tilt and viewing angles. The proximity sensor, initially designed for near-field communication and display brightness adjustment, is utilized to detect distance to other persons. This approach provides comprehensive environmental monitoring while avoiding the need for entirely new dedicated security hardware.
Solution Approach 2:
The invention merges security monitoring functions with existing device subsystems. Image processing for face detection is integrated with the camera system, and sensor data from the gyroscope, accelerometer, and proximity sensor are combined to create a comprehensive environmental assessment. This consolidation reduces overall system complexity by utilizing and integrating existing components rather than adding separate dedicated security systems.
Applied Scientific Principles
This section explains which scientific principles are used to turn an abstract innovation direction into a practical engineering solution.
Function Achieved in This Case
Effectively protects sensitive information from unauthorized access by automatically obscuring it in unsafe viewing environments, maintaining information security and preventing visual hacking.
Implementation Method 1
The application accesses data from a gyroscope of the mobile device to detect roll and pitch viewing angles of the screen of the mobile device
Implementation Method 2
The application accesses data from a proximity sensor to detect a distance between the screen of the mobile device and a face of the user
Implementation Method 3
The application activates a camera to take an image of the user and detect a number of faces visible in the image
Data Source
Figure 1A
Figure 1B
Figure 2A
AI summary
Protecting a mobile device from visual hacking comprises analyzing a displayed window/page containing data fields and parsing the fields to identify when sensitive information is displayed. Data from a gyroscope of the mobile device is used to detect roll and pitch viewing angles of the mobile device, data from a proximity sensor is used to detect a distance between the mobile device and the user, and a camera is used to take an image of the user and detect a number of faces visible in the image. The sensitive information is obscured when any predefined conditions are not met, including: determining that the roll and pitch viewing angles are not within pitch and roll threshold values, or that the detected distance is not within distance threshold values, or that more than a single face is detected in the image.