AI Screen Recording Analysis for Remote Session Threat Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In remote connection environments, monitoring user behavior for malicious activities is challenging due to limited direct access and encryption, making it difficult to detect unauthorized actions effectively.
Innovation Solution
Implementing an AI model to analyze screen recordings from remote connections, extracting visual elements, and categorizing behaviors to initiate intervention actions when malicious intent is detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional monitoring methods are used in remote connection environments, then system security can be maintained, but monitoring efficiency and detection accuracy deteriorate due to limited direct access and encryption
Solution Approach 1:
The patent introduces an intermediary AI model that acts as a mediator between the encrypted remote connection environment and the monitoring system. The AI model receives screen recordings from the remote environment, processes visual information to detect malicious behaviors, and provides monitoring capabilities without requiring direct access to the encrypted communication channels, thus maintaining security while improving monitoring efficiency
Solution Approach 2:
The patent replaces traditional mechanical monitoring approaches (direct access, manual monitoring) with an AI-based visual analysis system. The AI model substitutes human monitors and direct system access with automated computer vision technology that can analyze screen recordings to detect malicious behaviors, significantly improving monitoring efficiency and scalability
2Measurement precision
If direct access to remote machines is implemented for monitoring, then monitoring accuracy improves, but system security and operational complexity worsen
Solution Approach 1:
The patent extracts the essential monitoring function from the complex remote access infrastructure by capturing screen recordings as a separate, standalone data source. The AI model processes these extracted visual elements to detect malicious behaviors, achieving monitoring accuracy without requiring direct access to or integration with the complex remote machine systems
Solution Approach 2:
The patent creates a visual copy of the remote environment through screen recordings, which are then fed to the AI model for analysis. This copy serves as a sufficient representation for monitoring purposes, eliminating the need for direct access to the original remote machines and reducing operational complexity while maintaining monitoring accuracy
Data Source
Figure 1
Figure 2
Figure 3A
AI summary
Systems and methods are provided for monitoring a remote connection environment. A screen recording of a user accessing a second computing machine via a first computing machine is captured. Visual elements are extracted from the screen recording. The screen recording and the extracted visual elements are provided to a computer model that has been trained using training screen recordings, training extracted text, and characterizations of behavior. An indication of whether the screen recording and the extracted text are indicative of a particular behavior in which the method is designed to intervene are received from the computer model. An intervention action is executed when the computer model indicates the particular behavior.