Remote Desktop Session Auditing Using AI Video-to-Text Detection

Solution Overview

Problem

Organizations struggle with detecting malicious behaviors in remote desktop sessions efficiently, leading to delayed responses and increased storage costs due to extensive recording requirements.

Innovation Solution

Implementing a generative AI model to generate text descriptions of user actions in remote desktop sessions, parsing these descriptions for prohibited or suspicious behaviors, and automatically terminating sessions or alerting administrators as needed.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If RD sessions are recorded for security monitoring, then malicious behaviors can be detected, but storage costs increase and detection speed decreases

Engineering Contradiction:

Improvedetection capabilityVSAvoidauditing time

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The patent extracts only the essential information from video recordings by generating text descriptions that capture key user actions and events. This selective extraction allows organizations to retain detection capability while dramatically reducing the volume of data that requires human review, thereby reducing auditing time without sacrificing reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent introduces an intermediary component (video-to-text generation system) that translates video recordings into structured text descriptions. This intermediary layer enables automated analysis of session content, allowing rapid scanning and filtering of potentially malicious behaviors without requiring administrators to manually review entire video recordings, thus resolving the contradiction between detection reliability and auditing speed.

Inventive Principle:

Principle #24Intermediary (Mediator)

2Reliability

If video recordings are retained for auditing purposes, then malicious behaviors can be reviewed, but storage requirements increase

Engineering Contradiction:

Improveaudit accuracyVSAvoidstorage space

Core Design Contradiction:

ReliabilityVSQuantity of substance

Solution Approach 1:

The patent extracts only the essential information from video recordings by generating text descriptions that capture key user actions and events. This selective extraction allows organizations to retain detection capability while dramatically reducing the volume of data that requires human review, thereby reducing auditing time without sacrificing reliability.

Inventive Principle:

Principle #2Taking out (Extraction)

Solution Approach 2:

The patent creates text-based copies of video content that preserve the essential audit information while occupying minimal storage space. These text descriptions serve as sufficient copies for audit purposes, eliminating the need to store large video files while maintaining audit accuracy through keyword search and automated analysis capabilities.

Inventive Principle:

Principle #26Copying

3Reliability

If administrators manually review all RD session recordings, then comprehensive security monitoring is achieved, but administrative workload increases

Engineering Contradiction:

Improvesecurity monitoringVSAvoidadministrative workload

Core Design Contradiction:

ReliabilityVSEase of operation

Solution Approach 1:

The patent enables the system to self-monitor and self-analyze session content by automatically generating text descriptions and searching for malicious keywords. This self-service capability performs the initial security screening without administrative intervention, allowing administrators to focus only on reviewing flagged incidents rather than manually examining every session recording, thereby maintaining security monitoring effectiveness while dramatically reducing workload.

Inventive Principle:

Principle #25Self-service

Solution Approach 2:

The patent introduces an intermediary component (video-to-text generation system) that translates video recordings into structured text descriptions. This intermediary layer enables automated analysis of session content, allowing rapid scanning and filtering of potentially malicious behaviors without requiring administrators to manually review entire video recordings, thus resolving the contradiction between detection reliability and auditing speed.

Inventive Principle:

Principle #24Intermediary (Mediator)

Data Source

PatentUS20250342388A1Remote desktop session recording and auditing using generative artificial intelligence

Publication Date: 2025.11.06 OMNISSA LLC

AI summary

A method of auditing user actions performed in remote desktop (RD) sessions, includes the steps of: acquiring a first video file that visually captures a plurality of first user actions that were performed in a first RD session by a remote device hosting the first RD session in response to instructions from a client device of the first RD session; generating a first text file describing the first user actions from the first video file, by using a generative artificial intelligence (AI) model that has been trained to generate text descriptions of user actions from video data capturing user actions in RD sessions; searching the first text file for keywords or phrases that have been identified as being associated with prohibited or suspicious actions; and in response to detecting one of the keywords or phrases in the first text file, terminating the first RD session.