Handshake Protocol Analysis for Early Malicious Connection Detection

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Existing cyber security software is limited in the types of data and files it can monitor, creating potential weak points that can be exploited by malicious actors, particularly in handshake protocols like HTTPS, TLS, and SSH.

Innovation Solution

A method using machine learning models to analyze handshake parameters from client and server connections, predicting maliciousness, and initiating corrective actions such as blocking or quarantining suspicious connections.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Measurement precision

If machine learning models are used to analyze handshake parameters, then detection accuracy of malicious connections is improved, but device complexity and computational resources increase

Engineering Contradiction:
Improvedetection accuracyVSAvoidsystem complexity
Core Design Contradiction:
Measurement precisionVSDevice complexity

Solution Approach 1:

The system extracts and analyzes features from handshake parameters during the connection establishment phase (handshake), before the actual data transmission begins. This preliminary analysis allows the machine learning model to predict malicious connections early, improving detection accuracy while limiting the scope of computational resources to only the handshake phase data.

Inventive Principle:
Principle #10Preliminary action

Solution Approach 2:

The system extracts specific features from handshake parameters (such as cipher suite selections, protocol versions, and handshake timing) and feeds only these extracted features to the machine learning model, rather than analyzing entire communication streams. This extraction approach improves detection precision while reducing the computational burden on the system.

Inventive Principle:
Principle #2Taking out (Extraction)

2Speed

If automated corrective actions are initiated, then response time to malicious connections is improved, but risk of false positives affecting legitimate connections increases

Engineering Contradiction:
Improveresponse timeVSAvoidconnection reliability
Core Design Contradiction:
SpeedVSReliability

Solution Approach 1:

The system implements a feedback mechanism where the machine learning model's predictions are continuously refined based on actual outcomes. When connections are blocked or allowed, the system learns from these decisions to improve future predictions, reducing false positives while maintaining fast automated response times for clearly malicious connections.

Inventive Principle:
Principle #23Feedback

Solution Approach 2:

The system adjusts decision thresholds and model parameters dynamically based on confidence levels. For high-confidence malicious predictions, immediate automated blocking occurs with fast response time. For lower-confidence cases, the system may apply different thresholds or require additional verification, balancing speed with reliability based on the specific parameters of each connection attempt.

Inventive Principle:
Principle #35Parameter changes

Data Source

PatentUS12615268B2Detecting malicious behavior from handshake protocols using machine learning
Publication Date: 2026.04.28 UAB 360 IT
  • US12615268B2 patent drawing
  • US12615268B2 patent drawing
  • US12615268B2 patent drawing

AI summary

A method of generating a machine learning model for detecting malicious connections between two or more computing devices includes executing, within a secure operating environment, a plurality of known malicious software applications and a plurality of known non-malicious software applications, generating a dataset of known handshake parameters by monitoring connections between the plurality of known malicious software applications and one or more target servers, and the plurality of known non-malicious software applications and the one or more target servers, training a machine learning model using the dataset of known handshake parameters to predict a maliciousness of a connection between two or more computing devices based on handshake parameters between the two or more computing devices, and distributing the machine learning model to one or more client devices.