Bayesian LLM Message Detection for Phishing Email Screening
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing detection systems struggle to identify malicious messages, particularly those generated by large language models (LLMs), as they obscure traditional phishing indicators, making them harder to detect and remediate.
Innovation Solution
A two-step Bayesian inference approach using two machine learning (ML) methods to determine the likelihood of a message being malicious and generated by an LLM, combining scores to enhance detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional detection methods are used to identify phishing emails, then detection simplicity is maintained, but detection accuracy deteriorates against LLM-generated phishing emails
Solution Approach 1:
The detection system is divided into two separate ML methods: the first ML method detects phishing characteristics, and the second ML method detects LLM-generated content characteristics. This segmentation allows each method to specialize in detecting specific features, improving overall detection accuracy while maintaining manageable complexity through modular architecture.
Solution Approach 2:
The system combines outputs from two different ML methods into a composite detection score using Bayesian inference. This composite approach integrates multiple detection signals (phishing indicators and LLM-generated content indicators) to create a more robust and accurate detection mechanism that overcomes the limitations of individual methods.
2Measurement precision
If two ML methods are combined using Bayesian inference, then detection accuracy improves, but computational complexity increases
Solution Approach 1:
Bayesian inference serves as an intermediary mechanism that combines the outputs of two ML methods. It provides a mathematical framework for integrating detection scores from the first ML method (phishing detection) and the second ML method (LLM-generated content detection), allowing the system to achieve improved accuracy through a structured and interpretable combination process.
3Adaptability or versatility
If LLM-generated content detection is added to phishing detection, then detection capability against sophisticated attacks improves, but system complexity worsens
Solution Approach 1:
The second ML method is designed to detect LLM-generated content characteristics that can be applied across various phishing scenarios. This multi-functional approach allows the system to handle both traditional phishing emails and sophisticated LLM-generated phishing emails using a unified detection framework, improving adaptability without requiring entirely separate detection systems.
Data Source
AI summary
A system and method are provided for detecting malicious messages using a two-step Bayesian approach. A discrimination engine determines for each of the messages a first score and a second score. The first score represents a likelihood that the respective messages are malicious messages, and the second score represents a likelihood that they were generated by a machine learning (ML) method, such as a large language model (LLM). Using a combination of these two scores, message with a high probability of being malicious message are discriminated and marked as such. For example, messages for which the first and second scores exceed respective thresholds are marked as suspicious.