Local Anomaly Detection in Text Data via Sender-Specific Rules
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current anomaly detection methods for electronic messages lack the ability to provide local, independent detection of anomalies in natural language content and fail to offer human-interpretable explanations for flagged messages, especially when dealing with imposters who hijack legitimate sender identities, and they consume bandwidth by communicating with servers for verification.
Innovation Solution
A computer-implemented method and apparatus that detects anomalous segments in electronic messages by creating a clustering model from the sender's previous messages, deriving anomaly detection rules, and generating human-interpretable explanations for anomalies, allowing for local processing and alerting recipients about suspicious messages without relying on external servers.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If anomaly detection methods communicate with external servers for verification, then detection accuracy is improved, but bandwidth consumption increases
Solution Approach 1:
The system creates and maintains local anomaly detection rules derived from the sender's historical message content. The processor can independently detect anomalies by comparing new message segments against these locally stored rules without needing to communicate with external servers, thereby achieving both accurate detection and reduced bandwidth consumption
Solution Approach 2:
The system performs preliminary analysis by creating anomaly detection rules from the sender's previous messages before actual anomaly detection is needed. This pre-processing step stores the learned patterns locally, enabling subsequent rapid independent detection without server communication
2Adaptability or versatility
If anomaly detection methods use global patterns, then detection coverage is improved, but ability to detect individual-specific anomalies deteriorates
Solution Approach 1:
The system creates anomaly detection rules specific to each sender by analyzing their individual historical message content. Each sender has their own set of learned patterns and linguistic characteristics stored locally, enabling the system to detect anomalies specific to that individual's writing style while maintaining the ability to handle multiple senders
3Productivity
If anomaly detection methods flag messages without explanations, then processing speed is improved, but user understanding deteriorates
Solution Approach 1:
The system generates natural language explanations that act as intermediaries between the automated anomaly detection process and the user. When an anomaly is detected, the system retrieves the relevant rule that was violated and presents it in human-readable form, explaining why the message segment is suspicious without sacrificing processing speed
Data Source
Figure 1~3
Figure 4
Figure 5
AI summary
A method, computer program and information processing apparatus for flagging anomalies in text data are proposed. For example, a method of flagging suspicious electronic messages comprises detecting an anomalous segment in natural language content of an electronic message purporting to be sent or written by a particular individual, the detection using anomaly detection rules derived from natural language message content of a set of electronic messages previously sent by the particular individual. For a detected anomalous segment, a segment of text in an electronic message of the set which is closest to text in the anomalous segment, and an associated rule of the anomaly detection rules which is satisfied by the closest segment, are identified. A human-interpretable anomaly explanation based on the anomalous segment and the associated rule is generated.