Contextually-Anomalous Sentence Detection Using BiLSTM-Attention
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection techniques in documents are inadequate as they require pre-defined rule sets for context information extraction, leading to insufficient reflection of specific context and high maintenance costs, and struggle with complex context classification.
Innovation Solution
An apparatus and method using a deep neural network to detect anomalous sentences by encoding document data into vectors, converting them into context embedding vectors, and calculating result values to determine contextual anomalies without relying on pre-defined rule sets, allowing for direct extraction of context information from data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of information
If pre-processing is performed to assign metadata reflecting context information, then additional context information is provided, but additional costs are required for maintenance and repair of dictionary or custom rules
Solution Approach 1:
The patent replaces manual rule-based context extraction with a deep learning model (BiLSTM-Attention) that automatically learns context representations from data. The model substitutes the mechanical process of manually creating and maintaining dictionaries and custom rules with an automated neural network that adapts to context patterns without human intervention.
Solution Approach 2:
The deep learning model performs self-service by automatically extracting and representing context information from input sentences without requiring external dictionaries or manually crafted rules. The Attention mechanism enables the model to self-determine which words or phrases are most relevant to context, eliminating the need for human-maintained context databases.
2Measurement precision
If explicit classification of context information is performed, then context categories are defined and anomalies detected, but it is difficult to create learning data for classifying complex context
Solution Approach 1:
The patent transforms the discrete classification task into a continuous representation learning problem. Instead of forcing sentences into predefined context categories, the BiLSTM-Attention model generates continuous context embedding vectors that capture nuanced contextual relationships. This parameter transformation from discrete classes to continuous vectors enables the model to handle complex contexts without requiring exhaustive labeled training data for each category.
Solution Approach 2:
The model adds a dimensional transformation by converting sentences into high-dimensional embedding vectors through the Attention mechanism. This dimensional elevation allows the model to represent complex contextual relationships in a continuous vector space, avoiding the need to explicitly define and classify every possible context category.
3Reliability
If rule-based context extraction is used, then specific context information can be reflected, but the method is insufficient to sufficiently reflect specific context information and requires additional maintenance costs
Solution Approach 1:
The patent introduces dynamics by using a trainable neural network model that adapts its context extraction behavior based on learned patterns from training data. Unlike static rule-based systems, the BiLSTM-Attention model dynamically adjusts its attention weights and context representations to accurately identify anomalous sentences, improving reliability without requiring complex manual rule configuration.
Data Source
AI summary
Disclosed are an apparatus and a method for detecting whether an anomalous sentence having a context different from that of other sentences exists in a document. The apparatus for detecting a contextually-anomalous sentence in a document according to the present invention includes: a sentence encoder for encoding individual sentences constituting document data by means of a predetermined rule (function) to generate encoding vectors; a context embedder neural network for converting the generated encoding vector into embedding vectors corresponding thereto; and a context anomaly detector neural network for detecting whether an anomalous sentence exists in the converted document data.