Graph Analysis for Unstructured Text Querying
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing text analysis systems struggle to accurately identify corporate, business, and industry risks such as regulatory, privacy, and cybersecurity risks in unstructured text due to issues like transcription errors, OCR errors, spelling variations, and the inability to handle 'fuzziness' in queries and results.
Innovation Solution
The system employs graph analysis and machine learning to identify items in unstructured text by providing a query string, defining relationships between terms, generating a graph of matches, and determining groups of nodes that match the query string, thereby mapping relevant text in the unstructured text.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If exact term matching is used for querying unstructured text, then the simplicity of the matching process is maintained, but the detection accuracy deteriorates due to transcription errors, OCR errors, spelling variations, and synonyms
Solution Approach 1:
The system transforms the query from exact term matching to a parameter-based approach by defining relationships between terms (such as proximity, order, and logical connections). This allows the system to match concepts rather than exact strings, accommodating transcription errors, OCR errors, spelling variations, and synonyms while maintaining query simplicity through a declarative relationship framework.
Solution Approach 2:
The patent introduces an intermediary layer between the query and the text search process. This intermediary consists of the relationship definitions that mediate between the query terms and the target text, enabling flexible matching through defined relationships (proximity, order, logical connections) rather than direct exact matching.
2Difficulty of detecting and measuring
If regular expressions are used for text parsing, then the ability to handle pattern matching is improved, but the flexibility and ease of implementation deteriorate due to limited applicability for unstructured content and complexity of debugging
Solution Approach 1:
The system segments the query into individual terms and defines relationships between them separately. This segmentation allows each term to be matched independently in the unstructured text, with relationships (proximity, order, logical connections) defining how they connect. This approach provides greater flexibility for unstructured content compared to rigid regular expressions while maintaining pattern matching capability.
Solution Approach 2:
The relationship definitions in the system are dynamic and configurable, allowing users to define custom proximity distances, ordering requirements, and logical connections based on the specific query needs. This dynamic approach provides adaptability for various unstructured content types, unlike static regular expression patterns that require complex rewriting for different scenarios.
3Adaptability or versatility
If multiple phrases with similar meanings are included in the query, then the comprehensiveness of the search is improved, but the accuracy of results deteriorates due to inability to exactly match search terms
Solution Approach 1:
The relationship-based query system provides a universal framework that can handle multiple phrases with similar meanings through a single unified approach. By defining relationships between terms (proximity, order, logical connections), the system can match any phrase that satisfies the relationship constraints, regardless of exact wording variations, synonyms, or transcription errors, thereby maintaining both comprehensiveness and accuracy.
Data Source
AI summary
Systems and methods for enhanced rule-based querying of unstructured text using graph analysis are provided. Some embodiments apply machine learning to the analysis of unstructured text to identify regulatory, privacy, and cybersecurity risks. In some embodiments, a query string is provided to identify desired content in the unstructured text. The query string is tokenized and the tokenized parts are converted into nodes of a query graph and relationships are defining between the nodes. After identifying matches between terms in target text with nodes of the query graph, a graph is generated using matched terms as nodes. Based on the defined relationships, a determination is made whether any group of nodes on the graph satisfy the query string. Any matched nodes are mapped to the target text, and this information is provided to an analyst or reviewer for further consideration or action.