Dynamic Query Templates for Unstructured Data Relevancy Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in efficiently and accurately filtering and identifying relevant data from unstructured database files, which are exacerbated by the breadth and scope of modern databases that lack identifying information, leading to resource-intensive and time-consuming operations.
Innovation Solution
A system that dynamically generates queries using pre-defined templates, filters them based on alert types, applies unstructured notes search engines to generate and transform scores, and determines relevancy scores to identify the most relevant data, thereby automating the process and optimizing resource usage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If manual filtering and identification of data in unstructured database files is performed, then accuracy of data identification can be maintained, but computing resource consumption and time required increase significantly
Solution Approach 1:
The system performs preliminary actions by generating multiple candidate queries using pre-defined templates before the actual data identification process. These candidate queries are prepared in advance and then evaluated against the unstructured database files, allowing the system to efficiently narrow down potential matches without manually examining every file, thus reducing time while maintaining accuracy.
Solution Approach 2:
The system introduces an intermediary mechanism in the form of candidate queries that mediate between the search objective and the unstructured database files. Instead of directly searching through all files, the system uses these intermediary queries to filter and identify relevant data, reducing the time required while preserving identification accuracy through systematic evaluation.
2Reliability
If comprehensive search of all unstructured database files is performed, then complete data coverage is achieved, but computing resource consumption increases
Solution Approach 1:
The system segments the comprehensive search process into multiple stages: generating candidate queries from templates, evaluating these candidates against database files, and then performing detailed analysis only on promising matches. This segmentation allows the system to maintain complete data coverage through thorough evaluation while significantly reducing computing resource consumption by avoiding exhaustive search of all files.
Solution Approach 2:
The system performs partial action by generating multiple candidate queries that may exceed the minimum needed for complete coverage. This excessive action in query generation is followed by efficient filtering and evaluation, ensuring that complete data coverage is achieved while the overall computing resource consumption is reduced through the selective and systematic processing of candidates.
3Productivity
If dynamic query generation is implemented, then processing efficiency and adaptability improve, but system complexity increases
Solution Approach 1:
The system implements dynamic query generation using a universal template-based approach where pre-defined query templates serve multiple purposes across different search scenarios. This multi-functionality allows the system to adapt to various search requirements dynamically while relying on a standardized set of templates, thereby improving processing efficiency without proportionally increasing system complexity.
Solution Approach 2:
The system achieves dynamic query generation by changing parameters within pre-defined templates rather than creating entirely new queries. By modifying template parameters based on search requirements, the system improves processing efficiency and adaptability while avoiding the complexity of building and managing a large library of custom query structures.
Data Source
AI summary
Systems, computer program products, and methods are described herein for automatically building dynamic queries for identifying data in unstructured datasets. The present disclosure is configured to receive a plurality of query templates associated with an alert type(s); identify an unstructured database note(s) based on a user identifier; apply the unstructured database note(s) and the plurality of query templates to an unstructured notes search engine; generate, by the unstructured notes search engine, at least one raw score for the at least one unstructured database note and for each alert type(s); transform, by the unstructured notes search engine, the at least one raw score to a relevancy score using a quantile transformation; and determine a highest relevancy score for each of the at least one unstructured database note for the user identifier.


