Machine Learning Document Clustering for Targeted Data Identification
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional hardware and software solutions are inadequate, inefficient, and inaccurate in distinguishing targeted data from other data within unstructured content, especially when dealing with large volumes of data spread across multiple sources, making it difficult to identify and retrieve specific data for a particular individual.
Innovation Solution
A method and system utilizing machine-learning models, such as classifier and clustering models, to process feature representations of documents, generate predictions, and cluster similar documents, reducing the number of documents that need to be analyzed by identifying and grouping documents with similar features, thereby facilitating efficient data retrieval and analysis.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional hardware and software solutions are used to identify targeted data in unstructured content, then the system can process data, but the accuracy and efficiency of distinguishing targeted data from other data is inadequate
Solution Approach 1:
The patent segments the data processing task into multiple stages: initial filtering to identify potentially relevant documents, followed by clustering to group similar documents, and finally targeted analysis of specific clusters. This segmentation allows the system to focus computational resources on promising areas while maintaining high accuracy in identifying targeted data.
Solution Approach 2:
The patent introduces machine learning models as intermediary components between the raw unstructured data and the final identification results. These models serve as mediators that learn patterns and relationships in the data, enabling more accurate identification of targeted data while improving processing efficiency through learned heuristics.
2Reliability
If all documents are analyzed individually to ensure complete data retrieval, then accuracy is maintained, but computational load and time requirements increase significantly
Solution Approach 1:
The patent performs preliminary actions by first filtering documents to identify those potentially containing targeted data, then clustering similar documents together before detailed analysis. This preliminary organization allows the system to maintain reliability by ensuring relevant documents are not missed, while significantly reducing the time required by avoiding analysis of clearly irrelevant documents.
Solution Approach 2:
The patent merges similar documents into clusters based on their content characteristics. By combining documents with similar features into groups, the system can analyze representative samples or use clustering results to infer information about multiple documents simultaneously, maintaining completeness while reducing overall processing time.
3Productivity
If machine learning models are used to process document features and generate predictions, then the speed and effectiveness of identifying targeted data is enhanced, but the device complexity increases
Solution Approach 1:
The patent extracts and processes specific feature representations from documents (such as text features, metadata features) separately from the full document content. By taking out and analyzing only the relevant features through machine learning models, the system achieves high processing speed while managing complexity through focused feature analysis rather than processing entire documents.
4Productivity
If clustering is used to group similar documents, then the number of documents requiring detailed review is reduced, but the complexity of the processing system increases
Solution Approach 1:
The patent applies partial action by performing clustering only on documents that have been pre-filtered as potentially relevant, rather than clustering all documents in the system. This approach maintains high efficiency by reducing the number of documents requiring detailed review, while managing complexity by limiting the scope of clustering operations to necessary subsets of data.
Data Source
AI summary
The present disclosure provides methods, systems, computing devices, computing entities, and/or the like for identifying and/or retrieving targeted data found in unstructured documents. In accordance with various aspects, a method is provided that comprises: receiving, a targeted data request identifying a data subject; processing a first feature representation of each document of a plurality of documents using a classifier machine-learning model to generate a prediction that the document contains the targeted data; generating a dataset that comprises each document having a prediction that satisfy a threshold; processing a second feature representation of each document of the dataset using a clustering machine-learning model to identify a document cluster for the document; and providing the document clusters so that an analysis can be performed on each document cluster to eliminate the document cluster as having targeted data and/or identify the targeted data associated with the data subject found in the document cluster.