Data Extraction Template Generation for Structured Communications
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing B2C communications, such as emails, lack efficient data extraction methods that can differentiate between non-confidential transient data and confidential or fixed boilerplate information, hindering the ability to automatically extract relevant data from structured communications.
Innovation Solution
A method involving clustering structured communications based on similarities, identifying transient structural paths, classifying them as semantic data types, and generating data extraction templates to extract non-confidential data while ignoring confidential and fixed information, using techniques like regular expressions, xPaths, and bipartite graphs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data extraction is performed on all structured communications without differentiation, then all data including confidential information is extracted, but user privacy and data security are compromised
Solution Approach 1:
The patent segments structured communications into distinct components: transient data (variable content like flight times, product names), fixed data (boilerplate content), and confidential data (personal information). By dividing the extraction process into separate handling for each segment type, the system efficiently extracts only non-confidential transient data while protecting sensitive information, thus resolving the contradiction between extraction efficiency and privacy protection.
Solution Approach 2:
The patent applies different processing qualities to different parts of the communication content. Transient data regions are extracted with high fidelity, fixed boilerplate regions are ignored or minimized, and confidential regions are protected or redacted. This local differentiation allows efficient extraction of useful data while maintaining privacy security, addressing the contradiction between productivity and harmful factors.
2Ease of operation
If traditional data extraction methods are used on structured communications, then simple data can be extracted, but the methods cannot differentiate between transient and fixed data or handle confidential information appropriately
Solution Approach 1:
The patent performs preliminary classification of data regions into transient, fixed, and confidential categories before the actual extraction process. This preliminary action enables the subsequent extraction to automatically differentiate between data types and apply appropriate handling, making the system both easy to operate and highly adaptable to different communication formats and content types.
Solution Approach 2:
The patent implements a dynamic extraction system that adapts to different structured communication formats by learning from training data. The system can dynamically adjust its extraction patterns based on the specific structure and content of each communication type, maintaining ease of operation while achieving high versatility across different data formats and confidentiality levels.
3Loss of information
If all data segments are processed equally during extraction, then comprehensive data collection is achieved, but processing time and computational resources increase significantly
Solution Approach 1:
The patent extracts only the essential transient data elements that provide value (such as flight times, product names, order numbers) while deliberately excluding fixed boilerplate content and confidential information. This selective extraction approach maintains data completeness for useful information while significantly reducing processing time and computational resources by ignoring irrelevant data segments.
Solution Approach 2:
The patent applies partial action by focusing extraction efforts only on transient data regions rather than processing the entire communication uniformly. This partial processing approach achieves sufficient data completeness for business purposes while avoiding the excessive processing time that would result from analyzing every character and element of the structured communication.
Data Source
AI summary
Methods, apparatus, and computer-readable media are provided for generating and applying data extraction templates. In various implementations, a corpus of structured communications such as emails may be grouped into clusters based on one or more similarities between the structured communications. A set of structural paths may be identified from structured communications of a particular cluster. One or more structural paths of the set may be classified as transient wherein a count of occurrences of one or more associated segments of text across the particular cluster satisfies a criterion. One or more transient paths may be assigned a semantic data type and/or a confidentiality designation based on various signals. A data extraction template may be generated to extract, from subsequent structured communications, segments of text associated with transient (and in some cases, non-confidential) structural paths.