Address Extraction from Communications Using Frequency Scoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data mining techniques face challenges in efficiently extracting and identifying addresses or strings from communications, particularly in determining the correspondence of data to a person across large datasets, due to the complexity and variability of data formats and sources.
Innovation Solution
A method involving a data processing system that assigns scores to strings based on frequency of usage, determines total sums for sequences, and extracts sequences exceeding a threshold value to identify addresses, utilizing Bayes' theorem and confidence scoring to determine the likelihood of strings being part of an address, and employs a repository to store and analyze observations for accurate data correlation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data mining techniques are used to extract addresses from large communication datasets, then data extraction capability is improved, but processing complexity and time consumption increase
Solution Approach 1:
The patent segments the address extraction process into distinct components: string scoring based on frequency analysis, sequence formation from scored strings, and threshold-based filtering. This segmentation allows each component to be optimized independently, reducing overall processing complexity while maintaining extraction capability.
Solution Approach 2:
The patent performs preliminary scoring of all strings in the communication dataset before actual address extraction. By pre-calculating frequency-based scores and storing them, the system avoids repeated analysis during extraction, significantly reducing processing time and complexity for subsequent queries.
2Measurement precision
If frequency analysis is applied to all strings in communications, then address identification accuracy is improved, but computational resources required increase
Solution Approach 1:
The patent changes the parameter of analysis from examining all possible string combinations to scoring individual strings based on their frequency of appearance in address contexts. This parameter transformation reduces computational resources while maintaining identification accuracy through the use of confidence thresholds.
Solution Approach 2:
The patent discards strings that fall below a predetermined confidence threshold after frequency analysis, avoiding further processing of unlikely candidates. This selective discarding recovers computational resources that would otherwise be wasted on analyzing non-address strings, while maintaining high accuracy through threshold optimization.
3Measurement precision
If confidence scoring with predetermined thresholds is used to filter extracted sequences, then data accuracy is improved, but the number of potential false negatives increases
Solution Approach 1:
The patent applies partial filtering by using confidence thresholds that capture the majority of true addresses while accepting some margin of error. Rather than requiring absolute certainty, the system uses predetermined thresholds that balance accuracy with minimizing false negatives, extracting sequences that meet the threshold while allowing for reasonable uncertainty.
Data Source
AI summary
Systems and methods to extract a string from a communication. A method includes: receiving a communication comprising a plurality of strings; assigning a score to each of the strings, wherein the score assigned to each of the strings corresponds to a frequency of usage of the respective string for a first function relative to an overall frequency of usage of the respective string; determining a respective total sum for each of a plurality of sequences in the communication, the respective total sum determined as a sum of the scores for each string in the respective sequence; and extracting a first sequence of the sequences from the communication based on the total sum for the first sequence. In one embodiment, the total sum includes an additional score for each of a starting word and an ending word of the first word sequence, wherein each respective additional score is associated with a probability that the starting (or ending) word is used as the first (or last word) of an address.