Transaction Data Parsing with Neural Entity Normalization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems face challenges in parsing and normalizing user account data, particularly transaction data, due to the lack of standardized formats across different entities, leading to messy, inconsistent, and convoluted data that is difficult to interpret and analyze for meaningful insights.
Innovation Solution
A method and system utilizing a masked language model and bidirectional parser, including a transformer encoder and linear-chain CRF, to tokenize and parse transaction data, extracting location and entity information through fuzzy string matching and neural networks, generating normalized transaction data in a standardized format.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional data parsing methods are used, then the system is simpler, but the data parsing accuracy and normalization quality are insufficient
Solution Approach 1:
The patent segments the data parsing task into multiple independent modules: location parsing module, entity parsing module, and normalization module. Each module handles specific aspects of the parsing task, allowing for improved accuracy through specialized processing while maintaining manageable system complexity through modular architecture.
Solution Approach 2:
The patent introduces a masked language model as an intermediary component that bridges the gap between raw transaction data and normalized output. This intermediary processes the data through tokenization and contextual understanding, enabling accurate parsing without requiring complex direct processing rules.
2Productivity
If manual data parsing is used, then the system is simpler, but the processing time and productivity are reduced
Solution Approach 1:
The patent replaces manual mechanical parsing processes with automated neural network-based processing. The masked language model and bidirectional parser automatically tokenize, analyze, and normalize data, dramatically increasing processing speed while the modular design keeps system complexity manageable.
Solution Approach 2:
The system performs self-service through automated parsing and normalization processes. The neural network models automatically learn patterns from transaction data and apply them consistently, eliminating the need for manual intervention and enabling high-speed processing without human operators.
3Reliability
If data is not normalized, then the system is simpler, but the data consistency and quality are poor
Solution Approach 1:
The patent applies parameter changes by transforming raw data into normalized formats through systematic transformations. The normalization module standardizes locations, entities, and transaction details according to predefined schemas, improving data consistency while the modular approach manages processing complexity.
Solution Approach 2:
The patent performs preliminary action by pre-defining normalization schemas and patterns before processing actual data. The system prepares transformation rules and contextual models in advance, enabling consistent and reliable data normalization without requiring complex real-time decision-making during processing.
Data Source
AI summary
Systems and methods for data parsing are disclosed. In one aspect, a method of parsing raw data associated with one or more transactions involves receiving a text string including raw data for a transaction, matching the text string to a plurality of locations within a location corpus to extract location information from the text string, and identifying a candidate entity from the text string based on a similarity score with respect to a plurality of entities within an entity corpus. The method further involves in response to the similarity score of the identified candidate entity being less than a threshold score, generating entity information using the tokens indicative of entity information, and generating normalized transaction data including the extracted location information and one of the identified candidate entity or the generated entity information.