Rule-Based Data Cleansing Formatting Control
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data cleansing methods using rule-based parsing struggle to control the ordering of parsed components and delimiting strings, making it difficult to format output data effectively.
Innovation Solution
A computer-implemented method for data cleansing that tokenizes and parses input data using a data dictionary and predefined parsing rules, allowing for precise control over the formatting of output data through formatting rules that include indexes and string literals, enabling flexible output formatting.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If rule-based parsing is used to parse data, then data can be tokenized and parsed according to predetermined rules, but it is difficult to control how the parsed components are ordered or what strings delimit the parsed components
Solution Approach 1:
The patent segments the formatting control into distinct components: parsing rules that handle data decomposition, and separate formatting rules that control output assembly. This allows independent optimization of parsing flexibility and formatting control without mutual interference.
Solution Approach 2:
The patent introduces an intermediary formatting rule layer between the parser and output. This intermediary component receives parsed data and applies formatting transformations, acting as a mediator that decouples the parsing logic from the formatting logic, thereby enabling precise control over output presentation.
2Productivity
If traditional parsing methods are used, then data processing can be performed, but the output data formatting cannot be precisely controlled
Solution Approach 1:
The patent applies preliminary action by pre-defining formatting rules that specify the exact structure, delimiters, and ordering of output data before the actual formatting process occurs. This preliminary specification ensures that when data is processed, the output formatting precision is automatically maintained without requiring post-processing adjustments.
Solution Approach 2:
The patent implements dynamics by making the formatting rules configurable and adaptable. The system can dynamically adjust formatting parameters such as delimiters, field ordering, and data types based on requirements, allowing the same parsing engine to produce precisely formatted output in various formats without sacrificing processing efficiency.
Data Source
AI summary
In one embodiment the present invention includes a computer-implemented method for data cleansing using rule based formatting. The method includes tokenizing and parsing a first input data and a second input data. The method further includes including a first token in a first output data if a first formatting rule component in a formatting rule is a first valid index to said first tokenized input data. The method further includes including a second token in a second output data if said first formatting rule component in the formatting rule is a second valid index to said second tokenized input data. The method further includes formatting said first output data and said second output data according to the formatting rule.