Transformer Model Transaction Classification via Data Preprocessing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current technologies face challenges in effectively classifying and resolving entities in transaction data, particularly in open banking transactions, due to inefficiencies in data preparation and model training processes.
Innovation Solution
A computer-implemented method involving data preparation operations such as removing numerical characters and accent words, followed by vocabulary training and model training using transformer models like DistilBert and Roberta, to enhance classification and entity resolution in transaction data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If traditional data preparation methods are used for transaction data, then the process is simpler, but the classification accuracy and entity resolution effectiveness deteriorate
Solution Approach 1:
The patent applies preliminary action by performing data preparation operations before model training. Specifically, it removes numerical characters, repeating special characters, and accent words from raw transaction data in advance, and performs vocabulary training to tokenize text and convert it into transformer model specific format. This preprocessing ensures that the classification model receives clean, standardized input, thereby improving classification accuracy and entity resolution effectiveness without increasing the complexity of the model itself.
Solution Approach 2:
The patent introduces an intermediary layer between raw data and the classification model. This intermediary consists of the data preparation pipeline that includes tokenization, removal of unwanted characters, and conversion into transformer model specific format. This intermediary processing layer mediates the transformation of raw transaction data into a format suitable for advanced transformer models, improving accuracy while managing complexity through structured preprocessing.
2Reliability
If advanced transformer models are used for classification, then the entity resolution effectiveness improves, but the model training time and computational resources increase
Solution Approach 1:
The patent applies preliminary action by performing vocabulary training and data preparation before the actual model training. The raw transaction data is preprocessed to remove numerical characters, repeating special characters, and accent words. Text is tokenized and converted into transformer model specific format in advance. This preliminary processing reduces the computational burden during model training, thereby reducing training time while maintaining entity resolution effectiveness.
Solution Approach 2:
The patent changes the parameters of the input data to optimize for transformer model processing. By removing specific character types (numerical, repeating special, accented) and converting text into a specific format, the data parameters are optimized for the transformer model's architecture. This parameter optimization enables faster training convergence and improved entity resolution effectiveness without requiring excessive computational resources or time.
3Measurement precision
If data preprocessing operations are performed to remove characters and tokenize text, then the classification model accuracy improves, but the data processing complexity increases
Solution Approach 1:
The patent applies segmentation by dividing the data processing into distinct, modular operations. The preprocessing pipeline is segmented into separate steps: removal of numerical characters, removal of repeating special characters, removal of accent words, tokenization of text, and conversion into transformer model specific format. Each segmentation handles a specific aspect of data cleaning, making the overall complex process manageable and maintainable while achieving high classification accuracy.
Solution Approach 2:
The patent performs all necessary data preprocessing operations in advance before model training begins. By removing unwanted characters, tokenizing text, and converting to the required format beforehand, the system establishes a clean foundation for training. This preliminary action ensures that the classification model focuses on learning meaningful patterns rather than dealing with data quality issues, thereby improving accuracy while keeping the processing complexity organized and systematic.
Data Source
AI summary
Techniques for training a classification model to improve the classification of open banking transactions are presented. The techniques include receiving raw training data from a data source. The raw training data includes historical transaction data made up of a plurality of individual transactions. The raw training data is input into the classification model. The raw training data is processed by performing a data preparation operation on the raw training data. The data preparation operation includes removing numerical characters, repeating special characters, and accent words from the textual data of each transaction. Vocabulary training is then performed on the processed training data, including tokenizing the text of each transaction and converting the tokenized text into a transformer model specific format. The classification model is then trained using a transformer model, which uses the tokenized text. The trained classification model is then stored in a database.