Neural Network Data Type Splitting in Character Strings
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing technologies face challenges in efficiently detecting and splitting multiple data types within character strings stored in data fields of a data file, especially when data types are concatenated without delimiters.
Innovation Solution
The use of a neural network model trained on a dataset of various data types and word samples allows for the detection and splitting of data types within character strings, enabling accurate matching of words to their respective data types.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional methods are used to detect and split data types in character strings without delimiters, then the process becomes complex and time-consuming, but using neural network models requires significant computational resources and training time
Solution Approach 1:
The patent applies preliminary action by pre-training the neural network model on a comprehensive dataset containing various data types and their characteristics before actual data processing. This training phase (steps 170-190) prepares the model in advance to quickly and accurately detect and split data types during runtime without requiring complex rule-based systems or extensive computational resources during the actual processing phase.
2Measurement precision
If manual data type detection and splitting is performed, then accuracy can be maintained, but the process becomes extremely time-consuming for large datasets
Solution Approach 1:
The patent replaces manual mechanical data type detection with an automated neural network-based system. The neural network model (steps 170-210) learns patterns and characteristics of different data types during training and automatically detects and splits data types in character strings, achieving both high accuracy and efficient processing of large datasets without manual intervention.
3Ease of operation
If delimiters are added to separate data types in character strings, then data type detection becomes easier, but the data structure becomes more complex and requires additional storage
Solution Approach 1:
The patent introduces an intermediary neural network model that acts as a mediator between the raw character strings and the data type detection process. Instead of modifying the data structure by adding delimiters, the neural network analyzes the character strings directly and identifies data type boundaries based on learned patterns, maintaining data compactness while enabling accurate detection.
Data Source
AI summary
A method for training a neural network model includes generating a training dataset with a plurality of data types and word samples belonging to each data type. A plurality of character strings stored in a plurality of data fields in a first data file are received where the plurality of character strings includes at least one word belonging to at least one data type in the plurality of data types. The at least one word from each of the plurality of character strings in each of the data fields are split and matched to the at least one data type using the neural network model. An ad hoc second data file with a plurality of data vectors is constructed based on a user selection of data field labels where each data vector includes words matched to a data type with a respective data field label.