Data Ingestion Using ML to Decouple Elements
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In computer networks, missing or erroneous data elements can cause bottlenecks and downtime, reducing the throughput and performance of network devices and the overall network.
Innovation Solution
A data ingestion process using a machine learning model to identify and remove dependencies between data elements, allowing them to be processed independently and grouped into data streams for parallel processing, thereby avoiding linkage issues and ensuring accurate value determination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If data elements are linked together to ensure data completeness, then data processing accuracy is improved, but network throughput and device performance deteriorate due to bottlenecks caused by missing data elements
Solution Approach 1:
The patent segments data elements into independent processable units by removing inter-element links. Each data element can be processed independently without waiting for other linked elements, eliminating bottlenecks while maintaining data integrity through alternative association methods in the database.
Solution Approach 2:
The patent extracts the linkage dependency from data elements, separating the need for data association from the processing flow. Data elements are taken out of sequential dependency chains and made independently processable, with associations reconstructed through database relationships rather than processing links.
2Reliability
If data elements are processed sequentially with linkage dependencies, then data accuracy is maintained, but processing time increases causing downtime
Solution Approach 1:
The patent performs preliminary action by pre-establishing database relationships and data associations before processing begins. This allows data elements to be processed independently without real-time linkage checks, maintaining accuracy through pre-configured relationships while eliminating processing delays.
Solution Approach 2:
The patent introduces a database as an intermediary that maintains data relationships without requiring direct element-to-element links during processing. The database mediates between data elements, preserving accuracy through centralized relationship management while enabling independent parallel processing.
3Productivity
If data elements are decoupled for parallel processing, then network throughput is improved, but data relationship integrity may be compromised
Solution Approach 1:
The patent creates copies of data relationships in the database that mirror the original data element associations. These copied relationships allow independent processing of data elements while preserving integrity information in the database, enabling throughput improvement without sacrificing relationship stability.
Solution Approach 2:
The database serves as an intermediary that stores and manages data relationships independently from the processing flow. This allows data elements to be decoupled for parallel processing while the database maintains relationship integrity, acting as a central repository that reconciles independent processing with data consistency.
Data Source
AI summary
A device configured to identify a first link between a value of a first data element in a first plurality of data elements and values of a first set of data elements in a second plurality of data elements and to remove the first link between the first data element and the first set of data elements. The device is further configured to input the data elements into a machine learning model that is configured to output a second link between the first data element and a second set of data elements. The device is further configured to create an entry in a relationship table that identifies the first data element and the second set of data elements. The device is further configured to generate a data stream with the first data element and the second set of data elements and to output the data stream.