Big Data Exchanger for Interconnecting Siloed Data Sources
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current big data systems face challenges in opening, interconnecting, and sharing data due to data islanding, lack of value measurement, inadequate monetization, and inefficient exchange mechanisms, which hinder the utilization of big data's potential value across industries and departments.
Innovation Solution
A system comprising a big data source, a big data exchanger, and a big data target, where the big data source collects and provides data to the exchanger, which processes and exports it to the target according to requests, utilizing components like data production, ownership, brokerage, and ingestion adaptation to manage and process various data types, ensuring interoperability and value extraction.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If big data are collected and stored from multiple sources, then the volume and variety of data increase, but data islanding occurs and sharing capability deteriorates
Solution Approach 1:
The patent introduces a data exchanger as an intermediary component that mediates between data sources and data targets. The data exchanger receives data from multiple sources, processes it through standardized interfaces, and distributes it to various targets, thereby breaking down data silos and enabling cross-industry and cross-departmental data sharing while maintaining the ability to handle large volumes of diverse data.
2Adaptability or versatility
If data processing capabilities are enhanced to handle various data types, then the system can process more complex data, but device complexity increases
Solution Approach 1:
The patent segments the data processing system into distinct functional modules: data sources, data exchanger, and data targets. Each module has specific responsibilities, with the data exchanger containing further sub-components for data reception, processing, and distribution. This segmentation allows the system to handle complex diverse data types while maintaining manageable system complexity through modular design.
Solution Approach 2:
The data exchanger is designed as a universal component that can handle multiple data types (structured, semi-structured, unstructured) and perform various processing operations (collection, cleaning, ETL, segmentation, feature extraction). This multi-functional design enables the system to process diverse data without requiring separate specialized systems for each data type, thereby managing complexity.
3Adaptability or versatility
If data exchange mechanisms are established between different industries, then data interconnection improves, but security and effectiveness of exchange deteriorate without proper standards
Solution Approach 1:
The patent establishes standardized data exchange parameters and interfaces that can be adjusted and configured for different industries and data types. These standards define data formats, protocols, and processing rules that ensure secure and effective exchange while maintaining flexibility to adapt to specific industry requirements, thereby achieving both interconnection and reliability.
Data Source
Figure 1~2
Figure 3~4
Figure 5
AI summary
A method and device for big data exchange is provided. The method includes that : a big data source collects various data and provides the various data to a big data exchanger; the big data exchanger receives the various data imported from the big data source, processes the various data to obtain processed data, and exports the various data and the processed data to a big data target according to a data request of the big data target; the big data target sends the data request to the big data exchanger and receives the various data and the processed data, corresponding to the data request, which are exported from the big data exchanger.