DBMS Structured Data Provision for Parallel Execution Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Developing analysis applications that combine structured and unstructured data for business analysis is costly and time-consuming due to the need for deep knowledge of both database management systems (DBMS) and parallel distributed processing infrastructures, and the complexity of dynamically changing combining methods based on data formats.
Innovation Solution
A DBMS that communicates with execution nodes managing unstructured data, allowing these nodes to create processed data by combining structured and unstructured data, and controlling the provision of structured data to execution nodes based on reference requests, enabling asynchronous and parallel processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If a parallel distributed processing infrastructure is used for combined analysis, then development costs are reduced, but processing response time increases
Solution Approach 1:
The DBMS pre-processed and prepared structured data in advance, converting it into a format suitable for distributed processing. This preliminary preparation of data structures and formats enables execution nodes to process data more quickly without requiring complex real-time data preparation, thus reducing overall processing response time while maintaining the simplicity of the distributed processing approach
2Productivity
If the DBMS provides structured data to execution nodes based on reference requests, then processing speed is improved, but system complexity increases
Solution Approach 1:
The DBMS acts as an intermediary between the analysis application and execution nodes, managing the provision of structured data based on reference requests. This intermediary role simplifies the interaction protocol by centralizing data management logic in the DBMS, which processes reference requests systematically and provides data to execution nodes in a controlled manner, thereby improving processing speed without proportionally increasing overall system complexity
Data Source
AI summary
A DBMS manages a database including a structured data group capable of communicating with a plurality of execution nodes that manage an unstructured data group and execute parallel distributed processing. Each of the plurality of execution nodes is capable of creating data to be processed based on at least one structured data of the structured data group managed by the DBMS and at least one unstructured data of the unstructured data group managed by the plurality of execution nodes, and performing processing using the data to be processed. Based on control information (information about controlling the provision of two or more structured data designated in a reference request sent to the DBMS), the two or more structured data are acquired by the DBMS from the database, and provided from the DBMS to two or more execution nodes among the plurality of execution nodes.