Unified HPC Data Aggregation via Proxy Microservice
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current high-performance computing (HPC) platforms face challenges in managing and aggregating access to multiple intensive calculation solutions, such as high-performance calculation, supervised learning, and quantum calculations, due to complexity and redundancy issues, lacking a unified and secure interface for users to access diverse solutions efficiently.
Innovation Solution
A system utilizing microservices, including an agent microservice and calculation microservices, generates aggregated data accessible through a unified interface, reducing data redundancy and enabling secure, transparent access to multiple intensive calculation solutions by coordinating data access and authorization across different platforms.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If multiple separate HPC solutions are accessed individually, then each solution can be optimized for its specific function, but the system complexity increases and data redundancy occurs
Solution Approach 1:
The patent introduces a portal as an intermediary layer between users and multiple HPC solutions. This portal provides a unified access point that aggregates data from various HPC platforms (quantum computing, deep learning, high-performance computing) without requiring users to interact with each solution separately. The portal acts as a mediator that handles authentication, data aggregation, and presentation, thereby reducing system complexity while maintaining access to diverse HPC solutions.
Solution Approach 2:
The portal is designed with universal functionality to access multiple types of HPC solutions through a single interface. It implements multi-functionality by supporting different HPC platforms (quantum computing, deep learning, traditional HPC) and providing common operations (job submission, monitoring, data access) across all solutions. This eliminates the need for separate access mechanisms for each HPC solution, reducing overall system complexity.
2Productivity
If data is aggregated from multiple HPC solutions, then access efficiency improves, but data redundancy and security constraints increase
Solution Approach 1:
The portal extracts only the necessary and relevant data from multiple HPC solutions rather than aggregating all available data. It selectively retrieves information based on user needs and portal configuration, such as extracting job status, results, or metadata while leaving detailed computation data in the source systems. This extraction approach improves access efficiency while minimizing data redundancy.
Solution Approach 2:
The portal implements local quality by customizing data aggregation based on specific user roles, permissions, and requirements. Different users receive different subsets of aggregated data according to their authorization levels and operational needs. This selective aggregation reduces unnecessary data transmission and storage while maintaining efficient access to relevant information.
3Ease of operation
If a unified interface is implemented for multiple HPC solutions, then ease of operation improves, but adaptability to new solutions decreases
Solution Approach 1:
The portal is designed as a dynamic system that can adapt its interface and functionality based on new HPC solutions being added. The architecture allows for dynamic registration of new HPC platforms, automatic detection of their capabilities, and adaptive presentation of their features through the unified interface. This dynamic adaptability maintains ease of operation while accommodating evolutionary changes in the underlying HPC solutions.
Solution Approach 2:
The portal architecture is segmented into modular components that can independently handle different HPC solutions. Each HPC platform is integrated through separate, standardized interface modules that can be added or removed without affecting the core portal functionality. This segmentation allows the unified interface to maintain consistency while adapting to new solutions through modular extensions.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a system and method for generating aggregated data accessible at the level of an aggregated interface (10) of a user client (2) within the framework of unified governance of a plurality of high-performance computing solutions (70), said generation method comprising the steps of: Receiving (210), by a proxy microservice (20), access command information from a user client (2) contained in a message from an aggregated interface (10), Determining (220) data accessible to the user client (2) based at least in part on the access command information, Determining (230) data to be aggregated from accessible data determined by each of the computing microservices (50, 51, 52, 53), Generating (240) aggregated data from the data to be aggregated from the accessible data, and Transmitting (250) the aggregated data, by the proxy microservice (20) to the aggregate user client interface (10).