Automated Application Discovery for Data Confidence Fabric Overlays
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
The process of overlaying a data confidence fabric (DCF) onto an ecosystem is complex and time-consuming, especially in large ecosystems with many applications, and existing methods do not account for pre-existing applications or their vertical use cases.
Innovation Solution
An automated application discovery process intercepts data packets to generate metadata, updates a DCF capabilities graph, and deploys DCF plugins to newly discovered applications, enabling them to process data with confidence metadata.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual insertion of DCF SDK invocations is performed in each application, then DCF functionality can be implemented, but the process becomes complex and time-consuming especially in large ecosystems with many applications
Solution Approach 1:
The system enables self-service by allowing the DCF overlay mechanism to automatically discover applications and deploy plugins without requiring manual intervention. The automated discovery process scans the ecosystem, identifies running applications, and autonomously configures DCF functionality, thereby eliminating the time-consuming manual insertion process while ensuring reliable DCF implementation.
Solution Approach 2:
The invention changes the operational parameters from manual configuration to automated discovery and deployment. By transforming the overlay process from a manual, application-by-application configuration task to an automated system-wide discovery and plugin deployment process, the solution dramatically reduces implementation time while maintaining functional reliability.
2Reliability
If manual overlay of DCF is performed, then DCF functionality is implemented, but the complexity increases especially in large ecosystems with many applications
Solution Approach 1:
The invention introduces an intermediary automated discovery process that mediates between the DCF overlay mechanism and the ecosystem applications. This intermediary automatically scans for applications, determines their DCF capability status, and manages plugin deployment, thereby simplifying the overall process complexity while ensuring reliable DCF functionality implementation across the entire ecosystem.
Solution Approach 2:
The automated discovery process serves multiple functions simultaneously: it scans for applications, identifies running processes, determines DCF capability compatibility, and deploys appropriate plugins. This multi-functional approach consolidates what would otherwise require multiple separate manual operations into a single automated process, reducing overall system complexity.
3Reliability
If traditional overlay methods are used, then DCF is implemented, but pre-existing applications and their vertical use cases are not accounted for
Solution Approach 1:
The system performs preliminary action by conducting automated discovery and assessment of pre-existing applications before deploying DCF functionality. The discovery process scans the ecosystem in advance, identifies running applications, and evaluates their compatibility with DCF, allowing the system to adaptively deploy appropriate plugins or configurations tailored to each application's specific vertical use case requirements.
Solution Approach 2:
The invention applies local quality by tailoring the DCF implementation to each specific application's characteristics and vertical use case. Rather than applying a uniform overlay approach, the system discovers each application's properties and deploys customized DCF plugins or configurations that are specifically adapted to that application's needs, thereby maintaining both reliability and adaptability.
Data Source
AI summary
One example method includes determining that an application is running on a node of a data confidence fabric; performing an investigation to determine if the application is new to the data confidence fabric; when it is determined that the application is new to the data confidence fabric, generating an update comprising metadata that identifies the application and the node on which the application is running; and automatically deploying, to the node of the data confidence fabric, an application-specific plug-in configured to annotate data received from the application that was determined to be new to the data confidence fabric.