Runtime Metadata Capture via Property Graph Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current systems fail to efficiently capture run-time metadata and its provenance and lineage, especially when dealing with conditional logic, leading to inefficiencies and inability to reflect data origins accurately.
Innovation Solution
A processor captures run-time metadata during events, generating a property graph that includes nodes representing input data, agents, and resulting data, enabling the visualization of data lineage and provenance, and facilitating efficient metadata processing and storage.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of time
If metadata is captured before data generation by examining existing code and design documentation, then metadata can be obtained in advance, but the actual run-time metadata including provenance and lineage cannot be captured accurately
Solution Approach 1:
The patent sets up metadata capture mechanisms in advance by instrumenting code with metadata capture points that automatically record provenance and lineage information during runtime execution, combining preliminary preparation with actual runtime capture to achieve both timing efficiency and accuracy
Solution Approach 2:
The patent introduces a metadata capture layer that acts as an intermediary between the execution engine and the data processing logic, automatically recording provenance and lineage information without interfering with the actual data processing operations
2Loss of information
If existing code is examined to capture metadata, then metadata can be obtained, but the process is time-consuming, inefficient, and costly
Solution Approach 1:
The patent implements self-service metadata capture where the system automatically instruments its own code to capture provenance and lineage information during runtime execution, eliminating the need for manual code examination and making the metadata capture process efficient and automated
Solution Approach 2:
The patent replaces manual mechanical code examination with automated runtime instrumentation that programmatically captures metadata during execution, substituting inefficient manual processes with automated computational methods
3Adaptability or versatility
If conditional logic is present in existing code, then flexible data processing is achieved, but systems cannot successfully reflect the provenance and lineage of generated data
Solution Approach 1:
The patent implements feedback mechanisms where metadata capture points continuously monitor and record the actual execution paths taken through conditional logic, capturing the real provenance and lineage information that results from dynamic decision-making during runtime
Data Source
AI summary
A processor captures, during the run-time of a first event, run-time metadata associated with the first event, the run-time metadata comprising a data lineage and a data provenance. The data lineage identifies input data existing before the first event and resulting data of the first event. The data provenance identifies an agent executing the first event. The processor then generates a property graph with the property graph comprising a plurality of nodes. The plurality of nodes comprises a first node, a second node, and a third node. The first node comprises an identification of the input data existing before the first event. The second node comprises an identification of the agent executing the first event, and the second node is coupled to the first node. The third node comprises an identification of the resulting data of the first event, and the third node is coupled to the second node.