Schema Inference via Depth-Dependent Thresholds for JSON Co-reference
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for inferring schema and co-referring types from JSON-formatted serialized data are inadequate, particularly due to the lack of clear hierarchical relationships and the tendency to fail when lower-level nodes with similar values are not considered in context, leading to incorrect merging and mapping issues.
Innovation Solution
A system and method that generates a tree structure from serialized data, computes pairwise similarity between nodes using a breadth-first approach, and applies a depth-dependent threshold to identify co-referring nodes, thereby inferring a Directed Acyclic Graph (DAG) that links children of co-referring nodes to their parents, effectively addressing the schema inference challenge in JSON data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If existing methods perform bottom-up matching of JSON nodes, then simple node similarity can be computed, but hierarchical context information is lost leading to incorrect merging of nodes that should remain distinct
Solution Approach 1:
The patent inverts the traditional bottom-up matching approach by implementing a top-down traversal of the JSON tree structure. This inversion allows the algorithm to first establish hierarchical relationships and context from parent nodes before comparing child nodes, thereby preventing incorrect merging while maintaining computational efficiency through systematic traversal.
Solution Approach 2:
The patent performs preliminary actions by first traversing the tree structure and identifying parent-child relationships before conducting the actual similarity comparison. This preliminary establishment of hierarchical context ensures that when nodes are compared for co-reference, their positional relationships are already known, preventing premature or incorrect merging decisions.
2Device complexity
If nodes with similar values at lower levels are merged based on value similarity alone, then matching simplicity is maintained, but semantic correctness deteriorates due to ignoring ancestor context
Solution Approach 1:
The patent applies local quality by making the matching criterion context-dependent: nodes at different positions in the hierarchy are evaluated with different thresholds and criteria. The similarity threshold is adjusted based on the depth and position of nodes in the tree, allowing more lenient matching at lower levels while maintaining strict semantic correctness through parent context validation.
3Measurement precision
If a strict similarity threshold is applied to identify co-referring nodes, then false positives are reduced, but false negatives increase causing missed co-reference identifications
Solution Approach 1:
The patent implements dynamic thresholding where the similarity threshold is not fixed but adapts based on the hierarchical context of the nodes being compared. The threshold varies depending on the depth in the tree and the semantic type of parent nodes, allowing the system to maintain high precision for critical nodes while being more permissive for less critical ones, thus balancing precision and completeness.
Data Source
AI summary
A system and method provide for identifying coreference from serialized data coming from different services. The method includes generating a tree structure from serialized data. The serialized data includes responses to queries from the different services. The responses each identify a hierarchical relationship between a respective set of objects. Nodes of the tree structure each have a name corresponding to a respective one of the objects. The tree structure is traversed in a breadth first manner and, for each node in the tree structure, a respective pairwise similarity is computed with each of the other nodes of the tree structure. The computed pairwise similarity is compared with a threshold to identify co-referring nodes that refer to a same entity. The threshold is a function of a depth of the node in the tree structure.