JSON Data Validation via Tree Graph Analysis

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

JSON-based services face parsing complexity and security threats due to textual representation, leading to resource-intensive parsing and vulnerability to Deny of Service attacks and fuzzing attacks, necessitating an efficient method for real-time JSON data validation.

Innovation Solution

A method utilizing a tree graph structure to classify JSON data at the TCP level, allowing early detection and validation of JSON properties without requiring heavy computing resources, and transparently protecting servers from potential threats by interrupting TCP connections if invalid data is detected.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If JSON data is parsed using traditional text-based parsing methods, then complete validation of JSON structure and semantics can be achieved, but parsing complexity increases and resource consumption rises

Engineering Contradiction:

ImproveJSON validation accuracyVSAvoidparsing complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent replaces traditional text-based JSON parsing mechanisms with a binary format-based processing system. JSON data is converted to binary representation, which can be processed more efficiently by computer systems, reducing parsing complexity while maintaining validation accuracy. The binary format allows for faster reading and validation without requiring complex text parsing logic.

Inventive Principle:

Principle #28Mechanics substitution (Replace mechanical system)

2Reliability

If traditional JSON parsing is performed on all incoming data, then complete validation can be achieved, but processing speed decreases and resource consumption increases

Engineering Contradiction:

ImproveJSON validation completenessVSAvoidprocessing speed

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The validation process is segmented into multiple stages: first, a quick binary format validation is performed to check basic structure and syntax; only data that passes this initial check proceeds to more comprehensive semantic validation. This segmented approach filters out obviously invalid data early, preserving processing speed while ensuring complete validation for data that requires it.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The system performs partial validation on all incoming JSON data through binary format checking, and excessive (complete) validation only on data that passes the initial check or requires it based on context. This selective validation approach maintains high processing speed for the majority of data while ensuring thorough validation where necessary.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If comprehensive JSON validation is performed, then security threats can be detected, but processing overhead increases and may impact network latency

Engineering Contradiction:

Improvesecurity detection capabilityVSAvoidnetwork latency

Core Design Contradiction:

ReliabilityVSLoss of time

Solution Approach 1:

The system performs preliminary binary format validation and structural checks before complete semantic validation. This preliminary action identifies and filters out malicious or invalid data early in the processing pipeline, detecting security threats without requiring the full overhead of comprehensive validation on every data packet. Network latency is minimized by avoiding unnecessary deep validation of obviously valid or clearly malicious data.

Inventive Principle:

Principle #10Preliminary action

Data Source

PatentEP3211853B1Real-time validation of JSON data applying tree graph properties

Publication Date: 2019.10.30 MITSUBISHI ELECTRIC R&D CENTRE EUROPE BV

Figure 1
Figure 2~3
Figure 4

AI summary

The invention relates to a method of processing in real-time a data stream exchanged between a source and a server over a telecommunications network, the method being carried out by a classifier, said classifier storing a first set of a least one test, and a list of existing TCP connections, the method comprising: - receiving (400) a TCP segment of the data stream, the TCP segment including data payload, said payload comprising at least a portion of a HTTP message; - identifying (401) a first TCP connection to which the TCP segment belongs or creating a new first TCP connection for the received TCP segment; - detecting (406), in the portion of the HTTP message, presence of JavaScript Object Notation, JSON, data ; - in the case where JSON data is detected, building (409) a tree graph based on the JSON data; - performing (410) at least one test of the first set on properties of the built tree graph to determine whether the JSON data is invalid; - if the JSON data is invalid, interrupting (411) the first TCP connection.