A syntax tree extraction method for json data

Abstract

The present invention provides a syntax tree extraction method of JSON data, comprising: sampling the JSON data in a given data set, and recording the JSON record structure tree; traversing all the JSON record structure trees to accumulate and generate a statistical structure tree; traversing the statistics The counter of each node in the structure tree divides the domains in the JSON record into sparse domains and non-sparse domains; classifies the sparse domains with the same characteristics together to form the classification of the domains with the same characteristics; based on non-sparse domains, the same The initial state of the syntax tree is generated by the classification of the domain of the feature: according to the initial state of the syntax tree, all JSON records in the given data set are parsed one by one, and the syntax tree corresponding to the data set is dynamically constructed. The present invention improves the organization efficiency of the syntax tree by classifying the sparse domains in the JSON data record (Record) according to the characteristics; by classifying the domains with the same characteristics, the columnar files of the parsed data are reduced Quantity, thereby improving the efficiency in the subsequent query process.

Description

technical field [0001] The present invention relates to a data processing method, specifically, to the field of JSON data storage and query, more specifically, to a method for extracting a syntax tree (metadata of tree-structured data) of JSON data, and based on this method Data processing method, query method and system. Background technique [0002] With the development of computer network and big data processing technology, traditional relational data can no longer meet the requirements of data definition and use in the network and big data environment. The semi-structured data represented by JSON can not only fully express the object (class) in the programming language, but also modify and expand the original data format according to the format change of the data, so it is used in the actual environment. widely used. [0003] In traditional relational databases, before importing records into the database, users need to pre-define the metadata of records in the form of ...

AI Technical Summary

Problems solved by technology

[0029] The above-mentioned fields are closely related to people's production and life, and require relatively high data processing efficiency. However, in the above-mentioned fields, JSON data records (Record) have low storage and query efficiency, and queries for sparse domains The response is slow, and parsing errors will occur during parsing and calculation for domains with the same domain name but different domain value types

Images