Data query method, device and storage medium
By parsing the query statement into an abstract syntax tree and generating a high-order function, the problem of insufficient flexibility in structured text data query in the existing technology is solved, efficient query of arbitrary structured text data is achieved, and the flexibility and efficiency of the query are improved.
Patent Information
- Application Number
- CN202210925540.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-08-03
- Publication Date
- 2025-09-23
- Estimated Expiration
- 2042-08-03
AI Technical Summary
In the existing technology, user-defined free queries are difficult to implement on structured text data, and query statements can only be processed for data with a specific structure, which lacks flexibility and efficiency.
Parse query statements into abstract syntax trees, generate high-order functions, and access structured text data through row-level data access handles. It supports multiple query statements and text data formats, including SQL and NoSQL, and optimizes the abstract syntax tree to improve query efficiency.
It realizes efficient and convenient query of arbitrary structured text data, improves the flexibility and scope of query, and meets users' diverse query needs.
Smart Images
Figure CN115114341B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of Internet technology, and in particular to a method, device, and storage medium for data query. Background Art
[0002] When performing data queries, network requests are typically sent to retrieve data from remote data sources, which are then processed and analyzed based on specific business needs. The data structure is typically consistent or inferable. The processing logic for this data is relatively fixed and can only serve specific, known needs.
[0003] Because the data is structured in a hierarchical tree, it's difficult to directly perform user-defined, free-form analysis on the data. Conventional designs prohibit highly flexible user-defined queries; users can only query and analyze data within a specific structure using predefined queries provided by the system. Summary of the Invention
[0004] The embodiments of the present application provide a data query method, device and storage medium, which solve the problem in the prior art that text queries cannot be performed based on query statements, and realize efficient query of formatted text data using commonly used query statements.
[0005] In a first aspect, an embodiment of the present invention provides a data query method, which includes: obtaining a query statement and structured text data; parsing the query statement into an abstract syntax tree; attaching the structured text data as a parameter to the abstract syntax tree; compiling the abstract syntax tree to generate a high-order function; executing the high-order function to generate a row-level data access handle; using the row-level data access handle to access the structured text data, obtain query results, and return tabular data of the query results.
[0006] In combination with the first aspect, in a possible implementation method, parsing the query statement into an abstract syntax tree includes: converting the query statement into a tag list according to preset parsing rules; traversing the tag list and generating the abstract syntax tree; performing a logical check on the abstract syntax tree, and encapsulating the syntax tree if the check is successful; and re-parsing if the check is unsuccessful.
[0007] In combination with the first aspect, in a possible implementation, attaching the structured text data as a parameter to the abstract syntax tree includes: assigning a value to a parameter unit of the syntax tree, where the value of the parameter unit is the structured text data.
[0008] In combination with the first aspect, in a possible implementation, the method further includes: optimizing the abstract syntax tree.
[0009] In combination with the first aspect, in a possible implementation method, optimizing the abstract syntax tree includes: loading an equivalent rewriting optimizer, a condition simplification optimizer, and a connection elimination optimizer to respectively rewrite query conditions, simplify query statements, and separate nested connection queries on the abstract syntax tree; traversing the above three optimizers, optimizing the abstract syntax tree, and modifying the abstract syntax tree; performing a logical check on the modified abstract syntax tree, and if the check is successful, encapsulating the syntax tree; if the check is unsuccessful, re-parsing.
[0010] In combination with the first aspect, in a possible implementation, the query statement includes an SQL statement and / or a NOSQL statement.
[0011] In combination with the first aspect, in a possible implementation, the structured text data includes JSON text data.
[0012] In a second aspect, an embodiment of the present invention provides a data query device, which includes: an input module for obtaining query statements and structured text data; a syntax tree construction module for parsing the query statement into an abstract syntax tree; a parameter reference unit for attaching the structured text data as a parameter to the abstract syntax tree; a function generation unit for compiling the abstract syntax tree to generate a high-order function; a handle generation unit for executing the high-order function to generate a row-level data access handle; and a result return unit for using the handle to format data and return tabular data.
[0013] In combination with the second aspect, in a possible implementation method, the syntax tree construction module is specifically used to: convert the query statement into a token list according to preset parsing rules; traverse the token list and generate the abstract syntax tree; perform a logical check on the abstract syntax tree, and encapsulate the syntax tree if the check is successful; if the check is unsuccessful, re-parse.
[0014] In conjunction with the second aspect, in a possible implementation, the parameter referencing unit is specifically used to: assign a value to a parameter unit of the syntax tree, where the value of the parameter unit is the structured text data.
[0015] In combination with the second aspect, in a possible implementation manner, the apparatus further includes an optimization module, and the optimization module is configured to optimize the abstract syntax tree.
[0016] In combination with the second aspect, in a possible implementation method, the optimization module is specifically used to: load an equivalent rewriting optimizer, a condition simplification optimizer, and a join elimination optimizer to respectively rewrite query conditions, simplify query statements, and separate nested join queries on the abstract syntax tree; traverse the above three optimizers, optimize the abstract syntax tree, and modify the abstract syntax tree; perform a logical check on the modified abstract syntax tree, and if the check is successful, encapsulate the syntax tree; if the check is unsuccessful, re-parse it.
[0017] In combination with the second aspect, in a possible implementation manner, the query statement of the input module includes an SQL statement and / or a NOSQL statement.
[0018] In combination with the second aspect, in a possible implementation, the structured text data of the input module includes JSON text data.
[0019] In a third aspect, an embodiment of the present invention provides a data query server, characterized in that it includes a memory and a processor; the memory is used to store computer-executable instructions; the processor is used to execute the computer-executable instructions to implement the method described in the first aspect or any one of the first aspects.
[0020] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores executable instructions, and when a computer executes the executable instructions, it can implement the method described in the first aspect or any one of the first aspects.
[0021] One or more technical solutions provided in the embodiments of the present invention have at least the following technical effects or advantages:
[0022] An embodiment of the present invention employs a data query method, comprising: obtaining a query statement and structured text data; parsing the query statement into an abstract syntax tree; attaching the structured text data as a parameter to the abstract syntax tree; compiling the abstract syntax tree to generate a high-order function; executing the high-order function to generate a row-level data access handle; using the row-level data access handle to access the structured text data, obtaining query results, and returning tabular data of the query results. The query statement is any existing query statement entered by the user, and the structured text data is the data required for the query. The query statement type and the format of the structured text are not restricted, making the query more convenient and adaptable, and enabling the acquisition of more query results. Abstracting the syntax tree and generating a high-order function and a row-level data access handle further transforms the query statement entered by the user into a statement that can query structured text, greatly improving the scope and efficiency of the query. This method effectively solves the problem in the prior art that query statements can only query structured table data but not structured text data, thereby achieving the goal of efficiently and conveniently querying any structured text according to user needs. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments of the present invention or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.
[0024] Figure 1 A flowchart of the steps of the data query method provided in an embodiment of the present application;
[0025] Figure 2 A flowchart of the steps of parsing an abstract syntax tree in the data query method provided in an embodiment of the present application;
[0026] Figure 3 A flowchart of the steps for optimizing the abstract syntax tree provided in an embodiment of the present application;
[0027] Figure 4 A flowchart of a specific embodiment method provided in the embodiments of the present application;
[0028] Figure 5 A schematic diagram of the structure of a data query device provided in an embodiment of the present application;
[0029] Figure 6 A schematic diagram of the structure of the data query server provided in an embodiment of the present application. DETAILED DESCRIPTION
[0030] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts are within the scope of protection of the present invention.
[0031] With the rapid development of informatization, databases are used more and more widely. Query statements are a set of operation commands specially established for databases. In a specific embodiment of the present application, a query is performed on JSON data. Since the structure of JSON data is a tree structure with a hierarchical relationship, it is difficult to directly implement free analysis defined by the end user on the JSON data.
[0032] In the existing method, querying JSON data is done by writing JSONpath, but JSONpath requires a clear understanding of the internal structure of the data. For users, querying JSON data requires mastering the writing method of JSONpath and understanding the internal structure of JSON data. Based on the above problems, the embodiment of the present invention provides a data query method, such as Figure 1 As shown, the method includes the following steps S101 to S106.
[0033] S101, obtaining query statements and structured text data.
[0034] S102: Parse the query statement into an abstract syntax tree.
[0035] S103: Attach the structured text data as a parameter to the abstract syntax tree.
[0036] S104: Compile the abstract syntax tree to generate a high-order function.
[0037] S105: Execute the high-order function to generate a row-level data access handle.
[0038] S106 , using the row-level data access handle to access the structured text data, obtain the query result, and return the table data of the query result.
[0039] The method provided in the embodiments of this application enables querying structured text data using ordinary query statements. When a user enters a query statement and structured text, the parser abstracts the query statement into an abstract syntax tree (ABST). The ABST has multiple structural units, one of which is a parameter reference. Through this unit, the structured text data is assigned as a parameter. The ABST is then compiled to generate high-order functions, ultimately generating row-level data access handles, for querying the text data.
[0040] In step S101, the query statement includes an SQL statement and / or a NOSQL statement. The structured text data includes JSON text data.
[0041] In step S102, the query statement is parsed into an abstract syntax tree, such as Figure 2 The process shown includes the following steps S201 to S203.
[0042] S201: Convert the query statement into a tag list according to preset parsing rules.
[0043] S202, traverse the token list and generate an abstract syntax tree.
[0044] S203, performing a logic check on the abstract syntax tree. If the check succeeds, the syntax tree is encapsulated; if the check fails, re-parsing is performed.
[0045] In a specific embodiment of the present application, an SQL statement is input, and in step S201, when converting into an abstract syntax tree, word segmentation is performed according to the syntax parser, and then syntax parsing is performed according to the parser. When abstracting, fields that the user does not care about are ignored. The abstract syntax tree is a grammatical structure mainly represented in the form of a tree. Each node on the tree represents a structure. Each structure parses the query statement at the current level and generates corresponding key information according to the rules of the query statement. The key information mainly includes the column information, table name information, connection information, filter information and grouping information that the query statement needs to query. The generation of the abstract syntax tree fully describes each step that the query statement entered by the user wants to execute.
[0046] In step S103, the structured text data is attached to the abstract syntax tree as a parameter, including assigning a value to a parameter unit of the syntax tree, where the value of the parameter unit is the structured text data. In this step, one of the structural units of the abstract syntax tree is a parameter reference. Step S103 uses the structured text data as the parameter value and assigns the parameter reference unit a value, thereby forming a complete parameter unit.
[0047] The above method also includes: optimizing the abstract syntax tree. Optimizing the abstract syntax tree, such as Figure 3 The process shown includes the following steps S301 to S303.
[0048] S301 , loading an equivalent rewriting optimizer, a condition simplification optimizer, and a join elimination optimizer, and performing query condition rewriting, query statement simplification, and nested join query separation on the abstract syntax tree, respectively.
[0049] The equivalent rewrite optimizer will traverse the input abstract syntax tree node by node from the outside to the inside. When it is found that the query field of the current node contains "*" or includes redundant columns, the output column of the child node will be simplified and rewritten according to the output column of the current node, so that the simplified output column of the child node just meets the needs of the parent node. The query statement simplification will traverse the input abstract syntax tree node by node from the outside to the inside. When it is found that the conditional expression of the current query node contains comparisons between constants, or the same field is used multiple times for or conditional judgments, etc., corresponding equivalent simplification replacements will be performed. For example, replace 1=1 in the query statement with true. Nested query separation is mainly to separate nested queries as much as possible when multiple structured texts are jointly queried. Step S301 improves query efficiency and reduces memory usage.
[0050] S302: traverse the above three optimizers, optimize the abstract syntax tree, and modify the abstract syntax tree. Each time an optimizer is traversed, the abstract syntax tree is optimized once.
[0051] S303, performing a logic check on the modified abstract syntax tree. If the check succeeds, the syntax tree is encapsulated; if the check fails, re-parsing is performed.
[0052] In a specific embodiment provided in this application, Figure 4 As shown, a SQL statement is entered to query a JSON file. The user prepares a JSON text file. This text can be the result of another network request, the contents of a file, or directly provided through code. The user enters the query SQL statement, which generates an abstract syntax tree. An object is created and the JSON text is placed into it. Combining the object and the abstract syntax tree, a row-level data access handle is generated. The row-level data access handle is used to obtain the column and row information of the JSON data, ultimately obtaining the query result of the entered SQL statement.
[0053] The embodiment of the present invention further provides a data query device 500. Figure 5 As shown, it includes: an input module 501, a syntax tree construction module 502, a parameter reference unit 503, a function generation unit 504, a handle generation unit 505 and a result return unit 506.
[0054] The input module 501 is used to obtain query statements and structured text data. The query statements of the input module 501 include SQL statements and / or NOSQL statements. The structured text data of the input module 501 include JSON text data.
[0055] The syntax tree construction module 502 is used to parse the query statement into an abstract syntax tree. The syntax tree construction module 502 includes: converting the query statement into a token list according to preset parsing rules; traversing the token list and generating an abstract syntax tree; performing a logical check on the abstract syntax tree, encapsulating the syntax tree if the check succeeds; and re-parsing if the check fails.
[0056] The parameter reference unit 503 is used to attach the structured text data as a parameter to the abstract syntax tree. The parameter reference unit 503 includes: assigning a value to the parameter unit of the syntax tree, where the value of the parameter unit is the structured text data.
[0057] The function generation unit 504 is used to compile the abstract syntax tree to generate a high-order function;
[0058] The handle generation unit 505 is used to execute the high-order function and generate a row-level data access handle;
[0059] The result returning unit 506 is used to format the data using the handle and return the table data.
[0060] The data query device 500 provided in the embodiment of the present application also includes an optimization module, which is used to optimize the abstract syntax tree. The optimization module is specifically used to: load the equivalent rewriting optimizer, the condition simplification optimizer, and the join elimination optimizer, respectively, to rewrite query conditions, simplify query statements, and separate nested join queries on the abstract syntax tree; traverse the above three optimizers, optimize the abstract syntax tree, and modify the abstract syntax tree; perform a logical check on the modified abstract syntax tree, and if the check is successful, encapsulate the syntax tree; if the check is unsuccessful, reparse it.
[0061] In the data query device 500 provided by the present application, a query statement is first input. In a specific example provided by the present application, an SQL statement is input. The syntax tree construction module 502 performs abstract syntax tree parsing on the provided SQL statement, modifies and verifies the finally generated abstract syntax tree, and assigns the formatted text to be queried as a parameter to the abstract syntax tree. The abstract syntax tree is compiled and executed to generate a row-level data access handle. Finally, the query statement result is returned in the format required by the user. This effectively solves the problem in the prior art that query statements can only query structured table data but cannot query structured text data, thereby realizing efficient and convenient query of query statements for any structure according to user needs.
[0062] The methods, devices, or modules described in this application can be implemented in the form of computer-readable program code. The controller can be implemented in any appropriate manner. For example, the controller can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) that can be executed by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to implement the same function of the controller in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be considered as structures within the hardware component. Or even, the means for implementing various functions may be considered to be both a software module for implementing the method and a structure within a hardware component.
[0063] The embodiment of the present invention also provides a data query server, such as Figure 6 As shown, the data query server includes a memory 601 and a processor 602. The memory 601 is used to store computer-executable instructions; the processor 602 is used to execute the computer-executable instructions to implement the data query method provided in the embodiment of the present application.
[0064] An embodiment of the present invention provides a computer-readable storage medium, which stores executable instructions. When a computer executes the executable instructions, it implements the data query method provided in the embodiment of the present application.
[0065] The above-mentioned storage medium includes, but is not limited to, random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions.
[0066] Although the present application provides method operation steps as described in the embodiments or flowcharts, more or fewer operation steps may be included based on conventional or non-creative work. The order of steps listed in this embodiment is only one way of executing the order of many steps and does not represent the only execution order. When the actual device or client product is executed, it can be executed in sequence or in parallel according to the method shown in this embodiment or the accompanying drawings (for example, in a parallel processor or multi-threaded processing environment).
[0067] The devices or modules described in the above embodiments can be implemented by computer chips or physical devices, or by products with certain functions. For ease of description, the above devices are described separately by function and are divided into various modules. When implementing this application, the functions of each module can be implemented in the same or multiple software and / or hardware. Of course, a module that implements a certain function can also be implemented by combining multiple sub-modules or sub-units.
[0068] Some modules in the apparatus described herein may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, classes, etc. that perform specific tasks or implement specific abstract data types. The present application may also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communications network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.
[0069] It can be seen from the description of the above implementation methods that those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary hardware. Based on this understanding, the technical solution of the present application can essentially or the part that contributes to the prior art can be embodied in the form of a software product, or it can be embodied through the implementation process of data migration. The computer software product can be stored in a storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a mobile terminal, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present application or certain parts of the embodiments.
[0070] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referenced to each other. Each embodiment focuses on the differences from other embodiments. All or part of this application can be used in many general or special computer system environments or configurations. For example: personal computers, server computers, handheld devices or portable devices, tablet devices, mobile communication terminals, multi-processor systems, microprocessor-based systems, programmable electronic devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.
[0071] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit the present application. Although the present application has been described in detail with reference to the aforementioned embodiments, a person of ordinary skill in the art should understand that the technical solutions described in the aforementioned embodiments can still be modified, or some or all of the technical features therein can be replaced by equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the present application.
Claims
1. A data query method, characterized in that: include: Obtain query statements and structured text data; Parsing the query statement into an abstract syntax tree; Attaching the structured text data as a parameter to the abstract syntax tree; The attaching the structured text data as a parameter to the abstract syntax tree comprises: assigning a value to a parameter unit of the syntax tree, wherein the value of the parameter unit is the structured text data; Compiling the abstract syntax tree to generate a high-order function; Executing the high-order function to generate a row-level data access handle; The structured text data is accessed using the row-level data access handle to obtain a query result, and tabular data of the query result is returned.
2. The method according to claim 1, characterized in that Parsing the query statement into an abstract syntax tree includes: Convert the query statement into a tag list according to preset parsing rules; Traversing the token list and generating the abstract syntax tree; Perform a logic check on the abstract syntax tree. If the check succeeds, encapsulate the syntax tree. If the check fails, reparse the tree.
3. The method according to claim 1, characterized in that Also includes: The abstract syntax tree is optimized.
4. The method according to claim 3, characterized in that The optimizing the abstract syntax tree comprises: Loading an equivalent rewriting optimizer, a condition simplification optimizer, and a join elimination optimizer to respectively perform query condition rewriting, query statement simplification, and nested join query separation on the abstract syntax tree; Traversing the above three optimizers, optimizing the abstract syntax tree, and modifying the abstract syntax tree; Performing a logic check on the modified abstract syntax tree, and encapsulating the syntax tree if the check succeeds; and re-parsing if the check fails.
5. The method according to claim 1, wherein The query statement includes an SQL statement and / or a NOSQL statement.
6. The method according to claim 1, characterized in that The structured text data includes JSON text data.
7. A data query device, characterized in that: include: Input module, used to obtain query statements and structured text data; A syntax tree building module, configured to parse the query statement into an abstract syntax tree; a parameter referencing unit, configured to attach the structured text data as a parameter to the abstract syntax tree; The attaching the structured text data as a parameter to the abstract syntax tree comprises: assigning a value to a parameter unit of the syntax tree, wherein the value of the parameter unit is the structured text data; A function generation unit, configured to compile the abstract syntax tree to generate a high-order function; A handle generation unit, configured to execute the high-order function and generate a row-level data access handle; The result returning unit is used to format the data using the handle and return the table data.
8. A data query server, characterized in that: including memory and processor; The memory is used to store computer-executable instructions; The processor is configured to execute the computer-executable instructions to implement the method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores executable instructions, and when a computer executes the executable instructions, the method according to any one of claims 1 to 6 can be implemented.