A data query method and device, electronic equipment, and storage medium

Abstract

Embodiments of the present application disclose a data query method, comprising: obtaining a memory table, the memory table being obtained by parsing structured data in an HTTP response; extracting a keyword in a SQL statement sent by a visualization system; and determining a query result in the memory table based on the keyword. Embodiments of the present application also provide a data query device, an electronic device and a storage medium.

Description

Technical Field

[0001] This application relates to the field of information technology, and relates to, but is not limited to, a data query method and apparatus, electronic equipment, and storage medium. Background Technology

[0002] In related technologies, relational databases or relational data warehouses store large volumes of data, and data analysis and visualization systems can provide SQL analysis and visualization for these large volumes. However, for small to medium volumes of data that are not stored in relational databases or relational data warehouses, dynamic analysis is not possible due to the lack of SQL analysis and visualization systems; only static displays are possible. Summary of the Invention

[0003] This application provides a data query method and apparatus, an electronic device, and a storage medium.

[0004] The technical solution of this application embodiment is implemented as follows:

[0005] On one hand, embodiments of this application provide a data query method, the method comprising: obtaining a memory table, the memory table being obtained by parsing structured data in an HTTP response; extracting keywords from an SQL statement sent by a visualization system; and determining query results in the memory table based on the keywords.

[0006] In another aspect, embodiments of this application provide a data query device, the device comprising an acquisition module, an extraction module, and a query module, wherein: the acquisition module is used to acquire a memory table, the memory table being obtained by parsing structured data in an HTTP response; the extraction module is used to extract keywords from an SQL statement sent by a visualization system; and the query module is used to determine query results in the memory table based on the keywords.

[0007] In another aspect, embodiments of this application provide an electronic device, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the program to implement the steps in the above-described method.

[0008] In another aspect, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in the above-described method.

[0009] The beneficial effects of the technical solutions provided in this application include at least the following:

[0010] In this embodiment, a memory table is obtained, which is obtained by parsing the structured data in the HTTP response; keywords are extracted from the SQL statement sent by the visualization system; and query results are determined in the memory table based on the keywords. Thus, on the one hand, after obtaining the HTTP response, the memory table storing the structured data can be obtained by parsing the structured data in the HTTP response. When it is necessary to query the structured data in the HTTP response, the query results can be obtained by extracting the keywords from the SQL statement and querying the memory table based on the keywords. This avoids storing the structured data in the HTTP response in a relational database. In the case of querying structured data, the database is first connected, and then the SQL statement is used to query the database. After obtaining the query results from the database, the query results are returned. After the data query transaction ends, the data stream is closed, and other complex data query processes are streamlined, thus simplifying the query process for small amounts of data in the HTTP response and improving data query efficiency.

[0011] On the other hand, based on the keywords, the query results are determined in the memory table. Thus, by determining the query method for the memory table according to different keywords, using different query methods as different query models, and using the set of models composed of different query models as the SQL parser, this method can reduce the coupling between SQL statements and structured data, which is beneficial for dynamic analysis of other small datasets, and also facilitates the visualization system in displaying the results of dynamic analysis. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort, wherein:

[0013] Figure 1 A flowchart illustrating a data query method provided in an embodiment of this application;

[0014] Figure 2 A flowchart illustrating a data query method provided in an embodiment of this application;

[0015] Figure 3 A flowchart illustrating a data query method provided in an embodiment of this application;

[0016] Figure 4 A flowchart illustrating a data query method provided in an embodiment of this application;

[0017] Figure 5 This is a schematic diagram of the composition structure of a data query device provided in an embodiment of this application;

[0018] Figure 6 This is a schematic diagram of the hardware entity of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0020] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0021] It should be noted that the terms "first, second, and third" used in the embodiments of this application are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, and third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0022] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which the embodiments of this application pertain. It should also be understood that terms such as those defined in general dictionaries should be understood to have a meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined as herein.

[0023] Figure 1 This is a flowchart illustrating a data query method provided in an embodiment of this application, such as... Figure 1 As shown, the method includes at least the following steps:

[0024] Step S101: Obtain the memory table, which is obtained by parsing the structured data in the HTTP response;

[0025] It should be understood that the structured data includes intrinsic keys and corresponding numerical values, and the format of the intrinsic keys and numerical values ​​is fixed. For example, in an HTTP response, the structured data can be an intrinsic key `name` and its corresponding numerical value `class1`, or it can be an intrinsic key `price` and its corresponding numerical value `2`.

[0026] In one feasible approach, parsing the structured data in an HTTP response to obtain an in-memory table can be achieved by storing the structured data in the HTTP response as an in-memory object and then converting the in-memory object into an in-memory table.

[0027] In one possible implementation, the memory table can be two-dimensional. The memory table may include metadata information describing attributes, or it may include data information stored in memory objects. For example, structured data is class1,2; class2,2; class2,2; class4,8; after parsing, a two-dimensional table is obtained. The data information in the two-dimensional table includes: Metadata information includes column names and the data type of those columns: [{"columnName":"name","columnType":"String",},{"columnName":"price","columnType":"Integer",}].

[0028] Step S102: Extract keywords from the SQL statement sent by the visualization system;

[0029] In one possible implementation, the visualization system can be a reporting system, which may include charts such as pie charts, bar charts, line charts, and column charts to represent data changes. For example, the visualization system can be a presentation (PowerPoint, PPT) or a spreadsheet (EXCEL).

[0030] In one feasible approach, the visualization system can generate SQL statements via drag-and-drop, send these statements to the data processing system, analyze the data based on the received SQL statements, and return the results to the visualization system, which then displays the results. For example, in the visualization system, dragging the name field to the X-axis and the price field to the Y-axis, and selecting the SUM function, generates the SQL statement: `select name, sum(price) from example group by name`. This SQL statement is then sent to the data processing system, which groups the data in the `example` table by name and sums the results by price, obtaining the analyzed results. These results are then returned to the visualization system, which displays the grouped and summed results as a chart.

[0031] In one possible approach, after receiving the SQL statement from the visualization system, the data processing system can extract keywords by performing string operations on the SQL statement.

[0032] In one feasible approach, after executing step S102, the data processing system extracts keywords from the SQL statement sent by the visualization system and can then establish a query model by saving these keywords. When the data processing system receives the next SQL statement, it can perform string operations on the SQL statement to extract keywords.

[0033] In one feasible approach, multiple query models can be combined into an SQL parser. When the data processing system receives the next SQL statement, it submits the SQL statement to the parser, which then transforms the SQL statement into a query model. This query model performs string manipulation on the SQL statement to extract keywords.

[0034] Step S103: Determine the query result in the memory table based on the keyword.

[0035] In one possible implementation, when the keywords include grouping aggregation, the query results can be obtained by performing grouping aggregation calculations on the in-memory table. In another possible implementation, the grouping set calculations can be performed using a query engine within a data processing system.

[0036] In this embodiment, a memory table is obtained, which is obtained by parsing the structured data in the HTTP response; keywords are extracted from the SQL statement sent by the visualization system; and query results are determined in the memory table based on the keywords. Thus, on the one hand, after obtaining the HTTP response, the memory table storing the structured data can be obtained by parsing the structured data in the HTTP response. When it is necessary to query the structured data in the HTTP response, the query results can be obtained by extracting the keywords from the SQL statement and querying the memory table based on the keywords. This avoids storing the structured data in the HTTP response in a relational database. In the case of querying structured data, the database is first connected, and then the SQL statement is used to query the database. After obtaining the query results from the database, the query results are returned. After the data query transaction ends, the data stream is closed, and other complex data query processes are streamlined, thus simplifying the query process for small amounts of data in the HTTP response and improving data query efficiency.

[0037] On the other hand, based on the keywords, the query results are determined in the memory table. Thus, by determining the query method for the memory table according to different keywords, using different query methods as different query models, and using the set of models composed of different query models as the SQL parser, this method can reduce the coupling between SQL statements and structured data, which is beneficial for dynamic analysis of other small datasets, and also facilitates the visualization system in displaying the results of dynamic analysis.

[0038] Figure 2 This is a flowchart illustrating a data query method provided in an embodiment of this application, such as... Figure 2 As shown, the method includes at least the following steps:

[0039] Step S201: Obtain the pre-defined interface and the HTTP response; the HTTP response includes structured data;

[0040] In one possible implementation, the interface is used to extract structured data from an HTTP response. For example, the content of the HTTP response includes: {"success":true,"code":200,"data":[{"name":"class1","price":2,},{"name":"class2","price":2,},{"name":"class2","price":2,},{"name":"class4","price":8}],"message":null}; in this HTTP response, the four sets of names and prices included in the "data" keyword are structured data.

[0041] Step S202: Based on the pre-defined interface, parse the structured data to obtain a memory table, which is obtained by parsing the structured data in the HTTP response;

[0042] In one possible implementation, step S202, based on the pre-defined interface, parses the structured data to obtain a memory table, including:

[0043] Step S2021: Based on the pre-defined interface, determine the implementation class of the interface;

[0044] In one possible implementation, based on the predefined interface, the implementation class of the interface can be determined by overriding the abstract methods in the predefined interface. In another possible implementation, the implementation class of the interface can be a static file. When a method in this static file is called, it performs the functions of extracting structured data from the HTTP response, converting the structured data into a memory object, and then converting the memory object into a memory table.

[0045] In one possible implementation, the method further includes: the data processing system obtaining the implementation class of the interface; in another possible implementation, obtaining the implementation class of the interface can be accomplished by the data processing system identifying the implementation class file of the interface submitted by the user.

[0046] Step S2022: Use the implementation class to parse the structured data to obtain a memory object;

[0047] In one possible implementation, the memory object is stored as a linked list, the predefined interface includes an abstract method for reading the memory table, and the implementation class includes methods for overriding the abstract method.

[0048] For example, the values ​​of the multiple memory objects include: class1, 2, class2, 2, class2, 2, class4, 8. During the storage of these memory objects, they are stored in the following form:<class1,2> ,<class2,2> ,<class2,2> ,<class4,8> The predefined interface may include: List <Map<String,Object> The `readTable(String data)` abstract method. This abstract method can be overridden by implementing a class to extract structured data from the HTTP response and convert that structured data into an in-memory object.

[0049] Step S2023: Convert the memory object into a memory table.

[0050] In one possible implementation, step S2022, parsing the structured data using the implementation class to obtain a memory object, includes: dynamically constructing a source code object based on the implementation class; dynamically loading the source code object; dynamically compiling the dynamically loaded source code object to obtain a compiled implementation class; and parsing the structured data using the compiled implementation class to obtain a memory object.

[0051] For example, firstly, an HTTP Reader interface is predefined, which is used to extract structured data from HTTP and convert it into an in-memory object. For example, the interface definition is: List <Map<String,Object> >readTable(String data). Then, write the implementation class for configuring and submitting the HTTP Reader interface. Finally, convert the structured data into an in-memory object according to the implementation methods in the implementation class.

[0052] Step S203: Extract keywords from the SQL statement sent by the visualization system;

[0053] Step S204: Determine the query result in the memory table based on the keyword.

[0054] In this embodiment, based on the pre-defined interface, the implementation class of the interface is determined; the structured data is parsed using the implementation class to obtain a memory object; and the memory object is converted into a memory table. Thus, structured data can be converted into a memory table through the pre-defined interface, facilitating the querying of structured data in HTTP responses, providing the ability to dynamically analyze structured data in HTTP responses, and enabling visualization systems to dynamically present structured data in HTTP responses. This solves the problem that structured data in HTTP responses can only be statically displayed, and improves the user experience of analyzing structured data in HTTP responses through visualization systems.

[0055] Figure 3 This is a flowchart illustrating a data query method provided in an embodiment of this application, such as... Figure 3 As shown, the method includes at least the following steps:

[0056] Step S301: Obtain the memory table, which is obtained by parsing the structured data in the HTTP response;

[0057] Step S302: Determine the query result corresponding to the SQL statement in the cache;

[0058] It should be understood that the data stored in the memory table is the structured data in the HTTP response. After multiple queries on the data in the memory table using SQL statements, the data processing system will store multiple query results in a cache. When querying the same data using the same SQL statement, the query operation can be simplified and the efficiency of data retrieval can be improved by directly reading the query results from the cache.

[0059] Step S303: If the query result is not present in the cache, extract Data Query Language (DQL) keywords from the SQL statement;

[0060] It should be understood that if the query result is not present in the cache, the data processing system needs to re-query the data in the memory table according to the query operation defined in the SQL statement to obtain the query result. During the data query process, since the memory table is only a single table and is not stored in the database, it cannot automatically execute the query operation based on the SQL statement. It is necessary to extract keywords from the SQL statement and query the data according to the operations defined by the keywords to obtain the query result.

[0061] It should be understood that in the data query process, since most of the SQL statements involved are related to the SELECT keyword, and the SELECT statement belongs to the Data Query Language (DQL), extracting the DQL keywords from the SQL statement can determine the operation that the SQL statement needs to perform in the memory table.

[0062] In one possible implementation, the basic structure of the Data Query Language (DQL) consists of a query block composed of a SELECT clause, a FROM clause, and a WHERE clause: SELECT <field name table> FROM WHERE <query condition>.

[0063] Step S304: Determine the query result in the memory table based on the keyword.

[0064] In one possible implementation, step S304, determining the query result in the memory table based on the keyword, includes:

[0065] Step S3041: Determine the execution order of the keywords;

[0066] Step S3042: Based on the execution order, process the memory table according to the key to obtain the processed memory table;

[0067] Step S3043: The processed memory table is determined as the query result.

[0068] In one possible implementation, the keyword includes at least one of the following first operations: filtering, grouping, aggregation, and joining, wherein step S3041, determining the execution order of the keyword, includes:

[0069] If the keyword includes at least one of the first operations, a first priority order is determined as the execution order of each of the first operations; the first priority order includes the execution order of the filtering, grouping, aggregation, and joining operations in sequence.

[0070] If the keyword includes other second operations in addition to at least one of the first operations, the second priority order is determined as the execution order of the operations in the keyword; the second priority includes the execution order of filtering, grouping, aggregation, joining and other second operations in sequence;

[0071] If the keyword includes, but does not include, at least one of the first operations, and only includes other second operations, a third priority order is determined as the execution order of the keyword; the third priority includes the execution order of other second operations determined according to a preset priority.

[0072] In one possible implementation, the operation corresponding to the keyword includes a first operation consisting of filtering, grouping, aggregation, and joining. The execution order of the first operation can be: filtering, grouping, aggregation, and joining are performed sequentially, and the data in the memory table is filtered, grouped, aggregated, and joined sequentially according to the execution order of the first operation, and the joined data is determined as the query result.

[0073] For example, 1) the memory table (two-dimensional table A) is filtered according to the filtering conditions specified by the filter keyword "filters" to obtain the filtered Memory Row Table. 2) Grouping based on the grouping keyword "dimensions": The first step is to obtain the intermediate results "Memory" from the two groups. and The second step is to aggregate tmp1 and tmp2 using the aggregation keyword `sum`, obtaining two intermediate results: [class2 4] and [class4 8]. The third step is to join the aggregated memory tables using the join keyword `union`, resulting in the joined memory table. Will This is confirmed as a query result, representing the total price for different names.

[0074] In one possible implementation, if the key does not include any of the first operations, the memory table is queried in the order of execution of the other second operations.

[0075] For example, the keyword includes the insertion keyword select into, and the second priority order can be: execute the select into operation, and determine the result of select into as the query result.

[0076] In one possible implementation, when the keywords include filtering, grouping, aggregation, joining, and insertion operations, the third priority order is determined as follows: filter, grouping, aggregation, joining, and insertion operations are executed in sequence.

[0077] For example, a query is performed on the in-memory table based on the filtering keyword `filters`, the grouping keyword `dimensions`, the aggregation keyword `sum`, the join keyword `union`, and the insertion keyword `select into`. After executing the join operation, the result of the join operation is obtained. Insert the class1,6 data into the result of the join operation to obtain The result of this insert operation is the query result.

[0078] In this embodiment, the execution order of the keywords is determined; based on the execution order, the memory table is processed according to the keywords to obtain a processed memory table; and the processed memory table is determined as the query result. Thus, the query method for the memory table can be determined according to different keywords and execution orders. When different query methods are viewed as different query models, this method can reduce the coupling between the query model and structured data. When the set of different query models is viewed as an SQL parser, this method can reduce the coupling between SQL statements and structured data. The SQL parser can analyze the query operations to be executed and the data to be queried, which is more conducive to the analysis of other small amounts of data and improves the applicability of the method.

[0079] In related technologies, relational databases or relational data warehouses store large volumes of data, and data analysis and visualization systems can provide SQL analysis and visualization for these large volumes. However, for small to medium volumes of data that are not stored in relational databases or relational data warehouses, dynamic analysis is not possible due to the lack of SQL analysis and visualization systems; only static displays are possible.

[0080] To address the aforementioned problems, this application provides a data processing method. Figure 4 This is a flowchart illustrating a data processing method provided in an embodiment of this application, as shown below. Figure 4 As shown, the method includes:

[0081] Step S401: The visualization system detects the user's drag-and-drop operation;

[0082] Step S402: The visualization system converts the drag-and-drop operation into an SQL statement;

[0083] The visualization system sends the SQL statement to the data processing system;

[0084] The data processing system receives the SQL statement;

[0085] Step S403: The data processing system determines the query result corresponding to the SQL statement in the cache;

[0086] Step S404: If the query result is not found in the cache, the SQL statement is sent to the SQL parser to obtain the keywords of the SQL statement;

[0087] The data processing system obtains a pre-defined interface;

[0088] Step S405: The data processing system dynamically compiles and loads the pre-defined interface to obtain the dynamically compiled implementation class;

[0089] Step S406: Parse the structured data in the HTTP response of the dynamically compiled implementation class to obtain the memory table;

[0090] Step S407: The data processing system calls the query engine to determine the query results based on the keywords of the memory table and the SQL statement;

[0091] Step S408: The data processing system sends the query results to the visualization system;

[0092] The visualization system presents the query results to the user.

[0093] To address the above problems, this application provides a data processing method, the method comprising:

[0094] Step S510: The data processing system parses the HTTP response based on a pre-defined interface to obtain the memory table;

[0095] In one possible implementation, step S510 involves the data processing system parsing an HTTP response based on a pre-defined interface to obtain a memory table, including:

[0096] Step S511: The data processing system obtains a pre-defined interface; the interface is used to extract structured data from the HTTP response and convert the structured data into a memory object.

[0097] In one possible implementation, the memory objects can be stored in the form of a linked list.

[0098] For example, a pre-defined HTTP Reader interface is obtained, which is used to extract structured data from the HTTP response and convert the structured data into an in-memory object. This interface includes an abstract method for reading a table, List. <Map<String,Object> >readTable(String data).

[0099] For example, the content of the HTTP response includes: {"success":true,"code":200,"data":[{"name":"class1","price":2,},{"name":"class2","price":2,},{"name":"class2","price":2,},{"name":"class4","price":8}],"message":null}; In this HTTP response, the 'data' keyword includes four sets of names and prices, which are structured data to be converted into in-memory objects.

[0100] Step S512: The data processing system obtains the implementation class of the interface;

[0101] In one possible implementation, the implementation class of the interface can be a static file. The methods in the implementation class override the abstract methods in the interface, enabling the abstract methods to perform their functions—extracting structured data from the HTTP response and converting it into in-memory objects—during the invocation of the implementation class. In another possible implementation, obtaining the implementation class of the interface can be accomplished by a data processing system identifying the user-submitted implementation class file.

[0102] Step S513: The data processing system obtains the memory object based on the HTTP response and using the implementation class.

[0103] In one possible implementation, step S513, where the data processing system executes the implementation class, includes: dynamically constructing a source code object based on the implementation class; dynamically loading the source code object; dynamically compiling the dynamically loaded source code object to obtain the compiled implementation class; and executing the dynamically compiled source code using reflection based on the HTTP response to obtain a memory object.

[0104] In one possible implementation, the memory object can be stored as key-value pairs, for example, a Map.<key,value> .

[0105] For example, using the implementation class, the four sets of names and prices included in the `data` keyword in the above HTTP response are converted into multiple memory objects, wherein the values ​​of the multiple memory objects include: class1, 2, class2, 2, class2, 2, class4, 8. The memory objects are stored in the following form:<class1,2> ,<class2,2> ,<class2,2> ,<class4,8> .

[0106] Step S514: Convert the memory object into a memory table;

[0107] In one possible implementation, the memory table can be two-dimensional, and the memory table can include metadata information describing attributes, or it can include data information stored in the memory object.

[0108] For example, memory objects<class1,2> ,<class2,2> ,<class2,2> ,<class4,8> After converting to a two-dimensional table, the data information in the converted two-dimensional table includes: Metadata information includes column names and the data type of those columns: [{"columnName":"name","columnType":"String",},{"columnName":"price","columnType":"Integer",}].

[0109] Step S520: The data processing system parses the query operation and obtains the SQL statement;

[0110] In one possible implementation, before step S520, in which the data processing system parses the query operation to obtain the SQL statement, the method further includes: the data processing system acquiring the query operation;

[0111] In one possible approach, the user drags and drops within a visualization system to generate a query, which is then sent to a data processing system.

[0112] For example, in a visualization system, suppose a user drags the `name` field onto the X-axis and the `price` field onto the Y-axis, and selects the SUM aggregate function, resulting in the SQL statement: `select name, sum(price) from example group by name`. The user then drags the `name` field into the visualization system's filter and enters the filter condition `name! = 'class1'`. The data processing system, after receiving the query from the visualization system, parses the filter condition and fields to obtain the SQL statement: `select name, sum(price) from example group by name where name! = 'class1'`.

[0113] For example, the filter criteria are obtained by the visualization system detecting user-defined operations, except for name!='class1'. In addition to filter criteria, simple conditional operators (>, <, =, !=, >=, <=); logical operators (&&, ||, !); fuzzy queries like, wildcards %, between and, is null / is not null, and all the filter criteria inherent in the SQL language itself.

[0114] In step S530, the data processing system determines the query result in the cache based on the SQL statement.

[0115] Step S531: If the query result exists in the cache, the data processing system sends the query result to the visualization system.

[0116] Step S532: If the query result is not present in the cache, the SQL statement is converted into a query model, which is used to parse Data Query Language (DQL) keywords from the SQL statement.

[0117] In one possible implementation, the data processing system includes an SQL parser that parses Data Query Language (DQL) keywords from the SQL statement.

[0118] For example, the input SQL statement is placed in a caching system for querying. If a query result exists in the cache, the result is sent to the visualization system. If no query result exists in the cache, the result is submitted to the SQL parser. The SQL parser transforms the SQL statement into a query model. Using the query model, keywords from the SQL statement are extracted, such as the `select` keyword, table names, column names, and filter conditions.

[0119] In one possible implementation, the query model includes fields corresponding to Data Query Language (DQL) keywords in the SQL statement.

[0120] In one possible implementation, the query model includes fact table fields corresponding to table names in the SQL statement, dimension fields corresponding to categories in the SQL statement, measure fields corresponding to aggregate functions in the SQL statement, and filter fields corresponding to filters in the SQL statement.

[0121] For example, the SQL statement is: `select name, sum(price) from example group by name where name != 'class1'`. The query model is: `{"fact_tables":"example","dimensions":["name"],"measures":[{"field":"price","aggType":"sum"}],"filters":["name != 'class1'"]}`. Here, the `fact_tables` field corresponds to the `example` table name in the SQL statement, the `dimensions` field corresponds to the `group by` category field, the `aggType` field corresponds to the `sum` aggregate function field, and the `filters` field corresponds to the `where` condition field.

[0122] In one possible implementation, after step S532, the method further includes: if the query result is not present in the cache, parsing the SQL statement using the filter fields of the query model to obtain a filter result; filtering the memory table based on the filter keywords to obtain memory tables that meet the filter conditions; parsing the SQL statement using the measure fields of the query model to obtain an aggregate function; performing an aggregation operation on the memory table based on the aggregate function to obtain an aggregated memory table; parsing the SQL statement using the join fields of the query model to obtain a join result; performing a join operation on the memory table based on the join result to obtain a joined memory table; if the SQL statement does not include preset fields in the query model, determining the joined memory table as the query result; and the data processing system sending the query result to the visualization system.

[0123] For example, 1) the memory table (two-dimensional table A) is filtered according to the filtering conditions specified by the filter keyword "filters" to obtain the filtered Memory Row Table. 2) Grouping based on the grouping keyword "dimensions": The first step is to obtain the intermediate results "Memory" from the two groups. The first step is to aggregate tmp1 and tmp2 using the aggregation keyword `sum`, resulting in two intermediate results: `[class2 4]` and `[class4 8]`. The second step is to perform a join operation on the aggregated memory table using the `union` keyword, resulting in the joined memory table. Will This is confirmed as a query result, representing the total price for different names.

[0124] Therefore, this method can be implemented through the following steps: receiving an SQL statement sent by the visualization system; determining the query result corresponding to the SQL statement in the cache; if the query result is not found in the cache; extracting keywords from the SQL statement; and determining the query result in the memory table based on the keywords. For example, the steps in the above method can be implemented using a Java Database Connectivity (JDBC) device, which shields the source of the query data, thereby reducing the coupling between various modules in the device and facilitating the design of data query methods.

[0125] In this embodiment, on the one hand, by converting the HTTP response containing structured data into an in-memory table; grouping and aggregating the data in the in-memory table according to keywords; and providing SQL queries to process the structured data in the HTTP response, coupling between modules is reduced, facilitating integration with multiple systems. On the other hand, it provides dynamic data analysis (On-Line Analytical Processing, OLAP) capabilities for the HTTP response containing structured data, dynamically displaying query results. Furthermore, by querying the structured data contained in the HTTP response using SQL statements, different query methods corresponding to different keywords can be used as different query models, and the set of models composed of different query models can be used as an SQL parser. This method reduces the coupling between SQL statements and structured data, solves the problem in related technologies that cannot query small amounts of data using SQL statements, and is beneficial for dynamic analysis of other small amounts of data, expanding the scope of application of this method.

[0126] Based on the foregoing embodiments, this application further provides a data query device. The control device includes various modules and units included in each module, which can be implemented by a processor in an electronic device; of course, it can also be implemented by specific logic circuits. In the implementation process, the processor can be a central processing unit (CPU), a microprocessor (MPU), a digital signal processor (DSP), or a field programmable gate array (FPGA), etc.

[0127] Figure 5 This is a schematic diagram of the composition structure of a data query device provided in an embodiment of this application, as shown below. Figure 5 As shown, the device 500 includes an acquisition module 501, an extraction module 502, and a query module 503, wherein:

[0128] The acquisition module 501 is used to acquire a memory table, which is obtained by parsing the structured data in the HTTP response;

[0129] The extraction module 502 is used to extract keywords from the SQL statements sent by the visualization system;

[0130] The query module 503 is used to determine the query result in the memory table based on the keyword.

[0131] In one possible implementation, the acquisition module 501 includes an acquisition unit and a parsing unit, wherein: the acquisition unit is used to acquire a pre-defined interface and the HTTP response; the HTTP response includes structured data; and the parsing unit is used to parse the structured data based on the pre-defined interface to obtain a memory table.

[0132] In one possible implementation, the extraction module 502 includes a query unit and an extraction unit, wherein: the query unit is used to determine the query result corresponding to the SQL statement in the cache; and the extraction unit is used to extract Data Query Language (DQL) keywords from the SQL statement if the query result does not exist in the cache.

[0133] In one possible implementation, the query module 503 includes a sorting unit, a processing unit, and a determining unit, wherein: the sorting unit is used to determine the execution order of the keywords; the processing unit is used to process the memory table corresponding to the keywords based on the execution order to obtain a processed memory table; and the determining unit is used to determine the processed memory table as the query result.

[0134] In one possible implementation, the keyword includes at least one of the following first operations: filtering, grouping, aggregation, and joining; the sorting unit is further configured to, when the keyword includes at least one of the first operations, determine a first priority order as the execution order of each of the first operations; the first priority order includes the execution order of the filtering, grouping, aggregation, and joining operations in sequence; when the keyword includes other second operations in addition to at least one of the first operations, determine a second priority order as the execution order of the operations in the keyword; the second priority includes the execution order of the filtering, grouping, aggregation, joining, and other second operations in sequence; when the keyword includes only other second operations and does not include at least one of the first operations, determine a third priority order as the execution order of the keyword; the third priority includes the execution order of other second operations determined according to a preset priority.

[0135] In one possible implementation, the parsing unit is further configured to determine the implementation class of the interface based on the pre-defined interface; parse the structured data using the implementation class to obtain a memory object; and convert the memory object into a memory table.

[0136] In one possible implementation, the memory object is stored as a linked list, the predefined interface includes an abstract method for reading the memory table, and the implementation class includes methods for overriding the abstract method.

[0137] In one possible implementation, the parsing unit is further configured to dynamically construct a source code object based on the implementation class; dynamically load the source code object; dynamically compile the dynamically loaded source code object to obtain a compiled implementation class; and use the compiled implementation class to parse the structured data to obtain a memory object.

[0138] It should be noted that the descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0139] It should be noted that, in the embodiments of this application, if the above methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of this application, or the part that contributes to the related technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (such as a computer) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0140] Correspondingly, embodiments of this application provide a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the methods described in the above embodiments.

[0141] Correspondingly, in this embodiment of the application, a chip is also provided, the chip including programmable logic circuits and / or program instructions, which, when the chip is running, are used to implement the steps in any of the methods described in the above embodiments.

[0142] Correspondingly, in this embodiment of the application, a computer program product is also provided, which, when executed by the processor of an electronic device, is used to implement the steps in any of the methods described in the above embodiments.

[0143] Based on the same technical concept, this application provides an electronic device for implementing the data query method described in the above method embodiments. Figure 6 This is a hardware entity diagram of an electronic device provided in an embodiment of this application, such as... Figure 6 As shown, the electronic device 600 includes a memory 610 and a processor 620. The memory 610 stores a computer program that can run on the processor 620. When the processor 620 executes the program, it implements the steps in any of the methods described in the embodiments of this application.

[0144] The memory 610 is configured to store instructions and applications executable by the processor 620, and can also cache data to be processed or already processed by the processor 620 and various modules in the electronic device. It can be implemented by flash memory or random access memory (RAM).

[0145] When the processor 620 executes the program, it implements the steps of any of the methods described above. The processor 620 typically controls the overall operation of the electronic device 600.

[0146] The aforementioned processor can be at least one of the following: Application Specific Integrated Circuit (ASIC), Digital Signal Processor (DSP), Digital Signal Processing Device (DSPD), Programmable Logic Device (PLD), Field Programmable Gate Array (FPGA), Central Processing Unit (CPU), Controller, Microcontroller, and Microprocessor. It is understood that other electronic devices can also implement the functions of the aforementioned processor, and this application does not specifically limit the specific implementation.

[0147] The aforementioned computer storage media / memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), magnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM), etc.; or it can be various electronic devices that include one or any combination of the above-mentioned memories, such as tablet devices, personal digital assistants, etc.

[0148] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0149] It should be understood that the phrase "one embodiment" or "an embodiment" throughout the specification means that a specific feature, structure, or characteristic related to the embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0150] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0151] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or units can be electrical, mechanical, or other forms.

[0152] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units. They may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of the embodiments of this application, depending on actual needs.

[0153] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0154] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause the device automatic test line to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0155] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0156] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0157] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A data query method, characterized in that, The method includes: Obtain the memory table, which is obtained by parsing the structured data in the HTTP response; Determine the query results corresponding to the SQL statement in the cache; If the query result is not present in the cache, extract the Data Query Language (DQL) keywords from the SQL statement; Determine the execution order of the keywords; Based on the execution order, the memory table is processed according to the key to obtain the processed memory table; The processed memory table is determined as the query result; The keyword includes at least one of the following first operations: filtering, grouping, aggregation, and joining; determining the execution order of the keyword includes: If the keyword includes at least one of the first operations, a first priority order is determined as the execution order of each of the first operations; the first priority order includes the execution order of the filtering, grouping, aggregation, and joining operations in sequence. If the keyword includes other second operations in addition to at least one of the first operations, the second priority order is determined as the execution order of the operations in the keyword; the second priority includes the execution order of filtering, grouping, aggregation, joining and other second operations in sequence; If the keyword includes, but does not include, at least one of the first operations, and only includes other second operations, a third priority order is determined as the execution order of the keyword; the third priority includes the execution order of other second operations determined according to a preset priority.

2. The method as described in claim 1, characterized in that, The process of obtaining the memory table includes: Obtain the pre-defined interface and the HTTP response; the HTTP response includes structured data; Based on the pre-defined interface, the structured data is parsed to obtain a memory table.

3. The method as described in claim 2, characterized in that, The process of parsing the structured data based on the pre-defined interface to obtain a memory table includes: Based on the pre-defined interface, determine the implementation class of the interface; The structured data is parsed using the implementation class to obtain a memory object; Convert the memory object into a memory table.

4. The method as described in claim 3, characterized in that, The step of parsing the structured data using the implementation class to obtain a memory object includes: Based on the implementation class, dynamically construct the source code object; Dynamically load the source code object; The dynamically loaded source code object is dynamically compiled to obtain the compiled implementation class. The structured data is parsed using the compiled implementation class to obtain a memory object.

5. A data query device, characterized in that, The device includes an acquisition module, an extraction module, and a query module, wherein: The acquisition module is used to acquire a memory table, which is obtained by parsing the structured data in the HTTP response; The extraction module includes a query unit and an extraction unit, wherein: the query unit is used to determine the query result corresponding to the SQL statement in the cache; the extraction unit is used to extract Data Query Language (DQL) keywords from the SQL statement when the query result does not exist in the cache; the keywords include at least one of the following first operations: filtering, grouping, aggregation, and joining; The query module includes a sorting unit, a processing unit, and a determining unit, wherein: the sorting unit is used to determine the execution order of the keywords; the processing unit is used to process the memory table according to the execution order, corresponding to the keywords, to obtain a processed memory table; and the determining unit is used to determine the processed memory table as the query result. The sorting unit is further configured to: determine a first priority order as the execution order of the first operations when the keyword includes at least one of the first operations; the first priority order includes the execution order of the filtering, grouping, aggregation, and joining operations in sequence; determine a second priority order as the execution order of the operations in the keyword in addition to including at least one of the first operations; the second priority order includes the execution order of the filtering, grouping, aggregation, joining, and other second operations in sequence; and determine a third priority order as the execution order of the keyword when the keyword does not include at least one of the first operations and only includes other second operations; the third priority order includes the execution order of other second operations determined according to a preset priority.

6. An electronic device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 4.

7. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the steps of the method described in any one of claims 1 to 4.

Images