Query processing method and device based on scene analysis, equipment and storage medium

Abstract

This paper relates to query technologies for heterogeneous data sources and multiple business scenarios, providing a query processing method, apparatus, device, and storage medium based on scenario analysis. The method includes: receiving query requests asynchronously sent by different application systems, where the query requests contain custom-configured scenario data features; matching rule paradigms from a set of rule paradigms based on the scenario data features; converting the rule paradigms into code entities; matching rule templates referenced in the rule paradigms from a set of rule templates; generating a query rule execution graph based on the rule templates; binding the code entities to nodes in the query rule execution graph to obtain a static query plan; generating an execution plan based on the static query plan; querying the corresponding heterogeneous data source according to the execution plan; and returning query results constrained by the corresponding rule paradigm. The embodiments described in this paper can improve the reusability and scalability of query services and reduce implementation costs in scenarios with heterogeneous data sources and diverse business scenarios.

Description

Technical Field

[0001] This specification relates to query technologies for heterogeneous data sources and multiple business scenarios, and in particular to a query processing method, apparatus, device, and storage medium based on scenario analysis. Background Technology

[0002] In recent years, with the continuous development of the computer software industry, more and more business data has been collected and stored in various databases. As business scenarios become increasingly diverse, this stored business data may be reused or processed on demand and provided to external systems or services. Due to the diversity of business scenarios and the differences in data storage engines, application developers often need to develop separate service modules—query services—to read raw data from the database, or process the data in the raw database and import it into their respective data storage engines for use in their specific business scenarios. Traditional data query services are highly coupled to business services; application developers need to obtain expected data metrics from heterogeneous data sources on demand, depending on different business scenarios.

[0003] Furthermore, due to the heterogeneity of data sources and the diversity of business scenarios, traditional application developers will develop different query services to obtain business data from the database. Before using the data, they need to clearly understand the data definitions and data structures in the database and be proficient in using data from different storage engines. At the same time, different application developers are unaware of whether their query services have duplicate data processing logic, which may lead to application developers repeatedly developing similar query services. Data providers, in turn, need to communicate and coordinate multiple times based on the needs of data users, significantly increasing software development costs. If the organization structure of the original data changes or is adjusted, the data provider needs to notify all query services, and each query service needs to assess the impact of the data structure change on the application; thus, the flexibility of the query services is affected. Therefore, how to improve the reusability and scalability of query services and reduce implementation costs in scenarios with heterogeneous data sources and diverse business scenarios has become an urgent technical challenge to be solved. Summary of the Invention

[0004] The purpose of the embodiments in this specification is to provide a query processing method, apparatus, device, and storage medium based on scenario analysis, so as to improve the reusability and scalability of query services and reduce implementation costs in scenarios with heterogeneous data sources and diverse business scenarios.

[0005] To achieve the above objectives, on the one hand, embodiments of this specification provide a query processing method based on scenario analysis, including:

[0006] Receive query requests sent asynchronously from different application systems, wherein the query requests carry custom-configured scenario data features;

[0007] Match rule paradigms from the rule paradigm set based on the scene data characteristics;

[0008] Convert the rule paradigm into code entities;

[0009] Match the rule templates referenced in the rule paradigm from the set of rule templates;

[0010] A query rule execution graph is generated based on the rule template; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes.

[0011] The code entity is bound to a node in the query rule execution graph to obtain a static query plan;

[0012] An execution plan is generated based on the static query plan;

[0013] Query the corresponding heterogeneous data source according to the execution plan;

[0014] Returns the query results constrained by the corresponding rule normal form.

[0015] In the scenario-based query processing method of this specification, the scenario data features include: scenario keywords, scenario hot search terms, or scenario tags.

[0016] In the scenario-based query processing method of this specification, matching rule paradigms from the rule paradigm set according to the scenario data characteristics includes:

[0017] The paradigm translation engine is invoked to parse the rule paradigms in the rule paradigm set, thereby obtaining the query scenario paradigm and query rule paradigm in each rule paradigm; each query scenario paradigm defines corresponding scenario data features, and each query rule paradigm defines corresponding query strategies;

[0018] Determine the semantic similarity between the scene data features and each of the query scene paradigms;

[0019] The rule paradigm to which the query scenario paradigm corresponding to the one with the highest semantic similarity belongs is taken as the target rule paradigm.

[0020] In the scenario-based query processing method of this specification, the rule paradigm is converted into code entities, including:

[0021] Convert the query rule paradigm in the target rule paradigm into query rule code entities.

[0022] In the scenario-based query processing method of this specification embodiment, each rule template includes: a rule header storing query rule metadata, and a rule body storing a complete description of the query rule; the rule body includes: field entity, aggregation type entity, calculation rule entity, query condition entity, query range entity, sorting method entity, and custom query entity.

[0023] In the scenario-based query processing method of this specification, the custom query entity is pre-constructed by application developers based on the rule paradigm of the rule paradigm set.

[0024] In the scenario-based query processing method of this specification, the generation of a query rule execution graph according to the rule template includes:

[0025] The rule parsing engine is invoked to parse the rule templates referenced in the rule paradigm;

[0026] Load and verify the correctness of the rule templates referenced in the rule paradigm;

[0027] Each query-related operation in the rule template is drawn as a node;

[0028] Connect the corresponding nodes with directed edges according to the dependencies between the query-related operations.

[0029] In the scenario-based query processing method of this specification, after returning the query results constrained by the corresponding rule paradigm, the method further includes:

[0030] Asynchronously optimize historical execution plans with the same scenario data characteristics and save the optimized execution plans;

[0031] When a query request carrying the same scenario data characteristics is received, the corresponding optimized historical execution plan is invoked to query the corresponding heterogeneous data source.

[0032] In the scenario-based query processing method of this specification, the rule paradigms in the rule paradigm set are in XML or JSON data format.

[0033] In the scenario-based query processing method of this specification, the rule templates in the rule template set are in XML or JSON data format.

[0034] In the scenario-based query processing method of this specification, the query result includes: a set of formatted data results that conform to the rule paradigm predefined by the application developer based on the rule paradigm set.

[0035] On the other hand, embodiments of this specification also provide a query processing device based on scenario analysis, including:

[0036] The receiving module is used to receive query requests sent asynchronously by different application systems, wherein the query requests carry custom-configured scenario data features;

[0037] The first matching module is used to match rule paradigms from the rule paradigm set based on the features of the scene data;

[0038] A conversion module is used to convert the rule paradigm into code entities;

[0039] The second matching module is used to match the rule templates referenced in the rule paradigm from the rule template set;

[0040] The first generation module is used to generate a query rule execution graph based on the rule template; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes.

[0041] The binding module is used to bind the code entity to the node in the query rule execution graph to obtain a static query plan;

[0042] The second generation module is used to generate an execution plan based on the static query plan;

[0043] The execution module is used to query the corresponding heterogeneous data source according to the execution plan;

[0044] The return module is used to return query results constrained by the corresponding rule normal form.

[0045] On the other hand, embodiments of this specification also provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein the computer program, when run by the processor, executes instructions for the above-described method.

[0046] On the other hand, embodiments of this specification also provide a computer storage medium storing a computer program thereon, which, when run by the processor of a computer device, executes instructions for the above-described method.

[0047] On the other hand, embodiments of this specification also provide a computer program product, which includes a computer program that, when run by the processor of a computer device, executes instructions for the above-described method.

[0048] As can be seen from the technical solutions provided in the embodiments of this specification above, in these embodiments, the query engine can match the corresponding rule paradigm from the rule paradigm set based on the custom-configured scenario data features carried in the query requests sent by different Web application systems (i.e., custom-configured scenario data features on the Web application system side), and convert the rule paradigm into a code entity; then, it matches the rule template referenced in the rule paradigm from the rule template set, generates a query rule execution graph based on the rule template, binds the code entity to the nodes in the query rule execution graph, and obtains a static query plan for querying heterogeneous data sources; thus, the query engine achieves universality for multiple different Web application systems, thereby improving the reusability of the query engine's query service. For business operation personnel on the Web application system side, by custom-configuring scenario data features, access to heterogeneous data sources can be achieved without understanding complex data access logic, and the Web application's awareness of heterogeneous data sources can also be shielded; moreover, application developers on the Web application system side do not need to develop separate query service modules for different business scenarios, or process the data in the original database and import it into their respective data storage engines to provide services for their respective business scenarios, thereby significantly reducing implementation costs. Furthermore, since rule paradigms and templates can be configured by rule maintainers, and scenario data features can be customized on demand by the web application system, the query engine's query service flexibility and scalability are improved. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. In the drawings:

[0050] Figure 1 The following diagram illustrates the application environment of query processing based on scenario analysis in some embodiments of this specification.

[0051] Figure 2 A flowchart of a query processing method based on scenario analysis in some embodiments of this specification is shown;

[0052] Figure 3 It shows Figure 2 The flowchart shown illustrates the method of matching rule paradigms from a set of rule paradigms based on scene data characteristics.

[0053] Figure 4 It shows Figure 2The flowchart shown illustrates the process of generating a query rule execution graph based on a rule template.

[0054] Figure 5 This specification illustrates a schematic diagram of the data structure of a rule template in an exemplary embodiment.

[0055] Figure 6 A schematic diagram of the code entity of a rule paradigm in an exemplary embodiment of this specification is shown;

[0056] Figure 7 This specification shows a structural block diagram of a query processing apparatus based on scenario analysis in some embodiments;

[0057] Figure 8 A structural block diagram of a computer device in some embodiments of this specification is shown.

[0058] [Explanation of Labels in the Attached Image]

[0059] 10. Web application systems;

[0060] 20. Query engine;

[0061] 30. Data source;

[0062] 71. Receiving module;

[0063] 72. First matching module;

[0064] 73. Conversion module;

[0065] 74. Second matching module;

[0066] 75. First generation module;

[0067] 76. Binding module;

[0068] 77. Second generation module;

[0069] 78. Execution module;

[0070] 79. Return to module;

[0071] 802. Computer equipment;

[0072] 804, Processor;

[0073] 806. Memory;

[0074] 808. Drive mechanism;

[0075] 810. Input / output interfaces;

[0076] 812. Input devices;

[0077] 814. Output devices;

[0078] 816. Presentation equipment;

[0079] 818. Graphical User Interface;

[0080] 820. Network interface;

[0081] 822. Communication link;

[0082] 824. Communication bus. Detailed Implementation

[0083] To enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of this specification.

[0084] Figure 1 The diagram illustrates an application environment for scenario-based query processing in some embodiments of this specification. This application environment includes multiple web application systems 10, a query engine 20, and multiple heterogeneous data sources 30. The query engine 20 can interact with multiple different web application systems 10, each providing different web services; moreover, each web application system 10 can provide one or more different web services. Therefore, the query engine 20 faces diverse business scenarios. Furthermore, the data sources 30 can include SQL-type data sources and non-SQL-type data sources; SQL-type data sources may include, but are not limited to, MySQL, Oracle, SQL Server, ClickHouse, etc.; non-SQL-type data sources may include, but are not limited to, Elastric, MongoDB, Redis, etc. Therefore, the query engine 20 also faces multi-source heterogeneous data source scenarios.

[0085] Continue parameters Figure 1As shown, the query engine 20 may include a query proxy server, a rule parsing engine, and a paradigm parsing engine. The rule paradigm set and rule template set can be stored separately and dynamically loaded from the outside by the query engine when needed. When the query proxy server receives query requests sent asynchronously by different Web application systems 10, it can perform scenario-based query processing by calling the paradigm parsing engine and the paradigm parsing engine. Specifically, the query engine 20 can receive query requests asynchronously sent by different web application systems 10, the query requests carrying custom-configured scenario data features; match rule paradigms from a set of rule paradigms based on the scenario data features; convert the rule paradigms into code entities; match rule templates referenced in the rule paradigms from a set of rule templates; generate a query rule execution graph based on the rule templates; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents a dependency relationship between nodes; bind the code entities to the nodes in the query rule execution graph to obtain a static query plan; generate an execution plan based on the static query plan; query the corresponding data source 30 according to the execution plan; and return query results constrained by the corresponding rule paradigm. In this way, the reusability and scalability of the query service can be improved and the implementation cost reduced in scenarios with heterogeneous data sources and diverse business scenarios.

[0086] This specification provides an example of a query processing method based on scenario analysis, which can be applied to the aforementioned query engine. (See also...) Figure 2 As shown, in some embodiments of this specification, the query processing method based on scenario analysis may include the following steps:

[0087] Step 201: Receive query requests sent asynchronously by different application systems, wherein the query requests carry custom-configured scene data features.

[0088] Step 202: Match rule paradigms from the rule paradigm set based on the scene data features.

[0089] Step 203: Convert the rule paradigm into code entities.

[0090] Step 204: Match the rule templates referenced in the rule paradigm from the rule template set.

[0091] Step 205: Generate a query rule execution graph based on the rule template; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes.

[0092] Step 206: Bind the code entity to the node in the query rule execution graph to obtain a static query plan.

[0093] Step 207: Generate an execution plan based on the static query plan.

[0094] Step 208: Query the corresponding heterogeneous data source according to the execution plan.

[0095] Step 209: Return the query results constrained by the corresponding rule normal form.

[0096] In the embodiments of this specification, the query engine can match the corresponding rule paradigm from the rule paradigm set based on the custom-configured scenario data features carried in the query requests sent by different web application systems (i.e., custom-configured scenario data features on the web application system side), and convert the rule paradigm into a code entity; then, it matches the rule template referenced in the rule paradigm from the rule template set, generates a query rule execution graph based on the rule template, and binds the code entity to the nodes in the query rule execution graph to obtain a static query plan for querying heterogeneous data sources; thus, the query engine achieves universality across multiple different web application systems, thereby improving the reusability of the query engine's query service. For business operations personnel on the web application system side, by custom-configuring scenario data features, access to heterogeneous data sources can be achieved without understanding complex data access logic, and the web application's awareness of heterogeneous data sources can also be shielded; moreover, application developers on the web application system side do not need to develop separate query service modules for different business scenarios, or process data in the original database and import it into their respective data storage engines for use in their respective business scenarios, thereby significantly reducing implementation costs. Furthermore, since rule paradigms and templates can be configured by rule maintainers, and scenario data features can be customized on demand by the web application system, the query engine's query service flexibility and scalability are improved.

[0097] Furthermore, since rule templates can be flexibly configured by rule maintainers, scenario data features can be customized by business operations personnel on the web application system side, and the query engine can also complete multi-dimensional data analysis queries based on the rule definitions of different scenarios.

[0098] In some embodiments of this specification, the query request may be initiated by the business operations personnel of each web application system after customizing and configuring the query scenario data features on the web application system side. The business operations personnel enter from the visual interface of the web application system front end, select the desired scenario data features (such as scenario keywords, scenario hot search terms, or scenario tags) according to the default query components provided by the web application system; the scenario data features are sent to the web application layer of the web application system, and the web application layer assembles the front end query request into a data message and sends it to the query engine for processing.

[0099] Scenario tags can be categorized according to different business domains, such as e-commerce, user experience, and intelligent marketing. In some embodiments of this specification, when using scenario tags as scenario data features, when matching rule paradigms from the rule paradigm set based on scenario data features, rule paradigms containing the scenario tag can be directly matched without semantic similarity calculation, thus improving query efficiency. In other embodiments of this specification, when using scenario keywords, scenario hot search terms, or combinations thereof as scenario data features, semantic similarity calculation is required; however, since scenario keywords and scenario hot search terms have relatively high flexibility, query flexibility is improved.

[0100] In some embodiments of this specification, the rule template set and rule paradigm set can be developed and maintained by the rule maintainers on the query engine side. The rule template set can contain multiple configurable and reusable rule templates. The rule paradigm set can contain multiple configurable and reusable rule paradigms. In the embodiments of this specification, the main purpose of introducing rule paradigms is that if business operators of different web application systems all follow the paradigms in the rule paradigm set for data queries, then application developers of different web application systems will not need to concern themselves with the underlying data storage structure. They only need to complete the rule construction according to the rule paradigm when customizing query entities and input it into the query engine, thereby further reducing implementation complexity and cost.

[0101] In some embodiments of this specification, each rule template can be semi-structured data based on XML or JSON data format to facilitate reading, writing, and maintenance, and to facilitate cross-platform use. The rule template provides the query engine with a basic description of data query rules; different query rules can be abstracted into independent rule templates (or rule models). Figure 5As shown, each rule template can include a rule header and a rule body; the rule header is used to store query rule metadata (such as the rule template's identifier, name, creation time, etc.); the rule body is used to store a complete description of the query rule (such as field entities, aggregation type entities, calculation rule entities, query condition entities, query range entities, sorting method entities, and custom query entities. These entities can all be abstracted into an object entity, and the combination of these object entities can construct a complete rule description).

[0102] The query entity comprises several key components: Field entities describe the basic information of the fields to be queried, and the query results will be generated within the given field range. Aggregation type entities define the aggregation rules for the queried field entities and assemble the corresponding aggregation logic according to these rules. Calculation rule entities define complex calculation rules between multiple fields (e.g., AND, OR, etc.) and translate these complex calculation logics into executable calculation statements for various databases. Query condition entities define the conditional expressions for the query statements, describing the filtering conditions for the query result set. Custom query entities allow application developers to develop custom query modules based on personalized business needs, and business operations personnel can configure specific query rules through simple visual configuration operations. Because custom query entities can be pre-built by application developers based on the rule paradigm set, application developers no longer need to worry about the diversity of heterogeneous data sources and can avoid repeatedly developing data query services with the same functionality.

[0103] In some embodiments of this specification, each rule paradigm can also be based on semi-structured data in XML or JSON format. Each rule paradigm can have two parts: Query Scene Style (QSS) and Query Rules Style (QRS). QSS defines the scene state on the query side (i.e., defines the scene description (scene data features) that can be used by the query side), while QRS defines the query strategy. Each rule paradigm can reference one or more rule templates. For example, in... Figure 6 The exemplary embodiment shown provides a code entity of a simple rule paradigm for query condition rules. This rule paradigm describes how, in scenarios requiring access to a MySQL database, business operators can dynamically configure the rule paradigm, combine different query rule templates, and define query constraints to complete a MySQL database query operation.

[0104] The "query scene" (including "query scene name", "scene attributes" and "scene resource description") forms the QSS; the "query rules" form the QRS.

[0105] Combination Figure 3 As shown in some embodiments of this specification, matching rule paradigms from a set of rule paradigms based on the scene data features may include the following steps:

[0106] Step 301: Call the paradigm translation engine to parse the rule paradigms in the rule paradigm set, and obtain the query scenario paradigm and query rule paradigm in each rule paradigm.

[0107] The rule paradigm in the rule paradigm set parsed by the query proxy server refers to the rule paradigm in the rule paradigm set parsed by the query proxy server.

[0108] Step 302: Determine the semantic similarity between the scene data features and each of the query scene paradigms.

[0109] Taking scene data features as scene keywords as an example, we can calculate the semantic similarity between the scene keywords and the query scene in each rule paradigm.

[0110] Step 303: Take the rule paradigm to which the query scenario paradigm corresponding to the one with the highest semantic similarity belongs as the target rule paradigm.

[0111] The target rule paradigm is the rule paradigm that best matches the data characteristics of the scenario.

[0112] Steps 302 and 303 can be implemented by the query proxy server or by the paradigm translation engine, depending on the specific requirements.

[0113] In some embodiments of this specification, converting the rule paradigm into code entities may specifically include: converting the query rule paradigm in the target rule paradigm into a query rule code entity (QRE). A QRE defines the complete lifecycle of a query; the query results, process, and constraints are all provided by the QRE. The query process may consist of one or more QREs.

[0114] For example, when the query rule paradigm in the target rule paradigm is an XML file, the XML file can be converted into query rule code entities; when the query rule paradigm in the target rule paradigm is a JSON file, the JSON file can be converted into query rule code entities. Since converting XML or JSON files into code entities are known techniques (e.g., an XML converter can convert XML files into C++ code, and a Java code generator can convert JSON files into Java code), and the embodiments in this specification do not involve improvements to this part, they will not be described further.

[0115] In some embodiments of this specification, matching a rule template referenced in a rule paradigm from a set of rule templates means that a rule paradigm that references a rule template can be matched from the set of rule templates based on the rule template name.

[0116] In the embodiments of this specification, the query rule execution graph may be, for example, a directed graph. Combined with... Figure 4 As shown in some embodiments of this specification, generating a query rule execution graph based on the rule template may include the following steps:

[0117] Step 401: Call the rule parsing engine to parse the rule templates referenced in the rule paradigm.

[0118] Step 402: Load and verify the correctness of the rule templates referenced in the rule paradigm.

[0119] In some embodiments of this specification, the validation content can be selected as needed. For example, it can validate the correctness of data types (such as strings, numbers, etc.) or the correctness of business logic. This specification does not limit this; if the rule template passes the validation, step 403 can be executed; otherwise, a corresponding prompt can be output for relevant personnel to handle.

[0120] Step 403: Draw each query-related operation in the rule template as a node.

[0121] In some embodiments of this specification, query-related operations may refer to a single operation or a group of user operations to implement a specific function (such as aggregation methods, calculation methods, calling methods, etc.). The appropriate node granularity can be selected as needed.

[0122] Step 404: Connect the corresponding nodes with directed edges according to the dependencies between the query-related operations.

[0123] In other embodiments of this specification, after returning the query results constrained by the corresponding rule paradigm, the method may further include: asynchronously optimizing historical execution plans with the same scenario data characteristics and saving the optimized execution plans; when a query request carrying the same scenario data characteristics is received, calling the corresponding optimized historical execution plan to query the corresponding heterogeneous data source (i.e., directly executing the query statement in the optimized historical execution plan); thus, it can help to further improve query efficiency.

[0124] In the embodiments of this specification, binding the code entity to the nodes in the query rule execution graph means establishing a mapping relationship between the nodes in the query rule execution graph and the code blocks in the code entity. For example, when the query-related operation corresponding to a node is an aggregation method, a mapping relationship is established between the code block corresponding to the aggregation method in the code entity and that node. After binding the code entity to the nodes in the query rule execution graph, a static query plan is obtained. Then, an execution plan (i.e., a query execution plan) can be generated based on the static query plan. Since query requests from different web applications are asynchronous and concurrent, the query engine may receive multiple query requests simultaneously or a large number of query requests in a short period of time. Therefore, the query engine can establish a mapping relationship between the nodes in the query rule execution graph and the code blocks in the code entity. Figure 1 The query execution plan cache queue is shown, and each query is processed one by one according to the first-in-first-out rule.

[0125] In the embodiments of this specification, a static query plan is a sequence of steps for query execution, used to describe how the query engine executes the query. An execution plan is the concrete implementation of the query plan, describing the specific executable operations required by the query engine to execute the query (including table reading methods, index usage, database connection methods, query result sorting methods, etc.). Since generating an execution plan from a static query plan is existing technology in the art, this specification does not involve improvements to the part of generating the execution plan from the query plan, and will not elaborate on this further.

[0126] In some embodiments of this specification, querying the corresponding heterogeneous data source according to the execution plan can mean that the query engine performs distributed query operations across multiple data sources according to the execution plan. After obtaining the query result dataset, the paradigm translation engine can return the required data set according to the data result set convention format defined in the rule paradigm, and the front end can perform dynamic data rendering. In other words, the query return result set can be formatted and configured by developers to conform to QRT constraints, so as to meet the needs of dynamically and flexibly obtaining the dataset structure required for testing and use at each stage.

[0127] Furthermore, it should be noted that during the requirements development process, application developers can use the query engine documentation to define standard query call interfaces. If the downstream system is a standard SQL database, the access method can be explicitly defined as SQL type in the request access method, and the query rules can be predefined according to the standard QRT in the query entity, so that the request messages of the front-end application can be queried and mapped according to the QRT parsing specification.

[0128] Furthermore, in the asynchronous optimization of historical execution plans with similar data characteristics mentioned above, if the downstream system is a non-standard SQL database, the standard request URL or database access API name and constraints need to be provided in the access method.

[0129] Although the process described above includes multiple operations that occur in a specific order, it should be clearly understood that these processes may include more or fewer operations, which may be executed sequentially or in parallel (e.g., using parallel processors or a multithreaded environment).

[0130] Corresponding to the scenario-based query processing method described above, this specification also provides a scenario-based query processing device, which can be configured on the query engine side described above. (Refer to...) Figure 7 As shown in some embodiments of this specification, the query processing apparatus based on scenario analysis may include:

[0131] The receiving module 71 is used to receive query requests sent asynchronously by different application systems, wherein the query requests carry custom-configured scene data features;

[0132] The first matching module 72 is used to match rule paradigms from the rule paradigm set according to the scene data features;

[0133] Conversion module 73 is used to convert the rule paradigm into code entities;

[0134] The second matching module 74 is used to match the rule templates referenced in the rule paradigm from the rule template set;

[0135] The first generation module 75 is used to generate a query rule execution graph according to the rule template; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes.

[0136] Binding module 76 is used to bind the code entity to the node in the query rule execution graph to obtain a static query plan;

[0137] The second generation module 77 is used to generate an execution plan based on the static query plan;

[0138] Execution module 78 is used to query the corresponding heterogeneous data source according to the execution plan;

[0139] Return module 79 is used to return query results constrained by the corresponding rule normal form.

[0140] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0141] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this specification are all information and data authorized and agreed upon by the user and fully authorized by all parties.

[0142] Embodiments of this specification also provide a computer device. For example... Figure 8 As shown, in some embodiments of this specification, the computer device 802 may include one or more processors 804, such as one or more central processing units (CPUs) or graphics processing units (GPUs), each of which may implement one or more hardware threads. The computer device 802 may also include any memory 806 for storing any kind of information such as code, settings, data, etc. In one specific embodiment, a computer program is stored on the memory 806 and can run on the processor 804. When the computer program is run by the processor 804, it can execute instructions of the query processing method based on scene analysis described in any of the above embodiments. Without limitation, for example, the memory 806 may include any type of RAM, any type of ROM, flash memory, hard disk, optical disk, etc. More generally, any memory can use any technology to store information. Further, any memory can provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 802. In one case, when the processor 804 executes associated instructions stored in any memory or combination of memories, the computer device 802 can perform any operation of the associated instructions. The computer device 802 also includes one or more drive mechanisms 808 for interacting with any memory, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0143] Computer device 802 may also include an input / output interface 810 (I / O) for receiving various inputs (via input device 812) and providing various outputs (via output device 814). A specific output mechanism may include a presentation device 816 and an associated graphical user interface 818 (GUI). In other embodiments, the input / output interface 810 (I / O), input device 812, and output device 814 may be omitted, and the device may function solely as a computer device within a network. Computer device 802 may also include one or more network interfaces 820 for exchanging data with other devices via one or more communication links 822. One or more communication buses 824 couple the components described above together.

[0144] Communication link 822 can be implemented in any way, such as via a local area network, a wide area network (e.g., the Internet), a point-to-point connection, or any combination thereof. Communication link 822 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc., governed by any protocol or combination of protocols.

[0145] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), computer-readable storage media, and computer program products according to some embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processor to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processor, create a mechanism for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processor to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] These computer program instructions may also be loaded onto a computer or other programmable data processor, causing a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0148] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0149] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0150] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by computer equipment. As defined in this specification, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0151] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, the embodiments of this specification can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, the embodiments of this specification can take the form of computer program products implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0152] The embodiments described in this specification can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. The embodiments of this specification can also be practiced in distributed computing environments where tasks are performed by remote processors connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0153] It should also be understood that, in the embodiments of this specification, the term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this document generally indicates that the preceding and following related objects have an "or" relationship.

[0154] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0155] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of the embodiments of this specification. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0156] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for query processing based on scene analysis, characterized in that, include: Receive query requests sent asynchronously from different application systems, wherein the query requests carry custom-configured scenario data features; Match rule paradigms from the rule paradigm set based on the scene data characteristics; Convert the rule paradigm into code entities; Match the rule templates referenced in the rule paradigm from the set of rule templates; A query rule execution graph is generated based on the rule template; each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes. The code entity is bound to a node in the query rule execution graph to obtain a static query plan; An execution plan is generated based on the static query plan; Query the corresponding heterogeneous data source according to the execution plan; Returns the query results constrained by the corresponding rule normal form.

2. The query processing method based on scenario analysis as described in claim 1, characterized in that, The scene data features include: scene keywords, scene trending search terms, or scene tags.

3. The query processing method based on scenario analysis as described in claim 1, characterized in that, Matching rule paradigms from the rule paradigm set based on the scene data features includes: The paradigm translation engine is invoked to parse the rule paradigms in the rule paradigm set, thereby obtaining the query scenario paradigm and query rule paradigm in each rule paradigm; each query scenario paradigm defines corresponding scenario data features, and each query rule paradigm defines corresponding query strategies; Determine the semantic similarity between the scene data features and each of the query scene paradigms; The rule paradigm to which the query scenario paradigm corresponding to the one with the highest semantic similarity belongs is taken as the target rule paradigm.

4. The query processing method based on scenario analysis as described in claim 3, characterized in that, Converting the rule paradigm into code entities includes: Convert the query rule paradigm in the target rule paradigm into query rule code entities.

5. The query processing method based on scenario analysis as described in claim 1, characterized in that, Each rule template includes: a rule header that stores the metadata of the query rule, and a rule body that stores the complete description of the query rule; the rule body includes: field entity, aggregation type entity, calculation rule entity, query condition entity, query range entity, sorting method entity, and custom query entity.

6. The query processing method based on scenario analysis as described in claim 5, characterized in that, The custom query entity is pre-built by application developers based on the rule paradigm of the rule paradigm set.

7. The query processing method based on scenario analysis as described in claim 1, characterized in that, Generate a query rule execution graph based on the rule template, including: The rule parsing engine is invoked to parse the rule templates referenced in the rule paradigm; Load and verify the correctness of the rule templates referenced in the rule paradigm; Each query-related operation in the rule template is drawn as a node; Connect the corresponding nodes with directed edges according to the dependencies between the query-related operations.

8. The query processing method based on scenario analysis as described in claim 1, characterized in that, After returning the query results constrained by the corresponding rule normal form, it also includes: Asynchronously optimize historical execution plans with the same scenario data characteristics and save the optimized execution plans; When a query request carrying the same scenario data characteristics is received, the corresponding optimized historical execution plan is invoked to query the corresponding heterogeneous data source.

9. The query processing method based on scenario analysis as described in claim 1, characterized in that, The rule paradigms in the rule paradigm set are in XML or JSON data format.

10. The query processing method based on scenario analysis as described in claim 1, characterized in that, The rule templates in the rule template set are in XML or JSON data format.

11. The query processing method based on scenario analysis as described in claim 1, characterized in that, The query results include a set of formatted data results that conform to the rule paradigm predefined by the application developer based on the rule paradigm set.

12. A query processing device based on scenario analysis, characterized in that, include: The receiving module is used to receive query requests sent asynchronously by different application systems, wherein the query requests carry custom-configured scenario data features; The first matching module is used to match rule paradigms from the rule paradigm set based on the features of the scene data; A conversion module is used to convert the rule paradigm into code entities; The second matching module is used to match the rule templates referenced in the rule paradigm from the rule template set; The first generation module is used to generate a query rule execution graph based on the rule template; Each node in the query rule execution graph represents a query-related operation, and each directed edge in the query rule execution graph represents the dependency relationship between nodes. The binding module is used to bind the code entity to the node in the query rule execution graph to obtain a static query plan; The second generation module is used to generate an execution plan based on the static query plan; The execution module is used to query the corresponding heterogeneous data source according to the execution plan; The return module is used to return query results constrained by the corresponding rule normal form.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, When the computer program is run by the processor, it executes the instructions of the method according to any one of claims 1-11.

14. A computer storage medium having a computer program stored thereon, characterized in that, When the computer program is run by the processor of the computer device, it executes the instructions of the method according to any one of claims 1-11.

15. A computer program product, characterized in that, The computer program product includes a computer program that, when run by the processor of a computer device, executes instructions according to any one of claims 1-11.

Images