Query processing method, electronic device, storage medium and program product

Abstract

Embodiments of the present application provide a query processing method, an electronic device, a storage medium and a program product, a first query request instruction triggered by an application program is received, the first query request instruction is obtained based on a preset unified query syntax structure, a first query converter corresponding to a data type identifier of a first target data set is determined, first conversion rule information corresponding to a data set identifier is obtained, the first conversion rule information is adapted to a first target storage engine storing the first target data set, and the first query request instruction is converted into a second query request instruction capable of being executed by the first target storage engine through the first query converter and the first conversion rule information; based on the scheme, unified access to data in different storage engines is realized efficiently.

Description

Technical Field

[0001] This application relates to the field of data query processing technology, and in particular to a query processing method, electronic device, storage medium and program product. Background Technology

[0002] In observability platforms of distributed systems, to support core capabilities such as monitoring, diagnosis, and operation and maintenance, it is typically necessary to process various types of observable data, including metrics, logs, tracing, events, and performance profiling. This observable data is stored in different types of backend storage engines. Because each storage engine exposes its own native, heterogeneous query interface, upper-layer applications must obtain the specific implementation details of the storage engine beforehand when accessing this data, including the specific engine type for data storage, the query protocol used, field naming rules, and the construction method of filter conditions. This results in the need to repeatedly implement a large amount of complex data access logic corresponding to each storage engine when querying data, significantly reducing the overall efficiency of cross-storage engine data access.

[0003] Currently, in order to simplify query logic, various types of observable data can be forcibly stored in the same storage engine (such as uniformly stored in a full-text search engine) to achieve a unified query interface. However, this approach sacrifices the flexibility of selecting a more suitable storage engine for different data types as needed, and it is difficult to take into account the differentiated storage needs of various types of data. Summary of the Invention

[0004] This application provides a query processing method, electronic device, storage medium, and program product to solve the problem of how to efficiently perform unified queries on data stored in different types of storage engines.

[0005] In a first aspect, embodiments of this application provide a query processing method, the method comprising: The system receives a first query request instruction triggered by an application. The first query request instruction is constructed based on a preset unified query syntax structure. The first query request instruction includes a data type identifier and a dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. Determine the first query converter corresponding to the data type identifier of the first target dataset; Obtain first transformation rule information corresponding to the dataset identifier, wherein the first transformation rule information is adapted to the first target storage engine storing the first target dataset; The first query request instruction is converted into a second query request instruction that can be executed by the first target storage engine using the first query converter and the first conversion rule information.

[0006] Secondly, embodiments of this application provide a query processing apparatus, including: The receiving module receives a first query request instruction triggered by the application. The first query request instruction is constructed based on a preset unified query syntax structure. The first query request instruction includes the data type identifier and dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. The query converter determination module determines the first query converter corresponding to the data type identifier of the first target dataset; The transformation rule acquisition module acquires first transformation rule information corresponding to the dataset identifier, and the first transformation rule information is adapted to the first target storage engine storing the first target dataset. The conversion module converts the first query request instruction into a second query request instruction that can be executed by the first target storage engine using the first query converter and the first conversion rule information.

[0007] Thirdly, embodiments of this application provide an electronic device, including: a memory and a processor; the memory is used to store one or more computer instructions; the processor is used to execute the one or more computer instructions to: perform the method as described in the first aspect.

[0008] Fourthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the method described in the first aspect.

[0009] Fifthly, embodiments of this application provide a computer program product, including: a computer program / instructions, which, when executed by a processor, can implement the method described in the first aspect.

[0010] In the query processing method provided in this application embodiment, the first query request instruction triggered by the application is constructed based on a preset unified query syntax structure. The first query request instruction includes the data type identifier and dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. In this way, the application only needs to initiate a query request instruction through the unified query syntax structure without needing to be aware of the differences between different storage engines. Based on the first query request instruction, a first query converter corresponding to the data type identifier of the first target dataset is determined, and first conversion rule information corresponding to the dataset identifier is obtained. Through the first query converter and the first conversion rule information, the first query request instruction is converted into a second query request instruction that can be executed by the first target storage engine. It can be seen that, according to the data type identifier and dataset identifier of the first target dataset to be queried in the first query request instruction, the corresponding query converter and conversion rule information can be automatically matched, and the first query request instruction constructed by the unified query syntax structure can be converted into a second query request instruction that can be executed by the target storage engine, thus achieving efficient unified access to data in different storage engines. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a schematic diagram of the structure of an observable platform according to one embodiment; Figure 2 A flowchart illustrating a query processing method provided in an embodiment of this application; Figure 3 A schematic diagram of a conversion rule information database provided in an embodiment of this application; Figure 4 This is a schematic diagram of the structure of an observable platform provided in an embodiment of this application; Figure 5 A flowchart of the observable platform query processing method provided in the embodiments of this application; Figure 6 A flowchart illustrating the dual-storage engine mode provided in this application embodiment; Figure 7 A flowchart illustrating the entity set-based query processing method provided in this application embodiment; Figure 8 A flowchart for generating query request instructions based on a method query interface provided in this application embodiment; Figure 9 A schematic diagram illustrating the relationship between an entity set, a dataset, and a method name, provided in an embodiment of this application; Figure 10 A schematic diagram illustrating the relationship between another entity set, dataset, and method name provided in an embodiment of this application; Figure 11 A flowchart illustrating the entity set association query method provided in this application embodiment; Figure 12 This is a schematic diagram illustrating the data query and processing principle of the association relationship between two entity sets provided in an embodiment of this application. Figure 13 This is a schematic diagram of the structure of a query processing device provided in an embodiment of this application; Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0013] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application. In addition, the timing of the steps in the following method embodiments is only an example and not a strict limitation.

[0014] It should be noted that, in the cases involving user information in the embodiments of this application, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments of this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use, and processing of related data must comply with the relevant laws, regulations, and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. In addition, the various models involved in this application (including but not limited to problem analysis models or other models) comply with relevant laws and standards.

[0015] The following explains the terms or concepts involved in the embodiments of this application: PromQL: Prometheus Query Language, is a functional query language specifically designed for time-series data, supporting filtering, aggregation, computation, and statistical analysis of time-series data.

[0016] DSL: Domain Specific Language, in this article refers to the specific query language used by the full-text search engine to construct complex search, filtering, and aggregation conditions.

[0017] API: Application Programming Interface, refers to the HTTP or gRPC interface exposed by the storage engine to receive query requests and return execution results. Different storage engines have different APIs in terms of request format, parameter passing methods, and response structure.

[0018] SPL: Search Processing Language, is a pipeline query language for streaming data processing that supports filtering, aggregation, and transformation of data.

[0019] like Figure 1 As shown, in the observability platform, multiple applications (application 1011, application 1012) need to access observable data stored in different backend storage engines. These backend storage engines include, but are not limited to: time-series database 1021 and full-text search engine 1022. The time-series database 1021 exposes a PromQL query interface to the applications for storing and querying metric data; the full-text search engine 1022 exposes a DSL or specific query syntax to the applications for storing and querying log data.

[0020] Because the query interfaces of different storage engines are incompatible, each application needs to be aware of the storage engine type when accessing different types of data. That is, it needs to know which storage engine the target data is stored in, learn and use different query languages ​​such as PromQL and DSL, and write corresponding query request commands for each storage engine. Thus, each application needs to maintain multiple sets of query logic corresponding to various storage engines, resulting in high complexity.

[0021] Currently, in order to simplify query logic, various types of observable data can be forcibly stored in the same storage engine (such as uniformly stored in a full-text search engine) to achieve a unified query interface. However, this approach sacrifices the flexibility of selecting a more suitable storage engine for different data types as needed, and it is difficult to take into account the differentiated storage needs of various types of data.

[0022] Based on this, embodiments of this application provide a query processing method. By defining a unified query syntax structure independent of the storage engine, the application only needs to construct a query request instruction based on the unified query syntax structure and carry a data type identifier and a dataset identifier in the query request instruction. Then, the method automatically routes to the corresponding query converter according to the data type identifier and combines the conversion rule information corresponding to the dataset identifier to automatically convert the query request instruction constructed by the unified query syntax structure into a query request instruction executable by the target storage engine. This shields the differences in query interfaces of different storage engines and enables unified querying of data that is scattered across multiple storage engines.

[0023] Figure 2 This is a flowchart illustrating a query processing method provided in an embodiment of this application, wherein the executing entity of the method is an electronic device. Figure 2 As shown, the method includes the following steps: 201. Receive the first query request instruction triggered by the application. The first query request instruction is constructed based on a preset unified query syntax structure. The first query request instruction includes the data type identifier and dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier.

[0024] 202. Determine the first query converter corresponding to the data type identifier of the first target dataset.

[0025] 203. Obtain the first transformation rule information corresponding to the dataset identifier. The first transformation rule information is adapted to the first target storage engine storing the first target dataset.

[0026] 204. Using the first query converter and the first conversion rule information, the first query request instruction is converted into a second query request instruction that can be executed by the first target storage engine.

[0027] The first query request triggered by the application is constructed based on a predefined unified query syntax structure. This unified query syntax structure is a standardized query expression independent of different storage engines, used to describe data query requirements and applicable to various storage engines. This unified query syntax structure can cover query capabilities for various data types, including metrics, logs, traces, events, and performance profiling.

[0028] The first query request instruction includes a data type identifier and a dataset identifier for the first target dataset to be queried. The first target dataset stores data corresponding to the data type identifier. The data type identifier indicates the data type of the dataset to be queried. For example, `metric_set` indicates the data type of a metrics dataset, `log_set` indicates the data type of a log set, `trace_set` indicates the data type of a trace dataset, `event_set` indicates the data type of an event dataset, and `profile_set` indicates the data type of a performance profiling dataset. The dataset identifier identifies the dataset. Under the same data type, different datasets can be divided according to dimensions such as scenario, data source, and time range. The dataset identifier can include the domain to which the dataset belongs and the dataset name. Furthermore, the first query request instruction can also include query filtering conditions. Therefore, the format of the first query request instruction can be: `.metric_set with(domain, name, query)`, `.log_set with(domain, name, query)`, etc.

[0029] A query converter is a data type-oriented conversion logic unit used to convert a first query request instruction constructed with a unified query syntax structure into the native query language of the storage engine corresponding to the data type of the first target dataset being queried. In some embodiments, query converters can be divided into two categories: dedicated query converters and general query converters. Dedicated converters are converters specifically implemented for specific data types or their special query modes. Dedicated converters encapsulate the conversion logic related to specific data types and can handle the conversion rules specific to those data types. For example, the MetricSet converter is specifically designed to handle query conversions of metric datasets (metric_set). It supports dual-mode conversion of instantaneous query mode and range query mode. Instantaneous query mode is suitable for scenarios where instantaneous values ​​are queried at the current moment, while range query mode is suitable for querying historical data over a period of time. General query converters refer to a class of converters with the same conversion logic template. Multiple data types share the same converter implementation. For example, log datasets, tracing datasets, and event datasets can share the same general query converter.

[0030] Upon receiving the first query request instruction triggered by the application, the system first parses the instruction, extracts its data type identifier, and determines the corresponding first query converter based on this identifier. After determining the first query converter, it further extracts the dataset identifier from the first query request instruction. Each dataset corresponds to a specific storage engine, and different storage engines may have different types, versions, field definitions, query protocols, and other attributes. Therefore, corresponding conversion rule information is pre-configured for each dataset. During the conversion process, the first query converter loads the first conversion rule information corresponding to the dataset identifier of the first target dataset. Based on this information, it converts the query filtering conditions in the first query request instruction into the corresponding syntax elements in the first target storage engine, ultimately assembling and generating a second query request instruction that can be executed by the first target storage engine.

[0031] In some embodiments, after determining the first query converter corresponding to the data type identifier of the first target dataset, the corresponding first transformation rule information is retrieved from the transformation rule information database using the dataset identifier (domain and name) in the first query request instruction as the key. For example... Figure 3 As shown, the transformation rule information base maintains the storage mapping relationship between the dataset and the target storage engine, field transformation rules, filter condition templates, and query generator configuration information. The storage mapping relationship includes the connection parameters for establishing the connection with the target storage engine. Furthermore, the storage mapping relationship can also include the default query filter conditions of the target storage engine. For example, the storage mapping relationship for a time-series database includes the connection parameters (region, project, store), and the storage mapping relationship for a full-text search engine includes the connection parameters (region, project, logstore) and the default query filter conditions. It should be noted that the same dataset can have storage mapping relationships with different types of target storage engines. Field transformation rules are the conversion rules between logical field names used by the application in the dataset and physical field names in the target storage engine. For example, the logical field name `service_name` is converted to the physical field name `exported_service`; the logical field name `start_time` is converted to the physical field name `_time_`; and the logical field name `duration` is converted to the physical field name `latency_ms`. The filtering template includes: storage-level default filtering conditions, variable placeholders, and entity set filtering conditions. The storage-level default filtering conditions are predefined filtering expressions in the target storage engine; variable placeholders can be in the format ${{variable}}, and are replaced with actual values ​​at runtime; entity-level filtering conditions support dynamic filtering by entity attributes. The query generator configuration information is associated with the target storage engine and may include: PromQL templates and SPL pipeline configurations.

[0032] The first transformation rule information includes: the storage mapping relationship between the first target dataset and the first target storage engine; the first field transformation rule between the logical field names used by the application in the first target dataset and the physical field names in the first target storage engine; the first filter condition template corresponding to the first target storage engine; and the first query generator configuration information. The first query generator configuration information describes the construction logic of query instructions corresponding to different fields in the first target dataset. The storage mapping relationship includes the connection parameters of the first target storage engine. Specifically, the storage mapping relationship describes the binding relationship between the first target dataset and the first target storage engine, indicating which storage engine the dataset is stored in. This storage mapping relationship may include: the type of storage engine and the connection parameters of the storage engine. The type of storage engine identifies the type of storage engine, and the connection parameters are the connection information required to access the storage engine, such as address, port, and authentication information. The first field conversion rule describes the correspondence between the logical field names used by the application in the unified query syntax structure and the actual physical field names stored in the first target storage engine. For example, applications often use semantically clear field names such as service_name, node_id, and timestamp, while storage engines may use different field naming conventions, such as exported_service, node, and @timestamp. The first field conversion rule decouples the field naming between the application and the storage engine. The first filter condition template describes how to convert the query filter conditions in the first query request instruction into a native filter syntax that the first target storage engine can recognize. The first query generator configuration information describes the construction logic of query instructions corresponding to different fields in the first target dataset, that is, it defines how to assemble the elements in the unified query syntax structure into the native query instructions of the target storage engine.

[0033] In some embodiments, converting a first query request instruction into a second query request instruction that can be executed by a first target storage engine using a first query converter and first conversion rule information may include: determining the first target storage engine based on the storage mapping relationship in the first conversion rule information using the first query converter, and converting the query parameters in the first query request instruction into the native query syntax of the first target storage engine based on the first field conversion rule and the first query generator configuration information in the first conversion rule information, so as to generate the second query request instruction.

[0034] In some embodiments, the query processing method supports two working modes: instruction return mode (i.e., DryRun mode) and result return mode. The system can select one mode based on the application's request parameters or configuration items to meet different usage scenario requirements. In instruction return mode, after the first query converter completes the conversion from the first query request instruction to the second query request instruction, it does not send the instruction to the first target storage engine. Instead, it directly returns the converted second query request instruction to the application in a readable form. This allows for previewing and verifying the second query request instruction to be sent to the first target storage engine before actual query execution, facilitating debugging, testing, and troubleshooting. In result return mode, the system executes the complete query processing flow: after completing the query instruction conversion, the second query request instruction is sent to the first target storage engine, which executes the query and returns the result data. The system then returns the result data to the application. Result return mode is the standard mode for normal queries, through which the application obtains the data it actually needs.

[0035] and Figure 1 In comparison, such as Figure 4 As shown, multiple applications (application 1011, application 1012) need to access observable data stored in different backend storage engines. These backend storage engines include, but are not limited to, time-series database 1021 and full-text search engine 1022. This embodiment introduces a query converter between the applications and different storage engines. The query converter provides a unified query syntax interface for all upper-layer applications by loading corresponding conversion rule information. Specifically, each application does not need to learn different native query languages ​​such as PromQL and full-text search DSL; it only needs to use query request instructions constructed using a unified query syntax structure. After receiving the query request instruction constructed using the unified query syntax structure, the query converter automatically selects the corresponding query converter based on the data type identifier and dataset identifier in the query request instruction, loads the conversion rule information, converts the query request instruction constructed using the unified query syntax structure into native query request instructions (such as PromQL, full-text search DSL, etc.) executable by each target storage engine, and sends them to the corresponding time-series database and full-text search engine for query execution.

[0036] In this embodiment, the first query request instruction triggered by the application is constructed based on a preset unified query syntax structure. The first query request instruction includes the data type identifier and dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. Thus, the application only needs to initiate a query request instruction through the unified query syntax structure, without needing to be aware of the differences between different storage engines. Based on the first query request instruction, a first query converter corresponding to the data type identifier of the first target dataset is determined, and first conversion rule information corresponding to the dataset identifier is obtained. Using the first query converter and the first conversion rule information, the first query request instruction is converted into a second query request instruction executable by the first target storage engine. Therefore, based on the data type identifier and dataset identifier of the first target dataset to be queried in the first query request instruction, the corresponding query converter and conversion rule information can be automatically matched, converting the first query request instruction constructed with the unified query syntax structure into a second query request instruction executable by the target storage engine, achieving efficient unified access to data in different storage engines.

[0037] In one optional embodiment, converting a first query request instruction into a second query request instruction executable by a target storage engine using a first query converter and first conversion rule information can be achieved by: inputting the first conversion rule information into the first query converter, so that the first query converter performs the following processing: 2041. Based on the storage mapping relationship, determine the first target storage engine, and connect to the first target storage engine according to the connection parameters.

[0038] 2042. According to the first field conversion rule, convert the target logical field name into the corresponding target physical field name.

[0039] 2043. Based on the first filtering condition template, convert the query filtering conditions into the target filtering expression corresponding to the first target storage engine.

[0040] 2044. Based on the configuration information of the first query generator, as well as the target physical field name and the target filter expression, generate a second query request instruction.

[0041] After obtaining the first transformation rule information, the first query converter parses the storage mapping relationship, which may include: the type identifier of the first target storage engine, the name of the first target storage engine, and connection parameters. Based on the type identifier and name of the storage engine in the storage mapping relationship, the first query converter determines which storage engine the first query request instruction is targeting, i.e., it determines the first target storage engine. Furthermore, the first query converter extracts the connection parameters from the storage mapping relationship and establishes a connection with the first target storage engine based on these connection parameters.

[0042] The first query converter parses the query filtering conditions in the first query request instruction and extracts the various logical field names involved. These logical field names are the field names with application semantics in the first query request instruction (such as service_name, node_id, timestamp, etc.). The first query converter obtains the first field conversion rule from the first conversion rule information. The first field conversion rule records the mapping relationship between the logical field names and the actual physical field names stored in the first target storage engine. For each logical field name extracted from the first query request instruction, the first query converter searches within the first field conversion rule to determine its corresponding physical field name and replaces the logical field name with the corresponding physical field name for use in subsequent steps.

[0043] The first query converter obtains the query filtering conditions from the first query request instruction and the first filtering condition template from the first conversion rule information, converting the query filtering conditions in the first query request instruction into a target filtering expression that can be recognized by the first target storage engine. The first filtering condition template may include: operator mapping rules and condition expression templates. The operator mapping rules map operators in the unified query syntax (such as =, !=, in, not_in, >, <, etc.) to the native operators of the target storage engine. The condition expression template defines how a single query filtering condition is constructed as a string expression or a structured object. Furthermore, the first filtering condition template may also include: condition combination rules. Condition combination rules define the combination logic (such as AND, OR, etc.) between multiple query filtering conditions. If the first query request instruction contains multiple query filtering conditions, the first query converter combines the filtering expressions corresponding to the multiple query filtering conditions according to the condition combination rules defined in the first filtering condition template to obtain the target filtering expression.

[0044] The first query converter obtains the first query generator configuration information from the first conversion rule information, and combines it with the target physical field name obtained in step 2042 and the target filtering expression obtained in step 2043 to generate a second query request instruction that can be executed by the first target storage engine.

[0045] In some embodiments, suppose an observability platform needs to query the P99 service latency metric for an order service in the APM (Application Performance Monitoring) domain. This metric measures the latency experienced by 99% of requests to the order service within the most recent hour. Figure 5 As shown, the specific process is as follows: 501. Receive the first query request command triggered by the application. metric_set with(domain="apm",name="service_latency_p99", query="service_name=order-service", step="60s"), where .metric_set is the data type identifier, domain="apm" indicates that the domain of the first target dataset is application performance monitoring, name="service_latency_p99" indicates that the name of the first target dataset is the service P99 latency metric, query="service_name=order-service" indicates the query filter condition (specifying that only data from the order service is queried), and step="60s" indicates that the sampling step size is 60 seconds, that is, one data point is returned every 60 seconds.

[0046] 502. By parsing the .metric_set identifier in the first query request instruction, the data type of the first target dataset in this query is identified as a metric dataset. Based on the data type of the first target dataset, the MetricSet dedicated query converter is selected as the first query converter to process the first query request instruction.

[0047] 503. The first query converter loads the corresponding first conversion rule information based on the domain="apm" and name="service_latency_p99" in the first query request instruction. The loaded first conversion rule information includes the following: Storage mapping relationship: indicator dataset and time series database; The first field conversion rule is: service_name → exported_service; First filter template: PromQL template; First query generator configuration information: PromQL template: histogram_quantile(0.99, sum by(le)(rate(http_request_duration_seconds_bucket{${{filter}}}[${{window}}])).

[0048] 504. The first query converter executes a query conversion process to convert the first query request instruction into a second query request instruction of the PromQL template, so that the second query request instruction can be executed by the time-series database; wherein the query conversion process includes: Field name conversion: service_name=order-service → exported_service=order-service; Variable substitution: ${{filter}} → exported_service='order-service', ${{window}} → 60s; Query generator configuration information adaptation: Assemble the complete PromQL expression, that is, generate the second query request instruction.

[0049] 505. Output the second query request command, which is: histogram_quantile(0.99,sum by(le)(rate(http_request_duration_seconds_bucket{exported_service='order-service'}[60s]))).

[0050] In this embodiment, the first query converter automatically completes storage engine determination, field mapping conversion, filter condition expression conversion, and query instruction assembly based on the first conversion rule information. This achieves automated conversion from a first query request instruction constructed with a unified query syntax to a second query request instruction that can be executed by the first target storage engine, without requiring the application to be aware of the details of the underlying storage engine.

[0051] In some embodiments, the first target dataset may be stored in multiple storage engines. For example, for the first indicator data in the indicator dataset, there may be two storage scenarios. In one scenario, the first indicator data is stored in a time-series database; in another scenario, the first indicator data is not pre-stored in a time-series database but needs to be aggregated and calculated in real time from a full-text search engine. In both scenarios, the application triggers a first query request instruction. The system automatically identifies whether the queried first indicator data is stored in a time-series database or needs to be aggregated and calculated in real time from a full-text search engine, and executes the corresponding storage engine's conversion logic, such as... Figure 6 As shown below, the details are explained in detail: 601. Receive the first query request instruction triggered by the application (.metric_set with (domain, name, query, step, group_by)).

[0052] 602. Based on the process of the above embodiment, obtain the configuration information of the first query generator. If the configuration information of the first query generator is a PromQL template, then enter the time-series database mode; if the configuration information of the first query generator is an SPL pipeline configuration, then enter the real-time calculation mode of the full-text search engine.

[0053] In time-series database mode: 6031. Convert the logical field names in the query filter conditions of the first query request instruction into PromQL tag selectors and replace the variable placeholders in the PromQL template.

[0054] 6032. Modify the PromQL aggregation expression according to the group_by parameter in the first query request instruction, such as changing sumby(a) to sum by(a, b).

[0055] 6033. Set the time range query window according to the step parameter. This time query window determines the sampling interval of the data points. For example, when the step size is 60 seconds, the query result will return a data point every 60 seconds.

[0056] 6034. Assemble all the transformation results into a complete PromQL query statement (second query request instruction), which can be directly sent to the time series database for execution.

[0057] In the real-time calculation mode of the full-text search engine: 6041. Determine the target data source to be queried in the full-text search engine.

[0058] 6042. Based on the query filtering conditions in the first query request instruction and combined with the first field conversion rules corresponding to the full-text search engine, convert the logical field names into actual physical field names in the full-text search engine, and construct the corresponding filtering expressions.

[0059] 6043. Construct an aggregation calculation expression based on the indicator definition, and perform real-time aggregation calculation on the filtered log data records.

[0060] 6044. Serialize the aggregation calculation results according to the step parameter to form a standard time series format.

[0061] As can be seen, the first query request instruction constructed based on the same unified query syntax does not need to know whether the data to be queried is pre-stored in the time-series database or needs to be calculated in real time from the full-text search engine. It directly converts the first query request instruction into a second query request instruction that can be executed by the time-series database or the full-text search engine according to the corresponding first transformation rule information.

[0062] In an alternative embodiment, the first query converter is further configured to perform the following processing: If the data type identifier of the first target dataset indicates that the first target dataset is a preset general dataset, then according to the logical field list and the first field conversion rule set in the first conversion rule information, the physical field name of each logical field name in the logical field list is converted, and an instruction is generated to output the data corresponding to the converted physical field name.

[0063] In some embodiments, log datasets, link tracing datasets, event datasets, and performance profiling datasets all correspond to a general query converter. The general query converter executes a unified conversion process. The differences between the above four dataset types are only reflected in the dataset type parameters, such as different logical field lists, different storage mapping relationships, and different default query filtering conditions. When the first query converter determines that the data type identifier of the first target dataset indicates that the dataset is a preset general dataset, it triggers the parameterized unified conversion process of the first query converter (general converter). The conversion process of the first query converter is as follows: obtain the storage mapping relationship from the first conversion rule information, determine the first target storage engine and its connection parameters; convert the query filtering conditions in the first query request instruction into target filtering expressions that the first target storage engine can execute, and merge them with the default query filtering conditions corresponding to the storage engine; according to the logical field list and the first field conversion rules set in the first conversion rule information, convert each logical field name in the logical field list into a physical field name, and generate an instruction for outputting the data corresponding to the converted physical field names. The logical field list set in the first conversion rule information contains analyzable logical fields, which are used to limit the range of fields returned by the query results; finally, assemble the above conversion results to form a second query request instruction.

[0064] In the foregoing embodiments, the application constructs the first query request instruction directly against the underlying dataset. In some scenarios, users often prefer to query higher-level entities, such as the "service" entity, the "host" entity, or the "alarm rule" entity, rather than directly querying the dataset containing storage service metrics or the dataset containing host logs.

[0065] Based on this, in one optional embodiment, a query processing method based on entity sets is provided, such as... Figure 7 As shown, the specific steps include: 701. Receive a third query request instruction, which includes the entity set identifier of the first entity set to be queried, entity filtering conditions, and a first method name. The first method name indicates the operation to be performed on the first entity set.

[0066] 702. Determine the first method processor corresponding to the first method name.

[0067] 703. Using the first method processor, based on the entity set identifier, entity filtering conditions, and the first method name, determine the data type identifier and query parameters of the second target dataset associated with the first entity set.

[0068] 704. Determine the second query converter corresponding to the data type identifier of the second target dataset.

[0069] 705. Obtain the second transformation rule information corresponding to the second target dataset.

[0070] Among them, the second transformation rule information is adapted to the second target storage engine that stores the second target dataset.

[0071] 706. Using the second query converter and the second conversion rule information, generate a fourth query request instruction that can be executed by the second target storage engine.

[0072] The third query request instruction includes the entity set identifier of the first entity set to be queried, entity filtering conditions, and the first method name. An entity set is a collection of managed objects or logical units; an entity set typically associates with one or more datasets. For example, the "service" entity set might associate with datasets of storage service metrics, storage service logs, and storage service call chains. The entity set identifier identifies the first entity set to be queried and may include the entity set's domain (e.g., "apm", "infrastructure") and entity set name (e.g., "service", "host"). Here, "apm" represents the application performance monitoring domain, "infrastructure" represents the infrastructure domain, "service" represents the service entity set, and "host" represents the host entity set. Entity filtering conditions are used to filter entities in the entity set that meet specific conditions. These conditions can be expressed as "service name equals order service," "host status equals running," or "availability zone belongs to Shanghai region," etc. The first method name indicates the operation that the application needs to perform on the entity set. The method name is a predefined operation identifier; different method names correspond to different processing logic and different underlying datasets.

[0073] In some embodiments, the third query request instruction can be: .entity_set with(domain='apm', name='apm.service', query, entity-callmethod_name(params)), where 'apm' and 'apm.service' are entity set identifiers, query is the entity filtering condition, entity-callmethod_name() is the first method name, and params are the method parameters in the first method name.

[0074] After receiving the third query request instruction, the system extracts the first method name. The system maintains a mapping table between method names and method processors, and finds and determines the corresponding first method processor based on the first method name. A method processor is a functional unit that implements the logic of a specific operation method. Each method processor encapsulates the following information: which underlying datasets the method needs to query, and how to construct the query parameters for the datasets based on entity filtering conditions.

[0075] After the first method processor is determined, the entity set identifier, entity filtering conditions, and first method name are passed to the first method processor as input parameters. The first method processor performs the following processing flow: based on the entity set identifier and the first method name, it determines which underlying datasets need to be queried, that is, it determines the data type identifier of the second target dataset associated with the first entity set.

[0076] Based on the entity filtering conditions, construct the query parameters required to query the second target dataset. The query parameters include the dataset identifier of the second target dataset and the query filtering conditions.

[0077] Based on the data type identifier of the second target dataset, the second query converter corresponding to the data type identifier of the second target dataset can be determined. Similar to the previous embodiments, different data type identifiers correspond to different query converters. As described above, query converters can be divided into two categories: dedicated query converters and general query converters, which will not be elaborated here.

[0078] The query parameters required for the second target dataset include the dataset identifier and query filtering conditions of the second target dataset. In some embodiments, after determining the second query converter corresponding to the data type identifier of the second target dataset, the corresponding second conversion rule information is retrieved from the conversion rule information database using the dataset identifier of the second target dataset as the key. The second conversion rule information includes: the storage mapping relationship between the second target dataset and the second target storage engine; the second field conversion rules between the logical field names used by the application in the second target dataset and the physical field names in the second target storage engine; the second filtering condition template corresponding to the second target storage engine; and the second query generator configuration information. The second query generator configuration information is used to describe the construction logic of query instructions corresponding to different fields in the second target dataset. The storage mapping relationship includes the connection parameters of the second target storage engine. The storage mapping relationship describes the binding relationship between the second target dataset and the second target storage engine, indicating which storage engine the dataset is stored in. This storage mapping relationship may include: the type of storage engine and the connection parameters of the storage engine. The type of storage engine identifies the type of storage engine, and the connection parameters are the connection information required to access the storage engine, such as address, port, authentication information, etc. The second field conversion rule describes the correspondence between the logical field names used by the application in the unified query syntax structure and the actual physical field names stored in the second target storage engine. For example, applications often use semantically clear field names such as service_name, node_id, and timestamp, while storage engines may use different field naming conventions, such as exported_service, node, and @timestamp. The second filter condition template describes how to convert the query filter conditions corresponding to the second target dataset into a native filter syntax that the second target storage engine can recognize. The second query generator configuration information describes the construction logic of query instructions corresponding to different fields in the second target dataset, that is, it defines how to assemble the elements in the unified query syntax structure into the native query instructions of the target storage engine.

[0079] Using the second query converter and the second transformation rule information, a fourth query request instruction that can be executed by the second target storage engine is generated, specifically including the following steps: 7061. Determine the second target storage engine based on the storage mapping relationship, and establish a connection with the second target storage engine based on the connection parameters.

[0080] 7062. According to the second field conversion rule, convert the logical field names in the query parameters into physical field names in the second target storage engine.

[0081] 7063. Based on the second filter condition template, convert the query filter conditions in the query parameters into filter expressions corresponding to the second target storage engine.

[0082] 7064. Based on the configuration information of the second query generator, and combined with the converted physical field names and filtering expressions, generate the fourth query request instruction.

[0083] Furthermore, if the third query request instruction is associated with multiple datasets, the system will generate a corresponding fourth query request instruction for each dataset in parallel or serially, and send them to their respective storage engines for execution.

[0084] This application's embodiments introduce entity sets and method processors, enabling applications to query based on entities rather than directly manipulating the underlying dataset. This makes the query expressions more closely resemble the application scenario, and users do not need to understand the underlying data storage structure, field naming, and query syntax, thus simplifying the query process.

[0085] In one optional embodiment, the data type identifier and query parameters of the second target dataset associated with the first entity set are determined based on the entity set identifier, entity filtering conditions, and the first method name, including the following steps: 7031. Based on the entity set identifier and the method parameters in the first method name, determine the dataset identifier and data type identifier of the second target dataset from the data association description information of the first entity set.

[0086] 7032. Based on the dataset identifier of the second target dataset, obtain the second field transformation rule, wherein the second field transformation rule is used to describe the mapping relationship between the field names of different entities in the first entity set and the logical field names in the second target dataset.

[0087] 7033. Based on entity filtering conditions and second field transformation rules, determine the query filtering conditions for the second target dataset.

[0088] 7034. Based on the dataset identifier and query filtering conditions of the second target dataset, determine the query parameters of the second target dataset.

[0089] Upon receiving the third query request instruction, the entity set identifier and the first method name of the first entity set are extracted, and the method parameters are parsed from the first method name. Based on the entity set identifier and method name (including method parameters), the second target dataset associated with the first entity set is searched and determined from the pre-configured data association description information. The data association description information records which underlying datasets need to be queried for a given combination of entity sets and methods.

[0090] Using the dataset identifier of the second target dataset as the key, the corresponding second transformation rule information is retrieved from the transformation rule information database. The second transformation rule information includes: the storage mapping relationship between the second target dataset and the second target storage engine; the second field transformation rules between the logical field names used by the application in the second target dataset and the physical field names in the second target storage engine; the second filter condition template corresponding to the second target storage engine; and the second query generator configuration information. The second query generator configuration information describes the construction logic of query instructions corresponding to different fields in the second target dataset. The storage mapping relationship includes the connection parameters of the second target storage engine.

[0091] Extract entity filtering conditions from the third query request instruction, and then convert the entity filtering conditions into query filtering conditions for the second target dataset according to the second field transformation rules.

[0092] Finally, based on the dataset identifier and query filtering conditions of the second target dataset obtained above, the query parameters of the second target dataset are obtained.

[0093] In this embodiment, the mapping to the underlying dataset is completed based on the entity set identifier and method parameters, and the entity filtering conditions are converted into the query filtering conditions of the dataset based on the second field conversion rule. This eliminates the need to be aware of the differences in the underlying data distribution and field naming, significantly simplifying the complexity of the query.

[0094] In some scenarios, certain metrics are particularly important, such as service request latency, throughput, error rate, and system saturation. These metrics are often referred to as key performance indicators (KPIs). When an application triggers a query for entity set-related data, it tends to pay more attention to and access these KPIs more frequently.

[0095] To meet the above requirements, a key indicator data processing flow is provided. When it is determined that the dataset associated with the entity set contains indicator data marked as key indicators, this key indicator data is automatically retrieved from the target storage engine and output, without the application needing to explicitly specify it in the query parameters or obtain it through additional query steps.

[0096] In one optional embodiment, the key indicator data processing flow includes: if it is determined that the second target dataset associated with the first entity set contains indicator data marked as key indicators, then the indicator data marked as key indicators is obtained from the second target storage engine and the indicator data is output, wherein the data type of the second target dataset is identified as the indicator type.

[0097] Key metrics refer to indicator data that have core reference value for assessing system status and application status. The specific meaning of key metrics may vary in different scenarios. Specifically, after determining the second target dataset associated with the first entity set, the data type identifier of the associated second target dataset is further checked to see if it is an indicator type. If the data type identifier is an indicator type, then based on the second transformation rule information corresponding to the second target dataset, it is determined whether there is indicator data marked as a key metric. If it is determined that the second target dataset contains indicator data marked as a key metric, then the indicator data marked as a key metric is retrieved from the second target storage engine and output to the application. In some embodiments, key metric data can be returned along with the query results, or it can be embedded as an independent output field in the returned results.

[0098] By identifying and acquiring key indicator data, the system can automatically identify and return key indicator data associated with entity sets. Applications do not need to specify these core indicators in the query parameters, which simplifies the query process.

[0099] In the foregoing embodiments, the application requires the user to construct a third query request instruction, including specifying an entity set identifier, entity filtering conditions, and a first method name. However, in real-world applications, users may not be aware of which method names a given entity set supports, nor the dataset types to which each method name is applicable. To improve query efficiency, this application provides an automatic query request instruction generation process based on a method query interface, such as... Figure 8 As shown, the specific steps include: 801. According to the preset method query interface, query the description information of multiple method names corresponding to the entity set identifier of the first entity set. Each method name description information includes the type identifier of the applicable dataset.

[0100] 802. Based on the entity set identifier of the first entity set, determine the type identifier of each dataset associated with the first entity set.

[0101] 803. Among multiple method names, determine the method name corresponding to each dataset associated with the first entity set.

[0102] 804. Based on the description information of the method names corresponding to each dataset associated with the first entity set and the query intent entered by the user, generate a third query request instruction, wherein the query intent includes the entity set identifier of the first entity set, entity filtering conditions and operation behavior.

[0103] The preset method query interface is a pre-provided interface by the system for querying method name description information. All method names and their description information supported by a specified entity set can be obtained through the preset method query interface.

[0104] The description information for a method name is metadata information recorded in the system that describes the method name. The description information for each method name includes at least the following: the method name identifier, the applicable dataset type identifier, the method function description, and the method parameter definitions.

[0105] After receiving a query request, the method query interface carries the entity set identifier of the first entity set in the query request. Based on the entity set identifier, it searches for all method names associated with the first entity set and returns the description information for each method name. Using the entity set identifier as the key, it searches for pre-configured data association description information, which records the mapping relationship between the entity set and the underlying dataset, including the data type identifier of each dataset.

[0106] The method names obtained in step 801 are matched with the type identifiers of each dataset associated with the first entity set obtained in step 802 to determine the method name corresponding to each dataset.

[0107] The query intent entered by the user can be a query statement in natural language. Parsing the query intent can extract the entity set identifier of the first entity set, the entity filtering conditions, and the description of the user's expected operation behavior.

[0108] In some embodiments, such as Figure 9 As shown, assume the entity set queried by the user is "Service". This entity set is associated with the following three datasets: a metrics dataset, a log dataset, and a performance profiling dataset. The metrics dataset, log dataset, and performance profiling dataset are each linked to the "Service" entity set through association relationships. Based on the datasets associated with the entity set, calling the `list_method()` function determines the list of method names. The basic methods are those common to all entity sets, including: `list_method`, `get_entity_set` for retrieving basic information about the entity set, `list_data_set` for listing all datasets associated with the entity set, and `list_related_entity_set` for listing other entity sets associated with the current entity set. Based on this, extended methods can be added according to the datasets associated with the entity set: if associated with the metrics dataset, add `get_metric` to query metrics data, `get_label_values` to retrieve metric label values, and `get_golden_metrics` to retrieve key metric data; if associated with the log dataset, add `get_log` to query log data; if associated with the performance profiling dataset, add `get_profile` to query performance profiling data.

[0109] like Figure 10As shown, the entity set queried by the user is "hosts," and the datasets associated with the "hosts" entity set are: the metrics dataset and the log dataset. Figure 9 The difference in the illustrated embodiment is that the dataset associated with the "host" does not include the performance profiling dataset. Therefore, the method names corresponding to the host entity set include: list_method, get_entity_set, list_data_set, list_related_entity_set, plus get_metric, get_label_values, and get_golden_metrics provided by the metric dataset.

[0110] The application can automatically query the method names supported by the entity set and their applicable scenarios through a pre-defined method query interface, without requiring the user to know the method names in advance. In addition, it can automatically match and generate a complete query request instruction based on the user's query intent, simplifying the application's query process.

[0111] In some scenarios, it is often necessary to query the relationship between two entity sets. For example, querying the deployment relationship between a service and a host, the relationship between a user and an order, or the triggering relationship between alarm rules and monitoring metrics. To meet these needs, embodiments of this application provide a method for querying entity set relationships, such as... Figure 11 As shown, the specific steps include: 1101. Receive the fifth query request instruction. The fifth query request instruction is used to query the association relationship data between the second entity set and the third entity set. The fifth query request instruction includes the entity set identifier of the second entity set, the entity filtering conditions of the second entity set, the entity set identifier of the third entity set, the entity filtering conditions of the third entity set, and the second method name corresponding to the query association relationship data.

[0112] 1102. Determine the second method processor corresponding to the second method name.

[0113] 1103. Using the second method processor, based on the entity set identifier of the second entity set, the entity filtering conditions of the second entity set, the entity set identifier of the third entity set, the entity filtering conditions of the third entity set, and the second method name corresponding to the query relationship data, the data type identifier and query parameters of the third target dataset are determined. The third target dataset stores the relationship data between the second entity set and the third entity set.

[0114] 1104. Determine the third query converter corresponding to the data type identifier of the third target dataset.

[0115] 1105. Obtain the third transformation rule information corresponding to the third target dataset. The third transformation rule information is adapted to the third target storage engine that stores the third target dataset.

[0116] 1106. Using the third query converter and the third conversion rule information, generate a sixth query instruction that can be executed by the third target storage engine.

[0117] The association data between the second entity set and the third entity set can be: data on the calling behavior between the second entity set and the third entity set, recording the calling behavior initiated by the calling entity set to the called entity set. The association data can include the following information: calling entity identifier (identifying the entity set that initiated the call), called entity identifier (identifying the entity set that received the call), calling metrics, calling time, etc.

[0118] The fifth query request instruction includes the entity set identifier of the second entity set, the entity filtering conditions of the second entity set, the entity set identifier of the third entity set, the entity filtering conditions of the third entity set, and the second method name corresponding to the query relationship data. Assuming the second entity set is the entity set initiating the call, and the third entity set is the entity set being called, the entity filtering conditions of the second entity set are used to filter entities in the caller's entity set that meet specific conditions, and the entity filtering conditions of the third entity set are used to filter entities in the callee's entity set that meet specific conditions. The second method name indicates the specific operation behavior for querying the call relationship data. The second method name may include "query call count," "query call latency distribution," "query call error rate," "query call chain," etc., and may also carry method parameters.

[0119] Upon receiving the fifth query request instruction, the system parses it and extracts the second method name. The system maintains a mapping table between method names and method handlers. Based on the extracted second method name, the system searches the mapping table to determine the corresponding second method handler. Once the second method handler is determined, the system calls it, passing the parameters from the fifth query request instruction as input. The second method handler then performs the following operations based on its internal processing logic: 11031. Based on the second method name, determine the third target dataset for storing the relational data, that is, determine the data type identifier of the third target dataset.

[0120] 11032. Based on the entity filtering conditions of the second entity set and the third entity set, construct the query parameters required to query the third target dataset. Specifically, this can be done by: converting the entity filtering conditions of the second entity set into query filtering conditions for the caller identifier field in the third target dataset; similarly, converting the entity filtering conditions of the third entity set into query filtering conditions for the callee identifier field in the third target dataset; extracting method parameters from the second method name and determining the data type identifier of the queried third target dataset; and then, based on the above information, determining the data type identifier and query parameters of the third target dataset.

[0121] Based on the data type identifier of the third target dataset, the third query converter corresponding to the data type identifier of the third target dataset can be determined. Similar to the previous embodiments, different data type identifiers correspond to different query converters. As described above, query converters can be divided into two categories: dedicated query converters and general query converters, which will not be elaborated here.

[0122] The query parameters required for the third target dataset include the dataset identifier and query filtering conditions of the third target dataset. In some embodiments, after determining the third query converter corresponding to the data type identifier of the third target dataset, the corresponding third transformation rule information is retrieved from the transformation rule information database using the dataset identifier of the third target dataset as the key. The third transformation rule information includes: the storage mapping relationship between the third target dataset and the third target storage engine, the third field transformation rules between the logical field names used by the application in the third target dataset and the physical field names in the third target storage engine, the third filtering condition template corresponding to the third target storage engine, and the third query generator configuration information. The third query generator configuration information is used to describe the construction logic of query instructions corresponding to different fields in the third target dataset. The storage mapping relationship includes the connection parameters of the third target storage engine. The storage mapping relationship describes the binding relationship between the third target dataset and the third target storage engine, i.e., indicating which storage engine the dataset is stored in. The storage mapping relationship may include: the type of storage engine and the connection parameters of the storage engine. The type of storage engine is used to identify the type of storage engine, and the connection parameters are the connection information required to access the storage engine, such as address, port, authentication information, etc. The third field transformation rule describes the correspondence between the logical field names used by the application in the unified query syntax structure and the physical field names actually stored in the third target storage engine. The third filter condition template describes how to convert the query filter conditions corresponding to the third target dataset into a native filter syntax that the third target storage engine can recognize. The third query generator configuration information describes the construction logic of query instructions corresponding to different fields in the third target dataset, that is, it defines how to assemble the elements in the unified query syntax structure into the native query instructions of the target storage engine.

[0123] Using the third query converter and third conversion rule information, a sixth query request instruction that can be executed by the third target storage engine is generated, specifically including the following steps: 11061. Determine the third target storage engine based on the storage mapping relationship, and establish a connection with the third target storage engine based on the connection parameters.

[0124] 11062. Based on the third field conversion rules, convert the logical field names in the query parameters into physical field names in the third target storage engine.

[0125] 11063. Based on the third filter condition template, convert the query filter conditions in the query parameters into the filter expression corresponding to the third target storage engine.

[0126] 11064. Based on the configuration information of the third query generator, combined with the converted physical field names and filtering expressions, generate the sixth query request instruction.

[0127] like Figure 12 As shown, Figure 12 The diagram illustrates the data query processing principle of the relationship between two entity sets. Entity relationship area 1201 shows the second entity set "Service A" and the third entity set "Service B". The relationship between the second and third entity sets is a call relationship where the second entity set calls the third entity set, and the corresponding associated dataset is the service call indicator dataset. Field mapping area 1202 shows the field conversion rules for the relationship, specifically: converting the entity field names corresponding to the second entity set into physical field names in the third target storage engine, and converting the entity field names corresponding to the third entity set into physical field names in the third target storage engine. The automatic orchestration area 1203 illustrates the process of querying entity set relationships, specifically: the user inputs the fifth query request command (get_relation_metric(dest_query='service B')), and the second method processor searches for the associated metric dataset (the third target dataset); according to the relationship mapping rules in the field mapping area 1202, the query parameters required to query the third target dataset are constructed, including the query filter conditions: source_service='A' AND target_service='B'; then the third query generator is called to generate the sixth query request command; finally, the sixth query request command is output.

[0128] In this embodiment, by introducing an entity set association query process, the application can directly query the association data between two related entity sets without needing to understand the specific storage structure and query syntax of the association data in the underlying storage engines, thus realizing unified access to entity set association queries.

[0129] In some embodiments, the operations and maintenance knowledge base dataset is used to store knowledge data such as troubleshooting experience, operations and maintenance manuals, and solutions to common problems. Some embodiments' knowledge retrieval methods typically only support single keyword matching, which is insufficient to meet users' needs to find relevant solutions by describing problems in natural language.

[0130] To address the aforementioned issues, some embodiments also provide a multi-mode retrieval and transformation scheme for the operations and maintenance knowledge base dataset. The operations and maintenance knowledge base dataset supports three retrieval modes: vector retrieval mode, keyword retrieval mode, and hybrid retrieval mode. The query converter automatically selects the corresponding transformation rule information based on the retrieval mode configuration in the transformation rule base, converting the query request command triggered by the application into a retrieval query executable by the target storage engine. All three retrieval modes expose the same query interface to the application; the application only needs to trigger the query request command and does not need to know which retrieval mode is used at the underlying level.

[0131] The query processing apparatus of one or more embodiments of this application will be described in detail below. Those skilled in the art will understand that these apparatuses can all be configured using commercially available hardware components through the steps taught in this solution.

[0132] Figure 13 This is a schematic diagram of a query processing device provided in an embodiment of this application. The device is applied to electronic devices, such as... Figure 13 As shown, the device includes: a receiving module 11, a query converter determination module 12, a conversion rule acquisition module 13, and a conversion module 14.

[0133] The receiving module 11 receives a first query request instruction triggered by the application. The first query request instruction is constructed based on a preset unified query syntax structure. The first query request instruction includes the data type identifier and dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. The query converter determination module 12 determines the first query converter corresponding to the data type identifier of the first target dataset; The transformation rule acquisition module 13 acquires first transformation rule information corresponding to the dataset identifier, and the first transformation rule information is adapted to the first target storage engine storing the first target dataset. The conversion module 14 converts the first query request instruction into a second query request instruction that can be executed by the first target storage engine using the first query converter and the first conversion rule information.

[0134] Figure 13 The device shown can execute the steps in the query processing method in the foregoing embodiments. For detailed execution process and technical effects, please refer to the description in the foregoing embodiments, which will not be repeated here.

[0135] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 14 As shown, in practice, this electronic device includes a memory 21 and a processor 22.

[0136] Memory 21 is used to store computer programs and can be configured to store various other data to support operation on the electronic device. Examples of this data include instructions for any application or method used to operate on the electronic device, data structures, contact data, phone book data, messages, pictures, videos, etc.

[0137] The processor 22, coupled to the memory 21, is used to execute the computer program in the memory 21 to implement the query processing method provided in the foregoing embodiments.

[0138] Furthermore, such as Figure 14 As shown, the electronic device also includes other components such as a communication component 23, a display 24, a power supply component 25, and an audio component 26. Figure 14 The diagram only shows some components and does not mean that the electronic device includes only these components. Figure 14 The components shown are as follows. The electronic device in this embodiment can be a terminal device such as a desktop computer, laptop computer, smartphone, or IoT device, or a server device such as a conventional server, cloud server, or server array.

[0139] The aforementioned memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as Static Random-Access Memory (SRAM), Electrically Erasable Programmable Read Only Memory (EEPROM), Erasable Programmable Read Only Memory (EPROM), Programmable Read-Only Memory (PROM), Read-Only Memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0140] The aforementioned communication component is configured to facilitate wired or wireless communication between the device containing the communication component and other devices. The device containing the communication component can access wireless networks based on communication standards, such as 2G, 3G, 4G / LTE, 5G, or combinations thereof. In one exemplary embodiment, the communication component receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel.

[0141] The aforementioned display includes a screen, which may include a Liquid Crystal Display (LCD) and a Touch Panel (TP). If the screen includes a Touch Panel, the screen can be implemented as a touchscreen to receive input signals from the user. The Touch Panel includes one or more touch sensors to sense touches, swipes, and gestures on the Touch Panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also the duration and pressure associated with the touch or swipe operation.

[0142] The aforementioned power supply components provide power to various components within the device in which they reside. These power supply components may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device in which they reside.

[0143] The aforementioned audio component can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC) configured to receive external audio signals when the device containing the audio component is in an operating mode, such as call mode, recording mode, or voice recognition mode. The received audio signals can be further stored in memory or transmitted via a communication component. In some embodiments, the audio component also includes a speaker for outputting audio signals.

[0144] Accordingly, embodiments of this application also provide a computer-readable storage medium storing a computer program, which, when executed by a processor, enables the processor to implement the steps in the above-described method embodiments. The computer-readable storage medium includes volatile or non-volatile components, or a combination thereof, and can be removable or non-removable. Examples of computer-readable storage media include, but are not limited to, phase-change random access memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), flash memory or other memory technologies, CD-ROM, Digital Video Disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium. Accordingly, this application also provides a computer program product, which includes a computer program or instructions that, when executed by a processor, cause the processor to implement the steps in the above method embodiments. It should be understood that each step or combination of steps in the above method flow can be implemented by the computer program or instructions. Furthermore, these computer programs or instructions can be applied to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device, enabling the processor of the general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing device to function as an apparatus for implementing the corresponding functions in the above method embodiments.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A query processing method, characterized in that, The method includes: The system receives a first query request instruction triggered by an application. The first query request instruction is constructed based on a preset unified query syntax structure. The first query request instruction includes a data type identifier and a dataset identifier of the first target dataset to be queried. The unified query syntax structure is applicable to different storage engines. The first target dataset stores data corresponding to the data type identifier. The first query request instruction also includes query filtering conditions, which include target logical field names. Determine the first query converter corresponding to the data type identifier of the first target dataset; Obtain first transformation rule information corresponding to the dataset identifier. The first transformation rule information is adapted to the first target storage engine storing the first target dataset. The first transformation rule information includes: a storage mapping relationship between the first target dataset and the first target storage engine; a first field transformation rule between logical field names used by the application in the first target dataset and physical field names in the first target storage engine; a first filter condition template and a first query generator configuration information corresponding to the first target storage engine; the first query generator configuration information is used to describe the construction logic of query instructions corresponding to different fields in the first target dataset; the storage mapping relationship includes the connection parameters of the first target storage engine. The first conversion rule information is input into the first query converter, so that the first query converter performs the following processing: Based on the storage mapping relationship, the first target storage engine is determined, and the first target storage engine is connected according to the connection parameters; According to the first field conversion rule, the target logical field name is converted into the corresponding target physical field name; Based on the first filtering condition template, the query filtering condition is converted into a target filtering expression corresponding to the first target storage engine; Based on the configuration information of the first query generator, the target physical field name, and the target filtering expression, a second query request instruction is generated.

2. The method according to claim 1, characterized in that, The first query converter is also used to perform the following processing: If the data type identifier of the first target dataset indicates that the first target dataset is a preset general dataset, then according to the logical field list set in the first conversion rule information and the first field conversion rule, the physical field name of each logical field name in the logical field list is converted, and an instruction for outputting the data corresponding to the converted physical field name is generated.

3. The method according to claim 1 or 2, characterized in that, The first target dataset includes a knowledge base dataset; the first transformation rule information includes a retrieval mode configuration; the method further includes: The target retrieval mode is determined based on the retrieval mode configuration, and the target retrieval mode includes vector retrieval mode, keyword retrieval mode, or hybrid retrieval mode; The first query request instruction is converted into the second query request instruction using the first query converter and the subset of conversion rules corresponding to the target retrieval mode.

4. The method according to claim 1 or 2, characterized in that, The method further includes: Receive a third query request instruction, the third query request instruction including the entity set identifier of the first entity set to be queried, entity filtering conditions and a first method name, the first method name indicating the operation behavior to be performed on the first entity set; A first method processor corresponding to the first method name is determined; the first method processor is a functional unit that encapsulates operation logic, and the information encapsulated by the first method processor includes: the underlying dataset that the method corresponding to the first method name needs to query, and the way to construct the query parameters of the dataset according to the entity filtering conditions; The first method processor determines the data type identifier and query parameters of the second target dataset associated with the first entity set based on the entity set identifier, the entity filtering conditions, and the first method name. Determine the second query converter corresponding to the data type identifier of the second target dataset; Obtain second transformation rule information corresponding to the second target dataset, wherein the second transformation rule information is adapted to the second target storage engine storing the second target dataset; The second query converter and the second conversion rule information are used to generate a fourth query request instruction that can be executed by the second target storage engine.

5. The method according to claim 4, characterized in that, The first method name includes method parameters; determining the data type identifier and query parameters of the second target dataset associated with the first entity set based on the entity set identifier, the entity filtering conditions, and the first method name includes: Based on the entity set identifier and the method parameters in the first method name, the dataset identifier and data type identifier of the second target dataset are determined from the data association description information of the first entity set; Based on the dataset identifier of the second target dataset, a second field conversion rule is obtained. The second field conversion rule is used to describe the mapping relationship between the field names of different entities in the first entity set and the logical field names in the second target dataset. Based on the entity filtering conditions and the second field transformation rules, the query filtering conditions for the second target dataset are determined; Based on the dataset identifier of the second target dataset and the query filtering conditions, the query parameters of the second target dataset are determined.

6. The method according to claim 4, characterized in that, The method further includes: If it is determined that the second target dataset associated with the first entity set contains indicator data marked as key indicators, then the indicator data marked as key indicators is obtained from the second target storage engine and the indicator data is output. The data type of the second target dataset is identified as indicator type.

7. The method according to claim 4, characterized in that, The method further includes: According to the preset method query interface, query the description information of multiple method names corresponding to the entity set identifier of the first entity set. Each method name description information includes the type identifier of the applicable dataset. Based on the entity set identifier of the first entity set, determine the type identifier of each dataset associated with the first entity set; Among the plurality of method names, determine the method name corresponding to each dataset associated with the first entity set; Based on the description information of the method names corresponding to each dataset associated with the first entity set and the query intent entered by the user, the third query request instruction is generated. The query intent includes the entity set identifier of the first entity set, the entity filtering conditions, and the operation behavior.

8. The method according to claim 1 or 2, characterized in that, The method further includes: Receive a fifth query request instruction, which is used to query the association relationship data between the second entity set and the third entity set. The fifth query request instruction includes the entity set identifier of the second entity set, the entity filtering conditions of the second entity set, the entity set identifier of the third entity set, the entity filtering conditions of the third entity set, and the second method name corresponding to querying the association relationship data. Determine the second method processor corresponding to the second method name; the second method processor is a functional unit that encapsulates operation logic, and the information encapsulated by the second method processor includes: the underlying dataset that the method corresponding to the second method name needs to query, and the way to construct the query parameters of the dataset according to the entity filtering conditions; The second method processor determines the data type identifier and query parameters of the third target dataset based on the entity set identifier of the second entity set, the entity filtering conditions of the second entity set, the entity set identifier of the third entity set, the entity filtering conditions of the third entity set, and the second method name corresponding to querying the association relationship data. The third target dataset stores the association relationship data between the second entity set and the third entity set. Determine the third query converter corresponding to the data type identifier of the third target dataset; Obtain the third transformation rule information corresponding to the third target dataset, wherein the third transformation rule information is adapted to the third target storage engine storing the third target dataset; Using the third query converter and the third conversion rule information, a sixth query instruction that can be executed by the third target storage engine is generated.

9. An electronic device, characterized in that, include: Memory and processor; The memory is used to store one or more computer instructions; The processor is configured to execute one or more computer instructions for performing the steps of the method according to any one of claims 1-8.

10. A computer-readable storage medium storing a computer program, characterized in that, When a computer program is executed by a processor, it is able to perform the steps of the method described in any one of claims 1-8.

11. A computer program product, characterized in that, include: A computer program / instruction that, when executed by a processor, enables the implementation of the steps in the method described in any one of claims 1-8.

Images