Multi-Trino cluster-based query routing method and system, and electronic equipment

Abstract

The invention provides a query routing method and system based on multiple Trino clusters and electronic equipment. The method comprises the following steps: configuring a plurality of Trino clusters; establishing a corresponding relationship between the query scene and the Trino cluster; wherein the Trino clusters corresponding to the same query scene are located in different machine rooms; and obtaining a to-be-processed query request, and routing the to-be-processed query request to the Trino cluster corresponding to the query scene to which the to-be-processed query request belongs for processing. According to the method, different cluster resources can be reasonably allocated, high availability of the Trino cluster is achieved, meanwhile, resource isolation under different query scenes can be achieved, and cluster collapse caused by resource scrambling is avoided.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular, to a query routing method, system and electronic device based on multiple Trino clusters. Background Art

[0002] With the rapid development of big data and distributed computing technologies, existing distributed computing frameworks such as Apache Spark, Flink, etc. have been widely applied. As a distributed OLAP SQL computing engine, Trino supports federated queries from multiple data sources such as Hive, Mysql, Iceberg, Doris, etc., and has significant advantages.

[0003] Trino mainly consists of two roles: Coordinator and Worker. The client sends query requests through JDBC. The Coordinator is responsible for receiving SQL, planning the query, and sending execution tasks to the Workers; the Workers are responsible for pulling and calculating data, and reporting the calculation results to the Coordinator; the Coordinator returns the results to the client. This architecture enables Trino to efficiently process complex data query tasks and meet the needs of modern enterprises for real-time data analysis.

[0004] However, in practical applications, due to the characteristics of the Trino architecture, the high availability of the cluster faces challenges in the case of a data center network failure or Coordinator downtime; at the same time, in the case of shared computing resources, resource contention between different services can easily lead to mutual influence. Summary of the Invention

[0005] In view of this, the purpose of the present invention is to provide a query routing method, system and electronic device based on multiple Trino clusters, which are used to achieve high availability of the Trino cluster and resource isolation between different query scenarios. To achieve the above purpose, the technical solutions adopted in the embodiments of the present invention are as follows:

[0006] In a first aspect, the present invention provides a query routing method based on multiple Trino clusters, the method comprising: configuring a plurality of Trino clusters; establishing a correspondence between query scenarios and Trino clusters; wherein, the Trino clusters corresponding to the same query scenario are located in different data centers; obtaining a query request to be processed, and routing the query request to be processed to the Trino cluster corresponding to the query scenario to which it belongs for processing.

[0007] In an alternative embodiment, the method further includes: establishing a correspondence between the client account and the query scenario; determining the query scenario to which the query request to be processed belongs based on the correspondence between the client account and the query scenario and the client account that sends the query request to be processed.

[0008] In an alternative embodiment, the method further includes: establishing a correspondence between the client account and the traffic limit information; determining whether to immediately submit the query request to be processed based on the correspondence between the client account and the traffic limit information.

[0009] In an alternative embodiment, determining whether to immediately submit the query request to be processed based on the correspondence between the client account and the traffic limit information includes: obtaining the total number of queries being processed on the Trino cluster under the client account that sends the query request to be processed; determining whether the total number of queries exceeds a preset query concurrency number based on the correspondence between the client account and the traffic limit information; if so, waiting to submit the request until it times out or the total number of queries is less than the limited query concurrency number, and then submitting the query request to be processed.

[0010] In an alternative embodiment, the method further includes: after obtaining the query request to be processed, first performing authentication on the client account corresponding to the query request to be processed.

[0011] In an alternative embodiment, the method further includes: monitoring the survival status of each Trino cluster; if any Trino cluster has an abnormality, shielding the any Trino cluster.

[0012] In an alternative embodiment, configuring a plurality of Trino clusters includes registering the addresses of the plurality of Trino clusters; in response to an access request from a client, allocating a Trino cluster from the plurality of Trino clusters for different query scenarios.

[0013] In an alternative embodiment, routing the query request to be processed to the Trino cluster corresponding to the query scenario to which it belongs includes: randomly submitting the query request to be processed to any one of the Trino clusters corresponding to the query scenario to which it belongs, or determining a target Trino cluster from the Trino clusters corresponding to the query scenario to which it belongs by using a load balancing strategy and then submitting the query request.

[0014] In a second aspect, the present invention provides a query routing system based on multiple Trino clusters, including: a client, a query proxy service, a data source, and a plurality of Trino clusters; the client is used to send query requests; the Trino clusters are used to process the query requests; the data source is used to provide data required for queries; the query proxy service is used to execute the query routing method based on multiple Trino clusters according to any one of claims 1 to 8.

[0015] In a third aspect, the present invention provides an electronic device, including a processor and a memory, the memory stores machine-executable instructions that can be executed by the processor, and the processor can execute the machine-executable instructions to implement the query processing method based on multiple Trino clusters according to any one of the foregoing embodiments.

[0016] The query routing method, system, and electronic device based on multiple Trino clusters provided by the embodiments of the present invention first configure a plurality of Trino clusters, and the Trino clusters corresponding to the same query scenario are located in different computer rooms. After obtaining a query request to be processed, the query request to be processed is routed to the Trino cluster corresponding to the query scenario to which it belongs for processing. The embodiments of the present invention locate the Trino clusters corresponding to the same query scenario in different computer rooms, achieving resource isolation and avoiding cluster crashes caused by resource contention. Finally, the query requests in different scenarios are routed to the corresponding Trino clusters for processing, which can reasonably allocate different cluster resources and achieve the effect of high availability of the Trino clusters.

[0017] To make the above objects, features, and advantages of the present invention more obvious and understandable, the following specifically enumerates preferred embodiments and, in conjunction with the accompanying drawings, provides detailed descriptions as follows. Description of the Drawings

[0018] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the drawings required for the embodiments. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as limiting the scope. For those of ordinary skill in the art, without creative efforts, other relevant drawings can also be obtained based on these drawings.

[0019] Figure 1 It is a deployment architecture diagram of an existing single Trino cluster.

[0020] Figure 2 It is a schematic flowchart of the query routing method based on multiple Trino clusters provided by the embodiments of the present invention.

[0021] Figure 3 It is an architecture diagram of the query routing system based on multiple Trino clusters provided by the embodiments of the present invention.

[0022] Figure 4 This is a functional module diagram of the query routing device based on multiple Trino clusters provided by an embodiment of the present invention.

[0023] Figure 5 This is a structural block diagram of an electronic device provided by an embodiment of the present invention. Detailed implementation manners

[0024] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all the embodiments. Usually, the components of the embodiments of the present invention described and illustrated herein can be arranged and designed in various different configurations.

[0025] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed present invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative efforts belong to the scope of protection of the present invention.

[0026] It should be noted that relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article or device including the said element.

[0027] Please refer to Figure 1 , Figure 1 This is an existing deployment architecture diagram of a single Trino cluster. A single Trino cluster can perform query analysis on various source data. If the computing power resources are tight, Worker nodes can be added for horizontal expansion to cope with the increasing query demands. However, Figure 1 the Trino cluster architecture shown has the following defects:

[0028] First, a single Trino cluster provides services externally only through a single Coordinator node, which is responsible for receiving query requests from clients. This architecture design has an obvious single point of failure risk: once the Coordinator node fails, the entire query system will completely break down and be unable to provide services continuously. In addition, when the cluster faces a shortage of computing resources, although horizontal scaling can be achieved by adding Worker nodes, this will undoubtedly further increase the cooperation pressure between the Coordinator node and the Worker nodes, resulting in a decline in system performance.

[0029] Second, with the development of the business, query scenarios have become increasingly diverse, covering various types such as monitoring scenarios, Adhoc scenarios, and reporting scenarios. In Figure 1 the Trino cluster shown, all businesses share the same resources. This resource sharing mode is prone to the backlog of query requests for monitoring and reporting scenarios due to the sudden resource occupation in the Adhoc scenario, thus affecting the normal operation of the business.

[0030] Considering the above technical defects, the embodiments of the present invention provide a query routing method based on multiple Trino clusters, which can automatically route requests for different business query scenarios to the corresponding clusters and achieve high availability of the Trino clusters.

[0031] Please refer to Figure 2 , Figure 2 which is a schematic flowchart of the query routing method based on multiple Trino clusters provided by the embodiments of the present invention. The execution subject of this method can be an electronic device, including steps S201 to S203, which are described as follows:

[0032] S201: Configure a number of Trino clusters;

[0033] S202: Establish the corresponding relationship between query scenarios and Trino clusters; among them, the Trino clusters corresponding to the same query scenario are located in different computer rooms;

[0034] S203: Obtain the query request to be processed and route the query request to the Trino cluster corresponding to the query scenario to which it belongs for processing.

[0035] The query routing method based on multiple Trino clusters provided by the embodiments of the present invention first configures a number of Trino clusters. The Trino clusters corresponding to the same query scenario are located in different computer rooms. After obtaining the query request to be processed, the query request to be processed is routed to the Trino cluster corresponding to the query scenario to which it belongs for processing. In the embodiments of the present invention, the Trino clusters corresponding to the same query scenario are located in different computer rooms, realizing resource isolation and avoiding cluster crashes caused by resource contention. Finally, the query requests in different scenarios are routed to the corresponding Trino clusters for processing, which can reasonably allocate different cluster resources and achieve the effect of high availability of the Trino clusters.

[0036] Next, the embodiments of the present invention will explain the above steps in detail and clearly.

[0037] In step S201, for each Trino cluster, the cluster address registration can be performed first. Specifically, the cluster address registration centrally manages the network address, port number, and related configuration information of the Trino cluster. Through address registration, the system can grasp the status and resource allocation of each Trino cluster in real time, providing basic data support for subsequent resource scheduling and query allocation.

[0038] After receiving the client access request, a Trino cluster is allocated for the query scenario of the client. In the embodiments of the present invention, the required cluster resources for different query scenarios are different. The proportion of cluster resources required for different queries can be determined according to the total number of Trino clusters, and then the different query scenarios are allocated according to the proportion of cluster resources. For example, for the three main query scenarios of monitoring, reporting, and Adhoc, Trino clusters of different scales can be deployed respectively. Among them, the Adhoc scenario may occupy 50% of the cluster resources, the reporting scenario occupies 30%, and the monitoring scenario occupies 20%.

[0039] Of course, the above proportions are only examples. In actual applications, relevant personnel can flexibly adjust the resource allocation strategy according to specific requirements. For example, if the real-time requirement of the monitoring scenario is higher, the proportion of its cluster resources can be appropriately increased; or during certain business peak periods, the cluster resource allocation of the Adhoc scenario can be dynamically adjusted to cope with sudden query loads. Through the flexible resource allocation strategy, the present invention can better adapt to changing business needs and improve the scalability and stability of the system.

[0040] In step S202, when establishing the correspondence between the query scenario and the Trino cluster, it is necessary to ensure that the Trino clusters corresponding to the same query scenario are distributed in different computer rooms. This layout design can effectively achieve resource isolation, avoid query backlogs caused by resource preemption, prevent interference between different query scenarios, and ensure the stability and query efficiency of the system.

[0041] To manage the correspondence between query scenarios and Trino clusters more efficiently, it is possible to gradually maintain the existing form using a data table. For example, use a data table named cluster_mapping to maintain the above correspondence. The cluster_mapping table records the specific correspondence between each Trino cluster and the query scenario. In this way, the system can clearly grasp the Trino cluster information associated with each query scenario, ensuring that query requests can be quickly and accurately routed to the appropriate Trino cluster for processing.

[0042] In step S203, after the system receives a query request to be processed, it will route it to the corresponding Trino cluster for processing according to the query scenario to which the query request belongs. To quickly and accurately determine the query scenario to which the query request to be processed belongs, the embodiments of the present invention provide the following specific implementation manners:

[0043] Step a1: Establish the correspondence between the client account and the query scenario;

[0044] In the embodiments of the present invention, when the system receives an access request from a client, it will assign a unique client account to the client and establish the correspondence between the account and the query scenario. This correspondence can be maintained in the form of a data table, and the data table is named user. The user table records the client account assigned when the client applies for a Trino account and the corresponding query scenario. In this way, the system can clearly identify the query scenario associated with each client account, providing basic data support for subsequent query routing.

[0045] Step a2: Determine the query scenario to which it belongs based on the correspondence between the client account and the query scenario and the client account that sends the query request to be processed.

[0046] When the system receives a query request to be processed, it first extracts the client account that sends the request. Subsequently, based on the correspondence between the client account and the query scenario, it quickly determines the query scenario to which the query request belongs. This process ensures that the query request can be accurately identified and assigned to the corresponding query scenario, thereby providing a basis for subsequent routing decisions.

[0047] In one embodiment of the present invention, to further enhance the security of the system, before determining the query scenario to which the query request to be processed belongs, the client account that sends the query request will be authenticated first. Through this authentication step, the system can ensure that the initiator of the query request has legitimate access rights, thereby effectively preventing unauthorized access and potential security risks. Only after the client account passes the authentication will the system continue to perform the identification of the query scenario and subsequent routing operations to ensure the security and reliability of the entire query processing flow.

[0048] In the prior art, due to the lack of reasonable control over the client request behavior, unreasonable requests from the client (such as high concurrency) are likely to cause an excessive load on the system and even trigger an avalanche effect, seriously affecting the stability and availability of the system. To solve this problem, in the embodiment of the present invention, a traffic restriction mechanism is introduced before submitting the query request to be processed to the corresponding Trino cluster, which specifically includes the following steps:

[0049] Step b1: Establish the corresponding relationship between the client account and the traffic restriction information;

[0050] In the embodiment of the present invention, when the system receives the access request from the client, it will first establish the corresponding relationship between the client account and the traffic restriction information. The traffic restriction information is used to control the request behavior of the client to ensure that it will not cause excessive pressure on the system due to unreasonable requests. The traffic restriction information may include, but is not limited to, key indicators such as the query concurrency number and the query frequency within a unit time. This corresponding relationship can also be maintained in the form of a data table, such as the user_limit table, so that the system can quickly query and apply relevant restriction conditions.

[0051] Step b2: Based on the corresponding relationship between the client account and the traffic restriction information, determine whether to immediately submit the query request to be processed.

[0052] When the system receives the query request to be processed, it first determines the client account corresponding to the request and obtains its traffic restriction indicators from the corresponding relationship of the traffic restriction information. Subsequently, the system will judge whether to immediately submit the query request according to these restriction conditions. If the traffic (such as the query concurrency number or the query frequency) under a certain client account exceeds the specified range in the traffic restriction information, the system will not submit the query request under this client account for the time being, thereby avoiding the occurrence of system overload or avalanche effect caused by unreasonable requests from the client.

[0053] As an example, assuming that the traffic restriction information is the query concurrency number, the implementation process of step b2 can be:

[0054] The first step: Obtain the total number of queries on the Trino cluster where the query request to be processed is being processed under the client account that sends the query request to be processed;

[0055] Step 2: Determine the correspondence between the client account and the traffic limit information, and check whether the total number of queries exceeds the preset query concurrency number.

[0056] Step 3: If so, wait for the submission request until it times out or the total number of queries is less than the restricted query concurrency number, and then submit the query request to be processed.

[0057] Through the above mechanism, the embodiments of the present invention can effectively control the request behavior of the client and ensure the stable operation of the system in high-concurrency scenarios.

[0058] Continue to refer to step S203. When submitting the query request to be processed to the corresponding Trino cluster, since one query scenario may correspond to multiple Trino clusters, it is necessary to further determine which Trino cluster the query request will be finally submitted to for processing. For this purpose, the embodiments of the present invention provide the following two implementation manners:

[0059] In one implementation manner, the embodiments of the present invention can randomly submit the query request to be processed to any Trino cluster corresponding to the query scenario. This random selection method can quickly complete the cluster selection process, and at the same time, disperse the request load through randomness to avoid local overload problems caused by centralized submission to a certain cluster.

[0060] In another implementation manner, the embodiments of the present invention can adopt a load balancing strategy to select a target Trino cluster from multiple Trino clusters corresponding to the query scenario, and then submit the query request to this cluster for processing. The load balancing strategy will dynamically select the cluster with the lowest current load according to the real-time monitored cluster load conditions (such as current CPU usage rate, memory occupancy rate, query queue length, etc.), so as to ensure the maximization of the overall performance and resource utilization rate of the system.

[0061] Through the above two implementation manners, the embodiments of the present invention can not only efficiently route the query request to the corresponding Trino cluster of the scenario, but also achieve load balancing among multiple clusters, avoid resource waste and local overload, thereby significantly improving the high availability of the Trino cluster and the overall stability of the system.

[0062] In one embodiment of the present invention, for each Trino cluster, in addition to allocating resources, the survival status of each Trino cluster can also be monitored in real time. Specifically, the system will regularly detect the health status of each Trino cluster, including but not limited to key indicators such as cluster response time, service availability, and node status. If any Trino cluster is abnormal (such as node failure, network problem, or performance degradation), the system will automatically block the Trino cluster and temporarily stop allocating new query requests to it.

[0063] This mechanism can effectively avoid query failures or delays caused by single cluster failures, ensuring the overall stability of the system and the query success rate. At the same time, by timely blocking abnormal clusters, the system can reallocate query requests to other normally operating clusters, further improving resource availability and system reliability.

[0064] In summary, the embodiments of the present invention have the following advantages compared with the conventional technology:

[0065] First, the embodiment of the present invention proposes a highly available Trino cluster deployment solution. By configuring multiple sets of Trino clusters and reasonably allocating cluster resources according to different query scenarios, high availability (HA) is achieved. At the same time, the Trino clusters corresponding to the same query scenario are distributed in different computer rooms, which effectively implements resource isolation and avoids query backlogs and mutual interference caused by resource preemption. In addition, the embodiment of the present invention also monitors the survival status of the Trino cluster in real time. Once a cluster abnormality is detected, it will be immediately shielded and the query request will be reallocated to ensure the stable operation of the system.

[0066] Secondly, when the client accesses, the embodiment of the present invention allocates the client account according to its query scenario and establishes the correspondence between the client account and the query scenario, which facilitates the rapid positioning of the subsequent query scenario. When the client initiates a query request, the system will authenticate and authorize the client account to ensure the legitimacy and security of the query request.

[0067] Furthermore, the embodiment of the present invention sets flow limit information for each client account. When processing a query request, the system will obtain the total number of queries currently running in the Trino cluster by the client account in real time. If the number of concurrent queries exceeds the limit, the request will be placed in a waiting queue until the current total number of runs is less than the limited number of concurrent queries or times out. This flow control mechanism effectively avoids cluster overload and avalanche effects caused by unreasonable requests from the client, significantly improving the performance and stability of the system.

[0068] Generally speaking, in the embodiments of the present invention, through the registration of the client account and the Trino cluster address, the following three groups of key corresponding relationships are maintained: the client account and the query scenario, the query scenario and the Trino cluster, and the client account and the traffic limit information. When the client initiates a query request, the system not only performs authentication and authorization, but also quickly distributes the query request according to the query scenario corresponding to the client account and routes it to the Trino cluster belonging to the corresponding query scenario for processing. This systematic management method not only improves the processing efficiency of query requests, but also ensures the reasonable allocation of resources and the efficient operation of the system.

[0069] In summary, the embodiments of the present invention are superior to the traditional technologies in terms of high availability, security, traffic control, and systematic management, and can effectively improve the performance of the Trino cluster and the user experience.

[0070] Based on Figure 2 the same inventive concept, the embodiments of the present invention also provide a query routing system 30 based on multiple Trino clusters. Please refer to Figure 3 , Figure 3 which is the architecture diagram of the query routing system based on multiple Trino clusters provided by the embodiments of the present invention. As Figure 3 shown, the system includes a client 310, a query proxy service 320, several Trino clusters 330, and a data source 340. Among them, the client 310 is responsible for initiating query requests; the query proxy service 320, as the core component, is responsible for processing, routing, and traffic control of query requests; the Trino clusters 330 are used to execute specific query tasks; and the data source 340 stores the underlying data required for queries.

[0071] In Figure 3 the example, each query scenario corresponds to two Trino clusters 330, and these Trino clusters 330 are respectively deployed in computer room A and computer room B. This architecture design aims to improve the high availability and fault tolerance of the system through resource isolation and redundant deployment.

[0072] It should be noted that Figure 3 the number of Trino clusters shown in

[0073] and the computer rooms to which they belong are only one example and do not limit the system architecture. In practical applications, the number of Trino clusters can be flexibly adjusted according to the query load and business requirements, and the deployment location of the clusters can also be selected according to the actual infrastructure conditions to meet the performance and reliability requirements in different scenarios. The query proxy service 320 is a JDBC backend service developed based on the Trino protocol, and can provide a proxy service for managing multiple sets of Trino clusters with a unified query gateway to achieve high availability, resource isolation, authentication, and query rate limiting. Specifically:

[0074] The query proxy service 320 can configure several Trino clusters; establish the correspondence between query scenarios and Trino clusters; where the Trino clusters corresponding to the same query scenario are located in different computer rooms; obtain the query request to be processed and route the query request to be processed to the Trino cluster corresponding to the query scenario to which it belongs for processing.

[0075] In an alternative embodiment, the query proxy service 320 can establish the correspondence between client accounts and query scenarios; and can also establish the correspondence between client accounts and traffic limit information.

[0076] It can be understood that the query proxy service 320 can be used to maintain the user table, cluster_mapping table, and user_limit table in the above embodiments.

[0077] In an alternative embodiment, the query proxy service 320 is also used to determine the query scenario to which it belongs based on the correspondence between the client account and the query scenario and the client account that sends the query request to be processed.

[0078] In an alternative embodiment, the query proxy service 320 is also used to determine whether to immediately submit the query request to be processed based on the correspondence between the client account and the traffic limit information. Specifically, it is used to: obtain the total number of queries on the Trino cluster where the query request is being processed under the client account that sends the query request to be processed; based on the correspondence between the client account and the traffic limit information, determine whether the total number of queries exceeds the preset query concurrency number; if so, wait to submit the request until it times out or the total number of queries is less than the limited query concurrency number, and then submit the query request to be processed.

[0079] In an alternative embodiment, when the query proxy service 320 receives the query request to be processed, it first authenticates the client account corresponding to the query request to be processed.

[0080] In an alternative embodiment, the query proxy service 320 is also used to monitor the survival status of each Trino cluster; if any Trino cluster has an abnormality, then shield any Trino cluster.

[0081] In an alternative embodiment, the query proxy service 320 is also used to register the addresses of several Trino clusters; in response to the access request of the client, allocate a Trino cluster for the query scenario of the client from several Trino clusters.

[0082] In an optional implementation, the query proxy service 320 is further used to randomly submit the query request to be processed to any Trino cluster corresponding to the query scenario for processing, or to determine a target Trino cluster from the Trino clusters corresponding to the query scenario using a load balancing strategy and then submit the query request.

[0083] The query proxy service 320 efficiently manages query requests and resource allocation through multiple background tasks. These tasks include regularly loading and updating the data of the user table, cluster_mapping table, and user_limit table and caching them to ensure that the query proxy service 320 can quickly access and process these key information. At the same time, the query proxy service 320 will obtain the total number of queries running in each Trino cluster for each client account in real time to implement traffic restriction and concurrency control. In addition, it will also detect the survival status of each Trino cluster in real time to dynamically adjust the routing strategy of the query request to ensure the high availability of the system.

[0084] When the client 310 accesses the query proxy service 320, the query proxy service 320 generates the corresponding account information and records the account information and the client's query scenario information in the user table. When the client 310 initiates a query request to the query proxy service 320, the query proxy service 320 first authenticates the client account. After successful authentication, the query proxy service 320 routes the query request to a Trino cluster of the corresponding scenario based on the usage scenario of the account and the mapping relationship in the cluster_mapping table.

[0085] When submitting a request to the Trino cluster, the query proxy service 320 will also compare the number of concurrent queries limited in the user_limit table with the total number of queries running under the current client account. If the current total number of queries exceeds the limit, the request will enter the polling waiting state until the current total number of queries is lower than the limit; otherwise, the request will be submitted directly. During the formal submission process, the query proxy service 320 can also use a random algorithm to randomly select a Trino cluster from multiple Trino clusters in the corresponding scenario to route the request, thereby achieving load balancing and high availability.

[0086] Based on Figure 2 With the same inventive concept, the embodiment of the present invention also provides a query routing device 40 based on multiple Trino clusters. Figure 4 , Figure 4 The functional module diagram of the query routing device based on multiple Trino clusters provided by the embodiment of the present invention includes: a configuration module 401, a creation module 402 and a routing module 403;

[0087] A configuration module 401 is used to configure a number of Trino clusters;

[0088] An establishment module 402 is used to establish a correspondence between a query scenario and a Trino cluster; among them, the Trino clusters corresponding to the same query scenario are located in different computer rooms;

[0089] A routing module 403 is used to obtain a query request to be processed and route the query request to be processed to the Trino cluster corresponding to the query scenario to which it belongs for processing.

[0090] It can be understood that the configuration module 401, the establishment module 402, and the routing module 403 can cooperate to execute Figure 1 each step in to achieve the corresponding technical effects.

[0091] In an optional implementation manner, the configuration module 401, the establishment module 402, and the routing module 403 are also used to execute each step in the above embodiments, which will not be elaborated here.

[0092] An embodiment of the present invention also provides an electronic device. Please refer to Figure 5 , Figure 5 which is a structural block diagram of the electronic device provided by the embodiment of the present invention, including: a memory 501, a processor 502, and a communication interface 503. The memory 501, the processor 502, and the communication interface 503 are directly or indirectly electrically connected to each other to realize data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0093] Optionally, the bus 504 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, Figure 5 only a thick line is used to represent it in, but it does not mean that there is only one bus or one type of bus.

[0094] In an embodiment of the present invention, the processor 502 may be a general-purpose processor, a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, and can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiments of the present invention. The general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the method disclosed in combination with the embodiments of the present invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor. The software module may be located in the memory 501, and the processor 502 reads the program instructions in the memory 501 and combines its hardware to complete the steps of the above method.

[0095] In an embodiment of the present invention, the memory 501 may be a non-volatile memory, such as a hard disk drive (HDD) or a solid-state drive (SSD), etc., or may also be a volatile memory, such as RAM. The memory may also be any other medium that can be used to carry or store the desired program executable code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory in the embodiments of the present invention may also be a circuit or any other device capable of implementing a storage function, for storing instructions and / or data.

[0096] The memory 501 can be used to store software programs and modules, such as the instructions / modules of the query routing device 40 based on multiple Trino clusters provided in the embodiments of the present invention, which can be stored in the memory 501 in the form of software or firmware, or solidified in the operating system (OS) of the electronic device 50. The processor 502 executes the software programs and modules stored in the memory 501, thereby performing various functional applications and data processing. The communication interface 503 can be used for signaling or data communication with other node devices.

[0097] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the foregoing method embodiments, and will not be described herein again.

[0098] It can be understood that Figure 5 The structure shown is only schematic, and the electronic device 50 may also include more or fewer components than those shown Figure 5 in the figure, or have a different configuration from that shown Figure 5 in the figure. Figure 5 Each of the components shown can be implemented by hardware, software, or a combination thereof.

[0099] The electronic device 50 may further include a network device and / or a user device. Among them, the network device includes, but is not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of hosts or network servers based on cloud computing (Cloud Computing).

[0100] Based on the above embodiments, the present application further provides a storage medium. A computer program is stored in the computer-readable storage medium. When the computer program is executed by a computer, the computer is caused to execute the query routing method based on multiple Trino clusters provided by the above embodiments.

[0101] Based on the above embodiments, the embodiments of the present invention further provide a computer program. When the computer program runs on a computer, the computer is caused to execute the query routing method based on multiple Trino clusters provided by the above embodiments.

[0102] Based on the above embodiments, the embodiments of the present invention further provide a chip. The chip is used to read the computer program stored in the memory and is used to execute the query routing method based on multiple Trino clusters provided by the above embodiments.

[0103] The embodiments of the present invention also provide a computer program product, including instructions. When it runs on a computer, the computer is caused to execute the query routing method based on multiple Trino clusters provided by the above embodiments.

[0104] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by instructions. These instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate a device for implementing the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0105] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured product including an instruction device, and the instruction device implements the functions specified in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0106] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are executed on the computer or other programmable apparatus to produce a computer-implemented process, thereby the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in one process or a plurality of processes and / or blocks Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps for implementing the functions specified in one block or a plurality of blocks.

[0107] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention should be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A query routing method based on multiple Trino clusters, characterized in that: The method comprises: Configure several Trino clusters; Establish the correspondence between query scenarios and Trino clusters; the Trino clusters corresponding to the same query scenario are located in different computer rooms; Obtain a pending query request, and route the pending query request to a Trino cluster corresponding to the query scenario for processing.

2. The query routing method based on multiple Trino clusters according to claim 1, characterized in that: The method further comprises: Establish the correspondence between client accounts and query scenarios; The query scenario is determined based on the correspondence between the client account and the query scenario and the client account that sends the query request to be processed.

3. The query routing method based on multiple Trino clusters according to claim 1, characterized in that: The method further comprises: Establish the correspondence between client account and traffic restriction information; Based on the correspondence between the client account and the traffic restriction information, determine whether to submit the pending query request immediately.

4. The query routing method based on multiple Trino clusters according to claim 3, characterized in that: Based on the correspondence between the client account and the traffic restriction information, determining whether to immediately submit the pending query request includes: Obtain the total number of queries on the Trino cluster that is processing query requests under the client account that sent the pending query request; The correspondence between the client account and the traffic limit information is used to determine whether the total number of queries exceeds the preset number of concurrent queries; If yes, wait for the request to be submitted until the timeout or the total number of queries is less than the limited number of concurrent queries, and then submit the pending query request.

5. The query routing method based on multiple Trino clusters according to claim 1, characterized in that: The method further comprises: After obtaining the pending query request, firstly authenticate the client account corresponding to the sending of the pending query request.

6. The query routing method based on multiple Trino clusters according to claim 1, characterized in that: The method further comprises: Monitoring the survival status of each of the Trino clusters; If any Trino cluster is abnormal, the Trino cluster is shielded.

7. The query routing method based on multiple Trino clusters according to claim 1, characterized in that: Configure several Trino clusters, including: Registering addresses of several Trino clusters; In response to the access request of the client, a Trino cluster is allocated for different query scenarios from the plurality of Trino clusters.

8. The query routing method based on multiple Trino clusters according to any one of claims 1 to 7, characterized in that: Routing the query request to be processed to the Trino cluster corresponding to the query scenario for processing, including: The query request to be processed is randomly submitted to any Trino cluster corresponding to the query scenario for processing, or a load balancing strategy is adopted to determine a target Trino cluster from the Trino clusters corresponding to the query scenario and then submit the query request.

9. A query routing system based on multiple Trino clusters, characterized in that: include: Client, query proxy service, data source and several Trino clusters; The client is used to send a query request; The Trino cluster is used to process the query request; The data source is used to provide data required for query; the query proxy service is used to execute the query routing method based on multiple Trino clusters as described in any one of claims 1 to 8.

10. An electronic device, characterized in that: It comprises a processor and a memory, wherein the memory stores machine executable instructions that can be executed by the processor, and the processor can execute the machine executable instructions to implement the query processing method based on multiple Trino clusters as described in any one of claims 1-8.

Images