Distributed query system, query method, device and storage medium
By introducing a distributed query system into the database query system, using the coordinated work of front-end nodes and computing nodes, priority is given to query requests with small computing volume, which solves the problem of large computing volume query requests occupying resources in the existing technology, and improves the problem of resource consumption by large computing volume query requests System response time and throughput.
Patent Information
- Application Number
- CN201910817916.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2019-08-30
- Publication Date
- 2025-02-21
- Estimated Expiration
- 2039-08-30
AI Technical Summary
Existing database query methods can easily lead to high-computing query requests occupying all computing resources, resulting in increased system response time and decreased throughput.
Design a distributed query system, including front-end nodes and computing nodes. The front-end node receives the query request, determines its type, and generates a query task, and distributes it to the target computing node. The computing node prioritizes tasks according to the query type to reduce system blockage.
By prioritizing the processing of query requests with smaller computational volume, the system blockage is reduced and the system response time and throughput is improved.
Smart Images

Figure CN112445834B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a distributed query system and query method, device and storage medium. Background Art
[0002] In today's information age, the application and research of information data has become a trend. Based on its advantages, databases are widely used in data storage, management, maintenance and query.
[0003] With the development of big data, database systems often need to provide data of different dimensions to different users. Due to different user query requirements, database systems may receive query requests with various computational requirements. The current database query method can easily cause large-volume query requests to occupy all computing resources, ultimately increasing the response time of the entire system and reducing system throughput. Summary of the invention
[0004] Multiple aspects of the present application provide a distributed query system and query method, device and storage medium to reduce system congestion, thereby improving system response time, and further helping to improve system throughput.
[0005] The embodiment of the present application provides a distributed query system, including: at least one front-end node and at least one computing node;
[0006] The at least one front-end node is used to receive a query request, determine the query type to which the query request belongs, generate at least one query task corresponding to the query request, and distribute the at least one query task to at least one target computing node among the at least one computing node; the at least one query task carries the query type to which the query request belongs;
[0007] The at least one target computing node is used to: when the query type to which the query request belongs is the first query type, preferentially execute the received query task.
[0008] The embodiment of the present application also provides a query method, which is applicable to a front-end node in a distributed query system, including:
[0009] receiving a query request;
[0010] Determining the query type to which the query request belongs;
[0011] generating at least one query task corresponding to the query request; the at least one query task carries the query type to which the query request belongs;
[0012] The at least one query task is distributed to at least one target computing node in the distributed query system, so that the at least one target computing node executes the received query task in a manner adapted to the query type to which the query request belongs.
[0013] The embodiment of the present application also provides a query method, which is applicable to a computing node in a distributed query system, including:
[0014] Receiving a query task issued by a front-end node in the distributed query system, wherein the query task carries a query type identifier; the query type identifier indicates the query type to which the query request corresponding to the query task belongs;
[0015] Determining, according to the query type identifier, the query type to which the query request belongs;
[0016] In a case where the query type to which the query request belongs is the first query type, the received query task is executed preferentially.
[0017] The embodiment of the present application also provides an electronic device, including: a memory, a processor, and a communication component;
[0018] Wherein, the memory is used for computer programs;
[0019] The processor is coupled to the memory and is configured to execute the computer program for:
[0020] receiving a query request through the communication component;
[0021] Determining the query type to which the query request belongs;
[0022] generating at least one query task corresponding to the query request; the at least one query task carries the query type to which the query request belongs;
[0023] The at least one query task is distributed to at least one target computing node in the distributed query system through the communication component, so that the at least one target computing node executes the received query task in a manner adapted to the query type to which the query request belongs.
[0024] The embodiment of the present application also provides an electronic device, including: a memory, a processor, and a communication component;
[0025] Wherein, the memory is used for computer programs;
[0026] The processor is coupled to the memory and is configured to execute the computer program for:
[0027] receiving, through the communication component, a query task issued by a front-end node in a distributed query system, wherein the query task carries a query type identifier; the query type identifier indicates the query type to which the query request corresponding to the query task belongs;
[0028] Determining, according to the query type identifier, the query type to which the query request belongs;
[0029] In a case where the query type to which the query request belongs is the first query type, the received query task is executed preferentially.
[0030] An embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute each step in the above method.
[0031] In an embodiment of the present application, a distributed query system is provided. The system includes a front-end node and a computing node. Among them, the front-end node can determine the query type to which the received query request belongs, and add the query type to which the query request belongs to at least one query task corresponding to the query request. In this way, the computing node can execute the query task in a manner adapted to the query type according to the query type to which the query request carried by the received query task belongs. Further, when the query type to which the query request belongs is the first query type, the computing node can give priority to executing the received query task, thereby helping to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0033] Figure 1a A schematic diagram of the structure of a distributed query system provided in an embodiment of the present application;
[0034] Figure 1b A schematic diagram of the structure of another distributed query system provided in an embodiment of the present application;
[0035] Figure 2 A flowchart of a query method provided in an embodiment of the present application;
[0036] Figure 3 A flowchart of another query method provided in an embodiment of the present application;
[0037] Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;
[0038] Figure 5 A schematic diagram of the structure of another electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0039] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.
[0040] The existing database query method can easily cause query requests with large computational complexity to occupy all computing resources, ultimately resulting in an increase in the response time of the entire system and a decrease in system throughput. In response to this technical problem, in some embodiments of the present application, a distributed query system is provided. The system includes a front-end node and a computing node. Among them, the front-end node can determine the query type to which the received query request belongs, and add the query type to which the query request belongs to at least one query task corresponding to the query request. In this way, the computing node can execute the query task in a manner adapted to the query type according to the query type to which the query request carried by the received query task belongs. Furthermore, when the query type to which the query request belongs is the first query type, the computing node can give priority to executing the received query task, which helps to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput.
[0041] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.
[0042] Figure 1a A schematic diagram of the structure of a distributed query system provided in an embodiment of the present application. Figure 1a As shown, the distributed query system includes: at least one front-end node 11 and at least one computing node 12. Figure 1a The number and implementation forms of the front-end nodes 11 and computing nodes 12 shown in the figure are for illustrative purposes only and do not constitute a limitation thereto.
[0043] In this embodiment, the number of front-end nodes 11 may be one or more. The number of computing nodes 12 may also be one or more. In each embodiment of the present application, multiple refers to 2 or more. The front-end node 11 and the computing node 12 may be a software module, an application, a service or a physical device that provides a query function based on a data source for the user. Multiple front-end nodes 11 may be deployed on different physical machines, or in different containers or virtual machines (VM). Of course, these containers or virtual machines may be deployed on the same physical machine or on multiple different physical machines. Of course, multiple computing nodes 12 may also be deployed on different physical machines, or in different containers or virtual machines. These containers or virtual machines may be deployed on the same physical machine or on multiple different physical machines.
[0044] Optionally, the front-end node 11 and the computing node 12 may be on different physical machines, or may be deployed in different containers or virtual machines. These containers or virtual machines may be deployed on the same physical machine, or may be deployed on multiple different physical machines.
[0045] Among them, a physical machine can be a single server device, a cloud server array, or a virtual machine (VM) running in a cloud server array. In addition, a physical machine can also refer to other computing devices with corresponding service capabilities, such as computers and other terminal devices (running service programs).
[0046] Optionally, different physical machines may be connected wirelessly or wired. For example, different physical machines may be connected by network cables or communication optical fibers. Alternatively, different physical machines may be connected via mobile network communication. Accordingly, the network standard of the mobile network may be any one of 2G (GSM), 2.5G (GPRS), 3G (WCDMA, TD-SCDMA, CDMA2000, UTMS), 4G (LTE), 4G+ (LTE+), 5G, WiMax, etc. Optionally, different physical machines may also be connected via Bluetooth, WiFi, infrared, etc.
[0047] In this embodiment, the distributed query system can provide query services to users. Among them, the distributed query system can provide various query services such as simple query services, joint query services and associated query services, but is not limited thereto. When a user needs to perform an associated query, an associated query request can be issued to the distributed query system. For example, the distributed query system can provide a query portal for users to issue query requests. Optionally, the query portal that the distributed query system can provide to users can be a web page, an application page, or a command window, etc.
[0048] Further, when the query request arrives at the distributed query system, the front-end node 11 receives the query request sent by the user and determines the query type to which the query request belongs. Further, the front-end node 11 generates at least one query task corresponding to the query request, wherein at least one query task carries the query type to which the query request belongs. Further, the front-end node 11 distributes the at least one query task to at least one target computing node in the at least one computing node 12.
[0049] The target computing node 12a is a part of or all of the at least one computing node. The number of the target computing nodes 12a may be determined by the number of computing nodes involved in the content to be queried by the query task. Figure 1a In the example, only two target computing nodes 12a are used. In an optional embodiment, the computing nodes involved in the content to be queried by each query task are different, that is, the content to be queried by a query task involves one computing node, and the number of target computing nodes 12a is equal to the number of query tasks.
[0050] Accordingly, the target computing node 12a may execute the query task in a manner adapted to the query type to which the query request carried by the received query task belongs, according to the query type to which the query request belongs. For example, if the query type to which the query request belongs is the first query type, the target computing node 12a preferentially executes the received query task. For another example, if the query type to which the query request belongs is the second query type, the target computing node 12a may add the received query task to the local cache queue, and schedule and execute the received query task according to the cache scheduling mechanism.
[0051] Among them, the target computing node 12a gives priority to executing the received query task means: if the target computing node 12a has no idle computing resources to execute the currently received query task of the first query type when receiving the query task of the first query type, that is, its computing resources are processing other query tasks that have arrived with priority, then after processing the task of the first query type that is currently being processed, the query task of the currently received first query type can be executed with priority; if the target computing node 12a has idle computing resources to process the currently received query task of the first query type when receiving the query task of the first query type, then the query task of the currently received first query type can be directly executed. It is worth noting that in the embodiment of the present application, the first type of query type is a preset query type that needs to be processed with priority; the second type of query type is other query types except the first type of query type. In different application requirements, query requests can be divided into different query types according to different classification criteria.
[0052] For example, in some application scenarios, query requests can be classified according to the amount of resources consumed. In this embodiment, a resource amount threshold can be preset, and the query type to which the query request that consumes less computing resources than the preset resource amount threshold belongs is defined as the first query type; correspondingly, the query type to which the query request that consumes more computing resources than or equal to the preset resource amount threshold belongs is defined as the second query type. In this application scenario, using the distributed query system provided in the embodiment of the present application, query requests that consume less computing resources can be given priority, which helps to reduce the query requests that consume more computing resources occupying all computing resources and blocking the system from processing and issuing query requests that consume less computing resources, thereby helping to improve the response time of the system, especially helping to improve the response time of the system to query requests that consume less resources, thereby improving the throughput of the system.
[0053] For another example, in some other application scenarios, the query requests can be classified according to their real-time requirements. In this embodiment, a delay threshold can be preset, and the query type to which the query request requiring the delay to be less than the preset delay threshold belongs is defined as the first query type; accordingly, the query type to which the query request not requiring the delay to be less than the preset delay threshold belongs is defined as the second query type; and so on, but not limited to this. In this application scenario, using the distributed query system provided in the embodiment of the present application, query requests with higher real-time requirements can be prioritized, which helps prevent query requests with lower real-time requirements from occupying all computing resources and blocking query requests with higher real-time requirements, thereby helping to improve the response time of the system, and in particular helps to improve the response time of the system to query requests with higher real-time requirements, thereby improving the throughput of the system.
[0054] The distributed query system provided in this embodiment includes a front-end node and a computing node. Among them, the front-end node can determine the query type to which the received query request belongs, and add the query type to which the query request belongs to at least one query task corresponding to the query request. In this way, the computing node can execute the query task in a manner adapted to the query type according to the query type to which the query request carried by the received query task belongs. Further, when the query type to which the query request belongs is the first query type, the computing node can give priority to executing the received query task, thereby helping to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput.
[0055] In particular, for a mixed load situation in which the target computing node 12a simultaneously has query tasks corresponding to query requests of the first query type and query tasks corresponding to query requests of the second query type, this query method can give priority to executing query tasks corresponding to query requests of the first query type, thereby helping to prevent the query requests of the second query type from occupying a large amount of computing resources and causing the target computing node 12a to be unable to execute query tasks corresponding to query requests of the first query type, thereby helping to improve the system's response time to query requests of the first query type, thereby helping to improve system throughput.
[0056] In this embodiment, for the case of multiple front-end nodes 11, in order to achieve load balancing between the front-end nodes 11. Optionally, as Figure 1b As shown, the distributed query system provided by the embodiment of the present application may also include: a load balancing node 13. Among them, the load balancing node 13 can balance the received query request to at least one front-end node 11 according to the load condition of at least one front-end node 11. Among them, the load balancing node 13 can be deployed on a physical machine. Regarding the implementation form of the physical machine, please refer to the above-mentioned related content, which will not be repeated here. For example, the load balancing node 13 can be implemented as a server load balance (SLB).
[0057] In an optional embodiment, the user may input a corresponding query statement to the distributed query system using a data source language supported by the distributed query system, thereby initiating a query request to the distributed query system. Optionally, the distributed query system of this embodiment may support a cache query language (Linkcache query language, LQL), a structured query language (Structured Query Language, SQL) or a domain-specific language (Domain Specified Language, DSL), etc., but is not limited thereto. The description language of the query request conforms to the grammatical format of the distributed query system. Optionally, a query type identifier may be added to the hint field in the query statement.
[0058] Based on the above analysis, the query request sent by the user may carry a query type identifier. The query type identifier indicates the query type to which the query request belongs; accordingly, the front-end node 11 may parse the query type identifier from the query request and add the query type identifier to at least one query task. Afterwards, the query task carrying the query type identifier information is sent to at least one target computing node 12a.
[0059] In other embodiments, the query request includes a filtering condition. Among them, the filtering condition includes a filtering condition related to the query table item and a screening condition related to the query data. Based on the filtering condition related to the query table item, information such as the target table item of this query, the table item to be associated, and the association path between the table item to be associated and the target table item can be determined. The front-end node 11 can also determine the query type to which the query request belongs based on the filtering condition of the query request. Optionally, the front-end node 11 can determine the query type to which the query request belongs based on the number of table items to be associated in this query and the amount of data to be queried. For example, if the number of table items to be associated in this query is greater than or equal to a preset number threshold, and the amount of data to be queried is greater than or equal to a preset data amount, then it is determined that the query type to which the query request belongs is the first query type; otherwise, it can be determined that the query type to which the query request belongs is the second query type.
[0060] In the embodiment of the present application, the computing node 12 stores table items and data to be queried. Optionally, the data stored on different computing nodes 12 may be the same or different. In some embodiments, the data stored on different computing nodes 12 back up each other; in other embodiments, different computing nodes 12 are distributed storages, and the data stored therein are different. Accordingly, the front-end node 11 stores the storage relationship between the content to be queried and the computing nodes.
[0061] In the embodiment of the present application, the query request includes a filter condition. Accordingly, the front-end node 11 performs syntax analysis on the query request, and can parse out the filter condition of the query. Optionally, a syntax analyzer such as Antlr or Yacc can be used to perform syntax analysis on the query request, but is not limited thereto.
[0062] Further, the front-end node 11 may split the filtering condition into at least one sub-filtering condition according to the storage relationship between the content to be queried and the computing node, wherein the data filtered by each sub-filtering condition is located on the same computing node. Further, the front-end node 11 generates the above-mentioned at least one query task according to at least one sub-filtering condition, and adds the identifier of the query type to which the query request belongs to each query task. Afterwards, the front-end node 11 distributes the at least one query task to at least one target computing node 12a. Optionally, the front-end node 11 may select a target computing node 12a from at least one computing node 12 according to the sub-filtering condition of each query task and the storage relationship between the content to be queried and the computing node; and assign at least one query task to the corresponding target computing node 12a.
[0063] Accordingly, at least one target computing node 12a parses the query type identifier from the received query task, and determines the query type to which the query request belongs according to the query type identifier.
[0064] In an embodiment of the present application, in order to implement priority processing of query requests of the first query type, multiple thread pools may be set in the computing node 12, some of which are specifically used to process query requests of the first query type. Based on this, if the target computing node 12a determines that the query type to which the query request belongs is the first query type, an idle thread is directly selected from the thread pool corresponding to the first query type, and the idle thread is called to execute the received query task. For ease of description and distinction, the idle thread in the thread pool corresponding to the first query type is defined as the first thread. In this way, the query tasks corresponding to the query requests of the first query type are no longer cached, but the idle threads are called to directly process the query tasks corresponding to the query requests of the first query type, which helps to improve the system's response time to the query requests of the first query type, thereby helping to improve the system throughput.
[0065] Furthermore, the computing node 12a is also provided with a local cache queue for caching query tasks corresponding to the second query type. Accordingly, if the target computing node 12a determines that the query type to which the query request belongs is the second query type, the received query task is added to the local cache queue, and the received query task is scheduled and executed according to the cache scheduling mechanism. The cache scheduling mechanism may be, but is not limited to, First Input First Output (FIFO) or the like.
[0066] Optionally, in the local cache queue, the query tasks may be sorted according to the arrival order of the corresponding query requests. Preferably, the earlier the query request arrives, the higher the ranking of the corresponding query task in the local cache queue is, and the higher the priority is for scheduling and execution.
[0067] Based on this, when the front-end node 11 determines that the query type to which the query request belongs is the second query type, it can generate a global identifier corresponding to the query request, wherein the global identifier indicates the arrival order of the query request. Further, the front-end node 11 carries the global identifier in at least one query task, and sends the query task carrying the global identifier to the corresponding target computing node 12a.
[0068] Optionally, the front-end node 11 may number the query requests in sequence according to the order in which the query requests arrive, and use the number as the global identifier of the query request. Alternatively, the front-end node 11 may use the arrival time of the query request as the global identifier of the query request. Further, in the case where there are multiple front-end nodes 11, since each front-end node 11 works independently when processing a query request, in order to ensure that when the subsequent target computing node 12a processes the query task corresponding to the query request, the position of these query tasks in the local cache queue of the target computing node 12a does not conflict, the front-end node 11 may generate a global identifier corresponding to the query request based on its own identifier and the time when the query request is received. Among them, the identifier of the front-end node 11 is information that can uniquely identify a front-end node, which can be allocated by the physical device cluster where the distributed query system is located when it is started. Further, the time when the front-end node 11 receives the query request can be expressed as long floating point data, accurate to milliseconds.
[0069] Further, in an embodiment of the present application, the local cache queue of the computing node 12 may include: an execution cache queue and a waiting cache queue. For the thread pool corresponding to the second query type, if there is an idle thread therein, the idle thread in the thread pool corresponding to the second query type may be called to process the query task in the execution cache queue. For ease of description and distinction, the idle thread in the thread pool corresponding to the second query type is defined as the second thread. Further, if the execution cache queue can accommodate new query tasks, the query tasks in the waiting cache queue may be added to the execution cache queue to wait for scheduling.
[0070] Based on this, in this embodiment, at least one target computing node 12a can schedule query tasks from the execution cache queue according to the cache scheduling mechanism, and when scheduling the currently received query task, select an idle second thread in the thread pool corresponding to the second query type; and call the second thread to execute the currently received query task.
[0071] Optionally, in this embodiment, a depth threshold of the execution cache queue may be preset, and the depth threshold is less than or equal to the length of the execution cache queue. Preferably, the preset depth threshold is less than the length of the execution cache queue. The execution cache queue may initiate an afull (full threshold) signal when the cache depth of the execution cache queue reaches the depth threshold, indicating that the data will reach saturation and will not be able to be accommodated, i.e., an overflow warning is issued.
[0072] Furthermore, in actual applications, due to poor network quality, the query task corresponding to the query request that arrives at the front-end node 11 earlier may arrive at the target computing node 12a later than the query task corresponding to the query request that arrives at the front-end node 11 later. In order to give priority to the query request sent by the user earlier, the global identifier corresponding to the query request may also be considered. Furthermore, the target computing node 12a may add the received query task to the execution cache queue or the waiting cache queue according to the global identifier corresponding to the query request and the cache depth of the execution cache queue.
[0073] Further, if the cache depth of the execution cache queue is less than a preset depth threshold, the target computing node 12a adds the received query task to the execution cache queue.
[0074] Further, in order to ensure that the query requests issued by the user earlier are given priority, in this embodiment, the target computing node 12a can parse the global identifier corresponding to the query request from the received query task, and compare the global identifier corresponding to the query request with the global identifier carried by the query task at the end of the execution cache queue. Further, if the global identifier corresponding to the query request is smaller than the global identifier carried by the query task at the end of the execution cache queue, the received query task is added to the execution cache queue. Optionally, the position of the currently received query task in the execution cache queue can be determined based on the global identifier carried by each query task in the execution cache queue. Preferably, the target computing node 12a adds the currently received query task to the first position, wherein the global identifier of the query task after the first position is smaller than the global identifier of the currently received query task, so as to ensure that the distributed query system gives priority to the query requests received earlier and reduce the impact of network quality on the query progress.
[0075] Accordingly, if the cache depth of the execution cache queue in the target computing node 12a is greater than or equal to the set depth threshold, but the global identifier carried by the received query task is less than the global identifier carried by the query task at the end of the execution cache queue, the set depth threshold is increased, and the currently received query task is added to the execution cache queue. This is because the global identifier carried by the received query task is less than the global identifier carried by the query task at the end of the execution cache queue, indicating that the time when the query request corresponding to the received query task arrives at the distributed query system is earlier than the time when the query request corresponding to the query task at the end of the execution cache queue arrives at the system, which can ensure that the distributed query system gives priority to query requests received earlier and reduce the impact of network quality on query progress.
[0076] Further, if the cache depth of the execution cache queue is greater than or equal to the preset depth threshold, and the global identifier corresponding to the query request is greater than or equal to the global identifier carried by the query task at the end of the execution cache queue, the target computing node 12a will add the received query task to the waiting cache queue, and when the cache depth of the execution cache queue is less than the preset depth threshold, the query task will be added to the execution cache queue.
[0077] Accordingly, in this embodiment, the target computing node 12a may also monitor the cache depth of the execution cache queue, and when the execution cache depth is less than a set depth threshold, schedule the query task in the waiting cache queue to the execution cache queue.
[0078] In an embodiment of the present application, at least one target computing node 12a may execute the query tasks received by each, and obtain the corresponding sub-query results. Further, at least one target computing node 12a may return the sub-query results obtained by each to the corresponding front-end node 11. Accordingly, the front-end node 11 may receive at least one sub-query result returned by at least one target computing node 12a, and assemble at least one sub-query result to obtain the query result corresponding to the query request received by it. Further, the front-end node 11 outputs the query result corresponding to the query request. Among them, the front-end node 11 that assembles at least one sub-query result may be the front-end node that receives the corresponding query request among at least one front-end node 11.
[0079] Optionally, the query request may include query items in addition to the filter conditions. The query items are used to describe the rule information for assembling the query results later. Accordingly, the front-end node 11 may assemble at least one sub-query result according to the query items, and then obtain the query result corresponding to the query request it receives.
[0080] In this embodiment, the manner in which the front-end node 11 outputs the query results is not limited. For example, in some embodiments, the query results corresponding to the query request can be played through the audio component thereon. In other embodiments, the front-end node 11 can display the query results corresponding to the query request on its human-computer interaction interface. In some other embodiments, the front-end node 11 is a server device, and the front-end node 11 sends the query results corresponding to the query request to the user's terminal device so that the user can obtain the query results corresponding to the query request. Among them, the front-end node 11 can send the query results corresponding to the query request corresponding to the keyword to be searched to the user's terminal device by email, short message, in-application message, etc.
[0081] It is worth noting that the distributed query system provided in the embodiment of the present application can be implemented as an online analytical processing (OLAP) system.
[0082] In addition to the distributed query system provided in the above embodiment, the embodiment of the present application also provides a query method. The following is an exemplary description from the perspectives of the front-end node and the computing node in the distributed query system.
[0083] Figure 2 The flowchart of a query method provided in the embodiment of the present application is shown in FIG. The method is applicable to the front-end node in the distributed query system. Figure 2 As shown, the method includes:
[0084] 201. Receive a query request.
[0085] 202. Determine the query type to which the query request belongs.
[0086] 203. Generate at least one query task corresponding to the query request; wherein the at least one query task carries the query type to which the query request belongs.
[0087] 204. Distribute at least one query task to at least one target computing node among the at least one computing node, so that the at least one target computing node executes the received query task in a manner adapted to the query type to which the query request belongs.
[0088] In this embodiment, the distributed query system includes at least one front-end node, and the execution subject of this embodiment can be any of the front-end nodes. The implementation form of the front-end node can refer to the relevant content of the above system embodiment, which will not be repeated here.
[0089] In this embodiment, when a query request arrives at the distributed query system, the front-end node receives the query request issued by the user and determines the query type to which the query request belongs. Further, the front-end node generates at least one query task corresponding to the query request, wherein at least one query task carries the query type to which the query request belongs. Further, the front-end node distributes the at least one query task to at least one target computing node in the distributed query system.
[0090] In this embodiment, the distributed query system includes at least one computing node. The target computing node is a part or all of the at least one computing node. The number of target computing nodes can be determined by the number of computing nodes involved in the content to be queried by the query task. In an optional embodiment, the computing nodes involved in the content to be queried by each query task are different, that is, the content to be queried by a query task involves one computing node, then the number of target computing nodes is equal to the number of query tasks.
[0091] Accordingly, the target computing node may execute the query task in a manner adapted to the query type to which the query request carried by the received query task belongs, according to the query type to which the query request belongs. For example, if the query type to which the query request belongs is the first query type, the target computing node preferentially executes the received query task. For the description of the target computing node preferentially executing the received query task, please refer to the relevant content of the above embodiment, which will not be repeated here.
[0092] For another example, if the query type to which the query request belongs is the second query type, the query task that the target computing node can receive is added to the local cache queue, and the received query task is scheduled and executed according to the cache scheduling mechanism. For the description of the first query type and the second query type, please refer to the relevant content of the above embodiment, which will not be repeated here.
[0093] In this embodiment, the front-end node can determine the query type to which the received query request belongs, and add the query type to which the query request belongs to at least one query task corresponding to the query request. In this way, the query method provided by this embodiment provides a basis for the computing node to execute the query task in a manner adapted to the query type according to the query type to which the query request carried by the received query task belongs. Further, when the query type to which the query request belongs is the first query type, the computing node can give priority to executing the received query task, thereby helping to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput.
[0094] In some embodiments, the query request sent by the user may carry a query type identifier. The query type identifier indicates the query type to which the query request belongs. Based on this, an optional implementation of step 202 is: parsing the query type identifier from the query request, and adding the query type identifier to at least one query task. Afterwards, the query task carrying the query type identifier information is sent to at least one target computing node.
[0095] In some other embodiments, the query request includes a filter condition. The filter condition includes a filter condition related to the query table item and a screening condition related to the query data. Based on this, another optional implementation of step 202 is: according to the filter condition of the query request, determine the query type to which the query request belongs. The specific implementation method can refer to the relevant content of the above system embodiment, which will not be repeated here.
[0096] In some other embodiments, in the local cache queue of the computing node, the query tasks may be sorted according to the order of arrival of the corresponding query requests. Preferably, the earlier the query request arrives, the higher the ranking of the corresponding query task in the local cache queue is, and the higher the priority is for scheduling and execution.
[0097] Based on this, when the front-end node determines that the query type to which the query request belongs is the second query type, it can generate a global identifier corresponding to the query request, wherein the global identifier indicates the arrival order of the query request. Further, the front-end node carries the global identifier in at least one query task, and sends the query task carrying the global identifier to the corresponding target computing node.
[0098] Optionally, the front-end node may number the query requests in sequence according to the order in which the query requests arrive, and use the number as the global identifier of the query request. Alternatively, the front-end node may use the arrival time of the query request as the global identifier of the query request. Further, in the case of multiple front-end nodes, since each front-end node works independently when processing a query request, in order to ensure that when the subsequent target computing node processes the query task corresponding to the query request, the position of these query tasks in the local cache queue of the target computing node does not conflict, the front-end node may generate a global identifier corresponding to the query request based on its own identifier and the time when the query request is received. Among them, the identifier of the front-end node is information that can uniquely identify a front-end node, which can be allocated by the physical device cluster where the distributed query system is located at startup. Furthermore, the time when the front-end node receives the query request can be expressed as long floating point data, accurate to milliseconds.
[0099] In an embodiment of the present application, at least one target computing node may execute the query tasks received by each, and obtain the corresponding sub-query results. Further, at least one target computing node may return the sub-query results obtained by each to the corresponding front-end node. Accordingly, the front-end node may receive at least one sub-query result returned by at least one target computing node, and assemble at least one sub-query result to obtain the query result corresponding to the query request received by it. Further, the front-end node outputs the query result corresponding to the query request. Among them, the specific implementation method of the front-end node outputting the query result corresponding to the query request can refer to the relevant content of the above-mentioned embodiment, which will not be repeated here.
[0100] Optionally, the query request may include query items in addition to the filter conditions. The query items are used to describe the rule information for assembling the query results later. Accordingly, the front-end node may assemble at least one sub-query result according to the query items, and then obtain the query result corresponding to the query request it receives.
[0101] It is worth noting that the specific implementation methods of the front-end node generating at least one query task corresponding to the query request and determining the target computing node can all be referred to the relevant contents of the above system embodiment, which will not be repeated here.
[0102] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute each step in the above-mentioned query method executed by the front-end node.
[0103] Figure 3 A flowchart of another query method provided in an embodiment of the present application. The method is applicable to computing nodes in a distributed query system. Figure 3 As shown, the method includes:
[0104] 301. Receive a query task sent by a front-end node in a distributed query system, where the query task carries a query type identifier; the query type identifier indicates the query type to which the query request corresponding to the query task belongs.
[0105] 302. Determine the query type to which the query request belongs according to the query type identifier.
[0106] 303. When the query type to which the query request belongs is the first query type, the received query task is preferentially executed.
[0107] In this embodiment, the computing node is a target computing node in the at least one computing node. The implementation form of the computing node can refer to the relevant content of the above embodiment, which will not be repeated here.
[0108] In this embodiment, when the query type to which the query request belongs is the first query type, the computing node can preferentially execute the received query task, thereby helping to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput. For the description of the first query type, please refer to the relevant content of the above embodiment, which will not be repeated here.
[0109] In this embodiment, the computing node may parse the query type identifier from the received query task; and determine the query type to which the query request belongs according to the query type identifier.
[0110] In an embodiment of the present application, in order to implement priority processing of query requests of the first query type, multiple thread pools may be set in the computing node, some of which are specifically used to process query requests of the first query type. Based on this, an optional implementation of step 303 is: if the target computing node determines that the query type to which the query request belongs is the first query type, then directly select an idle first thread from the thread pool corresponding to the first query type, and call the first thread to execute the received query task. In this way, the query tasks corresponding to the query requests of the first query type are no longer cached, but the idle threads are called to directly process the query tasks corresponding to the query requests of the first query type, which helps to improve the system's response time to the query requests of the first query type, thereby helping to improve the system throughput.
[0111] Furthermore, the computing node is also provided with a local cache queue for caching query tasks corresponding to the second query type. Accordingly, if the computing node determines that the query type to which the query request belongs is the second query type, the received query task is added to the local cache queue, and the received query task is scheduled and executed according to the cache scheduling mechanism. The cache scheduling mechanism may be FIFO, etc., but is not limited thereto.
[0112] Correspondingly, if the query type to which the query request belongs is the second query type, the received query task may be added to the local cache queue, and the received query task may be scheduled and executed according to the cache scheduling mechanism.
[0113] Optionally, in the local cache queue, the query tasks may be sorted according to the arrival order of the corresponding query requests. Preferably, the earlier the query request arrives, the higher the ranking of the corresponding query task in the local cache queue is, and the higher the priority is for scheduling and execution.
[0114] Based on this, when the front-end node determines that the query type to which the query request belongs is the second query type, it can generate a global identifier corresponding to the query request, and carry the global identifier in at least one query task. The global identifier indicates the arrival order of the query request. For the specific implementation method of the front-end node generating the global identifier corresponding to the query request, please refer to the relevant content of the above embodiment, which will not be repeated here.
[0115] Further, in an embodiment of the present application, the local cache queue of the computing node may include: an execution cache queue and a waiting cache queue. For the thread pool corresponding to the second query type, if there is an idle thread therein, the idle thread in the thread pool corresponding to the second query type may be called to process the query task in the execution cache queue. Further, if the execution cache queue can accommodate new query tasks, the query tasks in the waiting cache queue may be added to the execution cache queue to wait for scheduling.
[0116] Based on this, in this embodiment, the computing node can schedule query tasks from the execution cache queue according to the cache scheduling mechanism, and when scheduling the currently received query task, select an idle second thread in the thread pool corresponding to the second query type; and call the second thread to execute the currently received query task.
[0117] Optionally, in this embodiment, a depth threshold of the execution cache queue may be preset, and the depth threshold is less than or equal to the length of the execution cache queue. Preferably, the preset depth threshold is less than the length of the execution cache queue.
[0118] Furthermore, the computing node may add the received query task to the execution cache queue or the waiting cache queue according to the global identifier corresponding to the query request and the cache depth of the execution cache queue.
[0119] Furthermore, if the cache depth of the execution cache queue is less than a preset depth threshold, the computing node adds the received query task to the execution cache queue.
[0120] Further, if the global identifier corresponding to the query request is smaller than the global identifier carried by the query task at the end of the execution cache queue, the received query task is added to the execution cache queue. Optionally, the position of the currently received query task in the execution cache queue can be determined based on the global identifier carried by each query task in the execution cache queue. Preferably, the target computing node adds the currently received query task to the first position, wherein the global identifier of the query task after the first position is smaller than the global identifier of the currently received query task, so as to ensure that the distributed query system gives priority to query requests received earlier and reduce the impact of network quality on query progress.
[0121] Accordingly, if the cache depth of the execution cache queue in the computing node is greater than or equal to the set depth threshold, but the global identifier carried by the received query task is smaller than the global identifier carried by the query task at the end of the execution cache queue, the set depth threshold is increased and the currently received query task is added to the execution cache queue.
[0122] Furthermore, if the cache depth of the execution cache queue is greater than or equal to a preset depth threshold, and the global identifier corresponding to the query request is greater than or equal to the global identifier carried by the query task at the end of the execution cache queue, the computing node will add the received query task to the waiting cache queue, and when the cache depth of the execution cache queue is less than the preset depth threshold, the query task will be added to the execution cache queue.
[0123] Accordingly, in this embodiment, the computing node may also monitor the cache depth of the execution cache queue, and when the execution cache depth is less than a set depth threshold, schedule the query task in the waiting cache queue to the execution cache queue.
[0124] In an embodiment of the present application, the computing node may execute the received query task and obtain the corresponding sub-query result. Further, the computing node may return the sub-query result obtained by each computing node to the corresponding front-end node. Accordingly, the front-end node may receive at least one sub-query result returned by at least one computing node, and assemble at least one sub-query result to obtain the query result corresponding to the query request received by it. Further, the front-end node outputs the query result corresponding to the query request.
[0125] Accordingly, an embodiment of the present application also provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by one or more processors, the one or more processors are caused to execute each step in the above-mentioned query method executed by the computing node.
[0126] It should be noted that the execution subject of each step of the method provided in the above embodiment can be the same device, or the method can be executed by different devices. For example, the execution subject of steps 201 and 202 can be device A; for another example, the execution subject of step 201 can be device A, and the execution subject of step 202 can be device B; and so on.
[0127] In addition, in some of the processes described in the above embodiments and the accompanying drawings, multiple operations appearing in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this document or may be executed in parallel, and the sequence numbers of the operations, such as 201, 202, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel.
[0128] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4 As shown, the electronic device includes: a memory 40a, a processor 40b and a communication component 40c. Among them, the memory 40a is used for computer programs.
[0129] The processor 40b is coupled to the memory 40a, and is used to execute a computer program for: receiving a query request through the communication component 40c; determining the query type to which the query request belongs; generating at least one query task corresponding to the query request; at least one query task carries the query type to which the query request belongs; and distributing at least one query task to at least one target computing node in the distributed query system through the communication component 40c, so that the at least one target computing node executes the received query task in a manner adapted to the query type to which the query request belongs.
[0130] In some embodiments, when determining the query type to which the query request belongs, the processor 40b is specifically configured to: parse a query type identifier from the query request, and add the query type identifier to at least one query task; the query type identifier indicates the query type to which the query request belongs.
[0131] In some other embodiments, the processor 40b is further used to: when the query type to which the query request belongs is the second query type, generate a global identifier corresponding to the query request, and carry the global identifier in at least one query task, the global identifier indicating the arrival order of the query requests.
[0132] Further, when the processor 40b generates the global identifier corresponding to the query request, it is specifically configured to: generate the global identifier corresponding to the query request according to its own identifier and the time when the query request is received.
[0133] In some other embodiments, the processor 40b is further used to: receive at least one sub-query result returned by at least one target computing node through the communication component 40c, assemble at least one sub-query result to obtain a query result corresponding to the query request; and output the query result corresponding to the query request.
[0134] In some optional embodiments, such as Figure 4 As shown, the electronic device may further include: a power supply component 40d and other components. Optionally, if the electronic device is a terminal device, it may further include: a display screen 40e, an audio component 40f and other optional components. Figure 4 The components are shown schematically only, and do not mean that the electronic device must include Figure 4 The components shown do not necessarily mean that the electronic device can only include Figure 4 Components shown.
[0135] The electronic device provided in this embodiment can be implemented as a front-end node in a distributed query system, which can determine the query type to which the received query request belongs, and add the query type to which the query request belongs to at least one query task corresponding to the query request. In this way, therefore, the query method provided in this embodiment provides a basis for the computing node to execute the query task in a manner adapted to the query type according to the query type to which the query request carried by the received query task belongs. Further, when the query type to which the query request belongs is the first query type, the computing node can give priority to executing the received query task, thereby helping to reduce the degree of system congestion, thereby helping to improve the system response time and system throughput.
[0136] Figure 5 This is a schematic diagram of the structure of another electronic device provided in an embodiment of the present application. Figure 5 As shown, the electronic device includes: a memory 50a, a processor 50b and a communication component 50c. Among them, the memory 50a is used for computer programs.
[0137] The processor 50b is coupled to the memory 50a, and is used to execute a computer program for: receiving a query task issued by a front-end node in a distributed query system through a communication component 50c, the query task carrying a query type identifier; the query type identifier indicates the query type to which the query request corresponding to the query task belongs; determining the query type to which the query request belongs based on the query type identifier; and when the query type to which the query request belongs is the first query type, executing the received query task preferentially.
[0138] In some embodiments, when the processor 50b preferentially executes the received query task, it is specifically used to: when the query type to which the query request belongs is the first query type, directly select an idle first thread from the thread pool corresponding to the first query type, and call the first thread to execute the received query task.
[0139] In some other embodiments, further, the memory 50a also includes: a memory and other storage spaces. Among them, the memory includes a local cache queue. Accordingly, the processor 50b is also used to: when the query type to which the query request belongs is the second query type, add the received query task to the local cache queue; and schedule and execute the received query task according to the cache scheduling mechanism.
[0140] Furthermore, the local cache queue includes: an execution cache queue and a waiting cache queue. Accordingly, when the processor 50b adds the received query task to the local cache queue, it is specifically used to: parse the global identifier corresponding to the query request from the received query task; and add the received query task to the execution cache queue or the waiting cache queue according to the global identifier corresponding to the query request and the cache depth of the execution cache queue.
[0141] Further, when the processor 50b adds the received query task to the execution cache queue or the waiting cache queue, it is specifically used to: if the cache depth of the execution cache queue is less than a preset depth threshold, then the received query task is added to the execution cache queue; if the cache depth of the execution cache queue is greater than or equal to the preset depth threshold, and the global identifier corresponding to the query request is less than the global identifier carried by the query task at the end of the execution cache queue, then the set depth threshold is increased and the received query task is added to the execution cache queue; if the cache depth of the execution cache queue is greater than or equal to the preset depth threshold, and the global identifier corresponding to the query request is greater than or equal to the global identifier carried by the query task at the end of the execution cache queue, then the received query task is added to the waiting cache queue.
[0142] Furthermore, the processor 50b is further configured to: monitor the cache depth of the execution cache queue; and when the cache depth of the execution cache queue is less than a set depth threshold, schedule the query task in the waiting cache queue to the execution cache queue.
[0143] In other embodiments, the processor 50b is further used to: schedule query tasks from the execution cache queue according to the cache scheduling mechanism, and when scheduling a received query task, select an idle second thread in the thread pool corresponding to the second query type, and call the second thread to execute the received query task.
[0144] In some optional embodiments, such as Figure 5 As shown, the electronic device may further include: a power supply component 50d and other components. Optionally, if the electronic device is a terminal device, it may further include: a display screen 50e, an audio component 50f and other optional components. Figure 5 The components are shown schematically only, and do not mean that the electronic device must include Figure 5 The components shown do not necessarily mean that the electronic device can only include Figure 5 Components shown.
[0145] The electronic device provided in this embodiment can preferentially execute the received query task when the query type to which the query request belongs is the first query type, thereby helping to reduce the degree of system congestion, thereby helping to improve system response time and system throughput.
[0146] In the embodiment of the present application, the processor can be any hardware processing device that can execute the logic of the above method. Optionally, the processor can be a central processing unit (CPU), a graphics processing unit (GPU) or a microcontroller unit (MCU); it can also be a field programmable gate array (FPGA), a programmable array logic device (PAL), a general array logic device (GAL), a complex programmable logic device (CPLD) and other programmable devices; or an advanced reduced instruction set (RISC) processor (ARM) or a system on chip (SOC), etc., but not limited to this.
[0147] In an embodiment of the present application, the memory is used to store a computer program and can be configured to store various other data to support operations on an electronic device. Among them, the processor can execute the computer program stored in the memory to implement the corresponding control logic. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a static random access memory (SRAM), an electrically erasable programmable read-only memory (EEPROM), an erasable programmable read-only memory (EPROM), a programmable read-only memory (PROM), a read-only memory (ROM), a magnetic memory, a flash memory, a magnetic disk or an optical disk.
[0148] In an embodiment of the present application, the communication component is configured to facilitate wired or wireless communication between the electronic device and other devices. The electronic device can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G, 5G or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component can also be implemented based on near field communication (NFC) technology, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology or other technologies.
[0149] In an embodiment of the present application, the display screen may include a liquid crystal display (LCD) and a touch panel (TP). If the display screen includes a touch panel, the display screen may be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundary of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation.
[0150] In an embodiment of the present application, a power supply assembly is configured to provide power to various components of an electronic device. The power supply assembly may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the device where the power supply assembly is located.
[0151] In an embodiment of the present application, the audio component may be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal may be further stored in a memory or sent via a communication component. In some embodiments, the audio component also includes a speaker for outputting an audio signal. For example, for an electronic device with a language interaction function, voice interaction with a user may be achieved through an audio component.
[0152] It should be noted that the descriptions such as “first” and “second” in this article are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit “first” and “second” to different types.
[0153] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0154] The present invention is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate a device for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0155] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device that implements the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0156] These computer program instructions may also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, whereby the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more processes in the flowchart and / or one or more boxes in the block diagram.
[0157] In a typical configuration, a computing device includes one or more processors, input / output interfaces, network interfaces, and memory.
[0158] Memory may include non-permanent storage in a computer-readable medium, in the form of random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.
[0159] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0160] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.
[0161] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.
Claims
1. A distributed query system, characterized in that: include: at least one front-end node and at least one computing node; The at least one front-end node is configured to receive a query request, determine a query type to which the query request belongs, generate at least one query task corresponding to the query request, and distribute the at least one query task to at least one target computing node among the at least one computing node; The at least one query task carries the query type to which the query request belongs; The at least one target computing node is used to: when the query type to which the query request belongs is a first query type, directly select an idle first thread from a thread pool corresponding to the first query type, and call the first thread to preferentially execute a currently received query task.
2. The system according to claim 1, characterized in that The at least one front-end node is further configured to: parse a query type identifier from the query request, and add the query type identifier to the at least one query task; The query type identifier indicates the query type to which the query request belongs; The at least one target computing node is further configured to: parse the query type identifier from the currently received query task, and determine the query type to which the query request belongs according to the query type identifier.
3. The system according to claim 1, characterized in that The at least one target computing node is further configured to: In a case where the query type to which the query request belongs is the second query type, the currently received query task is added to the local cache queue, and the currently received query task is scheduled and executed according to the cache scheduling mechanism.
4. The system according to claim 3, characterized in that The at least one front-end node is further used to: generate a global identifier corresponding to the query request when the query type to which the query request belongs is the second query type, and carry the global identifier in the at least one query task, wherein the global identifier indicates the arrival order of the query requests.
5. The system according to claim 4, characterized in that The at least one front-end node is specifically configured to generate a global identifier corresponding to the query request according to its own identifier and the time when the query request is received.
6. The system according to claim 4, characterized in that The local cache queue of the at least one target computing node includes: an execution cache queue and a waiting cache queue; The at least one target computing node is specifically used for: Parsing the global identifier corresponding to the query request from the currently received query task; According to the global identifier corresponding to the query request and the cache depth of the execution cache queue, the currently received query task is added to the execution cache queue or the waiting cache queue.
7. The system according to claim 6, characterized in that The at least one target computing node is specifically used to: If the cache depth of the execution cache queue is less than a preset depth threshold, adding the currently received query task to the execution cache queue; If the cache depth of the execution cache queue is greater than or equal to the preset depth threshold, and the global identifier corresponding to the query request is less than the global identifier carried by the query task at the end of the execution cache queue, then increase the preset depth threshold and add the currently received query task to the execution cache queue; If the cache depth of the execution cache queue is greater than or equal to the preset depth threshold, and the global identifier corresponding to the query request is greater than or equal to the global identifier carried by the query task at the end of the execution cache queue, the currently received query task is added to the waiting cache queue.
8. The system according to claim 6, characterized in that The at least one target computing node is further configured to: The cache depth of the execution cache queue is monitored; and when the cache depth of the execution cache queue is less than a preset depth threshold, the query task in the waiting cache queue is dispatched to the execution cache queue.
9. The system according to claim 6, characterized in that The at least one target computing node is specifically used to: The query task is scheduled from the execution cache queue according to the cache scheduling mechanism, and when the currently received query task is scheduled, an idle second thread is selected from the thread pool corresponding to the second query type, and the second thread is called to execute the currently received query task.
10. The system according to any one of claims 1 to 9, characterized in that: The at least one front-end node is further used to: receive at least one sub-query result returned by the at least one target computing node, and assemble the at least one sub-query result to obtain a query result corresponding to the query request; And output the query result corresponding to the query request.
11. The system according to any one of claims 1 to 9, characterized in that: Also includes: Load balancing nodes; The load balancing node is used to balance the query request to the at least one front-end node according to the load condition of the at least one front-end node.
12. A query method, applicable to a front-end node in a distributed query system, characterized in that: include: receiving a query request; Determining the query type to which the query request belongs; generating at least one query task corresponding to the query request; The at least one query task carries the query type to which the query request belongs; Distribute the at least one query task to at least one target computing node in the distributed query system, so that when the query type to which the query request belongs is a first query type, the at least one target computing node directly selects an idle first thread from a thread pool corresponding to the first query type, and calls the first thread to preferentially execute the currently received query task.
13. A query method, applicable to a computing node in a distributed query system, characterized in that: include: Receiving a query task issued by a front-end node in the distributed query system, wherein the query task carries a query type identifier; The query type identifier indicates the query type to which the query request corresponding to the query task belongs; wherein the front-end node determines the query type to which the query request belongs and generates the query task; and determines the query type to which the query request belongs according to the query type identifier; In the case that the query type to which the query request belongs is the first query type, an idle first thread is directly selected from the thread pool corresponding to the first query type, and the first thread is called to preferentially execute the currently received query task.
14. The method according to claim 13, characterized in that Also includes: In a case where the query type to which the query request belongs is the second query type, adding the currently received query task to the local cache queue; According to the cache scheduling mechanism, the currently received query task is scheduled and executed.
15. An electronic device, characterized in that: The electronic device is implemented as a front-end node in a distributed query system, and the electronic device includes: a memory, a processor and a communication component; Wherein, the memory is used for computer programs; The processor is coupled to the memory and is configured to execute the computer program for: receiving a query request through the communication component; Determining the query type to which the query request belongs; generating at least one query task corresponding to the query request; the at least one query task carries the query type to which the query request belongs; The at least one query task is distributed to at least one target computing node in the distributed query system through the communication component, so that when the query type to which the query request belongs is a first query type, the at least one target computing node directly selects an idle first thread from a thread pool corresponding to the first query type, and calls the first thread to preferentially execute the currently received query task.
16. An electronic device, characterized in that: The electronic device is implemented as a computing node in a distributed query system, and the electronic device includes: a memory, a processor, and a communication component; Wherein, the memory is used to store computer programs; The processor is coupled to the memory and is configured to execute the computer program for: Receiving, through the communication component, a query task issued by a front-end node in a distributed query system, the query task carrying a query type identifier; the query type identifier indicates the query type to which the query request corresponding to the query task belongs; wherein the front-end node determines the query type to which the query request belongs, and generates the query task; Determining, according to the query type identifier, the query type to which the query request belongs; In the case that the query type to which the query request belongs is the first query type, an idle first thread is directly selected from the thread pool corresponding to the first query type, and the first thread is called to preferentially execute the currently received query task.
17. The device according to claim 16, characterized in that The processor is further configured to: In a case where the query type to which the query request belongs is the second query type, adding the currently received query task to the local cache queue; According to the cache scheduling mechanism, the currently received query task is scheduled and executed.
18. A computer-readable storage medium storing computer instructions, characterized in that: When the computer instructions are executed by one or more processors, the one or more processors are caused to execute the steps in the method according to any one of claims 12 to 14.
Citation Information
Patent Citations
Method for achieving distributed database query priority management based on peer deployment
CN104391918A
Data query method and device, electronic equipment and storage medium
CN110134707A