Method, device and database for selecting computing nodes

Abstract

Embodiments of the present specification provide a method, device, and database for selecting a computing node, wherein the method for selecting a computing node includes: obtaining multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost; for each computing node, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis of the factor values ​​corresponding to the multiple computing cost factors is performed to predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node; and selecting a computing node that executes the computing request according to the comprehensive computing costs corresponding to the computing request when it is executed on each computing node.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and more particularly to a method for selecting a computing node. One or more embodiments of this specification also relate to an apparatus for selecting a computing node, a database, a computing device, and a computer-readable storage medium. Background Art

[0002] Computing nodes are nodes that participate in data computing in scenarios such as federated computing and data lake computing, such as databases and computing engines.

[0003] Since data may be scattered in different databases, different clusters, and different networks, when a computing request is received, the data is generally pulled to a computing node for computing. Since the capabilities of each computing node are different, it is necessary to select a suitable computing node to perform the calculation in order to improve computing efficiency. Summary of the invention

[0004] In view of this, an embodiment of this specification provides a method for selecting a computing node. One or more embodiments of this specification also relate to an apparatus for selecting a computing node, a database, a computing device, and a computer-readable storage medium to solve the technical defects existing in the prior art.

[0005] According to the first aspect of the embodiments of the present specification, a method for selecting a computing node is provided, comprising: obtaining multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, the multiple computing cost factors including any two or more factors of node type, node load, and transmission cost; for each computing node, performing weighted synthesis on the factor values ​​corresponding to the multiple computing cost factors according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and predicting the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node; and selecting a computing node that executes the computing request according to the comprehensive computing costs corresponding to the computing request when it is executed on each computing node.

[0006] Optionally, obtaining multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node respectively includes: obtaining the network transmission rate corresponding to each computing node respectively; obtaining the network load idle rate corresponding to each computing node respectively; obtaining the amount of data transmitted between nodes respectively corresponding to the computing request when executed on each computing node; and for each computing node respectively, obtaining the transmission cost corresponding to the targeted computing node based on the corresponding transmission cost being equal to the amount of data transmitted between nodes divided by the product of the corresponding network transmission rate and the corresponding network load idle rate.

[0007] Optionally, the acquisition of multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node includes: for each computing node, acquiring the CPU load of the targeted computing node and the CPU load weight corresponding to the computing type of the computing request and the targeted computing node; for each computing node, acquiring the memory load of the targeted computing node and the memory load weight corresponding to the computing type of the computing request and the targeted computing node; for each computing node, weightedly combining the corresponding CPU load and the corresponding memory load according to the corresponding weights to obtain the node load of the targeted computing node.

[0008] Optionally, for each computing node respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, the factor values ​​corresponding to the multiple computing cost factors are weighted combined to predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node, including: for each computing node respectively, according to the factor weights corresponding to the node type, node load, and transmission cost, when the computing request is executed on the targeted computing node, the matching value between the node type and the computing request, the node load, and the transmission cost are weighted combined to predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node.

[0009] Optionally, before obtaining multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, it also includes: disassembling the received joint computing request to obtain multiple disassembled computing requests; after selecting the computing node to execute the computing request, it also includes: sending the multiple computing requests to the selected computing nodes respectively; receiving the execution results of the computing request from each selected computing node; performing joint computing corresponding to the joint computing request on the received execution results to obtain a joint computing result.

[0010] Optionally, before obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, it also includes: compressing the computing request by encoding the computing request to obtain an encoded computing request; after selecting the computing node to execute the computing request, it also includes: sending the encoded computing request to the selected computing node for execution; receiving the execution result of the computing request from the selected computing node; and decoding the execution result to obtain a decoded execution result.

[0011] According to the second aspect of the embodiments of this specification, there is provided a device for selecting a computing node, comprising: an acquisition module, configured to acquire multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost. A comprehensive calculation module, configured to perform weighted synthesis of the factor values ​​corresponding to the multiple computing cost factors for each computing node according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and predict the comprehensive computing cost corresponding to the computing request when the computing request is executed on the targeted computing node. A selection module, configured to select a computing node that executes the computing request according to the comprehensive computing costs corresponding to the computing request when the computing request is executed on each computing node.

[0012] According to the third aspect of the embodiments of this specification, a database is provided, including: a computing engine node and a plurality of computing nodes. The computing engine node is configured to obtain a plurality of computing cost factors of a computing request and factor values ​​corresponding to each computing node, wherein the plurality of computing cost factors include any two or more factors of node type, node load, and transmission cost; for each computing node, according to the factor weights corresponding to the plurality of computing cost factors and the computing nodes targeted, weighted synthesis is performed on the factor values ​​corresponding to the plurality of computing cost factors, and the comprehensive computing cost corresponding to the computing request when the computing request is executed on the computing node targeted is predicted; according to the comprehensive computing cost corresponding to the computing request when each computing node is executed, the computing node that executes the computing request is selected; and the computing request is sent to the selected computing node. The computing node is configured to receive the computing request sent by the computing engine node and execute the computing request.

[0013] According to a fourth aspect of an embodiment of the present specification, a computing device is provided, comprising: a memory and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions: obtaining multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, the multiple computing cost factors including any two or more factors of node type, node load, and transmission cost; for each computing node, respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis of the factor values ​​corresponding to the multiple computing cost factors is performed to predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node; according to the comprehensive computing costs corresponding to the computing request when it is executed on each computing node, a computing node that executes the computing request is selected.

[0014] According to a fifth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores computer instructions, and when the instructions are executed by a processor, the steps of the method for selecting a computing node described in any embodiment of this specification are implemented.

[0015] An embodiment of the present specification provides a method for selecting a computing node. Since the method obtains multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, and the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, for each computing node, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, the factor values ​​corresponding to the multiple computing cost factors are weighted and integrated, and the comprehensive computing cost corresponding to the computing request when executed on the targeted computing node can be accurately predicted. According to the comprehensive computing cost corresponding to the computing request on each computing node, the computing node that executes the computing request is selected. It can be seen that the method provided in the embodiment of the present specification comprehensively considers the situation of two or more factors such as database type characteristics, transmission cost, node load, etc., and determines an efficient computing route through weighted integration, which can obtain better computing performance and improve computing efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 is a flowchart of a method for selecting a computing node provided by an embodiment of this specification;

[0017] Figure 2a This is a schematic representation of the computing node type provided by an embodiment of this specification;

[0018] Figure 2b It is a representation of the comprehensive calculation cost provided by an embodiment of this specification;

[0019] Figure 2c This is a directional representation of network load provided by an embodiment of this specification;

[0020] Figure 2d It is a schematic diagram of transmission cost calculation provided by an embodiment of this specification;

[0021] Figure 2e This is a schematic diagram of computing node load provided by an embodiment of this specification;

[0022] Figure 2f This is a coding mapping representation provided by an embodiment of this specification;

[0023] Figure 3 It is a schematic diagram of a database structure provided by an embodiment of this specification;

[0024] Figure 4is a functional schematic diagram of a monitoring system provided by an embodiment of this specification;

[0025] Figure 5 is a process flow chart of a method for selecting a computing node provided by an embodiment of this specification;

[0026] Figure 6 It is a structural diagram of a device for selecting a computing node provided by an embodiment of this specification;

[0027] Figure 7 is a structural schematic diagram of a device for selecting a computing node provided by another embodiment of the present specification;

[0028] Figure 8 It is a schematic diagram of a database structure provided by an embodiment of this specification;

[0029] Fig. 9 It is a structural block diagram of a computing device provided by an embodiment of this specification. DETAILED DESCRIPTION

[0030] Many specific details are described in the following description to facilitate a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the connotation of this specification, so this specification is not limited to the specific implementation disclosed below.

[0031] The terms used in one or more embodiments of this specification are only for the purpose of describing specific embodiments, and are not intended to limit one or more embodiments of this specification. The singular forms of "a", "said" and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms, unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0032] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, this information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0033] In this specification, a method for selecting a computing node is provided. This specification also relates to an apparatus for selecting a computing node, a computing device, and a computer-readable storage medium, which are described in detail one by one in the following embodiments.

[0034] Figure 1 A flowchart of a method for selecting a computing node provided according to an embodiment of the present specification is shown, including steps 102 to 106.

[0035] Step 102: Obtain multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node respectively, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost.

[0036] For example, assuming that the multiple computing cost factors include node type, node load, and transmission cost. The computing nodes include: computing node 1, computing node 2, and computing node 3. Then, the three cost factors of node type, node load, and transmission cost may be obtained, which correspond to three factor values ​​of computing node 1, three factor values ​​of computing node 2, and three factor values ​​of computing node 3, respectively.

[0037] The node type is used to represent the database type of the node itself. For example, HBase database, ADB database, etc. HBase is suitable for point query calculation; ADB database is suitable for join calculation, and has the association advantage of SQL join. Taking HBase type node as an example, if the calculation request is a point query calculation, the factor value corresponding to the node type factor of the calculation request on the HBase type calculation node is the matching value "0" of the two. If the calculation request is a join calculation, the factor value corresponding to the node type factor of the calculation request on the HBase type calculation node is the matching value "1" of the two. Specifically, for example, you can set it as follows Figure 2a The computing node type table shown is used to record the node type of each computing node.

[0038] Wherein, the node load is used to represent the load of the computing resources of the computing node. For example, computing resource CPU load, computing resource memory load, IO resource load, computing resource weighted load, etc. Wherein, computing resource CPU load is a percentage used to describe the CPU usage of the computing node. For example, a database is configured with 100 CPU cores, and 50 are currently used, then the CPU load is 50%. Computing resource memory load is a percentage used to describe the memory usage of the computing node. For example, a database is configured with 100G memory, and 50G is currently used, then the load is 50%. IO resource load is a percentage used to describe IO usage. Computing resource weighted load is a percentage used to describe the overall load of computing resources, which is obtained by weighted synthesis of computing resource CPU load and memory load. It is understandable that the node load may include the load of other computing resources such as the data source docked with the computing node, which will not be repeated here.

[0039] It should be noted that the embodiments of this specification do not limit which computing resource load conditions are used to represent the node load. For example, the weighted load of computing resources can be used to represent the node load. Under this implementation, the corresponding weight size can be configured on demand for the load of the corresponding computing resource based on the resource requirements of the computing request, for example, for CPU-intensive computing requests, the calculation requires more CPU, for memory-intensive computing requests, the calculation requires more memory, and for IO-intensive computing requests, the calculation requires more IO resources. In the above example, since the node load is a factor that affects the comprehensive computing cost as a weighted calculation, and the node load is affected by the calculation type of the computing request, it is possible to select a computing node that is more suitable for executing the computing request.

[0040] The transmission cost is used to indicate the network resource usage of the data required for the transmission of the computing request between computing nodes. For example, the transmission cost factor of the computing request corresponding to the factor value on the computing node can be determined based on the amount of data required for the computing request, the network load of the directional transmission between nodes, and the network transmission rate.

[0041] Step 104: For each computing node, perform weighted synthesis of the computing costs corresponding to the multiple computing cost factors according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node.

[0042] For each computing node, the factor weights corresponding to each of the plurality of computing cost factors on the computing node may be set according to the characteristics of the computing node and the needs of the scenario. The sum of the factor weights corresponding to the plurality of computing cost factors is "1".

[0043] For example, for each computing node, according to the factor weights corresponding to the node type, node load, and transmission cost, the node type and the matching value of the computing request, the node load, and the transmission cost when the computing request is executed on the computing node are weighted and integrated to predict the comprehensive computing cost corresponding to the computing request when the computing request is executed on the computing node. According to this embodiment, the calculation method of the comprehensive computing cost R can be expressed by the following formula:

[0044] R=a1*c1+a2*c2+a3*c3

[0045] Among them, c1 represents the factor value corresponding to the node type, a1 represents the factor weight corresponding to the node type, c2 represents the factor value corresponding to the node load, a2 represents the factor weight corresponding to the node load, c3 represents the factor value corresponding to the transmission cost, a3 represents the factor weight corresponding to the transmission cost, and a1+a2+a3=1.

[0046] For example, Figure 2b The comprehensive calculation cost table shown is based on Figure 2b The factor values ​​corresponding to the three factors of node type, node load, and transmission cost as shown in the figure and the preset factor weights are: the node type factor weight is 0.9, the node load factor weight is 0.08, and the transmission cost factor weight is 0.02. Figure 2b The comprehensive computing costs corresponding to the three computing nodes shown.

[0047] Step 106: Select a computing node that executes the computing request according to the comprehensive computing costs corresponding to each computing node when the computing request is executed.

[0048] For example, the computing request may be executed on each computing node according to its corresponding comprehensive computing cost, and the computing node with the minimum cost may be selected to execute the computing plan. Figure 2b As shown in the comprehensive computing cost table, since computing node 1 has the smallest comprehensive computing cost, computing node 1 can be selected to execute the computing request.

[0049] It can be seen that since the method obtains multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, and the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, therefore, for each computing node, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, the factor values ​​corresponding to the multiple computing cost factors are weighted and integrated, which can accurately predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node, and select the computing node that executes the computing request according to the comprehensive computing cost corresponding to the computing request on each computing node. The method provided in the embodiment of this specification comprehensively considers the situation of two or more factors such as database type characteristics, transmission cost, node load, etc., and determines an efficient computing route through weighted integration, which can obtain better computing performance and improve computing efficiency.

[0050] In one or more embodiments of the present specification, considering that the transmission cost is not only affected by the network load, but also by the amount of data transmitted between nodes, in order to be able to calculate the transmission cost more accurately, the acquisition of multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node may include: acquiring the network transmission rate corresponding to each computing node; acquiring the network load idle rate corresponding to each computing node; acquiring the amount of data transmitted between nodes corresponding to the computing request when each computing node is executed; and for each computing node, obtaining the transmission cost corresponding to the targeted computing node based on the corresponding transmission cost being equal to the amount of data transmitted between the corresponding nodes divided by the product of the corresponding network transmission rate and the corresponding network load idle rate.

[0051] For example, the calculation method of the transmission cost can be expressed by the following formula:

[0052] Transmission cost = amount of data transmitted between nodes / (network transmission rate*network load idle rate), where network load idle rate = (1-network load rate).

[0053] The network load rate refers to the network load between two computing nodes and is a percentage. For example, if a line has a rated capacity of 100Mbps and at a certain moment, the data volume on the network is only 20Mbps, then the network load rate is 20%.

[0054] Among them, the amount of data transmitted between nodes can be obtained based on the histogram information. For example, the amount of data may be 100M, 2G, etc. A histogram, also known as a quality distribution graph, is a statistical report graph that represents the data distribution by a series of vertical stripes or line segments of varying heights. Generally, the horizontal axis represents the data type, and the vertical axis represents the distribution. In a database, a histogram can represent the distribution of a column's statistical information. Column statistics are statistical information about columns in a database table, such as the maximum value, minimum value, and distribution of the column.

[0055] In order to obtain network load information, you can set Figure 2c The network load directed table shown is used to record the network load of directed transmission between each computing node and refresh it regularly or on demand. Figure 2c The network load directed table shown is Figure 2d The network transmission rate and data transmission volume in the transmission cost calculation diagram shown in FIG. 1 and the calculation formula of the transmission cost can be obtained as follows: Figure 2d The transmission cost is shown.

[0056] In this embodiment, the transmission cost is accurately calculated by integrating factors such as network load and the amount of data transmitted between nodes, making the comprehensive computing cost prediction more accurate, thereby being able to determine a more efficient computing route, obtain better computing performance, and improve computing efficiency.

[0057] In one or more embodiments of the present specification, in order to more accurately obtain the node load, the node load is calculated by weighting the CPU and memory resources. Specifically, for example, the method of obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node may include: obtaining the CPU load of the computing node targeted and the CPU load weight corresponding to the computing type of the computing request and the computing node targeted for each computing node; obtaining the memory load of the computing node targeted and the memory load weight corresponding to the computing type of the computing request and the computing node targeted for each computing node; and weighting and combining the corresponding CPU load and the corresponding memory load according to the corresponding weights for each computing node to obtain the node load of the computing node targeted.

[0058] For example, the calculation method of the node load can be expressed by the following formula:

[0059] Node load = CPU load * CPU load weight + memory load * memory load weight, where CPU load weight + memory load weight = 1.

[0060] Specifically, for example, a computing node load table may be set to obtain the node loads corresponding to the computing requests when they are executed on each computing node. Figure 2e The computing node load table shown in the figure records the CPU and memory load of the computing node and the weight corresponding to the CPU load, which can be refreshed regularly or on demand. The weights corresponding to the CPU and memory can be set accordingly according to the matching degree between different computing types and computing nodes. Figure 2e The calculation node load table shown in Figure 1 and the calculation formula for the node load can be obtained as follows: Figure 2e Node loads shown.

[0061] In this embodiment, the weights corresponding to the CPU load and the memory load are determined according to the characteristics of the computing type of the computing request. For example, for CPU-intensive computing types, more CPU resources are required, and the corresponding CPU load weight is correspondingly higher. For memory-intensive computing types, more memory resources are required, and the corresponding memory load weight is higher. Therefore, the node load suitable for the computing type can be more accurately calculated through resource weighted comprehensive calculation. Further, the prediction of the comprehensive computing cost is more accurate, so that a more efficient computing route can be determined, better computing performance can be obtained, and computing efficiency can be improved.

[0062] In one or more embodiments of the present specification, in order to be able to fully select a computing node to perform the calculation of a computing request based on the capabilities, transmission costs, node loads, etc. that the database is good at, the joint computing request can be disassembled, and the multiple disassembled computing requests can be sent to the computing nodes that are compatible with the disassembled computing requests. Specifically, before obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, it can also include: disassembling the received joint computing request to obtain the multiple disassembled computing requests. After selecting the computing node to execute the computing request, it can also include: sending the multiple computing requests to the selected computing nodes respectively; receiving the execution results of the computing request from each selected computing node; performing the joint calculation corresponding to the joint computing request on the received execution results to obtain the joint calculation results.

[0063] In this embodiment, since the joint computing request is disassembled, the multiple disassembled computing requests are sent to the computing nodes suitable for the disassembled computing requests respectively, so that there is no need to pull all the data to one computing node. It is possible to fully select the appropriate computing node to execute the disassembled computing request based on the database's capabilities, transmission costs, node loads, etc., thereby obtaining better computing performance and improving computing efficiency.

[0064] In one or more embodiments of the present specification, in order to further improve the computing efficiency, the data is compressed according to the coding, so that the compressed data transmission is more efficient and the computing efficiency is higher. Specifically, for example, before obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, it can also include: compressing the computing request by encoding the computing request to obtain the encoded computing request. After selecting the computing node to execute the computing request, it can also include: sending the encoded computing request to the selected computing node for execution; receiving the execution result of the computing request from the selected computing node; decoding the execution result to obtain the decoded execution result.

[0065] For example, you can set Figure 2f The coding mapping table shown is used to set the mapping relationship between actual data and coding. The coding mapping table can be preset, and the data is stored according to the coding. In this embodiment, the coding value can be used to store the data to achieve the purpose of compressing the data. The coding value can also be used to participate in the calculation during the calculation. After the result is returned, it is decoded and converted into the original value, so that the compressed data transmission is more efficient and the calculation efficiency is higher.

[0066] In one or more embodiments of this specification, in order to facilitate the above Figure 2a-2f The information in the information table shown is updated in time. The method can also be used as follows Figure 3 Each computing node in the networking system of the database shown is provided with a monitoring system for monitoring the load of each computing node, the network load of each network, etc., so as to timely refresh the relevant information in the above information table. Figure 3 The database shown in the figure, the method for selecting a computing node provided in the embodiment of this specification can be applied to a computing node that serves as a query engine. Figure 4 The monitoring system functional diagram shown in the figure can monitor the network database and refresh it to the computing node load table and the network load directed table in real time or on demand.

[0067] The following combination Figure 5 , taking the application of the method for selecting computing nodes provided in this specification in the CBO optimizer for query plan optimization as an example, the method for selecting computing nodes is further explained. CBO, evaluates the execution cost of each plan based on the actual data distribution and organization, and thus selects the execution plan with the minimum cost. Among them, Figure 5 A flowchart of a processing procedure of a method for selecting a computing node provided by an embodiment of the present specification is shown, and specific steps include step 502 to step 526.

[0068] Step 502: Receive the query request sent.

[0069] For example, the following query request is received from the query engine:

[0070] select*from(select*from db1.t1 where id<1001)a join(select sum(c1),sum(c2)from db2.t1 where name like'test%'group by c1,c2)b join(select*fromdb3.t1 where address='xxxx')c on a.id=b.id and a.id=c.id

[0071] Step 504: Decompose the query request to obtain a number of sub-queries.

[0072] For example, by breaking down the above query request, we can get the following three sub-queries:

[0073] Subquery a: The corresponding query type is CPU-intensive, with an estimated data volume of 1G and a CPU load weight of 0.8;

[0074] Subquery b: The corresponding query type is memory-intensive, with an estimated data volume of 1M and a CPU load weight of 0.2;

[0075] Subquery c: The corresponding query type is CPU intensive, the estimated data volume is 1M, and the CPU load weight is 0.8.

[0076] Step 506: Obtain the matching value between the subquery type and the node type of each computing node.

[0077] Step 508: Evaluate the data size of the subquery.

[0078] Step 510: Count network load in real time.

[0079] Step 512: Real-time statistics of node load.

[0080] Step 514: Calculate the transmission cost according to the data volume of the sub-query and the network load.

[0081] Step 516: For each computing node, according to the factor weights corresponding to the node type, node load, and transmission cost, the matching value of the sub-query and the node type, the node load, and the transmission cost are weighted and combined to predict the comprehensive computing costs corresponding to several sub-queries on each computing node.

[0082] Step 518: Select computing nodes corresponding to the sub-queries respectively according to the comprehensive computing costs corresponding to the sub-queries at the respective computing nodes.

[0083] Step 520: Send the subquery to the corresponding computing node.

[0084] Step 522: Receive the query result returned by the computing node.

[0085] Step 524: restore the query result according to the encoding and decoding.

[0086] Step 526: Perform a joint calculation corresponding to the query request on the query results of the plurality of sub-queries to obtain a joint query result.

[0087] In this embodiment, after receiving a query request, multiple sub-queries are decomposed according to CBO, and the data size of each sub-query, the network at that time, and the node load are evaluated, so that the cost is calculated based on the weighted comprehensive calculation of these factors, and the computing node used to execute the sub-query is selected based on the cost, which can determine an efficient computing route, obtain better computing performance, and improve computing efficiency.

[0088] Corresponding to the above method embodiment, this specification also provides an apparatus embodiment for selecting a computing node. Figure 6 FIG. 1 is a schematic diagram showing a structure of a device for selecting a computing node provided by an embodiment of the present specification. Figure 6 As shown, the device includes: an acquisition module 602, a comprehensive calculation module 604 and a selection module 606.

[0089] The acquisition module 602 may be configured to acquire multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost.

[0090] The comprehensive calculation module 604 can be configured to perform weighted synthesis on the factor values ​​corresponding to the multiple computing cost factors for each computing node respectively according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node.

[0091] The selection module 606 may be configured to select a computing node for executing the computing request according to the comprehensive computing costs corresponding to the computing request when each computing node executes the computing request.

[0092] Since the device obtains multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, and the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, therefore, for each computing node, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, the factor values ​​corresponding to the multiple computing cost factors are weighted and integrated, so that the comprehensive computing cost corresponding to the computing request when executed on the targeted computing node can be accurately predicted, and the computing node that executes the computing request is selected according to the comprehensive computing cost corresponding to the computing request on each computing node. It can be seen that the device provided in the embodiment of this specification comprehensively considers the situation of two or more factors such as database type characteristics, transmission cost, node load, etc., and determines an efficient computing route through weighted integration, which can obtain better computing performance and improve computing efficiency.

[0093] Figure 7 FIG. 2 shows a schematic diagram of a device for selecting a computing node provided by another embodiment of the present specification. Figure 7 As shown, the acquisition module 602 may include: a transmission rate acquisition submodule 6021, a network load acquisition submodule 6022, a data volume acquisition submodule 6023 and a transmission cost calculation submodule 6024.

[0094] The transmission rate acquisition submodule 6021 may be configured to acquire the network transmission rates corresponding to the respective computing nodes.

[0095] The network load acquisition submodule 6022 may be configured to acquire the network load idle rates corresponding to the respective computing nodes.

[0096] The data volume acquisition submodule 6023 may be configured to acquire the data volume of data transmitted between nodes corresponding to each computing node when the computing request is executed.

[0097] The transmission cost calculation submodule 6024 can be configured to obtain the transmission cost corresponding to the computing node for each computing node respectively, according to the corresponding transmission cost which is equal to the amount of data transmitted between the corresponding nodes divided by the product of the corresponding network transmission rate and the corresponding network load idle rate.

[0098] In this embodiment, the transmission cost is accurately calculated by integrating factors such as network load and the amount of data transmitted between nodes, making the comprehensive computing cost prediction more accurate, thereby being able to determine a more efficient computing route, obtain better computing performance, and improve computing efficiency.

[0099] In one or more embodiments of this specification, Figure 7As shown, the acquisition module 602 may include: a CPU load acquisition submodule 6025, a memory load acquisition submodule 6026 and a node load integration submodule 6027.

[0100] The CPU load acquisition submodule 6025 may be configured to acquire, for each computing node, the CPU load of the computing node and the CPU load weight corresponding to the computing type of the computing request and the computing node.

[0101] The memory load acquisition submodule 6026 may be configured to acquire, for each computing node, the memory load of the computing node and the memory load weight corresponding to the computing type of the computing request and the computing node.

[0102] The node load synthesis submodule 6027 may be configured to perform weighted synthesis of the corresponding CPU load and the corresponding memory load according to the corresponding weights for each computing node, to obtain the node load of the targeted computing node.

[0103] In this embodiment, the weights corresponding to the CPU load and the memory load are determined according to the characteristics of the computing type of the computing request. For example, for CPU-intensive computing types, more CPU resources are required, and the corresponding CPU load weight is correspondingly higher. For memory-intensive computing types, more memory resources are required, and the corresponding memory load weight is higher. Therefore, the node load suitable for the computing type can be more accurately calculated through resource weighted comprehensive calculation. Further, the prediction of the comprehensive computing cost is more accurate, so that a more efficient computing route can be determined, better computing performance can be obtained, and computing efficiency can be improved.

[0104] In one or more embodiments of this specification, Figure 7 As shown, the comprehensive calculation module 604 can be configured to perform weighted comprehensive calculation of the matching value between the node type and the calculation request, the node load, and the transmission cost when the calculation request is executed on the targeted computing node, according to the factor weights corresponding to the node type, the node load, and the transmission cost, respectively, for each computing node, and predict the comprehensive computing cost corresponding to the calculation request when it is executed on the targeted computing node.

[0105] In one or more embodiments of this specification, Figure 7 As shown, the device may further include: a computing request decomposing module 608 , a computing request sending module 610 , a result receiving module 612 and a joint computing module 614 .

[0106] The computing request disassembly module 608 may be configured to disassemble the received joint computing request to obtain multiple disassembled computing requests.

[0107] The computing request sending module 610 may be configured to send the multiple computing requests to the selected computing nodes respectively.

[0108] The result receiving module 612 may be configured to receive the execution result of the computing request from each selected computing node.

[0109] The joint calculation module 614 may be configured to perform a joint calculation corresponding to the joint calculation request on the received execution result to obtain a joint calculation result.

[0110] In this embodiment, since the joint computing request is disassembled, the multiple disassembled computing requests are sent to the computing nodes suitable for the disassembled computing requests respectively, so that there is no need to pull all the data to one computing node. It is possible to fully select the appropriate computing node to execute the disassembled computing request based on the database's capabilities, transmission costs, node loads, etc., thereby obtaining better computing performance and improving computing efficiency.

[0111] In one or more embodiments of this specification, Figure 7 As shown, the device may further include: an encoding module 616, which compresses the computing request by encoding the computing request to obtain an encoded computing request. The computing request sending module 610 may be configured to send the encoded computing request to the selected computing node for execution. The result receiving module 612 may be configured to receive the execution result of the computing request from the selected computing node. The device may further include: a decoding module 618, which may be configured to decode the execution result to obtain a decoded execution result.

[0112] In this embodiment, the encoded values ​​can be used to store data to achieve the purpose of compressing the data. The encoded values ​​can also be used in calculations. After the results are returned, they are decoded and converted into original values, thereby making the compressed data transmission more efficient and the calculation efficiency higher.

[0113] The above is a schematic scheme of a device for selecting a computing node in this embodiment. It should be noted that the technical scheme of the device for selecting a computing node and the technical scheme of the method for selecting a computing node belong to the same concept, and the details of the technical scheme of the device for selecting a computing node that are not described in detail can all be referred to the description of the technical scheme of the method for selecting a computing node.

[0114] Corresponding to the above method embodiment, this specification also provides a database embodiment, Figure 8 FIG. 1 shows a schematic diagram of a database structure provided by an embodiment of this specification. Figure 8As shown, the database includes: a computing engine node 802 and a plurality of computing nodes such as computing node 804, computing node 806 and computing node 808;

[0115] The computing engine node 802 is configured to obtain multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost; for each computing node, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors to predict the comprehensive computing cost corresponding to the computing request when the computing request is executed on the targeted computing node; according to the comprehensive computing costs corresponding to the computing request when each computing node is executed, the computing node that executes the computing request is selected; and the computing request is sent to the selected computing node;

[0116] Any one or more computing nodes among the computing node 804 , the computing node 806 and the computing node 808 are configured to receive the computing request sent by the computing engine node 802 and execute the computing request.

[0117] For example, computing node 804 , computing node 806 , and computing node 808 may be databases such as database 1 , database 2 , and database 3 .

[0118] Fig. 9 The structure block diagram of a computing device 900 provided according to an embodiment of the present specification is shown. The components of the computing device 900 include but are not limited to a memory 910 and a processor 920. The processor 920 is connected to the memory 910 via a bus 930, and a database 950 is used to store data.

[0119] The computing device 900 also includes an access device 940 that enables the computing device 900 to communicate via one or more networks 960. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 940 may include one or more of any type of network interface (e.g., a network interface card (NIC)) that is wired or wireless, such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a World Wide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0120] In one embodiment of the present specification, the above components of the computing device 900 and Fig. 9Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Fig. 9 The computing device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0121] The computing device 900 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or PC. The computing device 900 may also be a mobile or stationary server.

[0122] The processor 920 is used to execute the following computer executable instructions:

[0123] Obtaining multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost;

[0124] For each computing node, respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors, and a comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node is predicted;

[0125] A computing node for executing the computing request is selected according to the comprehensive computing costs respectively corresponding to the computing request when each computing node executes the computing request.

[0126] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the method for selecting computing nodes described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be found in the description of the technical scheme of the method for selecting computing nodes described above.

[0127] An embodiment of the present specification further provides a computer-readable storage medium storing computer instructions, which are used when executed by a processor to:

[0128] Obtaining multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost;

[0129] For each computing node, respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors, and a comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node is predicted;

[0130] A computing node for executing the computing request is selected according to the comprehensive computing costs respectively corresponding to the computing request when each computing node executes the computing request.

[0131] The above is a schematic scheme of a computer-readable storage medium of this embodiment. It should be noted that the technical scheme of the storage medium and the technical scheme of the method for selecting a computing node described above belong to the same concept, and the details not described in detail in the technical scheme of the storage medium can be referred to the description of the technical scheme of the method for selecting a computing node described above.

[0132] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0133] The computer instructions include computer program codes, which may be in source code form, object code form, executable files or some intermediate forms, etc. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0134] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0135] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0136] The preferred embodiments of this specification disclosed above are only used to help explain this specification. The optional embodiments do not describe all the details in detail, nor do they limit the invention to only the specific implementation methods described. Obviously, many modifications and changes can be made according to the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that technicians in the relevant technical field can well understand and use this specification. This specification is only limited by the claims and their full scope and equivalents.

Claims

1. A method for selecting a computing node, comprising: Obtain multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, respectively, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, wherein the node load of each computing node is obtained by weighted synthesis based on the CPU load and the corresponding memory load corresponding to the computing node according to the corresponding weights, the weights corresponding to the CPU load and the memory load are determined according to the computing type of the computing request, and the factor value corresponding to the transmission cost factor on the computing node is determined according to the amount of data required for the computing request, the network load for directed transmission between nodes, and the network transmission rate; For each computing node, respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors, and a comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node is predicted; A computing node for executing the computing request is selected according to the comprehensive computing costs respectively corresponding to the computing request when each computing node executes the computing request.

2. According to the method of claim 1, the step of obtaining multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node respectively comprises: Obtaining the network transmission rate corresponding to each computing node; Obtaining the network load idle rate corresponding to each computing node; Obtaining the amount of data transmitted between nodes corresponding to each computing node when the computing request is executed; For each computing node, the corresponding transmission cost is equal to the data volume of the corresponding inter-node transmission data divided by the product of the corresponding network transmission rate and the corresponding network load idle rate, so as to obtain the transmission cost corresponding to the computing node.

3. According to the method of claim 1, the step of obtaining multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node respectively comprises: For each computing node, respectively, obtain the CPU load of the computing node and the CPU load weight corresponding to the computing type of the computing request and the computing node; For each computing node, respectively, obtain the memory load of the computing node and the memory load weight corresponding to the computing type of the computing request and the computing node; For each computing node, the corresponding CPU load and the corresponding memory load are weighted and integrated according to the corresponding weights to obtain the node load of the targeted computing node.

4. The method according to claim 1, wherein for each computing node, weighted synthesis of the factor values ​​corresponding to the multiple computing cost factors is performed according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and the predicted comprehensive computing cost corresponding to the computing request when the computing request is executed on the targeted computing node comprises: For each computing node, according to the factor weights corresponding to the node type, node load, and transmission cost, the matching value between the node type and the computing request, the node load, and the transmission cost are weighted combined when the computing request is executed on the targeted computing node to predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node.

5. The method according to claim 1, before obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, further comprises: Decomposing the received joint computing request to obtain multiple decomposed computing requests; After selecting the computing node to execute the computing request, the method further includes: sending the plurality of computing requests to the selected computing nodes respectively; receiving the execution result of the computing request from each selected computing node; The received execution result is subjected to the joint calculation corresponding to the joint calculation request to obtain the joint calculation result.

6. The method according to claim 1, before obtaining the multiple computing cost factors of the computing request and the factor values ​​corresponding to each computing node, further comprises: The computing request is compressed by encoding the computing request to obtain an encoded computing request; After selecting the computing node to execute the computing request, the method further includes: Send the encoded computing request to the selected computing node for execution; Receiving an execution result of the computing request from the selected computing node; The execution result is decoded to obtain a decoded execution result.

7. A device for selecting a computing node, comprising: An acquisition module is configured to acquire multiple computing cost factors of a computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, wherein the node load of each computing node is obtained by weighted synthesis based on the CPU load and the corresponding memory load corresponding to the computing node according to the corresponding weights, the weights corresponding to the CPU load and the memory load are determined according to the computing type of the computing request, and the factor value corresponding to the transmission cost factor on the computing node is determined according to the amount of data required for the computing request, the network load for directed transmission between nodes, and the network transmission rate; The comprehensive calculation module is configured to perform weighted comprehensive calculation on the factor values ​​corresponding to the multiple computing cost factors according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes for each computing node, and predict the comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node; The selection module is configured to select a computing node that executes the computing request according to the comprehensive computing costs corresponding to the computing request when each computing node executes the computing request.

8. A database comprising: Compute engine node and multiple compute nodes; The computing engine node is configured to obtain multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost; for each computing node, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, and the comprehensive computing cost corresponding to the computing request when the computing request is executed on the targeted computing node is predicted; according to the comprehensive computing costs corresponding to the computing request when each computing node is executed, the computing node that executes the computing request is selected; and the computing request is sent to the selected computing node, wherein the node load of each computing node is obtained by weighted synthesis based on the CPU load and the corresponding memory load corresponding to the computing node according to the corresponding weights, the weights corresponding to the CPU load and the memory load are determined according to the computing type of the computing request, and the factor value corresponding to the transmission cost factor on the computing node is determined according to the amount of data required for the computing request, the network load for directed transmission between nodes, and the network transmission rate; The computing node is configured to receive a computing request sent by the computing engine node and execute the computing request.

9. A computing device comprising: Memory and processor; The memory is used to store computer executable instructions, and the processor is used to execute the computer executable instructions: Obtain multiple computing cost factors of the computing request and factor values ​​corresponding to each computing node, respectively, wherein the multiple computing cost factors include any two or more factors of node type, node load, and transmission cost, wherein the node load of each computing node is obtained by weighted synthesis based on the CPU load and the corresponding memory load corresponding to the computing node according to the corresponding weights, the weights corresponding to the CPU load and the memory load are determined according to the computing type of the computing request, and the factor value corresponding to the transmission cost factor on the computing node is determined according to the amount of data required for the computing request, the network load for directed transmission between nodes, and the network transmission rate; For each computing node, respectively, according to the factor weights corresponding to the multiple computing cost factors and the targeted computing nodes, weighted synthesis is performed on the factor values ​​corresponding to the multiple computing cost factors, and a comprehensive computing cost corresponding to the computing request when it is executed on the targeted computing node is predicted; A computing node for executing the computing request is selected according to the comprehensive computing costs respectively corresponding to the computing request when each computing node executes the computing request.

10. A computer-readable storage medium storing computer instructions, which, when executed by a processor, implement the steps of the method for selecting a computing node according to any one of claims 1 to 6.

Images