Cluster scheduling method, device and equipment

Abstract

The present application relates to the technical field of load balancing, and discloses a cluster scheduling method, device and equipment, including: parsing a received cluster scheduling request to determine the request type corresponding to the cluster scheduling request; determining the election information of the target cluster; selecting a target node in the target cluster based on the request type and the election information; sending the cluster scheduling request to the target node, so that the target node performs a data reading service or a data writing service according to the cluster scheduling request. Because when performing cluster scheduling, the cluster scheduling request will be parsed first to determine the request type of the cluster scheduling request, and a node suitable for executing the cluster scheduling request-related service will be selected from the target cluster as the target node according to the request type, and the request will be distributed to the target node, ensuring that the request can be directly and reasonably distributed during scheduling, avoiding the phenomenon of forwarding requests within the cluster.

Description

Technical Field

[0001] The present application relates to the field of load balancing technology, and in particular to a cluster scheduling method, device and equipment. Background Art

[0002] Nowadays, with the development of business, single-point databases can no longer meet actual needs. Today's databases generally use clustered databases (such as etcd clusters. etcd is a distributed key-value pair storage database that achieves reliable distributed collaboration through master-slave elections, distributed locks, and write barriers). That is, a cluster is formed by multiple devices equipped with databases to provide database services in the form of a cluster. In order to ensure that the clustered database can provide services to the outside world normally, a load balancer is generally set between the end side (that is, the side that accesses the database) and the database cluster. Requests are received through the load balancer and distributed to the database cluster.

[0003] However, current load balancers generally do not distinguish between read and write request types. Taking the etcd cluster as an example, the load balancer will directly dispatch requests to the cluster nodes (also called instances) in the etcd cluster. If a write request is dispatched to a slave node, the slave node needs to forward the write request to the master node for processing. The write request is forwarded by the slave node, which will reduce the request response speed. The forwarding of the request from the slave node will also waste server computing resources and network resources.

[0004] In addition, the load balancer will also dispatch some read requests to the master node in this scheduling mode. As a result, the master node will process all write requests and some read requests, causing write requests and read requests to compete for the resources of the master node, resulting in a higher load on the master node and a lower request response speed, which also reduces the overall load capacity of the clustered database. Summary of the invention

[0005] The main purpose of the present application is to provide a cluster scheduling method, device and equipment, aiming to solve the technical problems that cluster scheduling in related technologies may cause low request response speed and waste of resources.

[0006] To achieve the above purpose, the present application proposes a cluster scheduling method, which includes:

[0007] Parsing the received cluster scheduling request to determine a request type corresponding to the cluster scheduling request, where the request type includes a read request and a write request;

[0008] Determine election information of a target cluster, the election information including a master node and at least one slave node of the target cluster;

[0009] Selecting a target node in the target cluster based on the request type and the election information;

[0010] The cluster scheduling request is sent to the target node, so that the target node executes a data reading service or a data writing service according to the cluster scheduling request.

[0011] In a possible implementation of the present application, parsing the received cluster scheduling request to determine the request type corresponding to the cluster scheduling request includes:

[0012] Parsing the received cluster scheduling request, and extracting a request keyword from the cluster scheduling request; the request keyword is a field related to a data reading or data writing service;

[0013] A request type corresponding to the cluster scheduling request is determined according to the request keyword.

[0014] In a possible implementation of the present application, parsing the received cluster scheduling request and extracting a request keyword from the cluster scheduling request includes:

[0015] Serializing the request data in the received cluster scheduling request to obtain request serialized data, wherein the serialization process is to convert the request data into data in a form that is convenient for storage or transmission; extracting first-level data from the request serialized data, wherein the first-level data is the outermost or root-level key-value pair data in the request serialized data; extracting business-related fields from the first-level data to obtain request keywords;

[0016] or,

[0017] The request data in the received cluster scheduling request is objectified to obtain a request data object, wherein the objectification is to convert the request data into structured data in object form; non-empty fields in the request data object are read to obtain valid fields, wherein the non-empty fields are attribute fields whose corresponding values ​​in the request data object are not empty; business-related fields are extracted from the valid fields to obtain request keywords.

[0018] In a possible implementation of the present application, determining the request type corresponding to the cluster scheduling request according to the request keyword includes:

[0019] If the request keyword includes the first key field and does not include the second key field, determining that the request type corresponding to the cluster scheduling request is a read type;

[0020] If the request keyword includes a first key field and a second key field, determining that the request type corresponding to the cluster scheduling request is a write type;

[0021] The data corresponding to the first key field corresponds to a database field in the database, and the data corresponding to the second key field corresponds to a field value of the database field in the database.

[0022] In a possible implementation of the present application, the step of determining the election information of the target cluster includes:

[0023] Select a node to be visited from the cluster nodes of the target cluster; send an election acquisition request to the node to be visited, so that the node to be visited feeds back election information;

[0024] or,

[0025] The election information sent by the target cluster is received. The election information is sent by an election reporting node in the target cluster when the master node election is completed in the target cluster. The election reporting node is a master node or a slave node in the target cluster.

[0026] In a possible implementation of the present application, selecting a target node in the target cluster based on the request type and the election information includes:

[0027] If the request type is a write type, the master node in the target cluster is selected as the target node;

[0028] If the request type is a read type, a target node is selected from at least one slave node in the target cluster.

[0029] In a possible implementation of the present application, if the request type is a read type, selecting a target node from at least one slave node in the target cluster includes:

[0030] If the request type is a read type, obtaining a slave node identification queue;

[0031] Extract the queue head element from the node identifier queue to obtain the target node identifier;

[0032] Selecting a target node from at least one slave node in the target cluster based on the target node identifier;

[0033] When the selection is completed, the target node identifier is added to the tail of the slave node identifier queue.

[0034] In a possible implementation of the present application, if the request type is a read type, selecting a target node from at least one slave node in the target cluster includes:

[0035] If the request type is a read type, then the last selected sequence number and the slave node sorting result are obtained, where the slave node sorting result is a sorting result generated by sorting at least one slave node in the target cluster according to a preset sorting rule;

[0036] Update the previous selection sequence number to obtain the current selection sequence number;

[0037] The slave node corresponding to the currently selected sequence number in the slave node sorting result is selected as the target node.

[0038] In addition, to achieve the above purpose, the present application also proposes a cluster scheduling device, the device comprising:

[0039] A parsing module, used to parse the received cluster scheduling request and determine the request type corresponding to the cluster scheduling request, wherein the request type includes a read request and a write request;

[0040] A determination module, configured to determine election information of a target cluster, wherein the election information includes a master node and at least one slave node of the target cluster;

[0041] A selection module, configured to select a target node in the target cluster based on the request type and the election information;

[0042] The sending module is used to send the cluster scheduling request to the target node, so that the target node executes a data reading service or a data writing service according to the cluster scheduling request.

[0043] In addition, to achieve the above objectives, the present application also proposes a cluster scheduling device, which includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the cluster scheduling method described above.

[0044] In addition, to achieve the above purpose, the present application also proposes a storage medium, which is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the cluster scheduling method described above are implemented.

[0045] In addition, to achieve the above-mentioned purpose, the present application also proposes a computer program product, wherein the computer program product includes a computer program, and when the computer program is executed by a processor, the steps of the cluster scheduling method described above are implemented.

[0046] One or more technical solutions proposed in this application have at least the following technical effects:

[0047] When performing cluster scheduling, the cluster scheduling request will be parsed first, and the request type of the cluster scheduling request will be determined based on the request keyword obtained by the parsing. According to the request type, a node suitable for executing the cluster scheduling request-related business is selected from the target cluster as the target node, and the request is distributed to the target node, ensuring that the request can be directly and reasonably distributed during scheduling to avoid the phenomenon of forwarding requests within the cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the related technologies, the drawings required for use in the embodiments or the related technical descriptions are briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0050] Figure 1 A schematic diagram of a process flow provided for the first embodiment of the cluster scheduling method of the present application;

[0051] Figure 2 A schematic diagram of a flow chart provided for the second embodiment of the cluster scheduling method of the present application;

[0052] Figure 3 A schematic diagram of a cluster scheduling process according to an embodiment of the present application;

[0053] Figure 4 This is a schematic diagram of the module structure of the cluster scheduling device according to an embodiment of the present application;

[0054] Figure 5 Schematic diagram of the device structure of the hardware operating environment involved in the cluster scheduling method in the embodiment of the present application.

[0055] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0056] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0057] In order to better understand the technical solution of the present application, a detailed description will be given below in conjunction with the accompanying drawings and specific implementation methods.

[0058] Based on this, the present application embodiment provides a cluster scheduling method, referring to Figure 1 , Figure 1 This is a flow chart of the first embodiment of the cluster scheduling method of the present application.

[0059] In this embodiment, the cluster scheduling method includes steps S10 to S40:

[0060] Step S10: Parse the received cluster scheduling request to determine the request type corresponding to the cluster scheduling request.

[0061] It should be noted that the executor of this embodiment can be the cluster scheduling device, and the cluster scheduling device can be a physical device, such as a load balancer set between the side and the database cluster. Of course, it can also be a virtual device, such as a virtual device software with load balancing function set between the side and the database cluster, or other electronic devices that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the cluster scheduling method of the present invention is explained by taking the cluster scheduling device as an example.

[0062] It should be noted that the cluster scheduling request may be a request sent to the cluster scheduling device by other business devices that need to perform data reading or data writing related business, wherein the business device may be a user device, or other server, etc. The request type may include two categories: read request and write request. When the cluster scheduling device receives the cluster scheduling request, the cluster scheduling device may parse the cluster scheduling request to determine whether its specific request category is a read request or a write request.

[0063] In actual use, there may be a corresponding service identifier in the cluster scheduling request. The cluster scheduling device may parse the cluster scheduling request, extract the service identifier therefrom, and determine whether it is related to the read service or the write service according to the service identifier, thereby determining the request type corresponding to the cluster scheduling request.

[0064] In a possible implementation of this embodiment, some cluster scheduling requests may not include a service identifier. In order to ensure that the request type corresponding to the cluster scheduling request can still be quickly determined in this case, step S10 in this embodiment may include:

[0065] Parsing the received cluster scheduling request and extracting request keywords from the cluster scheduling request;

[0066] A request type corresponding to the cluster scheduling request is determined according to the request keyword.

[0067] It should be noted that the request keyword can be a field related to data reading or data writing services, and the number and / or type of request keywords contained in requests of different request types are different. Based on this, in actual use, it is possible to determine whether the cluster scheduling request is specifically related to data reading or data writing based on the number and / or type of request keywords, thereby determining the request type corresponding to the cluster scheduling request.

[0068] In actual use, the cluster scheduling request can be parsed in a pre-agreed request format, and fields related to data reading or data writing services can be extracted therefrom to obtain request keywords.

[0069] In a possible implementation of this embodiment, in order to facilitate data parsing, the step of parsing the received cluster scheduling request and extracting the request keyword from the cluster scheduling request in this embodiment may include:

[0070] Serialize the request data in the received cluster scheduling request to obtain request serialization data;

[0071] Extracting first level data from the request serialized data;

[0072] Business-related fields are extracted from the first-level data to obtain request keywords.

[0073] It should be noted that the request data can be extracted from the request body of the cluster scheduling request. Of course, the request data of some requests are spliced ​​in the access link of the request. At this time, the access link of the cluster scheduling request can also be intercepted to obtain the request data. The serialization process can be to convert the cluster scheduling request into data in a form that is easy to store or transmit. For example, the request data in the cluster scheduling request is serialized and converted into data in json or xml format, and the data obtained after the conversion is used as the request serialization data.

[0074] It is understandable that when a request is transmitting data, the data may be transmitted in a manner that is difficult to process directly, such as a data stream. Such data is difficult to process directly. Therefore, the request data can be first extracted from the cluster scheduling request, and then the request data can be serialized to generate request serialized data that is easy to analyze and process.

[0075] It should be noted that the first-level data can be the outermost or root-level key-value pair data in the request serialization data. For example, assuming the request serialization data is {"name": "Zhang San", "age": 30, "email": "zs@abc.com", "address": {"street": "123", "city": "Beijing", "country": "China"}, "hobbies": ["reading", "traveling", "photography"]}, then the fields such as name, age, email, address, hobbies, and the data combinations corresponding to each field are the first-level data.

[0076] In actual use, to extract business-related fields from the first-level data and obtain request keywords, the fields in the first-level data that are related to data reading or data writing operations can be used as request keywords.

[0077] For example, taking a key-value pair database as an example, the keywords related to data reading or data writing include key and value. At this time, assuming the first-level data is {"time": "1673082950", "from": "user1", "key": "100"}, then the request keyword extracted from the first-level data at this time is key.

[0078] In a possible implementation manner of this embodiment, for the convenience of data parsing, the step of parsing the received cluster scheduling request and extracting request keywords from the cluster scheduling request may include:

[0079] Perform objectification processing on the request data in the received cluster scheduling request to obtain a request data object;

[0080] Read the non-empty fields in the request data object to obtain valid fields;

[0081] Extract business-related fields from the valid fields to obtain request keywords.

[0082] It should be noted that the objectification processing can be to convert the request data into structured data in object form. The non-empty fields are the attribute fields in the request data object whose corresponding values are not empty.

[0083] It is understandable that when a request is transmitting data, the data may be transmitted in a manner that is difficult to process directly, such as a data stream. Such data is difficult to process directly. Therefore, the request data can be first extracted from the cluster scheduling request, and then the request data can be objectified and converted into structured object data that is easy to process, and then processed.

[0084] In actual use, each valid field can be matched with a business-related field, and the successfully matched valid field is used as a request keyword.

[0085] For example: Assume that the corresponding type of the requested data object is service, which contains fields including time, from, key, value, isfake, etc., among which the values ​​corresponding to the value and isfake fields are empty. In this case, the valid fields are time, from, key, and business-related fields (that is, keywords related to data reading or data writing) include key and value. In this case, the key match is successful, and the obtained request keyword is key.

[0086] In a possible implementation manner of this embodiment, in order to accurately determine the request type corresponding to the cluster scheduling request, the step of determining the request type corresponding to the cluster scheduling request according to the request keyword in this embodiment may include:

[0087] If the request keyword includes the first key field and does not include the second key field, determining that the request type corresponding to the cluster scheduling request is a read type;

[0088] If the request keyword includes a first key field and a second key field, it is determined that the request type corresponding to the cluster scheduling request is a write type.

[0089] It should be noted that the data corresponding to the first key field corresponds to the database field in the database, and the data corresponding to the second key field corresponds to the field value of the database field in the database.

[0090] In actual use, when the key-value database performs data query, the relevant data required to be used actually only includes the data corresponding to the database field in the database. Therefore, if the request keyword includes the first key field and does not include the second key field, it can be determined that the business corresponding to the cluster scheduling request is a data query business. At this time, it can be determined that the request type corresponding to the cluster scheduling request is a read type;

[0091] When writing data to the key-value database, the data required to be used will include both the data corresponding to the database fields in the database and the data corresponding to the field values ​​of the database fields in the database. Therefore, if the request keyword includes the first key field and the second key field, it can be determined that the business corresponding to the cluster scheduling request is a data writing business. At this time, it can be determined that the request type corresponding to the cluster scheduling request is a write type.

[0092] For example: assuming that the first key field is key and the second key field is value, if the request keyword corresponding to the cluster scheduling request only includes key, then the request type of the cluster scheduling request can be determined to be a read request; if the request keyword corresponding to the cluster scheduling request includes both key and value, then the request type of the cluster scheduling request can be determined to be a read request.

[0093] Step S20: Determine election information of the target cluster, where the election information includes a master node and at least one slave node of the target cluster.

[0094] It should be noted that the target cluster can be a key-value storage database cluster, such as an etcd cluster, a zookeeper cluster, a redis cluster, etc. The target cluster can include multiple nodes. During the operation of the cluster, the nodes in the cluster will be divided into a master node and at least one slave node through elections. At this time, the target cluster will generate election information to represent the division of master nodes and slave nodes.

[0095] In a possible implementation of this embodiment, the cluster scheduling device may actively obtain election information, and the step S20 may include:

[0096] Select a node to be accessed from the cluster nodes of the target cluster;

[0097] A selection acquisition request is sent to the node to be visited, so that the node to be visited feeds back election information.

[0098] It should be noted that when selecting a node to be visited, a node can be randomly selected from the target cluster as the node to be visited. Of course, if the visit also requires the consumption of corresponding resources in the node, when selecting a node to be visited, the node with the lowest currently occupied resources in the target cluster can be selected as the node to be visited. Among them, the occupied resources can be the currently occupied hardware resources (such as memory, disk read and write) and / or computing resources (such as the computing power of the computing core, etc.) in the node.

[0099] In actual use, if the cluster scheduling device sends a selection request to the node to be accessed, the node to be accessed will feedback the election information value to the cluster scheduling device, and the election information is the relevant information for electing the master node in the target cluster. The cluster scheduling device can determine which node in the target cluster is the master node based on the election information, and use other nodes in the target cluster except the master node as slave nodes.

[0100] In some cases, not all nodes in the target cluster may provide external services (for example, some nodes are offline or in hibernation). In this case, if nodes other than the master node in the target cluster are used as slave nodes, subsequent requests may be distributed to such nodes that do not provide external services. In order to avoid this situation, the election information can include relevant information of both the master node and the slave node. In this case, the cluster scheduling device can directly determine which node in the target cluster is the master node and which nodes in the target cluster are slave nodes based on the election information.

[0101] In specific applications, the cluster scheduling device can periodically obtain election information, for example: with a period of 5 seconds, select the node to be visited from the cluster nodes of the target cluster every 5 seconds, send a selection request to the node to be visited, and receive the election information fed back by the node to be visited, and then determine the master node and at least one slave node of the target cluster through the election information.

[0102] In a possible implementation of this embodiment, in order to ensure rapid synchronization when the master and slave nodes in the target cluster change, the election information may also be actively sent by the target cluster to the cluster scheduling device, and the step S20 may include:

[0103] Receive election information sent by the target cluster.

[0104] It should be noted that the election information is sent by the election reporting node in the target cluster when the master node election is completed in the target cluster. The election reporting node is the master node or the slave node in the target cluster.

[0105] In actual use, in order to avoid repeated reporting and reduce unnecessary performance consumption, when the master node election is completed in the target cluster, a node can be selected in the cluster as the election reporting node. For example, the master node is directly used as the election reporting node, one is randomly selected from at least one slave node as the election reporting node, or the slave node with the lowest occupied resources is used as the election reporting node.

[0106] The specific implementation method of determining the master node and at least one slave node of the target cluster according to the election information is the same as described above and will not be repeated here.

[0107] In specific applications, if the election information is obtained periodically, if the target cluster re-elects the master node within two consecutive cycles, the master and slave nodes determined by the cluster scheduling device will be inconsistent with the actual master and slave nodes within a certain period of time; and if the election information is actively reported, the election information may fail to be reported due to network fluctuations and other reasons, and the master and slave nodes determined by the cluster scheduling device will be inconsistent with the actual master and slave nodes. In order to avoid this defect, the above two methods of obtaining election information and determining the master and slave nodes of the target cluster can be combined, that is, the cluster scheduling device can periodically obtain election information from the target cluster, and when the master node election is completed in the target cluster, the election information will also be reported to the cluster scheduling device.

[0108] Step S30: Select a target node in the target cluster based on the request type and the election information.

[0109] It should be noted that the target node may be a master node or a slave node selected in the target cluster to perform business processing.

[0110] In actual use, the master node and slave node divided in the target cluster can be determined according to the election information. Then, based on the request type, a node suitable for executing the data reading service or data writing service corresponding to the cluster scheduling request is selected from the master node and slave node in the target cluster as the target node.

[0111] Step S40: Send the cluster scheduling request to the target node, so that the target node executes a data reading service or a data writing service according to the cluster scheduling request.

[0112] What is required is that after receiving the cluster scheduling request, the target node can execute related data reading services or data writing services according to the data in the cluster scheduling request, and then feedback the business data of the completed business to the cluster scheduling device, and the cluster scheduling device can forward the business data to the device that initiated the cluster scheduling request.

[0113] This embodiment provides a cluster scheduling method. When performing cluster scheduling, the cluster scheduling request is first parsed, and the request type of the cluster scheduling request is determined according to the request keyword obtained by the parsing. According to the request type, a node suitable for executing services related to the cluster scheduling request is selected from the target cluster as the target node, and the request is distributed to the target node, ensuring that the request can be directly and reasonably distributed during scheduling to avoid the phenomenon of forwarding requests within the cluster.

[0114] Based on the first embodiment of the present application, in the second embodiment of the present application, the same or similar contents as those in the above-mentioned embodiment 1 can refer to the above introduction, and will not be repeated later. Figure 2, step S30, step S30, including steps S301 and S302:

[0115] Step S301: If the request type is a write type, a master node in the target cluster is selected as a target node.

[0116] It should be noted that after the election information of the target cluster is determined, the specific division of the master node and the slave node in the target cluster can be determined according to the election information.

[0117] In order to avoid dirty data or data conflicts, in the database cluster, generally only the master node is allowed to write data, while other slave nodes can synchronize the data written by the master node. If the request type corresponding to the cluster scheduling request is a write type, then in the target cluster, only the master node can execute the business corresponding to the cluster scheduling request. Therefore, the master node in the target cluster can be selected as the target node.

[0118] Step S302: If the request type is a read type, a target node is selected from at least one slave node in the target cluster.

[0119] It should be noted that in the target cluster, both the master node and the slave node actually have the relevant functions of data query. If the request type corresponding to the cluster scheduling request is a read type, it means that all nodes in the target cluster can execute the business corresponding to the cluster scheduling request. However, in order to increase the load of the database cluster, it is necessary to avoid read requests and write requests competing for the resources of the master node, that is, it is necessary to avoid the master node executing both data query and data writing businesses. Therefore, if the request type corresponding to the cluster scheduling request is a read type, one of at least one slave node in the target cluster can be selected as the target node.

[0120] Among them, when selecting the target node from at least one slave node in the target cluster, it is necessary to ensure that the load levels of each slave node are the same or similar as much as possible. In order to achieve this effect, polling, minimum resource occupancy, minimum number of connections, shortest response time and other selection methods can be used to select the target node from at least one slave node in the target cluster.

[0121] For ease of understanding, now combined Figure 3 This is for illustration only, but not for limitation. Figure 3 The diagram is a schematic diagram of the cluster scheduling process of this embodiment.

[0122] like Figure 3 As shown, the load balancer is provided with a cluster detection module and a read-write separation module;

[0123] Among them, the cluster detection module: based on the Lua script, the load balancer periodically detects etcd election information, and saves the Leader (i.e., the master node) changes and updates them to the load balancer Lua script global variables. The subsequent Leader election information will be used as a parameter for the read-write separation module to execute the scheduling strategy.

[0124] Read-write separation module: Parses the HTTP content of etcd requests based on Lua scripts, and distinguishes the read and write types of requests by parsing the json-level keywords of the HTTP body (i.e. requestbody) content. If the first level of the json in the body content has the "key" keyword but no "value" keyword, the request is a read request; if the first level of the json in the body content has the "key" and "value" keywords, the request is a write request.

[0125] The cluster scheduling process is as follows:

[0126] 1. The etcd cluster performs Leader election. The etcd cluster elects one Leader instance and multiple Follower instances.

[0127] 2. During the initialization phase of the load balancer, the cluster detection module and the read-write separation module are loaded. The cluster detection module periodically detects the etcd cluster Leader election information and updates it to the load balancer Lua script global variables.

[0128] 3. The load balancer receives the etcd request (such as Figure 3 The read-write separation module parses the request to distinguish the read and write types. If it is a etcd read type request, the request is evenly distributed (such as using polling to distribute requests) to each Follower instance (i.e., slave node). If it is an etcd write type request, the request is dispatched to the Leader instance (i.e., master node). This ensures that the etcd cluster instance directly processes the request without the phenomenon of transferring requests within the etcd cluster.

[0129] This embodiment provides a cluster scheduling method. When the request type is a write type, the master node in the target cluster is selected as the target node; when the request type is a read type, the target node is selected from at least one slave node in the target cluster, thereby ensuring that read requests and write requests do not compete for the resources of the master node, so that the master node only needs to execute related services of data writing, thereby indirectly improving the load capacity of the target cluster.

[0130] In a feasible implementation, step S302 may include steps S3021-3024:

[0131] Step S3021: If the request type is a read type, obtain a slave node identification queue.

[0132] It should be noted that the slave node identification queue may be an ordered queue storing the node identification of each slave node in the target cluster, wherein the node identification may be a unique identification of the slave node in the target cluster.

[0133] Step S3022: extracting the queue head element from the node identifier queue to obtain the target node identifier.

[0134] It should be noted that the element at the head of the queue in the node identification queue can be used as the target node identification, wherein, in order to avoid the same node being selected repeatedly, the target node identification can also be removed from the node identification queue after obtaining the target node identification. The head of the queue can be the position of the first element in the node identification queue, and when removing an element from the queue, it is always removed from the head of the queue.

[0135] Step S3023: Select a target node from at least one slave node in the target cluster based on the target node identifier.

[0136] It should be noted that the target node identifier may be compared with the node identifiers of each slave node in the target cluster, and the slave node in the target cluster whose node identifier is consistent with the target node identifier may be selected as the target node.

[0137] Step S3024: When the selection is completed, the target node identifier is added to the tail of the slave node identifier queue.

[0138] It should be noted that, since the business is ongoing, in order to ensure that the slave node currently selected as the target node can still be selected as the target node again, and to prevent the slave node from being frequently selected as the target node, the target node identifier can be added to the tail of the slave node identifier queue. The tail of the queue can be the position of the last element in the slave node identifier queue. When adding a new element to the queue, it is always added to the tail of the queue.

[0139] In actual application, if the election information is re-acquired, the slave node identification queue can be rebuilt according to the slave nodes determined by the election information.

[0140] In this embodiment, the target node is selected from at least one slave node in the target cluster by combining queues, so that the probability of each slave node being selected as the target node is kept as consistent as possible, ensuring that read-type requests can be evenly distributed to each slave node in the target cluster as much as possible.

[0141] In a feasible implementation, step S302 may include steps S3021'-3023':

[0142] Step S3021': If the request type is a read type, then obtain the last selected sequence number and the slave node sorting result.

[0143] It should be noted that the slave node sorting result can be a sorting result generated by sorting at least one slave node in the target cluster according to a preset sorting rule, wherein the preset sorting rule can be set according to the node identifier or the node name. If the slave node remains unchanged, at least one slave node in the target cluster is sorted according to the preset sorting rule, and the generated sorting result is consistent.

[0144] In actual use, the last selected sequence number may be the selection sequence number used when the target node was selected last time.

[0145] Step S3022': Update the previous selection sequence number to obtain the current selection sequence number.

[0146] In actual use, the last selected sequence number may be increased by one, and the last selected sequence number after the increase by one is used as the current selected sequence number.

[0147] Step S3023': Select the slave node corresponding to the currently selected sequence number in the slave node sorting result as the target node.

[0148] It should be noted that the execution process of selecting the slave node corresponding to the current selection sequence number in the node sorting result as the target node can be to select the slave node whose sorting sequence number in the node sorting result is consistent with the current selection sequence number as the target node.

[0149] In specific applications, after selecting the target node, the current selection sequence number can be used as the new previous selection sequence number.

[0150] In actual use, since the selection sequence number is continuously incremented, it is very likely to exceed the maximum value of the sorting sequence number of the node sorting result as it increments. In order to ensure that the target node can be correctly selected, before executing step S3023', the current selection sequence number can be compared with the maximum value of the sorting sequence number. If the current selection sequence number is less than or equal to the maximum value of the sorting sequence number, the slave node corresponding to the current selection sequence number in the slave node sorting result is selected as the target node;

[0151] If the current selection number is greater than the maximum value of the sorting number, the current selection number is modified to the minimum value of the sorting number, and then the slave node corresponding to the current selection number modified from the node sorting result is selected as the target node.

[0152] For example: the sorting sequence number of the node sorting result is 1-n, n is the maximum sorting sequence number, and 1 is the minimum sorting sequence number. Assuming that the current selected sequence number is s, if s≤n, then the slave node corresponding to s in the node sorting result is directly selected as the target node; if s>n, then set s=1, and then select the slave node corresponding to s in the node sorting result as the target node.

[0153] In actual application, if the election information is obtained again, the slave node sorting result can be reconstructed according to the slave nodes determined by the election information.

[0154] In this embodiment, the target node is selected from at least one slave node in the target cluster by combining the sorting results, so that the probability of each slave node being selected as the target node is kept as consistent as possible, thereby ensuring that read-type requests can be evenly distributed to each slave node in the target cluster.

[0155] This application also provides a cluster scheduling device, please refer to Figure 4 , the cluster scheduling device comprises:

[0156] The parsing module 10 is used to parse the received cluster scheduling request and determine the request type corresponding to the cluster scheduling request, where the request type includes a read request and a write request;

[0157] A determination module 20, configured to determine election information of a target cluster, wherein the election information includes a master node and at least one slave node of the target cluster;

[0158] A selection module 30, configured to select a target node in the target cluster based on the request type and the election information;

[0159] The sending module 40 is used to send the cluster scheduling request to the target node, so that the target node executes a data reading service or a data writing service according to the cluster scheduling request.

[0160] The cluster scheduling device provided by the present application adopts the cluster scheduling method in the above embodiment, which can solve the technical problem that cluster scheduling in related technologies causes low request response speed and waste of resources. Compared with related technologies, the beneficial effects of the cluster scheduling device provided by the present application are the same as the beneficial effects of the cluster scheduling method provided by the above embodiment, and other technical features in the cluster scheduling device are the same as the features disclosed in the above embodiment method, which will not be repeated here.

[0161] The present application provides a cluster scheduling device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the cluster scheduling method in the above-mentioned embodiment 1.

[0162] Reference below Figure 5 , which shows a schematic diagram of the structure of a cluster scheduling device suitable for implementing an embodiment of the present application. The cluster scheduling device in the embodiment of the present application may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 5 The cluster scheduling device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0163] like Figure 5 As shown, the cluster scheduling device may include a processing device 1001 (such as a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM: Read Only Memory) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM: Random Access Memory) 1004. In RAM1004, various programs and data required for the operation of the cluster scheduling device are also stored. The processing device 1001, ROM1002 and RAM1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the I / O interface 1006: an input device 1007 including, for example, a touch screen, a touchpad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output device 1008 including, for example, a liquid crystal display (LCD: Liquid Crystal Display), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the cluster scheduling device to communicate with other devices wirelessly or by wire to exchange data. Although the figure shows a cluster scheduling device with various systems, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0164] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a ROM 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0165] The cluster scheduling device provided by the present application adopts the cluster scheduling method in the above embodiment, which can solve the technical problems that cluster scheduling in related technologies causes low request response speed and waste of resources. Compared with related technologies, the beneficial effects of the cluster scheduling device provided by the present application are the same as the beneficial effects of the cluster scheduling method provided by the above embodiment, and other technical features in the cluster scheduling device are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0166] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0167] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0168] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the cluster scheduling method in the above-mentioned embodiment.

[0169] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM: Random Access Memory), a read-only memory (ROM: Read Only Memory), an erasable programmable read-only memory (EPROM: Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM: CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency: Radio Frequency), etc., or any suitable combination of the above.

[0170] The computer-readable storage medium may be included in the cluster scheduling device; or may exist independently without being assembled into the cluster scheduling device.

[0171] The above-mentioned computer-readable storage medium carries one or more programs. When the above-mentioned one or more programs are executed by the cluster scheduling device, the cluster scheduling device: parses the received cluster scheduling request, determines the request type corresponding to the cluster scheduling request, and the request type includes a read request and a write request; determines the election information of the target cluster, and the election information includes the master node and at least one slave node of the target cluster; selects a target node in the target cluster based on the request type and the election information; sends the cluster scheduling request to the target node, so that the target node performs a data reading service or a data writing service according to the cluster scheduling request.

[0172] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0173] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0174] The modules involved in the embodiments described in this application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0175] The readable storage medium provided in this application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the above-mentioned cluster scheduling method, and can solve the technical problems that cluster scheduling in related technologies causes low request response speed and waste of resources. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as the beneficial effects of the cluster scheduling method provided in the above-mentioned embodiment, and will not be repeated here.

[0176] The present application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the steps of the cluster scheduling method as described above are implemented.

[0177] The computer program product provided by this application can solve the technical problem that cluster scheduling in related technologies causes low request response speed and wastes resources. Compared with related technologies, the beneficial effects of the computer program product provided by this application are the same as the beneficial effects of the cluster scheduling method provided by the above embodiment, which will not be repeated here.

[0178] The above descriptions are only some embodiments of the present application, and are not intended to limit the patent scope of the present application. All equivalent structural changes made using the contents of the present application specification and drawings under the technical concept of the present application, or direct / indirect applications in other related technical fields are included in the patent protection scope of the present application.

Claims

1. A cluster scheduling method, characterized in that: The cluster scheduling method comprises: Parsing the received cluster scheduling request to determine a request type corresponding to the cluster scheduling request, where the request type includes a read request and a write request; Determine election information of a target cluster, the election information including a master node and at least one slave node of the target cluster; Selecting a target node in the target cluster based on the request type and the election information; Sending the cluster scheduling request to the target node, so that the target node performs a data reading service or a data writing service according to the cluster scheduling request; The parsing of the received cluster scheduling request to determine the request type corresponding to the cluster scheduling request includes: Parsing the received cluster scheduling request, and extracting a request keyword from the cluster scheduling request; the request keyword is a field related to a data reading or data writing service; The request type corresponding to the cluster scheduling request is determined according to the number and / or type of the request keywords.

2. The cluster scheduling method according to claim 1, characterized in that: The step of parsing the received cluster scheduling request and extracting a request keyword from the cluster scheduling request includes: Serializing the request data in the received cluster scheduling request to obtain request serialized data, wherein the serialization process is to convert the request data into data in a form that is convenient for storage or transmission; extracting first-level data from the request serialized data, wherein the first-level data is the outermost or root-level key-value pair data in the request serialized data; extracting business-related fields from the first-level data to obtain request keywords; or, The request data in the received cluster scheduling request is objectified to obtain a request data object, wherein the objectification is to convert the request data into structured data in object form; non-empty fields in the request data object are read to obtain valid fields, wherein the non-empty fields are attribute fields whose corresponding values ​​in the request data object are not empty; business-related fields are extracted from the valid fields to obtain request keywords.

3. The cluster scheduling method according to claim 1, characterized in that: The determining the request type corresponding to the cluster scheduling request according to the number and / or type of the request keywords includes: If the request keyword includes the first key field and does not include the second key field, determining that the request type corresponding to the cluster scheduling request is a read type; If the request keyword includes a first key field and a second key field, determining that the request type corresponding to the cluster scheduling request is a write type; The data corresponding to the first key field corresponds to a database field in the database, and the data corresponding to the second key field corresponds to a field value of the database field in the database.

4. The cluster scheduling method according to claim 1, characterized in that: The determining of the election information of the target cluster includes: Select a node to be visited from the cluster nodes of the target cluster; send an election acquisition request to the node to be visited, so that the node to be visited feeds back election information; or, The election information sent by the target cluster is received. The election information is sent by an election reporting node in the target cluster when the master node election is completed in the target cluster. The election reporting node is a master node or a slave node in the target cluster.

5. The cluster scheduling method according to any one of claims 1 to 4, characterized in that: The selecting a target node in the target cluster based on the request type and the election information includes: If the request type is a write type, the master node in the target cluster is selected as the target node; If the request type is a read type, a target node is selected from at least one slave node in the target cluster.

6. The cluster scheduling method according to claim 5, characterized in that: If the request type is a read type, selecting a target node from at least one slave node in the target cluster includes: If the request type is a read type, obtaining a slave node identification queue; Extract the queue head element from the node identifier queue to obtain the target node identifier; Selecting a target node from at least one slave node in the target cluster based on the target node identifier; When the selection is completed, the target node identifier is added to the tail of the slave node identifier queue.

7. The cluster scheduling method according to claim 5, characterized in that: If the request type is a read type, selecting a target node from at least one slave node in the target cluster includes: If the request type is a read type, then the last selected sequence number and the slave node sorting result are obtained, where the slave node sorting result is a sorting result generated by sorting at least one slave node in the target cluster according to a preset sorting rule; Update the previous selection sequence number to obtain the current selection sequence number; The slave node corresponding to the currently selected sequence number in the slave node sorting result is selected as the target node.

8. A cluster scheduling device, characterized in that: The device comprises: A parsing module, used to parse the received cluster scheduling request and determine the request type corresponding to the cluster scheduling request, wherein the request type includes a read request and a write request; A determination module, configured to determine election information of a target cluster, wherein the election information includes a master node and at least one slave node of the target cluster; A selection module, configured to select a target node in the target cluster based on the request type and the election information; A sending module, used for sending the cluster scheduling request to the target node, so that the target node performs a data reading service or a data writing service according to the cluster scheduling request; The parsing module is also used to parse the received cluster scheduling request and extract request keywords from the cluster scheduling request; the request keywords are fields related to data reading or data writing services; and the request type corresponding to the cluster scheduling request is determined according to the number and / or type of the request keywords.

9. A cluster scheduling device, characterized in that: The device comprises: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program is configured to implement the steps of the cluster scheduling method according to any one of claims 1 to 7.

Images