Cluster container expansion method, device, computer equipment and storage medium

Abstract

The present invention discloses a method for expanding a cluster container, which is applied to the field of computer technology and is used to improve the efficiency of cluster container expansion. The method provided by the present invention includes: receiving an expansion request for expanding a cluster, performing domain name configuration on the cluster to be expanded according to domain name configuration information, and obtaining domain name information of the cluster to be expanded; obtaining node domain name information of the cluster to be expanded according to the domain name information of the cluster to be expanded and a preset container deployment file; performing node initialization on the cluster to be expanded based on the node domain name information, obtaining a routing node, performing shard configuration according to the routing node, activating the shard node, and using the shard node as the expansion space of the cluster container.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cluster container expansion method, device, computer equipment and storage medium. Background Art

[0002] Big data has become an important information resource in some large online live streaming platforms, news recommendation systems and other application scenarios. In actual big data storage application scenarios, when there is a significant increase in data and huge concurrency, traditional relational databases need to create multiple storage tables to ensure that application data can be stored in a timely manner. However, this method is cumbersome and inconvenient to modify later, resulting in low storage efficiency.

[0003] In order to improve the efficiency of storing application data, the existing technology generally adopts a cluster-based method to store data. However, the existing cluster capacity expansion method manages resources based on a pre-set configuration file. The resource size occupied by each container is fixed from the beginning of the declaration and creation, and cannot be flexibly and dynamically adjusted. If you want to adjust it, you must close the old container and recreate it manually. For applications that need to adjust the storage space, the efficiency of storing application data is not high.

[0004] Therefore, it is urgent to propose a cluster container expansion method that improves expansion efficiency. Summary of the invention

[0005] The present invention provides a cluster container expansion method, device, computer equipment and storage medium to improve the efficiency of expanding the container and quickly expand the storage space.

[0006] A method for expanding capacity of a cluster container, comprising:

[0007] receiving an expansion request for expanding a cluster, and determining, based on the expansion request, whether the expansion request is a first expansion, and obtaining a determination result;

[0008] If the judgment result is that the extension request is the first extension, performing domain name configuration on the cluster to be extended according to the domain name configuration information to obtain domain name information of the cluster to be extended;

[0009] According to the domain name information of the cluster to be expanded and the preset container deployment file, container deployment is performed on the cluster to be expanded to obtain node domain name information of the cluster to be expanded;

[0010] Based on the node domain name information, perform node initialization on the cluster to be expanded, obtain the routing node of the cluster to be expanded, and configure the sharding node through the routing node;

[0011] The shard node is activated, and the shard node is used as an expansion space of the cluster to be expanded.

[0012] A device for expanding a cluster container, comprising:

[0013] An expansion request receiving module, used for receiving an expansion request for expanding a cluster, and judging whether the expansion request is a first expansion based on the expansion request, and obtaining a judgment result;

[0014] A domain name configuration module, configured to, if the judgment result is that the extension request is the first extension, perform domain name configuration on the cluster to be extended according to the domain name configuration information to obtain domain name information of the cluster to be extended;

[0015] A container deployment module, used to perform container deployment on the cluster to be expanded according to the domain name information of the cluster to be expanded and a preset container deployment file, and obtain node domain name information of the cluster to be expanded;

[0016] A shard node configuration module, configured to initialize nodes of the cluster to be expanded based on the node domain name information, obtain routing nodes of the cluster to be expanded, and configure shard nodes through the routing nodes;

[0017] The shard node activation module is used to activate the shard node and use the shard node as the expansion space of the cluster to be expanded.

[0018] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned cluster container expansion method when executing the computer program.

[0019] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned cluster container expansion method are implemented.

[0020] The method, device, computer equipment and storage medium for expanding the capacity of a cluster container provided by the present invention, when receiving an extension request for expanding a cluster, perform domain name configuration on the cluster to be expanded according to domain name configuration information, obtain domain name information of the cluster to be expanded, obtain node domain name information of the cluster to be expanded based on the domain name information of the cluster to be expanded and the container deployment file, initialize the nodes in the cluster to be expanded according to the node domain name information, obtain routing nodes, wherein the routing nodes are used to configure shard nodes, wherein the routing nodes configure the current shard nodes to respond to the current extension request, and reserve memory in the container of the current shard node for the next container expansion; configure and activate the shard nodes through the routing nodes, and use the deployment of the shard nodes as a response to the current extension request. The shard nodes are deployed through the domain name configuration information and the container deployment file, simplifying the process of declaring the extended memory and improving the efficiency of expanding the storage space. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0022] Figure 1 It is a schematic diagram of an application environment of a cluster container expansion method in one embodiment of the present invention;

[0023] Figure 2 is a flow chart of a method for expanding capacity of a cluster container in one embodiment of the present invention;

[0024] Figure 3 is a flow chart of a method for expanding capacity of a cluster container in another embodiment of the present invention;

[0025] Figure 4 is a flow chart of a method for expanding capacity of a cluster container in another embodiment of the present invention;

[0026] Figure 5 is a flow chart of a method for expanding capacity of a cluster container in another embodiment of the present invention;

[0027] Figure 6 is a flow chart of a method for expanding capacity of a cluster container in another embodiment of the present invention;

[0028] Figure 7 is a flow chart of a method for expanding capacity of a cluster container in another embodiment of the present invention;

[0029] Figure 8 is a schematic structural diagram of a device for expanding a cluster container in one embodiment of the present invention;

[0030] Fig. 9 is a schematic diagram of a computer device in one embodiment of the present invention. DETAILED DESCRIPTION

[0031] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0032] The cluster container expansion method provided in the embodiment of the present invention can be applied in the following aspects: Figure 1 In the application environment, the embodiment of the present invention can be applied in the application container engine environment.

[0033] like Figure 1 As shown, the application scenario of the method for expanding the capacity of a cluster container provided by an embodiment of the present invention includes a cluster composed of multiple machines, where the first machine is expanded to obtain multiple machines, and the cluster is composed of multiple machines. Among them, the first machine includes an application layer, a driver layer, a routing node, a configuration node, and a sharding node. The second machine to the Nth machine include an application layer, a driver layer, a routing node, a configuration server, and a sharding node.

[0034] Among them, the machine refers to a server with storage capacity, a network server, a network device, etc. Further, the application layer is the seventh layer of the seven-layer OSI model of the network transmission protocol, also known as the application entity (AE), which mainly provides an interface between applications on any end of the network, such as remote access and management, email, virtual middle end, and directory services and other functions. It provides specific application services for application elements, such as file transport access and management (FTAM), electronic document processing (MHS), virtual terminal protocol (VAP), etc. At the same time, the application process provides common application services, such as contact control service element (ACSE), reliable transport service element (RTSE) and remote operation service element (ROSE).

[0035] In the application scenario of this embodiment, when the application needs to expand storage due to the increase of business data, the application layer sends an expansion request through the virtual terminal protocol and transmits it to the driver layer. The driver layer sends the expansion request to the routing node of the cluster, and the routing node forwards the expansion request accordingly. The first cluster refers to the cluster container of the embodiment of the present invention. After multiple expansions, the Nth cluster is obtained. In the first cluster, a configuration server is set. In clusters other than the first cluster, the configuration server includes a master configuration node, a replica configuration node, and an arbitration configuration node. The configuration server is used to store metadata and identity authentication configuration information of the sharding nodes. Among them, in the embodiment of the present invention, there are multiple clusters, routing nodes, configuration nodes, and sharding nodes.

[0036] The routing node is a query interface provided for clients and all shard nodes. It is the entry point for data set access requests. All requests are coordinated and processed through the routing node. It is a request distribution center responsible for forwarding corresponding data requests to the corresponding data nodes. Sharding nodes are nodes that store data, which are conducive to managing clusters with massive data sets and high-throughput operations. The shard nodes in the cluster are subsets of the data of the entire cluster. A shard node contains a master node, a replica node, and an arbitration node.

[0037] The master node is responsible for providing external read and write services. When the master node fails and becomes unavailable, the replica node will be voted to be elected as a new master node. The data of the replica node and the master node are completely synchronized. Furthermore, there can be multiple replica nodes. The arbitration node is mainly responsible for monitoring the status of other nodes and voting to elect the master node. It is generally not used to store data.

[0038] It should be noted that the cluster container expansion method provided in the embodiment of the present invention is executed by a server, and accordingly, the cluster container expansion device is disposed in the server.

[0039] In one embodiment, if Figure 2 As shown, a cluster container expansion method is provided, and the method is applied in Figure 1 The machine in is taken as an example to illustrate, including the following steps S101 to S105.

[0040] S101, receiving an expansion request for expanding a cluster, and based on the expansion request, determining whether the expansion request is a first expansion, and obtaining a determination result.

[0041] Specifically, the expansion request is a request initiated by the client application to expand the storage space when the client application finds that the current storage space is insufficient during data storage. Further, after receiving the expansion request, it is necessary to determine whether the currently initiated expansion request is the first request to request storage expansion.

[0042] Furthermore, the judgment can be made according to a preset judgment method, specifically: read the variable parameter for managing the number of extensions in the preset domain name configuration file and obtain the value of the variable parameter. Each time an extension is made, the value of the variable parameter will be added up, that is, the variable parameter is used to judge whether the current extension request is the first time. If the current value of the variable parameter is 0, it can be concluded that no extension was made before receiving this extension request, and it is determined that the current extension request is initiated for the first time.

[0043] S102: If the result of the determination is that the extension request is the first extension, a domain name configuration is performed on the cluster to be extended according to the domain name configuration information to obtain domain name information of the cluster to be extended.

[0044] Initial expansion refers to the first time an application expands the infrastructure, specifically storage space.

[0045] Specifically, the domain name configuration information includes the mapping relationship between the host and the domain name. In the application environment of the application container engine, when deploying or running a new container, it is necessary to configure the domain name for the new container according to the domain name configuration information to ensure that the server can access the container.

[0046] Furthermore, the domain name configuration information during the first extension is described using Table 1 as an example.

[0047] Table 1 Domain name information of the cluster during the first expansion

[0048]

[0049]

[0050] As shown in Table 1, when expanding for the first time, you need to configure the sharding nodes, routing nodes, and configuration servers. The routing nodes configure the master nodes, replica nodes, and arbitration nodes in the sharding nodes, such as the master nodes, replica nodes, and arbitration nodes of sharding node 1 in Table 1. After configuring the sharding node that needs to be expanded, you need to set the replica node of the next sharding node so that you can expand directly according to the replica node of the next sharding node when expanding next time. For example, sharding node 2 in Table 1.

[0051] In the process of configuring sharding nodes, routing nodes and configuration servers, a container name with a corresponding port number is assigned to each node so that each node can work. The domain name information and container name of the sharding nodes, routing nodes and configuration servers are shown in the "Domain Name: Port Number" column and the "Container Name" column in Table 1.

[0052] Specifically, the domain name configuration information can be an address format configuration file, which maps the domain name through the IP address and the custom domain name information, and writes the address format configuration file to the storage location of the address format configuration file to reduce the coupling between the cluster node and the IP address, which is reasonable for subsequent cluster expansion, migration and maintenance.

[0053] The domain name information of the cluster to be expanded is the domain name information of the cluster node to be expanded, and specifically records the domain name of each expanded cluster node, for example, cluster 1:192.168.1.1. cluster 1 is the name of the expanded cluster node, and 192.168.1.1 is the domain name of the cluster node.

[0054] Among them, the address format configuration file is a file responsible for the fast resolution of IP addresses and domain names in the Linux system, and is saved in the corresponding storage directory in ASCII format. Different Linux versions may have different files. The address format configuration file contains the mapping between IP addresses and host names, including the aliases of host names. In the absence of a domain name server, all network programs on the system resolve the IP address corresponding to a host name by querying this file, otherwise the DNS service program needs to be used to resolve it. Usually, the commonly used domain name and IP address mapping can be added to the address format configuration file for fast and convenient access.

[0055] S103: Perform container deployment on the cluster to be expanded according to the domain name information of the cluster to be expanded and the preset container deployment file, and obtain the node domain name information of the cluster to be expanded.

[0056] Specifically, in the application container engine environment, the container deployment file refers to a collection of instructions that can automatically deploy the application container engine, and can integrate the cumbersome operations in the application container engine into one command.

[0057] The node domain name information refers to the IP address information of each node that performs functions in the cluster, specifically the domain name information of each node and the corresponding port number. As an example, it can be node name-domain name:port number.

[0058] In this embodiment, the container deployment file defines the mirror addresses corresponding to different cluster nodes, file mapping, startup commands, network types and port numbers, and assigns different port numbers to services of different nodes to name the nodes and achieve unique identification in the same domain name address.

[0059] In a cluster node, there are also routing nodes and configuration nodes. Furthermore, in other extended cluster nodes, there are also master nodes, replica nodes and arbitration nodes. These nodes are assigned fixed port numbers according to the node domain name information.

[0060] The routing node provides a query interface for client applications and all sharded clusters. It is the entry point for data set storage requests. All requests are coordinated and processed through the routing node. It is a request distribution center responsible for forwarding corresponding data requests to the corresponding data nodes.

[0061] As an optional manner, in this embodiment, only three routing nodes are set in the cluster node, so that cluster resources can be managed more conveniently.

[0062] The configuration node is the configuration server of the cluster, which is used to store the metadata and authentication configuration information of the sharded cluster. The metadata contains the state and organization of all data and components in the sharded cluster, as well as the information and definition scopes on each sharded cluster. The routing node instance caches data and routes read and write operations to the correct shard. The configuration information is loaded through the configuration node when it is first started or shut down and restarted. After that, if the configuration information is updated, the routing node will update its own status.

[0063] Furthermore, sharding is a method of distributing data at multiple levels, which is conducive to managing clusters with massive data sets and high-throughput operations. A shard node includes a master node, a replica node, and an arbitration node, and there is only one master node, and only the master node provides read and write services to the outside world. When the master node fails and is unavailable, the replica node will be voted to be elected as a new master node. The data of the replica node and the master node are completely synchronized. The arbitration node is mainly responsible for monitoring the status of other nodes in the shard and voting to elect the master node. This node generally does not store data.

[0064] Specifically, the container deployment file may be a command used to automatically start the container in the application container engine environment, for example, the docker-compose.yml file in the Docker environment.

[0065] S104, based on the node domain name information, perform node initialization on the cluster to be expanded, obtain the routing node of the cluster to be expanded, and configure the sharding node through the routing node.

[0066] Specifically, according to the node domain name information, a node with a fixed form of domain name information is set as a routing node, and nodes with other fixed forms of domain name information are set as master configuration nodes and replica configuration nodes. In this embodiment, when the configuration server in the container is expanded for the first time, the node with a specific port number is set as the master configuration node. As the number of expansions increases, in the entire cluster, the node with another specific port number is set as a replica configuration node. Furthermore, an arbitration configuration node needs to be set. The main function of the master configuration node is to store the metadata of the sharded cluster and verify the configuration information. The replica configuration node stores the metadata of the replica set, etc., and the arbitration configuration node is used to configure the arbitration node.

[0067] After setting a node as a routing node, you need to add the routing node to the corresponding routing node container. After setting a node as a configuration node, you need to add the primary configuration node and the replica configuration node to the corresponding configuration server container.

[0068] Furthermore, the number of primary configuration nodes and replica configuration nodes needs to be determined according to the variable parameter of the number of expansions in the domain name configuration file.

[0069] As shown in Table 1, the domain name of the routing node may be cluster1:27017, and the domain name of the main configuration node may be cluster1:27011.

[0070] After setting up the routing node, configure the sharding node through the routing node. You need to set the master node, replica node, and arbitration node in the sharding node, as well as the replica node of the next sharding node.

[0071] When the next cluster expansion is needed, the replica node of the next shard and node configured during this expansion needs to be configured into the shard node of the next expansion, and the corresponding storage space and related storage data are reserved for the next expansion to improve the efficiency of cluster expansion. The replica node can be added to the shard node of the next expansion by configuring the domain name configuration information, which reduces the cumbersome process of manual setting and improves the efficiency of capacity expansion.

[0072] S105, activating the shard node, and using the shard node as the expansion space of the cluster to be expanded.

[0073] Specifically, the master node, the replica node, and the arbitration node are added to the shard container corresponding to the shard node, that is, the shard node is activated, and the shard node serves as the extended storage space of the cluster to be expanded.

[0074] In this embodiment, when an extension request for cluster extension is received, the domain name of the cluster to be extended is configured according to the domain name configuration information to obtain the domain name information of the cluster to be extended, and the node domain name information of the cluster to be extended is obtained based on the domain name information of the cluster to be extended and the container deployment file, and the nodes in the cluster to be extended are initialized according to the node domain name information to obtain the routing node, wherein the routing node is used to configure the sharding node, wherein the routing node configures the current sharding node to respond to the current extension request, and reserves memory in the container of the current sharding node for the next container extension; the sharding node is configured and activated by the routing node, and the deployment of the sharding node is used as a response to the current extension request. The sharding node is deployed through the domain name configuration information and the container deployment file, the process of declaring the extended memory is reduced, and the efficiency of expanding the storage space is improved.

[0075] In this embodiment, if Figure 3 As shown, as an optional implementation, the following is also provided:

[0076] S1: If the result of the judgment is that the extension request is not the first extension, the domain name configuration is performed on the cluster to be extended according to the domain name configuration information to obtain the extended domain name information.

[0077] Specifically, by reading the variable parameter for managing the number of extensions in the preset domain name configuration file and obtaining the value of the variable parameter, the value of the variable parameter will be added up each time an extension is made, that is, the variable parameter is used to determine whether the current extension request is the first time. If the current value of the variable parameter is not 0, it can be concluded that the current extension request is not initiated for the first time.

[0078] S2: Generate an extended container deployment file based on the extended domain name information, and generate extended node domain name information based on the extended container deployment file.

[0079] Specifically, the domain name configuration file is read, and the extended domain name information is obtained according to the domain name mapping relationship. The corresponding domain name and port number, image address, file mapping, startup command and network type in the extended domain name information are used to generate a container deployment file for generating the extended node domain name information.

[0080] Furthermore, the content of the domain name configuration information is as follows:

[0081] [First_IP] 192.168.1.1 192.168.1.2

[0084] [Expand_IP]

[0085] It can be concluded that [First_IP] refers to the domain name information of the shard node during the first expansion. According to the mapping relationship of the domain name, in this expansion, the domain name information of the newly added shard node should be added to the variable [Expand_IP], which can be:

[0086] [First_IP] 192.168.1.1 192.168.1.2

[0089] [Expand_IP]

[0090] Cluster3 192.168.1.3

[0091] Cluster4 192.168.1.4

[0092] The domain name information of the extended node needs to obtain the domain name information of the routing node and configuration node, as well as the master node, replica node, etc. Among them, the domain name information of routing node 3 is cluster3:27017, and the domain name information of routing node 4 is cluster4:27017.

[0093] S3, based on the domain name information of the extended node, perform node initialization on the cluster to be extended, and obtain the extended routing node and extended configuration server of the cluster to be extended, wherein the extended configuration server includes a replica configuration node and an arbitration configuration node.

[0094] The domain name information of the extended node needs to obtain the domain name information of the routing node and configuration node, as well as the master node, replica node, etc. Among them, the domain name information of routing node 3 is cluster3:27017, and the domain name information of routing node 4 is cluster4:27017.

[0095] S4, configure the extended sharding node by extending the routing node to obtain the master node and arbitration node of the extended sharding node.

[0096] Specifically, in this embodiment, Table 2 is taken as an example to illustrate the domain name configuration information of cluster deployment when it is not the first expansion.

[0097] Table 2 Cluster information table for non-first expansion deployment

[0098]

[0099] Among them, when the extension request is not the first extension, the domain name information of the current extension request is configured based on the domain name configuration information of the previous extension request, and the domain name configuration information of the current extension request is described according to Table 1 and Table 2.

[0100] Among them, the replica node of the shard node 3 in Table 1 is added to the shard node 3 as the replica node of the shard node 3 in Table 2, so that the shard node 3 completes the expansion.

[0101] In the current expansion request, as shown in Table 2, only shard node 3 and shard node 4 are expanded. The replica node of shard node 4 is pre-configured when shard node 3 is expanded. When shard node 4 is expanded, the pre-configured replica node of shard node 4 is added to shard node 4 to complete the expansion of shard node 4.

[0102] When configuring the master node, replica node, and arbitration node of shard node 4, pre-configure the replica node of shard node 5 so that when the next expansion request is made, the replica node of shard node 5 is added to shard node 5, so that shard node 5 completes the expansion.

[0103] Assign a corresponding domain name to each shard node, and add the domain name to the corresponding container through the container deployment file, so that the shard node is expanded. As shown in Table 1, the domain name of each node in shard node 4 is added to its corresponding container, so that shard node 4 is expanded.

[0104] Specifically, after obtaining the routing node of the extended cluster, the sharding configuration is performed through the routing node of the extended cluster. Routing node 3 sets the master node, replica node and arbitration node of chluster 3 through the port number of each node, and routing node 4 sets the master node, replica node and arbitration node of chluster 4 through the port number of each node.

[0105] S5: Add the replica node in the previous shard node to the extended shard node, activate the extended shard node, and use the extended shard node as the expansion space of the cluster to be extended.

[0106] Specifically, when each expansion node is configured, the domain name information of the replica node of the next shard node will be additionally set. When the next expansion is performed, the replica node will be added to the shard node of the next expansion.

[0107] Furthermore, the domain name information of the replica node of routing node 3 is cluster2:20002, and the domain name information of the replica node of routing node 4 is cluster3:20002.

[0108] In this embodiment, when the extension request is not the first extension, the domain name configuration information of the extension node is obtained according to the domain name configuration information, and then the node initialization and shard configuration are performed according to the corresponding node domain name information, without human intervention, which reduces the process occupancy of the extension cluster and improves the efficiency of the extension. In addition, by reserving a replica node of the previous shard node and adding the replica node to the next shard node, after retaining the architecture of the shard node, the storage container of the next shard node is pre-started, which saves the process time of starting the container when the extension is performed, and further speeds up the time of storing incremental data.

[0109] In this embodiment, if Figure 4 As shown, as a preferred implementation, in step S102, after determining that the extension request is the first extension, and before the step of performing domain name configuration on the cluster to be extended according to the domain name configuration information to obtain the domain name information of the cluster to be extended, the following steps S10 to S11 are also included, which are described in detail as follows:

[0110] S10, generating a routing image file for a routing node service, generating a configuration image file for a configuration node service, and generating a shard image file for a shard data node service.

[0111] Specifically, the routing node service refers to the service of providing routing distribution requests, and the configuration node service refers to the service of providing node configuration requests.

[0112] A mirror is a special file system, an object-oriented class, equivalent to a template, which does not contain any dynamic data and its content will not be changed after it is built.

[0113] S11, uploading and saving the routing image file, the configuration image file and the fragment image file to a preset image file storage space.

[0114] Specifically, in a Docker environment, when expanding, it is necessary to generate image files for the routing node service and the configuration node service, and upload the image files to a specific image repository for storage.

[0115] In this embodiment, the routing node service and the configuration node service generate image files and upload them to the image warehouse for storage. In the application scenario of this embodiment, the data in the application is made into an image file. When the container needs to be expanded and started, the data in the image file needs to be pulled from the image warehouse, ensuring that the subsequent processes of cluster expansion can be implemented smoothly.

[0116] In this embodiment, if Figure 5 As shown, as a preferred implementation, in step S102, the step of configuring the domain name of the cluster to be deployed according to the domain name configuration information to obtain the domain name of the cluster to be expanded includes:

[0117] S1021, read the domain name control variable in the domain name configuration information, and obtain the domain name information corresponding to the domain name control variable.

[0118] Specifically, the control variable read includes the number of current cluster expansions and the domain name of the cluster, and the domain name information of the current cluster node to be expanded is obtained according to the domain name mapping condition.

[0119] S1022: Determine the domain name information of the cluster to be expanded based on the domain name information and the address mapping method.

[0120] Further, the domain name information of the cluster to be expanded is obtained according to the domain name information and the mapping method. Specifically, in the domain name configuration information, the domain name of the previous shard node is 192.168.1.2, and the domain name of the current shard node can be obtained as 192.168.1.3 according to the one-by-one incremental mapping method.

[0121] In this embodiment, domain name information is automatically allocated to the cluster to be expanded through domain name information and domain name mapping method, ensuring that the nodes in the cluster to be expanded are allocated to corresponding port numbers, facilitating sharding configuration, speeding up the process of starting the container of the sharding node, and further improving the expansion efficiency of the container.

[0122] In this embodiment, if Figure 6 As shown, as an optional implementation method, in step S104, based on the node domain name information, the node of the cluster to be expanded is initialized, the routing node of the cluster to be expanded is obtained, and the specific implementation process of the step of configuring the shard node through the routing node is as follows:

[0123] S1041, based on the port number setting information in the node domain name information, set the port number for each node, wherein the port number setting information includes a routing node port number set and a configuration server port number set.

[0124] When the domain name configuration is performed on the node through the domain name configuration information, the port number setting information in the domain name configuration information can be read, and the port number setting information includes the mapping between the port number and the node function. Specifically, it includes a routing node port number set and a configuration server port number set, wherein the routing node port number set specifies that a certain port number or a sequence of port numbers can perform the function of the routing node, and the configuration server port number set specifies that a certain port number or a sequence of port numbers can perform the function of the configuration server, and the configuration server includes a primary configuration node, a replica configuration node, and an arbitration configuration node.

[0125] S1042: Set a node whose port number belongs to a routing node port number set as a routing node, and set a node whose port number belongs to a configuration server port number set as a configuration server, wherein the configuration server includes a master configuration node.

[0126] Specifically, a port number in the routing node port number set is allocated to a certain node, that is, the node is set as a routing node, and a port number in the configuration server port number set is allocated to another node, that is, the node is set as a configuration server node.

[0127] In a specific implementation, the port number of the routing node is 27017, the port number of the primary configuration node is 27011, the port number of the replica configuration node is 27012, and the port number of the arbitration configuration node is 27013.

[0128] S1043, configuring shard nodes through routing nodes, wherein the shard nodes include the master node, the current replica node and the arbitration node of the current shard node, and the replica node of the next shard node.

[0129] Specifically, the domain name information of the current shard node is cluster2:192.168.1.2, and the domain name information of the next shard node is cluster3:192.168.1.3. The routing node needs to configure a master node, replica node, arbitration node for cluster2, and a replica node for cluster3.

[0130] After the configuration is complete, the domain name of cluster3's master node is cluster3:20001, the replica node is cluster2:20002, and the arbitration node is cluster3:20003. cluster2:20002 is the current replica node, and cluster4:20002 is the replica node of the next shard node.

[0131] The replica nodes are synchronized with the data stored in the primary node.

[0132] Furthermore, after the routing node is obtained according to the setting in the previous step, the routing node configures the sharding node, specifically setting the node with a fixed port number as the master node, the replica node, and the arbitration node.

[0133] Specifically, the node with the port number 20001 is set as the master node, the node with the port number 20002 is set as the replica node, and the node with the port number 20003 is set as the arbitration node.

[0134] In this embodiment, node configuration is performed through port number setting information, and the architecture of the shard node is deployed according to the assigned port number without the need for human intervention, which further speeds up the efficiency of configuring the shard node, thereby improving the steps for capacity expansion.

[0135] Further, such as Figure 7 As shown, as an optional method, in step S104, the sharding node includes multiple basic nodes, each basic node is distinguished according to the port number, and the step of configuring the sharding node through the routing node includes:

[0136] S40, generating a priority corresponding to each node according to a preset priority setting method.

[0137] Specifically, the priority setting method includes setting the port number within a certain sequence range as the first priority and the port number within another sequence range as the second priority in the port number setting information. The priority level can be preset, specifically the first priority is 10 and the second priority is 5. The priority setting can be based on the port number of the node or the node memory, etc., which is not specifically limited here.

[0138] S41, sorting the priorities, and based on the obtained sorting result, configuring the node with the first priority as the master node of the current shard node, and configuring the node with the second priority as the replica node of the current shard node.

[0139] Specifically, according to the set priorities, the nodes with the first priority are set as the master nodes, and the nodes with the second priority are set as the replica nodes.

[0140] In this embodiment, the master node, replica node and even arbitration node are allocated by setting priorities. During this process, no human intervention is required, that is, the architecture of the shard node can be deployed according to the preset settings, thereby improving the efficiency of deploying the shard node.

[0141] Further, as an optional method, in step S1043, the step of configuring the sharding node through the routing node includes:

[0142] Configure the current shard node, add the master node, the current replica node, and the arbitration node to the current shard node, so that the current shard node takes effect.

[0143] Specifically, the master node, the current replica node, and the arbitration node are added to the container of the current shard node, so that the current shard node works, thereby achieving the purpose of expanding the storage space of the cluster.

[0144] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0145] In one embodiment, a device for expanding a cluster container is provided, and the device for expanding a cluster container corresponds one-to-one to the method for expanding a cluster container in the above embodiment. Figure 8 As shown, the capacity expansion device of the cluster container includes the following modules:

[0146] An expansion request receiving module 91 is used to receive an expansion request for expanding a cluster, and based on the expansion request, determine whether the expansion request is a first expansion, and obtain a determination result;

[0147] A domain name configuration module 92 is used to configure the domain name of the cluster to be expanded according to the domain name configuration information to obtain the domain name information of the cluster to be expanded if the result of the judgment is that the expansion request is the first expansion;

[0148] The container deployment module 93 is used to perform container deployment on the cluster to be expanded according to the domain name information of the cluster to be expanded and a preset container deployment file, and obtain the node domain name information of the cluster to be expanded;

[0149] The shard node configuration module 94 is used to initialize the nodes of the cluster to be expanded based on the node domain name information, obtain the routing nodes of the cluster to be expanded, and configure the shard nodes through the routing nodes;

[0150] The shard node activation module 95 is used to activate the shard nodes and use the shard nodes as expansion space of the cluster to be expanded.

[0151] In this embodiment, the device for expanding the capacity of the cluster container further includes the following modules:

[0152] The extended domain name configuration module is used to perform domain name configuration on the cluster to be extended according to the domain name configuration information to obtain extended domain name information if the judgment result is that the extension request is not the first extension.

[0153] The extended node configuration module generates an extended container deployment file based on the extended domain name information, and generates extended node domain name information based on the extended container deployment file.

[0154] The extended node initialization module is used to perform node initialization on the cluster to be extended based on the extended node domain name information, and obtain the extended routing node of the cluster to be extended.

[0155] The extended sharding node configuration module is used to configure the extended sharding node by extending the routing node to obtain the master node and arbitration node of the extended sharding node.

[0156] The extended shard node activation module is used to add the replica node in the previous shard node to the extended shard node, activate the extended shard node, and use the extended shard node as the expansion space of the cluster to be extended.

[0157] In this embodiment, the domain name configuration module also includes the following units:

[0158] The image file generation unit is used to generate a routing image file for a routing node service, generate a configuration image file for a configuration node service, and generate a shard image file for a shard data node service.

[0159] The image file storage unit is used to upload and save the routing image file, the configuration image file and the fragment image file to the preset image file storage space.

[0160] In this embodiment, the domain name configuration module also includes the following units:

[0161] The domain name information generating unit is used to read the domain name control variable in the domain name configuration information to obtain the domain name information corresponding to the domain name control variable.

[0162] The domain name information determining unit is used to determine the domain name information of the cluster to be expanded based on the domain name information and the address mapping method.

[0163] In this embodiment, the shard node configuration module includes the following units:

[0164] The port number setting unit is used to set the port number for each node based on the port number setting information in the node domain name information, wherein the port number setting information includes a routing node port number set and a configuration server port number set.

[0165] The node configuration unit is used to set a node whose port number belongs to a routing node port number set as a routing node, and set a node whose port number belongs to a configuration server port number set as a configuration server, wherein the configuration server includes a main configuration node.

[0166] The shard node configuration unit is used to configure the shard nodes through the routing nodes, wherein the shard nodes include the primary node, the current replica node and the arbitration node of the current shard node, and the replica node of the next shard node.

[0167] Furthermore, the shard node configuration unit includes the following subunits:

[0168] The priority setting subunit is used to generate the priority corresponding to each node according to a preset priority setting method.

[0169] The priority sorting subunit is used to sort the priorities, and based on the obtained sorting results, configure the node with the first priority as the master node of the current shard node, and configure the node with the second priority as the replica node of the current shard node.

[0170] The node configuration subunit is used to configure the current shard node, add the master node, the current replica node, and the arbitration node to the current shard node, so that the current shard node takes effect.

[0171] The meaning of "first" and "second" in the above modules / units is only to distinguish different modules / units, and is not used to define which module / unit has a higher priority or other limiting meanings. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules clearly listed, but may include other steps or modules that are not clearly listed or inherent to these processes, methods, products or devices. The division of modules in this application is only a logical division, and there may be other division methods when implemented in actual applications.

[0172] For the specific definition of the expansion device of the cluster container, please refer to the definition of the expansion method of the cluster container above, which will not be repeated here. Each module in the above-mentioned expansion device of the cluster container can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0173] In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Fig. 9 As shown. The computer device includes a processor, a memory, a network interface and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data involved in the method for expanding the capacity of a cluster container. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a method for expanding the capacity of a cluster container is implemented.

[0174] In one embodiment, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the cluster container expansion method in the above embodiment are implemented, for example: Figure 2 Alternatively, when the processor executes the computer program, the functions of each module / unit of the device for expanding the capacity of the cluster container in the above embodiment are realized, for example Figure 8 The functions of modules 91 to 95 are shown in Figure 9. To avoid repetition, they will not be described here.

[0175] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the computer device, and various interfaces and lines are used to connect various parts of the entire computer device.

[0176] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the computer device by running or executing the computer program and / or module stored in the memory and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, video data, etc.), etc.

[0177] The memory may be integrated into the processor or may be arranged separately from the processor.

[0178] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for expanding the capacity of the cluster container in the above embodiment are implemented, such as Figure 2 Steps S101 to S105 and other extensions of the method and related steps are shown. Alternatively, when the computer program is executed by the processor, the functions of each module / unit of the capacity expansion device of the cluster container in the above embodiment are realized, for example Figure 8 The functions of modules 91 to 95 are shown in Figure 9. To avoid repetition, they will not be described here.

[0179] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0180] Those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above.

[0181] The embodiments described above are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. Such modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention.

Claims

1. A method for expanding the capacity of a cluster container, characterized in that: include: receiving an expansion request for expanding a cluster, and determining, based on the expansion request, whether the expansion request is a first expansion, and obtaining a determination result; If the judgment result is that the extension request is the first extension, performing domain name configuration on the cluster to be extended according to the domain name configuration information to obtain domain name information of the cluster to be extended; According to the domain name information of the cluster to be expanded and the preset container deployment file, container deployment is performed on the cluster to be expanded to obtain node domain name information of the cluster to be expanded; Based on the port number setting information in the node domain name information, set a port number for each node, wherein the port number setting information includes a routing node port number set and a configuration server port number set; Set a node whose port number belongs to the routing node port number set as a routing node, and set a node whose port number belongs to the configuration server port number set as a configuration server, wherein the configuration server includes a master configuration node; Configure shard nodes through the routing nodes, wherein the shard nodes include a master node, a current replica node, and an arbitration node of a current shard node, and a replica node of a next shard node; The sharding node includes a plurality of basic nodes, and the basic nodes are distinguished according to port numbers; The step of configuring the sharding node through the routing node includes: According to a preset priority setting method, a priority corresponding to each of the nodes is generated; and the master node, the replica node, and even the arbitration node are allocated by setting the priority; The priorities are sorted, and based on the obtained sorting result, the node with the first priority is configured as the master node of the current shard node, and the node with the second priority is configured as the replica node of the current shard node; The shard node is activated, and the shard node is used as an expansion space of the cluster to be expanded.

2. The cluster container expansion method according to claim 1, characterized in that: If the judgment result is that the extension request is the first extension, and before the step of configuring the domain name of the cluster to be extended according to the domain name configuration information to obtain the domain name information of the cluster to be extended, the method further includes: Generates the routing image file of the routing node service, generates the configuration image file of the configuration node service, and generates the shard image file of the shard data node service; The routing image file, the configuration image file and the fragment image file are uploaded and saved to a preset image file storage space.

3. The cluster container expansion method according to claim 1, characterized in that: The step of configuring the domain name of the cluster to be deployed according to the domain name configuration information to obtain the domain name information of the cluster to be expanded includes: Read the domain name control variable in the domain name configuration information to obtain the domain name information corresponding to the domain name control variable; Based on the domain name information and the address mapping method, the domain name information of the cluster to be expanded is determined.

4. The cluster container expansion method according to claim 1, characterized in that: The step of configuring the sharding node through the routing node includes: Perform node configuration on the current shard node, add the master node, the current replica node, and the arbitration node to the current shard node, so that the current shard node takes effect.

5. The cluster container expansion method according to claim 1, characterized in that: After the step of determining whether the extension request is a first extension based on the extension request and obtaining a determination result, the method further includes: If the judgment result is that the extension request is not the first extension, performing domain name configuration on the cluster to be extended according to the domain name configuration information to obtain extended domain name information; Generate an extended container deployment file based on the extended domain name information, and generate extended node domain name information based on the extended container deployment file; Based on the extended node domain name information, the node of the cluster to be extended is initialized to obtain an extended routing node of the cluster to be extended; Configure the extended sharding node through the extended routing node to obtain the master node and arbitration node of the extended sharding node; The replica node in the previous shard node is added to the extended shard node, and the extended shard node is activated, and the extended shard node is used as the expansion space of the cluster to be extended.

6. A device for expanding the capacity of a cluster container, characterized in that: The device comprises: An expansion request receiving module, used for receiving an expansion request for expanding a cluster, and judging whether the expansion request is a first expansion based on the expansion request, and obtaining a judgment result; A domain name configuration module, configured to, if the judgment result is that the extension request is the first extension, perform domain name configuration on the cluster to be extended according to the domain name configuration information to obtain domain name information of the cluster to be extended; A container deployment module, used to perform container deployment on the cluster to be expanded according to the domain name information of the cluster to be expanded and a preset container deployment file, and obtain node domain name information of the cluster to be expanded; A sharding node configuration module, configured to set a port number for each node based on the port number setting information in the node domain name information, wherein the port number setting information includes a routing node port number set and a configuration server port number set; Set a node whose port number belongs to the routing node port number set as a routing node, and set a node whose port number belongs to the configuration server port number set as a configuration server, wherein the configuration server includes a master configuration node; Configure shard nodes through the routing nodes, wherein the shard nodes include a master node, a current replica node, and an arbitration node of a current shard node, and a replica node of a next shard node; The sharding node includes a plurality of basic nodes, and the basic nodes are distinguished according to port numbers; The step of configuring the sharding node through the routing node includes: According to a preset priority setting method, a priority corresponding to each of the nodes is generated; and the master node, the replica node, and even the arbitration node are allocated by setting the priority; The priorities are sorted, and based on the obtained sorting result, the node with the first priority is configured as the master node of the current shard node, and the node with the second priority is configured as the replica node of the current shard node; The shard node activation module is used to activate the shard node and use the shard node as the expansion space of the cluster to be expanded.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the cluster container expansion method according to any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the cluster container expansion method according to any one of claims 1 to 5 are implemented.

Images