Cloud native database, database expansion method, database reduction method and device

Abstract

The present invention discloses a cloud native database, a database expansion method, a database reduction method and a device, and relates to the field of computer technology. A specific implementation of the method includes: configuration information and a state management directory are stored in the management system, the configuration information includes: topology information of each working node and storage information of the shared storage, the state management directory includes: node status of each available working node; multiple computing nodes are used to process database-related calculations; each computing node has a local storage of a configuration file corresponding to the computing node; each computing node has a local storage of cluster status information corresponding to the computing node; the shared storage is used to store database data. This implementation can achieve flexible management of cloud native databases.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a cloud native database, a database expansion method, a database reduction method and a device. Background Art

[0002] Using distributed file storage technology, cloud-native databases can easily store PB or even EB-level data, respond quickly in seconds or even sub-seconds, and ensure reliable data replication, backup, and fast retrieval even in the event of an outage or failure. However, the cloud-native database solutions in existing technologies cannot achieve flexible management of databases. Summary of the invention

[0003] In view of this, an embodiment of the present invention provides a cloud native database, a database expansion method, a database reduction method and a device, which can realize flexible management of the cloud native database.

[0004] In a first aspect, an embodiment of the present invention provides a cloud native database, including:

[0005] Manage systems, multiple computing nodes, and shared storage;

[0006] The management system stores configuration information and a state management directory, wherein the configuration information includes: topology information of each of the working nodes, and the state management directory includes: node status of each available working node;

[0007] The computing node is used to process database-related calculations; the computing node locally stores a configuration file corresponding to the computing node, the configuration file is generated according to the configuration information in the management system, and the management system manages the computing node using the configuration file;

[0008] The computing node locally stores cluster status information corresponding to the computing node, the cluster status information is generated according to the status management directory in the management system, and the computing node uses the cluster status information to provide cluster task services;

[0009] The shared storage is used to store database data.

[0010] Optionally, the management system is used to: obtain the node status of each of the working nodes; update the status management directory according to the node information of the working nodes;

[0011] The working node is used to obtain the state management directory and update local cluster state information using the state management directory.

[0012] In a second aspect, an embodiment of the present invention provides a method for expanding database capacity, which is applied to the above-mentioned cloud native database, and the method includes:

[0013] Receiving a capacity expansion instruction for the native database;

[0014] In response to the capacity expansion instruction, the management system creates a node resource for the capacity expansion node;

[0015] The management system generates configuration information; wherein the configuration information includes: topology information of the expansion node and topology information of each of the computing nodes;

[0016] The expansion node obtains the configuration information, and uses the configuration information to generate a configuration file corresponding to the expansion node; wherein the management system uses the configuration file corresponding to the expansion node to manage the expansion node;

[0017] When the expansion node is in an available state, the management system obtains the node state of the expansion node and updates the state management directory;

[0018] The expansion node obtains the state management directory, and uses the state management directory to generate cluster state information corresponding to the expansion node; wherein the expansion node uses the cluster state information corresponding to the expansion node to provide cluster task services.

[0019] Optionally, after the management system generates the configuration information, the management system further includes:

[0020] The computing node obtains the configuration information, and generates a configuration file corresponding to the computing node using the configuration information;

[0021] The management system obtains the node status of the expansion node and updates the status management directory, further comprising:

[0022] The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services.

[0023] Optionally, after receiving the capacity expansion instruction for the native database, the method further includes:

[0024] In response to the capacity expansion instruction, the management system creates storage resources for the target storage;

[0025] loading database data on the target storage;

[0026] Performing alignment processing on the database data on the target storage;

[0027] The management system generates configuration information; wherein the configuration information includes storage information of the target storage.

[0028] In a third aspect, an embodiment of the present invention provides a method for scaling down a database, which is applied to the above-mentioned cloud native database, and the method includes:

[0029] Receiving a scaling-down instruction for the native database;

[0030] In response to the shrinking instruction, shutting down the removed node;

[0031] The management system deletes the topology information of the removed node from the configuration information;

[0032] The management system deletes the node information of the removed node from the state management directory.

[0033] Optionally, after the management system deletes the topology information of the removed node from the configuration information, the management system further includes:

[0034] The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services;

[0035] After the management system deletes the node information of the removed node from the state management directory, the management system further includes:

[0036] The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services.

[0037] Optionally, the receiving a scaling-down instruction for the native database further includes:

[0038] In response to the shrink instruction, determining a target storage corresponding to the shrink instruction;

[0039] The management system deletes the storage information of the target storage from the configuration information.

[0040] In a fourth aspect, an embodiment of the present invention provides a database expansion device, which is applied to the above-mentioned cloud native database, and the device includes:

[0041] An instruction receiving module, used for receiving an expansion instruction for the native database;

[0042] A resource creation module, configured to cause the management system to create node resources for the expanded node in response to the expansion instruction;

[0043] A first generating module, used for the management system to generate configuration information; wherein the configuration information includes: topology information of the expansion node and topology information of each of the computing nodes;

[0044] A second generating module is used for the expansion node to obtain the configuration information, and generate a configuration file corresponding to the expansion node using the configuration information; wherein the management system manages the expansion node using the configuration file corresponding to the expansion node;

[0045] A directory updating module, configured to enable the management system to obtain the node status of the expansion node and update the status management directory when the expansion node is in an available state;

[0046] The third generation module is used for the expansion node to obtain the state management directory and use the state management directory to generate cluster state information corresponding to the expansion node; wherein the expansion node uses the cluster state information corresponding to the expansion node to provide cluster task services.

[0047] In a fifth aspect, an embodiment of the present invention provides a database shrinking device, which is applied to the above-mentioned cloud native database, including:

[0048] An instruction receiving module, used for receiving a shrinking instruction for the native database;

[0049] A shutdown module, configured to shut down the removed node in response to the shrinking instruction;

[0050] A first deleting module, configured for the management system to delete the topology information of the removed node from the configuration information;

[0051] The second deleting module is used for the management system to delete the node information of the removed node from the state management directory.

[0052] In a sixth aspect, an embodiment of the present invention provides an electronic device, including:

[0053] one or more processors;

[0054] a storage device for storing one or more programs,

[0055] When the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any of the above embodiments.

[0056] In a seventh aspect, an embodiment of the present invention provides a computer-readable medium having a computer program stored thereon, and when the program is executed by a processor, the method described in any of the above embodiments is implemented.

[0057] An embodiment of the above invention has the following advantages or beneficial effects: using the configuration information in the management system, a configuration file is generated locally on each computing node, and the configuration file includes the topology information of each working node. The management system manages each computing node through the configuration file. Using the state management directory in the management system, cluster state information is generated locally on each computing node, and the cluster state information only includes computing nodes in an available state. Each computing node provides cluster services through the cluster state information. Therefore, the management system can achieve flexible management of cloud native databases using the configuration information and state management directory stored in the management system.

[0058] The further effects of the above-mentioned non-conventional optional manner will be described below in conjunction with the specific implementation manner. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The accompanying drawings are used to better understand the present invention and do not constitute an improper limitation of the present invention.

[0060] Figure 1 is a schematic diagram of the structure of a database system provided by an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of the structure of another database system provided by an embodiment of the present invention;

[0062] Figure 3 It is a schematic diagram of a process of a database expansion method provided by an embodiment of the present invention;

[0063] Figure 4 It is a schematic diagram of a process of a database shrinking method provided by an embodiment of the present invention;

[0064] Figure 5 It is a schematic diagram of a principle of scaling up and down a cloud native database provided by an embodiment of the present invention;

[0065] Figure 6 It is a schematic diagram of a process of scaling up and down a database provided by an embodiment of the present invention;

[0066] Figure 7 It is a structural schematic diagram of a database expansion device provided by an embodiment of the present invention;

[0067] Figure 8 It is a structural schematic diagram of a database shrinking device provided by an embodiment of the present invention;

[0068] Fig. 9 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0070] Cloud-native database technology uses MPP (massively parallel processing) to process storage structures for multiple operations simultaneously. This not only accelerates the expansion and contraction of storage and computing resources, but also realizes better compression and query characteristics brought by storage in data column format. In addition, compared with data warehouses deployed locally, cloud solutions are more scalable in business intelligence, can accelerate analytical operations, quickly go online, and provide data integration, observability, and the entire ecosystem. It supports on-demand access and elastic expansion features, effectively improving the expansion efficiency of data warehouses, reducing expansion costs and operation and maintenance difficulties. It is widely used in scenarios such as real-time BI dashboards, real-time user portraits, real-time data analysis, and real-time monitoring. The existing technology lacks solutions that can flexibly control cloud-native databases, and there is no elastic expansion and contraction solution for cloud-native databases.

[0071] Based on this, the present invention provides a cloud native database that can flexibly manage and control the cloud native database. Figure 1 FIG. 1 is a schematic diagram of the structure of a database system provided by an embodiment of the present invention. Figure 1 As shown, the cloud native database includes: a management system, multiple computing nodes and shared storage. The management system stores configuration information and a state management directory, the configuration information includes: topology information of each working node and storage information of the shared storage, and the state management directory includes: node status of each available working node;

[0072] The computing node is used to process database-related calculations; the computing node locally stores a configuration file corresponding to the computing node, the configuration file is generated according to the configuration information in the management system, and the management system manages the computing node using the configuration file;

[0073] The local storage of the computing node contains cluster status information corresponding to the computing node, the cluster status information is generated according to the status management directory in the management system, and the computing node uses the cluster status information to provide cluster task services. The shared storage is used to store database data.

[0074] Cloud native databases use a storage and computing separation architecture. Compute nodes are used to process database-related calculations, and shared storage is used to store database data. Storage and computing resources can be flexibly and quickly expanded and reduced as needed.

[0075] The management system is used to manage cloud native databases, such as: allocating and managing computing node resources, allocating and managing shared storage resources, managing cluster task services, load balancing, fault tolerance, etc. You can develop your own management system based on your needs. The management system can also be built using an existing platform. For example, you can combine Kubernetes and Zookeeper to build a management system.

[0076] Each computing node generates a configuration file of the computing node locally using the configuration information in the management system. The configuration file includes the topology information of each working node. The topology information may include: node identification, host / host name, network address, switch port number, number of computing nodes, upper node identification, lower node identification, and computing node topology map. The management system uses the configuration file to manage the computing nodes, including: starting up the computing nodes, shutting down the computing nodes, restarting the computing nodes, assigning tasks to the computing nodes, assigning network resources to the computing nodes, etc.

[0077] In an embodiment of the present invention, each node generates a configuration file using the configuration information on the management system. The management system manages the life cycle of each node using the configuration file of each node. The configuration file includes all computing nodes. Each node generates cluster status information using the status management directory on the management system. The computing node uses the cluster status information to provide cluster services to the outside. The cluster status information only includes nodes in an available state. The management system can achieve flexible management and control of the database through the configuration information and status management directory stored thereon.

[0078] In one embodiment of the present invention, the management system is used to: obtain the node status of each of the working nodes; update the status management directory based on the node information of the working nodes; the working nodes are used to: obtain the status management directory, and use the status management directory to update the locally stored cluster status information.

[0079] The cluster status information only includes nodes in the available state. Therefore, the management system continuously obtains the node information of all nodes and continuously updates the status management directory. Specifically, if the management system finds that a previously unavailable working node becomes an available working node, the working node is added to the status management directory. If the management system finds that a previously available working node becomes an unavailable working node, the working node is deleted from the status management directory.

[0080] Figure 2 FIG. 1 is a schematic diagram of the structure of another database system provided by an embodiment of the present invention. Figure 2 As shown in the figure, the computing nodes are separated from the shared storage, and the computing nodes can be expanded and shrunk horizontally in seconds. Due to the use of shared storage, data does not need to be redistributed, and the storage nodes, i.e. the distributed file storage system, can be expanded infinitely. In this database system, Kubernetes and Zookeeper are combined to build a management system. All components are deployed on Kubernetes, and the life cycle of the data warehouse is managed by the operator, including elastic expansion and contraction, failover, etc. Zookeeper is used as a coordination service node to provide distributed coordination task services.

[0081] Figure 3 FIG. 1 is a schematic diagram of a process of a database expansion method provided by an embodiment of the present invention. Figure 3 As shown, the method includes:

[0082] Step 301: Receive a capacity expansion instruction for a native database.

[0083] Step 302: In response to the capacity expansion instruction, the management system creates node resources for the capacity expansion node.

[0084] The expansion node is a newly added computing node. Creating node resources for the expansion node may include: determining a node identifier for the expansion node, configuring a domain name for the expansion node, allocating a container for the expansion node, and determining a container role.

[0085] Step 303: The management system generates configuration information; wherein the configuration information includes: topology information of the expansion node and topology information of each computing node.

[0086] Add the topology information of the expansion information to the configuration information in the management system.

[0087] Step 304: the expansion node obtains configuration information, and uses the configuration information to generate a configuration file corresponding to the expansion node; wherein the management system uses the configuration file corresponding to the expansion node to manage the expansion node.

[0088] The expansion node uses the configuration information in the management system to generate a configuration file corresponding to the expansion node locally. The management system uses the configuration file to manage the expansion node, including: starting up the expansion node, shutting down the expansion node, restarting the expansion node, assigning tasks to the expansion node, and allocating network resources to the expansion node.

[0089] Step 305: When the expanded node is in an available state, the management system obtains the node state of the expanded node and updates the state management directory.

[0090] Step 306: The expansion node obtains the state management directory, and uses the state management directory to generate cluster state information corresponding to the expansion node.

[0091] At this point, the process of adding the expanded node as a new computing node is completed. The expanded node and the original computing nodes in the cluster provide cluster services as a cluster as a whole.

[0092] In one embodiment of the present invention, after the management system generates configuration information, it also includes: the computing node obtains the configuration information, and uses the configuration information to generate a configuration file corresponding to the computing node; after the management system obtains the node status of the expanded node and updates the status management directory, it also includes: the computing node obtains the status management directory, and uses the status management directory to update the cluster status information corresponding to the computing node; wherein the computing node uses the cluster status information corresponding to the computing node to provide cluster task services.

[0093] The above method is the processing flow of the original computing nodes in the cluster when adding new computing nodes. The original computing nodes continuously use the configuration information and status management directory in the management system to update the local configuration files and cluster status information of the original computing nodes. On the one hand, it ensures the accuracy of the local information of the computing nodes, making it easier for the management system to manage each calculation. On the other hand, it also ensures that the newly added expansion nodes quickly join the cluster, provide distributed services for the database, and improve the performance of the database.

[0094] In one embodiment of the present invention, after receiving the expansion instruction for the native database, it also includes: in response to the expansion instruction, the management system creates storage resources for the target storage; loads the database data on the target storage; aligns the database data on the target storage; the management system generates configuration information; wherein the configuration information includes the storage information of the target storage. Through the method of the embodiment of the present invention, a shared storage can be quickly added to the database, and the shared storage is used to store the database data. By aligning the database data on the target storage, the query performance of the database can be further improved.

[0095] The cloud-native database adopts a distributed query design, which can distribute queries to multiple computing nodes at the same time, greatly improving the query performance. The query performance is positively correlated with the number of computing nodes, but traditional databases cannot scale up and down online in real time, and the data cannot be automatically balanced after horizontal expansion. The cloud-native real-time data warehouse uses shared storage, separates computing and storage, and supports elastic scaling of clusters.

[0096] Figure 4 FIG. 1 is a schematic diagram of a process of a database shrinking method provided by an embodiment of the present invention. Figure 4 As shown, the method includes:

[0097] Step 401: Receive a scaling-down instruction for a cloud-native database.

[0098] Step 402: In response to the shrink instruction, shut down the removed node.

[0099] The removed node is a working node in the cloud native database. The shrink command is used to remove the removed node from the cloud native database, thereby reducing the data size of the working node of the cloud native database.

[0100] Step 403: The management system deletes the topology information of the removed node from the configuration information.

[0101] Step 404: The management system deletes the node information of the removed node from the state management directory.

[0102] The management system deletes the information about the removed node from the configuration information and status management directory. The management system no longer manages the removed node, and the removed node no longer provides services for the cloud native database. At this point, the cloud native database no longer contains the removed node, and the process of removing the removed node from the cloud native database is complete.

[0103] In one embodiment of the present invention, after the management system deletes the topology information of the removed node from the configuration information, it also includes: the computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services; after the management system deletes the node information of the removed node from the state management directory, it also includes: the computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services.

[0104] The above method is the processing flow of the original computing nodes in the cluster when removing computing nodes. The original computing nodes continuously use the configuration information and status management directory in the management system to update the local configuration files and cluster status information of the original computing nodes. On the one hand, the accuracy of the local information of the computing nodes is guaranteed, which facilitates the management system to manage each calculation. On the other hand, it also ensures that the removed nodes are quickly removed from the cluster without affecting the normal operation of other working nodes in the database.

[0105] In one embodiment of the present invention, receiving a scaling-down instruction for a native database further includes: determining, in response to the scaling-down instruction, a target storage corresponding to the scaling-down instruction; and deleting, by the management system, storage information of the target storage from the configuration information. By using the method of the embodiment of the present invention, shared storage can be quickly deleted for the database, and the shared storage is used to store database data.

[0106] The cloud-native database adopts a distributed query design, which can distribute queries to multiple computing nodes at the same time, greatly improving the query performance. The query performance is positively correlated with the number of computing nodes, but traditional databases cannot scale up and down online in real time, and the data cannot be automatically balanced after horizontal expansion. The cloud-native real-time data warehouse uses shared storage, separates computing and storage, and supports elastic scaling of clusters.

[0107] In order to make the method of the embodiment of the present invention easier to understand, the implementation method of a specific embodiment is explained below. Figure 5 FIG. 1 is a schematic diagram of a cloud native database expansion and contraction principle provided by an embodiment of the present invention. Figure 5 As shown in the figure, cluster initialization and creation are controlled by the operator in Kubernetes, and expansion and contraction are also controlled by the operator. The cluster topology information is stored in the Kubernetes configmap. ConfigMap is an API object used to save non-confidential data in key-value pairs. It is synchronized to the local configuration file and the shared configuration file by Kubernetes; all computing nodes load the configuration file, and the data file is loaded from the shared storage. When it is in the ready state, the state is registered in Zookeeper. When the computing node crashes (damaged) or is unavailable, the computing node will be automatically deleted (Zookeeper feature); all computing nodes monitor the cluster status in Zookeeper to complete the creation of the cluster or horizontal expansion and contraction.

[0108] Figure 6 FIG. 1 is a schematic diagram of a process of scaling a database provided by an embodiment of the present invention. Figure 6As shown in the figure, the user inputs the expansion or reduction command, and the expansion and reduction are also controlled by the operator. The cluster topology information is saved in the Kubernetes configuration information (config). Each computing node loads the configuration file to obtain the node information. After the expansion node is available, the node information is registered in Zookeeper. Zookeeper returns the cluster information to each node, and returns the cluster information and the successful expansion / reduction information.

[0109] The database expansion method uses the operator to expand the cluster by modifying the configuration file. The following is the process of adding a new node when the database is expanded:

[0110] Step S01: The user initiates a capacity expansion command, and the operator creates new node resources, which are allocated by the Kubernetes cluster;

[0111] Step S02: The operator configures the domain name of the expansion node, and each computing node accesses each other through the domain name;

[0112] Step S03: the operator generates a new configuration file to configmap;

[0113] Step S04: Kubernetes maps the value of the newly generated configmap to the node configuration file, and the cluster loads new configuration information such as clusters and macros;

[0114] Step S05: the database data adopts shared storage, and metadata migration is not required, and metadata can be directly loaded from the disk;

[0115] Step S06: After the original data is loaded, if it is a replicatedmergetree table engine, the data will be copied from the memory of other nodes, and the data will be aligned through the full part information on Zookeeper to complete the newly generated shared storage data when loading the data;

[0116] Step S07: After the table data is loaded and the node is in the ready or available state, if the node status management directory does not exist, it is created, otherwise it is not created; the directory is the root directory of the node status management "db / cluster / status";

[0117] Step S08: Register your own node (Ephemeral type) status such as db / cluster / status / {replica}:active to the zookeeper directory db / cluster / status;

[0118] Step S09: Start the cluster_updating_task task, the main task of which is to watch the increase of nodes under the zookeeper directory db / cluster / status;

[0119] Step S10: When the computing node monitors the zookeeper status management directory, it updates the cluster status information in the memory;

[0120] Step S12: The computing nodes and expansion nodes are updated to the clusters referenced in the query and write processes. The query, write and other tasks are processed according to the new cluster topology.

[0121] At this point, the cluster is operating normally and capacity expansion is completed without the need for data redistribution or metadata migration, and users are unaware of the capacity expansion.

[0122] The following process is the processing flow of the original computing nodes during database expansion, including:

[0123] Step S21: the operator generates a new configuration file to configmap;

[0124] Step S22: Kubernetes maps the value of the newly generated configmap to the node configuration file, and the cluster loads the new configuration information clusters every 2 seconds;

[0125] Step S23: the cluster_updating_task task watches (monitors) the increase or decrease of working nodes under the zookeeper directory db / cluster / status;

[0126] Step S24: Update the cluster status information in the memory;

[0127] Step S25: Update the clusters referenced in the query, write and other processes. The query, write and other tasks are processed according to the new cluster topology.

[0128] At this point, the cluster is operating normally and capacity expansion is complete.

[0129] It should be noted that: if the node is damaged before it is ready or available during the expansion process, it will not affect the cluster status in memory and will not affect the database operations. Kubernetes will try to create a new one again without affecting the cluster. If the node is damaged after it is ready or available, it is a failover process, which is equivalent to scaling down the database and then expanding it.

[0130] The database shrinking method uses the operator to shrink the cluster by modifying the configuration file. The following is the process of removing nodes when shrinking the database, including:

[0131] Step S31: the operator shuts down the node to be removed;

[0132] Step S32: The removed node receives the shutdown command. Since shared storage is used, the shared data on the distributed file storage does not need to be cleaned up. The residual information and residual files of Zookeeper are cleaned up, and the Zookeeper client is closed. At this time, other clusters watch the cluster status change;

[0133] Step S33: the computing node is shut down, and the status of the shut down computing node on Zookeeper is erased by Zookeeper;

[0134] Step S34: Kubernetes reclaims node resources.

[0135] At this point, the database cluster has completed scaling down.

[0136] The following process is the processing flow of other computing nodes during database shrinking, including:

[0137] Step S41: The user initiates a reduction command, and the operator generates a new configuration file to configmap;

[0138] Step S42: Kubernetes maps the value of the newly generated configmap to the node configuration file, and the cluster loads the new configuration information clusters every 2 seconds;

[0139] Step S43: the cluster_updating_task task monitors the reduction of nodes under the zookeeper directory db / cluster / status;

[0140] Step S44: Update the cluster status information in the memory;

[0141] Step S45: Update the clusters referenced in the query, write and other processes. The query, write and other tasks are processed according to the new cluster topology.

[0142] At this point, the database cluster is providing normal services and has completed scaling down.

[0143] In an embodiment of the present invention, a method for elastically scaling a cloud-native database is provided. The notable features of a real-time data warehouse are low-frequency batch writing and high-frequency querying. Query performance is the key to determining whether an analytical database can be successful. The real-time data warehouse adopts a distributed query design, which can distribute queries to multiple nodes at the same time, greatly improving the query performance. Query performance is positively correlated with the number of nodes, but traditional data warehouses cannot be scaled online in real time, and data cannot be automatically balanced after horizontal expansion. Cloud-native databases use shared storage, separate computing and storage, and support elastic scaling of cluster levels.

[0144] Figure 7 FIG. 1 is a schematic diagram of a database expansion device provided by an embodiment of the present invention. Figure 7 As shown, the device comprises:

[0145] The instruction receiving module 701 is used to receive an expansion instruction for the native database;

[0146] A resource creation module 702, configured to, in response to the capacity expansion instruction, cause the management system to create a node resource for the expanded node;

[0147] The first generating module 703 is used for the management system to generate configuration information; wherein the configuration information includes: the topology information of the expansion node and the topology information of each of the computing nodes;

[0148] The second generating module 704 is used for the expansion node to obtain the configuration information and generate a configuration file corresponding to the expansion node using the configuration information; wherein the management system manages the expansion node using the configuration file corresponding to the expansion node;

[0149] The directory updating module 705 is used for the management system to obtain the node status of the expansion node and update the status management directory when the expansion node is in an available state;

[0150] The third generating module 706 is used for the expansion node to obtain the state management directory and generate cluster state information corresponding to the expansion node using the state management directory; wherein the expansion node provides cluster task services using the cluster state information corresponding to the expansion node.

[0151] Optionally, the second generating module 704 is further used for:

[0152] The computing node obtains the configuration information, and generates a configuration file corresponding to the computing node using the configuration information;

[0153] The third generating module 706 is further used for:

[0154] The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services.

[0155] Optionally, the device further comprises:

[0156] The storage management module 707 is used for creating storage resources for the target storage by the management system in response to the capacity expansion instruction;

[0157] loading database data on the target storage;

[0158] Performing alignment processing on the database data on the target storage;

[0159] The management system generates configuration information; wherein the configuration information includes storage information of the target storage.

[0160] Figure 8 FIG. 1 is a schematic diagram of a database shrinking device provided by an embodiment of the present invention. Figure 8 As shown,

[0161] The instruction receiving module 801 is used to receive a shrinking instruction for the native database;

[0162] A shutdown module 802, configured to shut down the removed node in response to the shrinking instruction;

[0163] A first deleting module 803, configured for the management system to delete the topology information of the removed node from the configuration information;

[0164] The second deleting module 804 is used for the management system to delete the node information of the removed node from the state management directory.

[0165] Optionally, the device further comprises:

[0166] The node management module 805 is used for the computing node to obtain the state management directory and use the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services;

[0167] The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; wherein the computing node uses the cluster state information corresponding to the computing node to provide cluster task services.

[0168] Optionally, the device further comprises:

[0169] A storage management module 806, configured to determine, in response to the shrink instruction, a target storage corresponding to the shrink instruction;

[0170] The management system deletes the storage information of the target storage from the configuration information.

[0171] An embodiment of the present invention provides an electronic device, including:

[0172] one or more processors;

[0173] a storage device for storing one or more programs,

[0174] When one or more programs are executed by one or more processors, the one or more processors implement the method of any of the above embodiments.

[0175] Reference below Fig. 9 , which shows a schematic diagram of the structure of a computer system 900 of a terminal device suitable for implementing an embodiment of the present invention. Fig. 9 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0176] like Fig. 9 As shown, the computer system 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the system 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0177] The following components are connected to the I / O interface 905: an input section 906 including a keyboard, a mouse, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

[0178] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 909, and / or installed from the removable medium 911. When the computer program is executed by the central processing unit (CPU) 901, the above-mentioned functions defined in the system of the present invention are executed.

[0179] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0180] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0181] The modules involved in the embodiments of the present invention may be implemented in software or hardware. The modules described may also be provided in a processor, for example, they may be described as: an instruction receiving module, a resource creation module, a first generation module, a second generation module, a directory update module, and a third generation module. The names of these modules do not, in some cases, constitute limitations on the modules themselves, for example, the instruction receiving module may also be described as a "module for receiving expansion instructions for the native database".

[0182] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0183] Receiving a capacity expansion instruction for the native database;

[0184] In response to the capacity expansion instruction, the management system creates a node resource for the capacity expansion node;

[0185] The management system generates configuration information; wherein the configuration information includes: topology information of the expansion node;

[0186] The expansion node obtains the configuration information, and generates a configuration file corresponding to the expansion node using the configuration information;

[0187] When the expansion node is in an available state, the management system obtains the node state of the expansion node and updates the state management directory;

[0188] The expansion node obtains the state management directory, and uses the state management directory to generate cluster state information corresponding to the expansion node.

[0189] According to the technical solution of the embodiment of the present invention, the configuration information in the management system is used to generate a configuration file in the expansion node, and the expansion node is managed through the configuration file to achieve flexible expansion of the cloud native database.

[0190] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A cloud native database, characterized in that: The cloud native database includes: a management system, multiple computing nodes and shared storage; The management system stores configuration information and a state management directory, wherein the configuration information includes: topology information of each working node and storage information of the shared storage, and the state management directory includes: node status of each available working node; The computing node is used to process database-related calculations; the computing node locally stores a configuration file corresponding to the computing node, the configuration file includes topology information of each working node, the configuration file is generated according to the configuration information in the management system, and the management system manages the computing node using the configuration file; The computing node locally stores cluster status information corresponding to the computing node, the cluster status information only includes nodes in an available state, the cluster status information is generated according to a state management directory in the management system, and the computing node uses the cluster status information to provide cluster task services; The shared storage is used to store database data.

2. The cloud native database according to claim 1, characterized in that: The management system is used to: obtain the node status of each of the working nodes; update the status management directory according to the node information of the working nodes; The working node is used to obtain the state management directory and use the state management directory to update the locally stored cluster state information.

3. A database expansion method, characterized in that: The method is applied to the cloud native database according to claim 1 or 2, and the method includes: Receiving a capacity expansion instruction for the cloud native database; In response to the capacity expansion instruction, the management system creates a node resource for the capacity expansion node; The management system generates configuration information; wherein the configuration information includes: topology information of the expansion node; The expansion node obtains the configuration information, and generates a configuration file corresponding to the expansion node using the configuration information; When the expansion node is in an available state, the management system obtains the node state of the expansion node and updates the state management directory; The expansion node obtains the state management directory, and uses the state management directory to generate cluster state information corresponding to the expansion node.

4. The method according to claim 3, characterized in that After the management system generates the configuration information, the further step includes: The computing node obtains the configuration information, and generates a configuration file corresponding to the computing node using the configuration information; The management system obtains the node status of the expansion node and updates the status management directory, further comprising: The computing node obtains the state management directory, and uses the state management directory to update cluster state information corresponding to the computing node.

5. The method according to claim 3, characterized in that: After receiving the capacity expansion instruction for the cloud native database, the method further includes: In response to the capacity expansion instruction, the management system creates storage resources for the target storage; loading database data on the target storage; Performing alignment processing on the database data on the target storage; The management system generates configuration information; wherein the configuration information includes storage information of the target storage.

6. A method for reducing the capacity of a database, characterized in that: The method is applied to the cloud native database according to claim 1 or 2, and the method includes: Receiving a scaling-down instruction for the cloud native database; In response to the shrinking instruction, shutting down the removed node; The management system deletes the topology information of the removed node from the configuration information; The management system deletes the node information of the removed node from the status management directory.

7. The method according to claim 6, characterized in that After the management system deletes the topology information of the removed node from the configuration information, the management system further includes: The computing node obtains the state management directory, and uses the state management directory to update the cluster state information corresponding to the computing node; After the management system deletes the node information of the removed node from the state management directory, the management system further includes: The computing node obtains the state management directory, and uses the state management directory to update cluster state information corresponding to the computing node.

8. The method according to claim 6, characterized in that The receiving a scaling-down instruction for the cloud native database further includes: In response to the shrink instruction, determining a target storage corresponding to the shrink instruction; The management system deletes the storage information of the target storage from the configuration information.

9. A database expansion device, characterized in that: The device is applied to the cloud native database according to claim 1 or 2, and the device includes: An instruction receiving module, used to receive an expansion instruction for the cloud native database; A resource creation module, configured to cause the management system to create node resources for the expanded node in response to the expansion instruction; A first generating module is used for the management system to generate configuration information; wherein the configuration information includes: topology information of the expansion node; A second generating module is used for the expansion node to obtain the configuration information and generate a configuration file corresponding to the expansion node using the configuration information; A directory updating module, configured to enable the management system to obtain the node status of the expansion node and update the status management directory when the expansion node is in an available state; The third generating module is used for the expansion node to obtain the state management directory and use the state management directory to generate cluster state information corresponding to the expansion node.

10. A database shrinking device, characterized in that: The device is applied to the cloud native database according to claim 1 or 2, and the device includes: An instruction receiving module, used to receive a scaling-down instruction for the cloud native database; A shutdown module, configured to shut down the removed node in response to the shrinking instruction; A first deleting module, configured for the management system to delete the topology information of the removed node from the configuration information; The second deleting module is used for the management system to delete the node information of the removed node from the state management directory.

11. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 3-5 or 6-8.

12. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method described in any one of claims 3-5 or 6-8 is implemented.

Images