A distributed cloud-native database management method and system

Abstract

The application relates to the technical field of computers, in particular to a distributed cloud native database management method and system. The method comprises the following steps: deploying multiple distributed database clusters based on a containerized deployment K8S framework; creating a unique database for each database cluster, and performing permission isolation at a user level; providing access to the database in a single cluster based on a memfire-cloud server provided by the K8S framework; performing user access and database access in a load balancing manner; expanding and shrinking the database cluster resources and storage space on demand; when a node fails and cannot be accessed, a K8S framework starts a new node, and database data information of the failed node is restored to the new node. The application can construct multiple distributed clusters on a K8S platform, and can add a database cluster in one key mode; the clusters are isolated from each other and do not interfere with each other, so that resource isolation between users is achieved, and database level isolation is achieved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer. BACKGROUND

[0002] Compared with single-machine databases, distributed databases are more reliable, have good scalability, and are easy to integrate with existing systems. Therefore, the popularity of distributed databases is unstoppable. Therefore, the management of distributed databases is particularly necessary.

[0003] Traditional distributed database installation must require a large number of real servers or virtual machines, which requires a high test environment. Each existing server is used as a node. Then the database installation package is copied to each node, and the necessary processes are started. The process parameters on each node are different, so it is easy to make mistakes during manual operation. In addition, when the node starts the corresponding process, the installation process is different between machines, which causes the environment dependency problem to be very prominent. Often, it is necessary to repeatedly generate or import dependency packages on multiple machines, which is tedious and time-consuming. When an exception occurs in the process of the machine, the abnormal machine cannot automatically solve the exception, and manual intervention is required to collect and analyze logs and manually troubleshoot. When the user needs to expand, the traditional distributed database management method is to find a new machine, then configure the corresponding database environment for it, and start the corresponding process through specific parameters. The operation is more troublesome.

[0004] With the rapid development of mobile communication and the increasing coverage and popularity of communication, companies are adjusting and changing the performance requirements of databases according to actual conditions. Generally, the urgent requirements of enterprises for distributed databases are concentrated on the stability and reliability of data in the database, the reduction of database operation and maintenance costs, and the simplicity of database expansion, contraction and upgrade operations. Therefore, the management system must be a "sustainable development" system that can maintain, expand and upgrade the system, and the maintenance personnel should be less, that is, the operation should be automated as much as possible.

[0005] In recent years, various enterprises have been developing towards application cloudization and virtualization, and the concepts of docker container technology and container orchestration technology kubernetes have also been hot. The advantage of containerization is lightweight, and a single machine can easily support hundreds of containers. The startup speed of the container is also very fast. "Agile" and "high performance" are the biggest advantages of containers compared to virtual machines. The application of container orchestration technology kubernetes can efficiently manage multiple containers and realize functions such as automatic deployment, automatic expansion and contraction, and maintenance of container clusters. Therefore, existing popular technologies can be used to improve the traditional database deployment method. SUMMARY

[0006] In view of the problems of the traditional distributed database deployment and installation mode, the embodiment of the present application aims to provide a management method and system for managing distributed databases using kubernetes container orchestration technology and docker virtualization technology.

[0007] To achieve the above object, the embodiment of the present application provides the following technical scheme:

[0008] In a first aspect, in an embodiment provided by the present application, a distributed cloud native database management method is provided,

[0009] A plurality of distributed database clusters are deployed based on a K8S (kubernetes, K8S is an abbreviation formed by replacing the eight characters "ubernete" in the middle of the name with 8) framework of containerization deployment;

[0010] Each database cluster creates a unique database, and performs permission isolation at the user level;

[0011] The memfire-cloud service provided by the K8S framework is used to provide access to the database in a single cluster;

[0012] A load balancing access method is used for user access and database access;

[0013] The database cluster resources and storage space are scaled as needed to ensure that the number of nodes remains at the target number and new space is applied;

[0014] When a node fails to access, the K8S framework starts a new node and restores the database data information of the failed node to the new node.

[0015] In some embodiments provided by the present application, the method for deploying a plurality of distributed database clusters without interfering with each other and creating a unique database, and creating a new database cluster, comprises:

[0016] Obtain the necessary information for creating a database cluster;

[0017] Change the value of the corresponding variable in Chart.yaml file according to the input of the memfire-cloud front end;

[0018] Generate a database cluster using Helm command, Chart.yaml file and dependency file;

[0019] If the generation of the database cluster is successful, the creation of the database cluster is successful, and the database is used normally;

[0020] If the generation of the database cluster fails, the creation of the database cluster fails, and the failure information is returned.

[0021] In some embodiments provided by the present application, after the creation of the database cluster is successful, the database in the cluster and the user associated with the database are created separately, and the permission isolation is performed at the user level, and the setting method of the permission isolation comprises: inputting information at the front end and transmitting the information to the back end to convert into an execution command of the database, creating a corresponding database and a database user, and assigning the corresponding user with the permission to access the database.

[0022] In some embodiments provided by the present application, the front end is used to view the database connection information and the performance information of a single database, and the read-write average delay of the database is displayed in the form of a chart, and the front end is also used for online access, automatically using the certificate information of the database and creating a terminal connected to the database in a browser tab.

[0023] In some embodiments provided by the present application, when the user access load balancing is performed in a load balancing access mode, the user accesses the cloud-memfire platform by using a browser, the service is provided by a plurality of nodes, when a node fails and cannot be accessed, a same node is restarted, the number of enabled nodes is maintained unchanged, and the node allocation reaches load balancing.

[0024] In some embodiments provided by the present application, when the database access is performed in a load balancing access mode, each access is randomly connected to a master node in the database cluster, and the master node allocation reaches load balancing.

[0025] In some embodiments provided by the present application, the method for expanding and shrinking the database cluster resource comprises: modifying a Charts.yaml file, updating the configuration number of tserver in the file by using helm, and reaching the expected number of nodes.

[0026] In some embodiments provided by the present application, the method for expanding and shrinking the storage space comprises: when the storage space is insufficient, automatically applying space based on the StorageClass function provided by the K8S framework.

[0027] In some embodiments provided by the present application, the elastic expansion by expanding and shrinking further comprises the expansion of the computing resource of the K8S platform, and the expansion method comprises: adding a new machine to the database cluster of the K8S framework, allocating the new node to the added new machine in the K8S framework, and using the resource of the new machine.

[0028] In a second aspect, in another embodiment provided by the present application, a distributed cloud native database management system is provided, which adopts the foregoing distributed cloud native database management method to manage a distributed database; the distributed cloud native database management system comprises a kubernetes platform, a memfiredb image, a dockerfile file and a front-end interface.

[0029] The kubernetes platform deploys a plurality of distributed database clusters based on a K8S framework of containerized deployment, and is used to divide the database on the basis of K8S platform containerized deployment, take the smallest management unit in the K8S framework as a node, provide a plurality of nodes on each machine, connect the nodes to each other, control the master node and tserver node of the database to communicate with each other, and is also used to connect the new node to the existing database when the capacity is expanded or reduced, and perform rolling update when the database version is upgraded.

[0030] The memfiredb image comprises installation files and corresponding environment dependencies of a distributed database memfire, and environment configuration is not required each time the capacity is expanded or reduced.

[0031] The dockerfile file is used to modify information in the memfiredb image and change data in the image.

[0032] The front-end interface is used for database and user interaction, and provides an interactive interface for user resource isolation, second-level database resource acquisition, data backup and recovery multi-copy mechanism, database encryption certificate connection, online database connection, real-time monitoring of database resource utilization, and minute-level rolling upgrade, automatic migration of a faulty node when a single node fails.

[0033] The technical solution provided by the present application has the following beneficial effects:

[0034] 1. The distributed cloud native database management method and system provided by the present application fully utilizes the advantages of containerized deployment, can construct a plurality of distributed clusters on a K8S platform, and can increase the database cluster in one key as long as the resources are sufficient, divides the database in a smaller granularity by taking a pod as the smallest unit, and greatly improves the utilization rate of single operating system resources.

[0035] 2. The distributed cloud native database management method and system provided by the present application connects the pods together, controls the master node and tserver node of the database, ensures that the two can communicate with each other, makes the function of the distributed database can be normally used, utilizes the characteristics of K8S to very simply connect the new node to the existing database when the capacity is expanded or reduced, and becomes a part of the database.

[0036] 3. The distributed cloud native database management method and system provided by the application can isolate database clusters from each other, and can not interfere with each other, can realize resource object isolation and resource quota isolation, and can realize database level isolation.

[0037] 4. The distributed cloud native database management method and system provided by the application can effectively reduce the use cost of users and improve the use efficiency of device resources on the basis of ensuring high security and high availability.

[0038] These aspects or other aspects of the application will be more apparent in the following description of the embodiments. It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the application. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical solutions in the embodiments of the application or the related art, the following will briefly introduce the drawings needed to be used in the exemplary embodiments or the related art description. The drawings are used to provide further understanding of the application, and constitute a part of the specification. The drawings together with the embodiments of the application are used to explain the application, and do not constitute a limitation on the application. In the drawings:

[0040] Figure 1 A flow chart of a distributed cloud native database management method provided by the embodiment of the application.

[0041] Figure 2 An access process schematic diagram of a distributed cloud native database management method provided by the embodiment of the application.

[0042] Figure 3 A database cluster building flow chart in a distributed cloud native database management method provided by the embodiment of the application.

[0043] Figure 4 A user access load balancing flow chart in a distributed cloud native database management method provided by the embodiment of the application.

[0044] Figure 5 A cluster expansion and contraction flow chart in a distributed cloud native database management method provided by the embodiment of the application.

[0045] Figure 6 A statefulset resource principle diagram in a distributed cloud native database management method provided by the embodiment of the application.

[0046] Figure 7 A database high availability schematic diagram in a distributed cloud native database management method provided by the embodiment of the application.

[0047] Figure 8 A flowchart of load balancing of connecting a database in a distributed cloud native database management method provided by an embodiment of the present application. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application is further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0049] In some of the processes described in the specification and claims of the present application and the above-mentioned drawings, a plurality of operations appearing in a specific order are included, but it should be clearly understood that these operations can be executed or in parallel without following the order in which they appear in this text. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes can include more or fewer operations, and these operations can be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this text are used to distinguish different messages, devices, modules, etc. and do not represent the order of precedence. Also, "first" and "second" are not different types.

[0050] The technical solutions in the exemplary embodiments of the present application will be described clearly and completely below in combination with the drawings in the exemplary embodiments of the present application. Obviously, the described exemplary embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0051] Traditional distributed database installation must require a large number of real servers or virtual machines, which requires a high test environment, and each real server is used as a node. Then copy the database installation package to each node, start the necessary processes, and the process parameters on each node are different, so it is easy to make mistakes in the manual operation process. In addition, when the node starts the corresponding process, the installation process is different between machines, resulting in a very prominent environment dependency problem, often requiring repeated generation or import of dependency packages on multiple machines, which is tedious and time-consuming. When an exception occurs in the process of the machine, the abnormal machine cannot automatically solve the exception, and manual collection and analysis of logs are required to manually troubleshoot. When the user needs to expand, the traditional distributed database management method is to find a new machine, then configure the corresponding database environment for it, and start the corresponding process through specific parameters, which is more troublesome to operate.

[0052] To solve the above problems, the embodiment of the present application provides a management method and system for managing distributed databases using kubernetes container orchestration technology and docker virtualization technology.

[0053] The technical solutions of the present application are further illustrated by specific embodiments in combination with the accompanying drawings.

[0054] Embodiment 1

[0055] Referring to Figure 1 , Figure 1 A flow chart of a distributed cloud native database management method provided by the present application. An embodiment of the present application provides a distributed cloud native database management method, comprising the following steps:

[0056] S1: deploying multiple distributed database clusters based on the K8S framework of containerized deployment.

[0057] In this embodiment, the multiple distributed database clusters are deployed without interference and create their own unique databases. Referring to Figure 2 , the overview of the distributed cloud native database memfiredb management method is shown, and it can be seen from the figure that the K8S framework of containerized deployment can deploy multiple distributed database clusters, each cluster does not interfere with each other, and can create its own unique database.

[0058] Referring to Figure 2 , the user can directly access the database cluster, or access through the memfire-cloud service provided on the K8S framework by logging in with a user password.

[0059] When creating a unique database, referring to Figure 3 , a method for creating a new database cluster is shown, which comprises:

[0060] S101, obtaining the necessary information for creating a database cluster;

[0061] S102, changing the value of the corresponding variable in Chart.yaml file according to the input of memfire-cloud front end;

[0062] S103, generating a database cluster using Helm command, Chart.yaml file and dependent file;

[0063] If the database cluster is generated successfully, the database cluster is created successfully, and the database is used normally;

[0064] If the database cluster fails to be generated, the database cluster fails to be created, and the failure information is returned.

[0065] In this embodiment, to create a new database cluster, only the cluster name, the number of tmaster nodes, the number of tserver nodes, the memfiredb version number and other necessary information need to be set to automatically create a whole cluster. In this distributed database management system, each Pod (Pod is the smallest management unit in K8S) is regarded as a node, and each machine (real server or virtual machine) can have multiple Pods, which can greatly improve the utilization of single operating system resources.

[0066] As described above, the principle of creating a cluster is to use the package manager Helm in K8S, and write the configured resources in the existing Chart.yaml file, such as the statefulset resource type management Pod, the Headless service service that enables stateful access between Pods, and the like. In addition, the values of specific variables need to be modified to control the creation of the database name and the number of master nodes, tserver nodes and memfiredb version number.

[0067] When a new database cluster is created, the connection information (IP, Port, certificate download link, etc.) of each cluster and the connection information of each database are provided on the front end of memfire-cloud, and the database in each cluster is logged in through the correct user and password. After obtaining various information from the first method, a third-party database client is used to directly connect the database cluster and perform all operations on the distributed database.

[0068] S2: Each database cluster creates a unique database, and performs permission isolation at the user level.

[0069] It should be particularly noted that after the database cluster is successfully created, the database in the cluster and the user associated with the database are created separately, and permission isolation is performed at the user level. The method for setting the permission isolation includes transmitting the information input on the front end to the back end to convert into a database execution command, creating a corresponding database and database user, and assigning the corresponding user with the permission to access the database.

[0070] Therefore, after the cluster is created, the operation of the cluster is further refined, and the database in the cluster is created separately, and the user associated with the database is also created at the same time. Each database has its own user, and each user only has the permission to access its own database. Permission isolation is performed at the user level to ensure the stability and security of the data. The function principle is that the information input on the front end is transmitted to the back end to convert into a database execution command, thereby creating a corresponding database and database user and assigning the corresponding user with the permission to access the database.

[0071] S3: Based on the memfire-cloud service provided by the K8S framework, access to the database in a single cluster is provided.

[0072] As in the access method in step S1 described above, the user password login method provides access through the memfire-cloud service provided on the K8S framework, and the memfire-cloud service provides access to the database in a single cluster. Referring to Figure 2 As shown in the front end of the memfire-cloud, the connection information (IP, Port, certificate download link, etc.) of each cluster and the connection information of each database are provided, and the database in a single cluster is accessed through correct user and password login.

[0073] S4: User access and database access are performed in a load balancing manner.

[0074] In this embodiment, the load balancing access method is used multiple times to ensure that the access can still be normal when some nodes fail. As Figure 4 As shown, the first access load balancing place is that the user accesses the cloud-memfire platform using a browser, and this service is provided by multiple pods of the same service.

[0075] When the user accesses the load balancing of the cloud-memfire platform using a browser, the service is provided by multiple nodes of the same service. When a node fails and cannot be accessed, a same node is restarted to maintain the number of enabled nodes unchanged, and the node allocation reaches load balancing.

[0076] As Figure 4 It can be seen that there are three cloud-memfire platforms in the figure, and in K8S, the number of three is managed by the deployment resource type to keep constant. If one of the pods fails due to some reason and is hung up, it will be detected and a same pod will be restarted to ensure that the number is always maintained at three. The restarted pod and the previous pod seem to be the same, but the IP address is actually changed, so the K8S service resource is used to detect the IP of the pod at any time, and the user only needs to connect the IP of the service. The connection will be automatically allocated to one of the pods, and the next connection will be allocated to another pod, thereby achieving the effect of load balancing.

[0077] In this embodiment, the second access load balancing place lies in the load balancing of connecting the database. When the load balancing access mode is used to access the database, the connection is randomly converted to the master node in the database cluster each time, and the master node is allocated to reach the load balancing.

[0078] As shown in Figure 5 , when the user accesses the database, the connection cluster IP is connected, the connection is randomly converted to the master node in the cluster, and the next connection is connected to the next master node.

[0079] The difference between the above two load balancing methods lies in the type of service, the former type uses NodePort, is based on ClusterIp, and opens a port on each Node, which can be accessed from all locations.

[0080] The latter type is LoadBalance, based on NodePort, and the cloud service provider has created a load balancing layer outside to guide the traffic to the corresponding Port. The LoadBalance type is not self-contained in K8S, and needs to be provided by the public cloud vendor.

[0081] S5: Scale the database cluster resources and storage space on demand to ensure that the number of nodes remains the target number and applies for new space.

[0082] In this embodiment, the elastic expansion feature is used a lot to expand resources on demand. The method for expanding and shrinking the database cluster resources is to modify the Charts.yaml file and use helm to update the configuration number of tserver in the file to make the number of nodes reach the expected number.

[0083] As shown in Figure 6 , the database cluster is expanded from 3 tserver nodes to 5 tserver nodes, and the method used is to modify the Charts.yaml file and then use helm to update the configuration number of tserver in the file, so that the number of nodes reaches the expected number. The deep principle is to use the statefulset resource in K8S to ensure that the number of nodes remains the target number.

[0084] Unlike the deployment resource, the former maintains the number of pods in a certain order, that is, tserver-0 is completed before tserver-1 is started to be created, and the name of the pod and its domain name are related. Through this domain name, a certain pod can be accessed fixedly.

[0085] Referring to Figure 7As shown, the domain name access requires the combination of Statefulset resource and headless service resource, and the domain name format of each Pod is: $<Pod Name>.$<service name>.$<namespace name>.svc.cluster.local. Only the number of tserver Pods in the statefulset resource needs to be expanded, and then the automatic expansion will be performed.

[0086] In the embodiment, the method for expanding the storage space is that, when the storage space is insufficient, the space is automatically applied based on the StorageClass function provided by the K8S framework. When the elastic expansion of the storage space is needed, if the storage space is insufficient, the new space is applied, the StorageClass function provided by the K8S is used, the space is automatically applied, and the process is automated.

[0087] It should be particularly noted that another elastic expansion function of the embodiment is that the K8S platform also supports elastic expansion when the computing resource of the K8S platform is insufficient. The method for expanding the computing resource of the K8S platform is that, a new machine is added to the database cluster of the K8S framework, the new node is distributed to the added new machine in the K8S framework, and the resource of the new machine is used.

[0088] If the computing resource of the K8S platform is insufficient, only the new machine needs to be added to the K8S cluster, and the new Pod is distributed to the machine in the K8S, and the resource of the machine is used. After the machine is added, all operations are automatically performed without manual intervention.

[0089] S6: When a node fails to access, the K8S framework starts a new node, and the database data information of the failed node is restored to the new node.

[0090] In the embodiment, referring to Figure 8 As shown, when the business access is performed, it is assumed that Server-Pod1 fails to access, the K8S starts a new Pod, the database restores the database data information to the Pod according to the information in each Master by using the Raft protocol.

[0091] In the distributed cloud native database management method, the communication between the database and the client used by the user uses TLS to encrypt the intra-cluster and client-to-server network communication, ensuring the privacy and integrity of the data transmitted through the network, and ensuring the security of the network communication between the servers. The data stored locally can be stored in the form of static encryption to ensure that the static data stored on the disk is protected. Different databases in the same cluster are created with their own users, and each user can only access the permissions of its own database and cannot access another database to ensure the security of database access.

[0092] The application provides a distributed cloud native database management method. The K8S platform is located on several machines, but fully utilizes the advantages of containerized deployment, and is smaller in granularity, with a pod as the minimum unit. Multiple distributed clusters can be built on one K8S platform, and as long as the resources are sufficient, the database cluster can be increased one-key. The clusters are isolated from each other and do not interfere with each other. The principle is that the namespace resource type provided by K8S is used to place each database cluster in different namespaces, which can not only isolate resource objects such as Service Deployment, but also isolate resource quotas such as CPU and Memory. When accessing memfire-cloud in the front end, multiple users can be registered, and the users can create their own databases. The users are invisible to each other, so as to achieve resource isolation between users. Different databases in the same cluster are created with their own users, and each user can only access the permissions of its own database and cannot access another database. Thus, database-level isolation is achieved.

[0093] It should be understood that although the above steps are described in a certain order, these steps are not necessarily executed in the above order. Unless otherwise specified herein, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, part of the steps of the present embodiment can include multiple steps or multiple stages, which are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily sequential, but can be executed in rotation or alternation with at least part of other steps or steps or stages in other steps.

[0094] Embodiment 2

[0095] In the preferred embodiment provided by the application, a distributed cloud native database management system includes a kubernetes platform, a memfiredb image, a dockerfile file, and a front-end interface. Wherein:

[0096] The kubernetes platform deploys multiple distributed database clusters based on the K8S framework of containerized deployment, divides the database on the basis of the K8S platform containerized deployment, takes the minimum management unit in the K8S framework as a node, provides multiple nodes on each machine, connects the nodes to each other, controls the master node and tserver node of the database to communicate with each other, and is also used for connecting the new node to the existing database when the capacity is expanded or reduced, and performing rolling update when the database version is upgraded.

[0097] In the embodiment, the kubernetes platform is the basis of running images, all the images are run on the kubernetes platform and are automatically managed by the kubernetes platform. On the basis of the K8S (kubernetes) platform containerized deployment, the database is divided in a smaller granularity, each Pod (the minimum management unit in the K8S) is taken as a node in the distributed database management system, and multiple Pods can be provided on each machine, so that the utilization rate of single operating system resources can be greatly improved.

[0098] In the embodiment, the main function of the distributed database management system is to connect the Pods to each other, control the master node and tserver node of the database, ensure that the two nodes can communicate with each other, so that the function of the distributed database can be normally used, and the K8S feature can be used to simply connect the new node to the existing database when the capacity is expanded or reduced, so that the new node becomes a part of the database. When the database version needs to be upgraded, the system can ensure that the rolling update is performed, so that the distributed database service is always available and the online update is ensured.

[0099] The approximate process of the rolling update is as follows: first, a pod is added, the image version is the new version, after the pod is available, an old version pod is deleted, the above two steps are repeated until the old version is completely deleted and the new version pod is completely available.

[0100] If an error occurs in the middle process, the above steps are reversed. The above update or rollback steps are all automatic operations.

[0101] The memfiredb image includes the installation file of the distributed database memfire and the corresponding environment dependency, and environment configuration is not required when the capacity is expanded or reduced each time.

[0102] In this embodiment, the memfiredb image contains the installation files of the distributed database memfire and the corresponding environment dependencies. Among them, the advantage of the docker image contained in the memfiredb image is that it is generated once and can be run on any environment. This solves the problem of troublesome database installation operations. The database can be deployed on the K8S platform only through the network or storage media such as USB flash drives, and also solves the problem of needing to configure the environment every time the capacity is expanded or reduced.

[0103] The Dockerfile file is used to modify the information in the memfiredb image and change the data in the image.

[0104] In this embodiment, the Dockerfile file is convenient for quickly modifying various information in the image. You only need to add corresponding field information in the file to directly change the data in the image, such as setting environment variables, setting the mount directory, setting the port exposed by the container, and the files required in the container, thereby forming a new database image, which is extremely convenient for image upgrades.

[0105] The front-end interface is used for interaction between the database and users, and is used to provide isolation of user resources, acquisition of database resources in seconds, multi-copy mechanism for data backup and recovery, database encryption certificate connection, database online connection, real-time monitoring of database resource utilization and minute-level rolling upgrades, and an interactive interface for automatic migration of faulty nodes when a single node fails.

[0106] In this embodiment, the front-end interface is memfire-cloud, a database management system used for database-user interaction. It provides user resource isolation, instantaneous database resource acquisition, multiple-copy data backup and recovery, database encryption certificate connection, online database connection, real-time monitoring of database resource utilization, minute-by-minute rolling upgrades, and automatic node migration in the event of a single node failure. While ensuring high security and high availability, it effectively reduces user costs and improves device resource efficiency.

[0107] In this embodiment, the distributed cloud native database management system is executed using the steps of a distributed cloud native database management method as described in the above embodiment. Therefore, the operation process of the distributed cloud native database management system is not described in detail in this embodiment.

[0108] In summary, the technical solution provided by the present invention has the following advantages:

[0109] 1. The distributed cloud native database management method and system provided by the application make full use of the advantages of container deployment, multiple distributed clusters can be built on a K8S platform, and as long as the resources are sufficient, the database cluster can be increased one-key, the database is divided into smaller granularity with the pod as the minimum unit, and the utilization rate of single operating system resources is greatly improved.

[0110] 2. The distributed cloud native database management method and system provided by the application connect the pods together, control the master node and tserver node of the database, and ensure that the two can communicate with each other so that the function of the distributed database can be normally used, and the characteristics of K8S can make it very simple to connect the new node to the existing database when the database is scaled, and become a part of the database.

[0111] 3. The distributed cloud native database management method and system provided by the application isolate the database clusters from each other, do not interfere with each other, can isolate resource objects, and can also isolate resource quotas, so as to achieve database-level isolation.

[0112] 4. The distributed cloud native database management method and system provided by the application can effectively reduce the use cost of users and improve the use efficiency of device resources on the basis of ensuring high security and high availability.

[0113] The above only describes the preferred embodiments of the application and does not limit the application, and any modification, equivalent replacement and improvement made within the spirit and principle of the application shall be included in the protection scope of the application.

Claims

1. A distributed cloud native database management method, characterized in that: include: Deploy multiple distributed database clusters based on the K8S framework for containerized deployment; After the database cluster is successfully created, each database cluster creates its own unique database and users associated with the database, and performs permission isolation at the user level; The memfire-cloud server provided by the K8S framework provides access to the database in a single cluster; Use load balancing access mode for user access and database access; Scale database cluster resources and storage space as needed, ensuring the number of nodes remains at the target number and applying for new space; When a node fails and becomes inaccessible, the K8S framework starts a new node and restores the database data of the failed node to the new node. The method of deploying multiple distributed database clusters without interfering with each other and creating their own unique databases to create a new database cluster includes: Obtain the necessary information to create a database cluster; Change the values ​​of corresponding variables in the Chart.yaml file according to the memfire-cloud front-end input; Generate a database cluster using Helm commands, Chart.yaml files, and dependency files. If the database cluster is generated successfully, the database cluster is created successfully and the database can be used normally; If the generation of the database cluster fails, the creation of the database cluster fails and a failure message is returned; When load balancing is used for user access, users use a browser to access the cloud-memfire platform. The service is provided by multiple nodes. When a node fails and becomes inaccessible, the service resource in K8S is used to constantly detect the node IP. When the user connects to the service IP, the connection will be automatically assigned to one of the nodes to restart the same node, maintaining the same number of enabled nodes and achieving load balancing. In addition, each access randomly switches the connection to the master node in the database cluster, achieving load balancing. The method for scaling the database cluster resources is as follows: modify the Charts.yaml file and use helm to update the number of tserver configurations in the file to increase the number of nodes to the expected number.

2. The distributed cloud native database management method according to claim 1, characterized in that: The method for setting up permission isolation includes transmitting input information from the front end to the back end and converting it into an execution command of the database, creating a corresponding database and database user, and granting the corresponding user permission to access the database.

3. The distributed cloud native database management method according to claim 2, characterized in that: The front end is used to view database connection information and performance information of a single database, and to display the average read and write latency of the database in the form of a chart. The front end is also used for online access, automatically using the database certificate information and creating a terminal connected to the database in a browser tab.

4. The distributed cloud native database management method according to claim 1, characterized in that: When using load-balanced access to connect to the database, each access is randomly transferred to the master node in the database cluster, and the master node distribution achieves load balancing.

5. The distributed cloud native database management method according to claim 1, characterized in that: The method of expanding or shrinking storage space is: when storage space is insufficient, space is automatically applied based on the StorageClass function provided by the K8S framework.

6. The distributed cloud native database management method according to claim 5, characterized in that: Elastic expansion through scaling also includes the expansion of computing resources on the K8S platform. The expansion method is: add new machines to the database cluster of the K8S framework, allocate new nodes to the newly added machines in the K8S framework, and use the resources of the new machines.

7. A distributed cloud-native database management system, characterized in that: The distributed cloud-native database management system adopts the distributed cloud-native database management method according to any one of claims 1 to 6 to manage a distributed database; the distributed cloud-native database management system includes: The Kubernetes platform deploys multiple distributed database clusters based on the containerized K8S framework. This is used to partition the database based on the containerized deployment of the K8S platform. The smallest management unit in the K8S framework is used as a node. Each machine has multiple nodes that connect to each other and control the communication between the master and tserver nodes of the database. It is also used to connect new nodes to the existing database during expansion and contraction, and to perform rolling updates when the database version is upgraded. Memfiredb image, including the installation files of the distributed database Memfire and the corresponding environment dependencies, no environment configuration is required each time the capacity is expanded or reduced; Dockerfile file, used to modify the information in the memfiredb image and change the data in the image; and The front-end interface is used for interaction between the database and users. It is used to provide isolation of user resources, acquisition of database resources in seconds, multi-copy mechanism for data backup and recovery, database encryption certificate connection, database online connection, real-time monitoring of database resource utilization and minute-level rolling upgrades, and an interactive interface for automatic migration of failed nodes in the event of a single node failure.

Images