Method, device and equipment for deploying and polling MongoDB cluster based on kubernetes and medium

Abstract

The invention relates to the field of deployment and inspection of MongoDB clusters based on kubernetes, and discloses a method, device, equipment and medium for deployment and inspection of MongoDB clusters based on kubernetes, the method comprises the following steps: S1, creating a deployment middleware management platform; s2, creating a MongoDB (MongoDB) instance; s3, performing MongoDB instance routing inspection; s301, putting into an inspection queue; and S302, carrying out tour inspection on the MongoDB instance. According to the method, the middleware management platform is created, then the MongoDB instance is created based on the middleware management platform, the MongoDB instance is polled by using the MongoDB polling device, and corresponding operation and maintenance operation is performed according to the state of the MongoDB instance to be polled, so that the safety problem of an existing container deployment mode and the problem that a host outside a Kubernetes cluster accesses MongoDB service are solved.

Description

Technical Field

[0001] The present invention relates to the field of Kubernetes-based deployment and inspection of MongoDB clusters, and in particular to a method, device, equipment and medium for Kubernetes-based deployment and inspection of MongoDB clusters. Background Art

[0002] Currently, there are several common ways to deploy MongoDB components on the market:

[0003] 1. The methods for deploying MongoDB components provided in the official documentation can be divided into two categories:

[0004] Physical deployment: Supports deployment on physical devices such as Red Hat, Ubuntu, Debian, Suse, Amazon, macOS, and Windows. Taking the commonly used Red Hat operating system as an example, the installation process is as follows:

[0005] 1) First install the dependent applications, download the tarball of the corresponding platform, unzip the tarball, set specific environment variables, and execute MongoDB Shell;

[0006] 2) Preparation before running: adjust the system's ulimit;

[0007] 3) Preparation before running: create data directory and log directory, grant specific permissions, and adjust configuration files;

[0008] 4) Configure SELinux;

[0009] Docker container deployment: obtain the Docker image and start the service;

[0010] 2. Deploy MongoDB to the Kubernetes cluster based on the open source Kubernetes template file;

[0011] 3. Deploy MongoDB to the Kubernetes cluster based on the open source mongodb operator;

[0012] 4. Deploy MongoDB to the Kubernetes cluster based on the open source docker-compose template file;

[0013] The above are four common MongoDB component deployment methods. Although they are widely used, they all have certain technical defects, as follows:

[0014] 1. The two deployment methods provided in the above official documents have the following technical defects:

[0015] a) The defects of physical machine mode deployment are as follows:

[0016] (1) The cost is usually high because specialized hardware equipment needs to be purchased and maintained. In addition, as the amount of data grows and performance requirements increase, hardware may need to be upgraded regularly;

[0017] (2) Compared with cloud environments or virtualized environments, MongoDB deployed on physical machines is less flexible. Once deployed, it is difficult to quickly adjust resource allocation or migrate to other environments;

[0018] (3) The management and maintenance of physical machines are relatively complex and require professional hardware knowledge and operation and maintenance skills. In addition, regular hardware maintenance and troubleshooting are required;

[0019] (4) In some cases, a physical machine may not be able to fully utilize its hardware resources. For example, when the workload of the MongoDB database is low, the server's CPU and memory resources may be idle, which will lead to resource waste and increased costs;

[0020] (5) Physical machines may be subject to certain limitations in scalability. Although MongoDB clusters can be expanded by adding more physical machines, this requires additional hardware investment and time costs. In contrast, cloud environments or virtualized environments usually have more flexible and rapid scalability options;

[0021] (6) This deployment model lacks monitoring measures, and it costs a lot to provide monitoring measures for the MongoDB cluster, and the degree of reusability is low.

[0022] b) The defects of Docker container deployment are as follows:

[0023] (1) As the number of MongoDB containers increases, managing and maintaining these containers may become complicated, requiring the handling of issues such as dependencies between containers, service discovery, and load balancing;

[0024] (2) The network configuration of Docker containers can be relatively complex, especially when communication is required across multiple containers or hosts. Cluster deployment of MongoDB requires additional network configuration and management.

[0025] (3) Logging and monitoring of Docker containers requires additional tools or configuration to effectively monitor the performance and health of MongoDB;

[0026] (4) A dedicated backup and recovery strategy needs to be developed for the MongoDB container to ensure data reliability and recoverability;

[0027] (5) The lifecycle management of Docker containers (such as starting, stopping, restarting, etc.) requires additional scripts or tools to automate, which increases the complexity of deployment.

[0028] 2. For the above MongoDB deployment method 2, MongoDB is deployed to the Kubernetes cluster based on the open source Kubernetes template file. The defects are as follows:

[0029] a) Backing up and restoring MongoDB data on Kubernetes may require additional configuration and tools to ensure data integrity and recoverability;

[0030] b) Managing MongoDB dependencies (such as plugins, extensions, etc.) in Kubernetes template files may require additional deployment methods, configuration, version compatibility, and security issues;

[0031] c) Although Kubernetes provides flexible configuration options, some advanced configuration options of MongoDB may not be directly configured in the Kubernetes template file;

[0032] d) With the help of some monitoring tools, you can monitor the status of the Pod and related data on resource consumption, but these monitoring components will not take corresponding operation and maintenance actions.

[0033] 3. For the above MongoDB deployment method 3, MongoDB is deployed to the Kubernetes cluster based on the open source MongoDB operator. This method provides room for customization, but it also has the following disadvantages:

[0034] a) Although open source projects usually have community support, the speed and quality of answering questions may vary depending on the community's activity and knowledge level.

[0035] b) Although Kubernetes supports dynamic expansion and reduction, the specific requirements of MongoDB (such as data sharding, replica set configuration, etc.) may limit this flexibility.

[0036] c) It is limited to the configuration and access methods of MongoDB replica sets and sharded clusters, and can only be accessed within the same Kubernetes cluster, and cannot be accessed from hosts outside the Kubernetes cluster;

[0037] d) The open source operator only provides MongoDB lifecycle management and lacks monitoring and operation and maintenance capabilities.

[0038] 4. For the above MongoDB deployment method 4, MongoDB is deployed to the Kubernetes cluster based on the open source docker-compose template file. The disadvantages of this deployment mode are the same as those of the Docker deployment method in the above MongoDB deployment method 1, and will not be repeated again.

[0039] In summary, the above MongoDB deployment methods 1 to 4 all have certain technical defects, and are likely to cause security issues in existing container deployment methods and inaccessibility of the Kubernetes cluster network. Based on this, the inventor has developed a method for deploying and inspecting MongoDB clusters based on Kubernetes. Summary of the invention

[0040] In order to solve the above technical problems, the present invention provides a method, device, equipment and medium for deploying and inspecting MongoDB clusters based on Kubernetes. The method creates a middleware management platform, then creates a MongoDB instance based on the middleware management platform, and uses a MongoDB inspection device to inspect the MongoDB instance, and then performs corresponding operation and maintenance operations according to the status of the MongoDB instance to be inspected, thereby solving the security problems of the existing container deployment method and the problem of hosts outside the Kubernetes cluster accessing MongoDB services.

[0041] In a first aspect, the present invention provides a method for deploying and inspecting a MongoDB cluster based on Kubernetes, which adopts the following technical solution:

[0042] A method for deploying and inspecting a MongoDB cluster based on Kubernetes includes the following steps:

[0043] S1. Create and deploy a middleware management platform;

[0044] The middleware management platform includes a tenant console, a management console, a central controller, a media warehouse, a resource pool, and a MongoDB inspection device, where the resource pool is one or more kubernetes clusters;

[0045] S2. Create a MongoDB instance.

[0046] S201. Enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, and configuration parameters;

[0047] Submit the basic instance to the central controller, which converts the basic instance into a Kubernetes workload template in YAML format, selects eligible nodes from the resource pool based on affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template;

[0048] S202, sending the final yaml format Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance according to the yaml template;

[0049] S203, when the MongoDB inspection device detects a newly created MongoDB instance, the MongoDB instance is added to the inspection list, and the MongoDB inspection device periodically polls the list of MongoDB instances to be inspected;

[0050] S3 and MongoDB instance inspection;

[0051] S301, adding to the inspection queue: the MongoDB inspection device deployed in the resource pool monitors the newly added MongoDB instance in real time and adds the newly created MongoDB instance to the inspection list in time;

[0052] S302, inspecting the MongoDB instances in the instance list: performing corresponding operation and maintenance operations according to the status of the MongoDB instances to be inspected, including the following situations:

[0053] a) Resource specification inspection: If the Pod of the MongoDB instance is not mounted with shared storage, the resource specification inspection is skipped. If it is found that the Kubernetes node resources where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, Kubernetes nodes that meet the specification requirements and node affinity rules are selected from other Kubernetes nodes, and rescheduling is triggered;

[0054] b) Connection number inspection: Check the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, monitor the trend of the number of connections, and if an abnormal increase in the number of connections is found, generate an analysis report based on the current connection status, issue an early warning in a timely manner, trigger data backup to protect data, and relieve server pressure by expanding the capacity of the MongoDB instance;

[0055] c) Security inspection: Check the complexity of passwords and password change cycles, and issue warnings in a timely manner when alarm thresholds are triggered;

[0056] d) Data integrity inspection: Check the status of the replica set by calling the status() function. When a replication delay problem is identified, the replication error is automatically repaired according to the error code, including solving network partitions and node failures.

[0057] e) Network status inspection: Get the access address. If there is a problem with the network connection access inside and outside the MongoDB cluster, back up the repl-set.cnf file, get the hostIp and Port parameters of all Pods, refresh the hostIp and Port recorded in the repl-set.cnf file, and reorganize the accessibility of the MongoDB cluster and observation service.

[0058] Preferably, in S1:

[0059] The tenant console is used to provide users with an entry point for creating middleware instances;

[0060] The management console is used to interact with the central controller through an interface and provides an entry point for middleware listing management and resource pool management;

[0061] The central controller is used for centralized management and control. It interacts with the media warehouse and resource pool through interfaces, uploads the media of the middleware instances listed on the management console to the media warehouse, transmits the management operations of the resource pool triggered by the management console to the resource pool, manages the MongoDB inspection devices in the resource pool, and sends the middleware instances defined by users in the tenant console to the resource pool.

[0062] The media repository is used to store middleware instances;

[0063] The resource pool is used to execute the instructions issued by the central controller and download the necessary media from the media warehouse to complete the deployment of the middleware instance;

[0064] The MongoDB inspection device is used for inspection.

[0065] Preferably, the deployment modes include stand-alone, replica set and sharding.

[0066] Preferably, the S302 further includes the following steps:

[0067] f) Expansion inspection: For a MongoDB instance in replica set or sharding mode, if a new Pod is detected after the expansion of the MongoDB instance, back up the repl-set.cnf file, add the hostIP and Port of the new Pod after the expansion to the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0068] Preferably, the S302 further includes the following steps:

[0069] g) Shrinkage inspection: For MongoDB instances in replica set or sharding mode, if it is detected that the MongoDB cluster has triggered a shrinkage, back up the repl-set.cnf file, remove the hostIP and Port of the Pod that has been reduced from the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0070] Preferably, the S302 further includes the following steps:

[0071] h) Piaoyi inspection: For MongoDB instances in replica set or sharding mode, if Piaoyi Pod is detected in the MongoDB cluster, back up the repl-set.cnf file, modify the hostIP and Port recorded in the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0072] Preferably, the S302 further includes the following steps:

[0073] i) Exit condition inspection: If a stopped MongoDB instance is detected, the storage resources occupied by the MongoDB instance are released, the access routes are cleaned up, and the MongoDB instance is removed from the inspection list.

[0074] In a second aspect, the present invention provides a device for deploying and inspecting MongoDB clusters based on Kubernetes, which adopts the following technical solution:

[0075] A device for deploying and inspecting a MongoDB cluster based on Kubernetes, comprising:

[0076] The middleware management platform creates a deployment component, which is used to create and deploy the middleware management platform. The middleware management platform includes a tenant console, a management console, a central controller, a media warehouse, a resource pool, and a MongoDB inspection device, wherein the resource pool is one or more kubernetes clusters;

[0077] Tenant console creation module, used to create tenant console creation;

[0078] Management console creation module, used to create a management console;

[0079] A central controller creation module is used to create a central controller;

[0080] A media warehouse creation module is used to create a media warehouse;

[0081] Resource pool creation module, used to create resource pools;

[0082] MongoDB inspection device creation module, used to create MongoDB inspection devices;

[0083] MongoDB instance creation component, used to create a MongoDB instance, including the following modules:

[0084] The basic instance creation module is used by users to enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, and configuration parameters;

[0085] The Kubernetes workload template module is used to submit the basic instance to the central controller. The central controller converts the basic instance into a Kubernetes workload template in YAML format, selects qualified nodes from the resource pool according to affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template.

[0086] The MongoDB instance creation module is used to send the final YAML-formatted Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance based on the YAML template;

[0087] A MongoDB instance list module is used to add a newly created MongoDB instance to the inspection list based on the MongoDB inspection device monitoring the newly created MongoDB instance. The MongoDB inspection device periodically polls the list of MongoDB instances to be inspected.

[0088] The MongoDB instance inspection component includes the following modules:

[0089] The inspection queue inclusion module is used to monitor the newly added MongoDB instances in real time based on the MongoDB inspection device deployed in the resource pool, and add the newly created MongoDB instances to the inspection list in time;

[0090] The inspection and maintenance module is used to perform corresponding operation and maintenance operations according to the status of the MongoDB instance to be inspected, and includes the following units:

[0091] Resource specification inspection unit, used for resource rule inspection. If the Pod of the MongoDB instance is not mounted with shared storage, the resource specification inspection is skipped. If it is found that the Kubernetes node resources where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, Kubernetes nodes that meet the specification requirements and node affinity rules are selected from other Kubernetes nodes, and rescheduling is triggered.

[0092] The connection number inspection unit is used to inspect the number of connections, detect the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, and monitor the trend of the number of connections. If an abnormal increase in the number of connections is found, an analysis report is generated based on the current connection status, an early warning is issued in time, and data backup is triggered to protect data, and the server pressure is relieved by expanding the capacity of the MongoDB instance;

[0093] Security inspection unit, used for security inspection, detecting the complexity of passwords, password change cycle, and issuing early warnings when alarm thresholds are triggered;

[0094] The data integrity inspection unit is used for data integrity inspection. It checks the status of the replica set by calling the status() function. When a replication delay problem is identified, it automatically repairs the replication error according to the error code, including solving network partitions and node failures.

[0095] The network status inspection unit is used to inspect the network status and obtain the access address. If problems are detected in the network connection access inside and outside the MongoDB cluster, the repl-set.cnf file is backed up, and the hostIp and Port parameters of all Pods are obtained. The hostIp and Port recorded in the repl-set.cnf file are refreshed, and the accessibility of the MongoDB cluster and the observation service is reorganized.

[0096] In a third aspect, the present invention provides an electronic device, which adopts the following technical solution:

[0097] An electronic device comprising:

[0098] One or more processors;

[0099] Memory;

[0100] One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute a method for deploying and inspecting a MongoDB cluster based on Kubernetes according to any of the above items.

[0101] In a fourth aspect, the present invention provides a computer-readable storage medium, which adopts the following technical solution:

[0102] A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for deploying and inspecting a MongoDB cluster based on Kubernetes described in any one of the above items is implemented.

[0103] In summary, the present invention includes the following beneficial technical effects:

[0104] 1. The present invention provides a set of middleware management platforms to manage the media, templates and configurations of the middleware, control the life cycle, service exposure and advanced operation and maintenance; only a small number of required parameters or no parameters need to be passed in from the interface to generate templates and configurations, establish MongoDB instances, avoid the risk of manual configuration modification, and use container technology to achieve resource isolation, eliminating vicious resource competition between processes. Compared with traditional deployment methods, the present invention omits the steps of modifying the operating system configuration and setting system settings, and solves the strong dependence on the operating system.

[0105] 2. Compared with the existing container deployment solutions, the present invention automatically selects servers through the resource specifications of MongoDB instances, thereby solving the risk of resource imbalance caused by manual selection of servers.

[0106] 3. The present invention uses a MongoDB inspection device to perform corresponding operation and maintenance operations according to the MongoDB instance status, such as automatically updating and maintaining the / var / mongodb / repl-set.conf file in the container, thereby solving the security problems of the existing container deployment method and the problem of hosts outside the cluster accessing MongoDB services.

[0107] 4. The present invention simplifies the deployment method and operation and maintenance process. The middleware management platform is responsible for managing the MongoDB medium, and provides users with a form interface for creating a service MongoDB instance. It shields the perception of different operating systems, CPU architectures, and Kubernetes technologies. By inputting fewer perceptual parameters, a MongoDB instance can be created, which simplifies the deployment and operation and maintenance process of the MongoDB instance.

[0108] 5. The present invention has automatic recovery capabilities. As the business load increases or decreases, the Pod of the MongoDB cluster may experience connection leakage, resource specification changes, security leakage risks, data integrity risks, capacity expansion, capacity reduction, and drift. When the MongoDB inspection device detects the above changes or problems, it can correct the MongoDB cluster in time, thereby ensuring the stability of the MongoDB cluster. BRIEF DESCRIPTION OF THE DRAWINGS

[0109] Figure 1 It is a system diagram of the method in an embodiment of the present invention.

[0110] Figure 2 It is an architectural diagram of the middleware management platform in the embodiment of the present application.

[0111] Figure 3 It is a system diagram of step S2 in an embodiment of the present application.

[0112] Figure 4 It is a system diagram of step S3 in the embodiment of the present application.

[0113] Figure 5 It is a system diagram of step S302 in an embodiment of the present application. DETAILED DESCRIPTION

[0114] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0115] An embodiment of the present invention discloses a method for deploying and inspecting a MongoDB cluster based on Kubernetes.

[0116] Reference Figure 1 The method for deploying and inspecting a MongoDB cluster based on Kubernetes includes the following steps:

[0117] S1. Create and deploy a middleware management platform;

[0118] Reference Figure 2 The middleware management platform includes a tenant console, a management console, a central controller, a media warehouse, a resource pool and a MongoDB inspection device, wherein the resource pool is one or more kubernetes clusters, and the MongoDB inspection device is embedded in the resource pool.

[0119] The tenant console is used to provide users with an entry point for creating middleware instances;

[0120] The management console is used to interact with the central controller through an interface and provides an entry point for middleware listing management and resource pool management;

[0121] The central controller is used for centralized management and control. It interacts with the media warehouse and resource pool through interfaces, and uploads the media of the middleware instances listed on the management console to the media warehouse; transmits the management operations of the resource pool triggered by the management console to the resource pool, including creating, updating, deleting, expanding nodes, and removing nodes, and manages the MongoDB inspection devices in the resource pool, and sends the middleware instances defined by users in the tenant console to the resource pool;

[0122] The media repository is used to store middleware instances;

[0123] The resource pool is used to execute the instructions issued by the central controller and download the necessary media from the media warehouse to complete the deployment of the middleware instance;

[0124] The MongoDB inspection device is used for inspection.

[0125] Deploy the middleware management platform, put the MongoDB middleware on the shelf (including images, load modules, and configuration modules), and then create a resource pool and MongoDB inspection device.

[0126] S2. Create a MongoDB instance.

[0127] Reference Figure 3 S201. After deploying the middleware management platform, putting the MongoDB middleware on the shelves, and creating the resource pool and MongoDB inspection device, enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, configuration parameters, where the deployment modes include stand-alone, replica set, and sharding.

[0128] Submit the basic instance to the central controller, which converts the basic instance into a Kubernetes workload template in YAML format, selects eligible nodes from the resource pool based on affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template;

[0129] S202, sending the final yaml format Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance according to the yaml template;

[0130] S203, when the MongoDB inspection device detects a newly created MongoDB instance, the MongoDB instance is added to the inspection list, and the MongoDB inspection device periodically polls the list of MongoDB instances to be inspected;

[0131] S3 and MongoDB instance inspection;

[0132] Reference Figure 4 , S301, adding to the inspection queue: the MongoDB inspection device deployed in the resource pool monitors the newly added MongoDB instance in real time and adds the newly created MongoDB instance to the inspection list in time;

[0133] S302, inspect the MongoDB instances in the instance list: perform corresponding operation and maintenance operations according to the status of the MongoDB instances to be inspected, refer to Figure 5 , including the following:

[0134] a) Resource specification inspection: If the Pod of the MongoDB instance is not mounted with shared storage, skip the resource specification inspection and check whether the Kubernetes node where the Pod of the MongoDB instance is located meets the resource rules. If it is found that the resources of the Kubernetes node where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, select Kubernetes nodes that meet the specification requirements and node affinity rules from other Kubernetes nodes and trigger rescheduling;

[0135] b) Connection number inspection: Check the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, monitor the trend of the number of connections, and if an abnormal increase in the number of connections is found, generate an analysis report based on the current connection status, issue an early warning in a timely manner, trigger data backup to protect data, and relieve server pressure by expanding the capacity of the MongoDB instance;

[0136] c) Security inspection: Check the complexity of passwords and password change cycles, and issue warnings in a timely manner when alarm thresholds are triggered;

[0137] d) Data integrity inspection: Check the status of the replica set by calling the status() function. When a replication delay problem is identified, the replication error is automatically repaired according to the error code, including solving network partitions and node failures.

[0138] e) Network status inspection: Get the access address and check whether there are any problems with the network connection inside and outside the MongoDB cluster. If there are any problems with the network connection access inside and outside the MongoDB cluster, back up the repl-set.cnf file, get the hostIp and Port parameters of all Pods, refresh the hostIp and Port recorded in the repl-set.cnf file, and reorganize the accessibility of the MongoDB cluster and observation service.

[0139] f) Expansion inspection: For MongoDB instances in replica set or sharding mode, check whether there are new Pods created after the expansion of the MongoDB instance. If it is detected that there are new Pods created after the expansion of the MongoDB instance, back up the repl-set.cnf file, add the hostIP and Port of the new Pod after the expansion to the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0140] g) Shrinkage inspection: For MongoDB instances in replica set or sharding mode, check whether the MongoDB cluster has triggered shrinkage. If it is detected that the MongoDB cluster has triggered shrinkage, back up the repl-set.cnf file, remove the hostIP and Port of the Pod that has been reduced from the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0141] h) Piaoyi inspection: For MongoDB instances in replica set or sharding mode, check whether there are piaoyi Pods in the MongoDB cluster. If piaoyi Pods are found in the MongoDB cluster, back up the repl-set.cnf file, modify the hostIP and Port recorded in the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

[0142] i) Exit condition inspection: Check whether there is a stopped MongoDB instance. If a stopped MongoDB instance is detected, release the storage resources occupied by the MongoDB instance, clean up the access route, and remove the MongoDB instance from the inspection list.

[0143] An embodiment of the present application also discloses a device for deploying and inspecting MongoDB clusters based on Kubernetes. The device for deploying and inspecting MongoDB clusters based on Kubernetes is applied to a method for deploying and inspecting MongoDB clusters based on Kubernetes disclosed in the above embodiment, and is used to implement a method for deploying and inspecting MongoDB clusters based on Kubernetes disclosed in the above embodiment.

[0144] The device for deploying and inspecting MongoDB clusters based on Kubernetes includes:

[0145] The middleware management platform creates a deployment component, which is used to create and deploy the middleware management platform. The middleware management platform includes a tenant console, a management console, a central controller, a media warehouse, a resource pool, and a MongoDB inspection device, wherein the resource pool is one or more kubernetes clusters;

[0146] A tenant console creation module, used to create a tenant console, wherein the tenant console is used to provide a user with an entry for creating a middleware instance;

[0147] A management console creation module is used to create a management console, wherein the management console is used to interact with the central controller through an interface and provide an entry for middleware listing management and resource pool management;

[0148] The central controller creation module is used to create a central controller, where the central controller is used for centralized management and control. It interacts with the media warehouse and resource pool through interfaces, uploads the media of the middleware instance listed on the management console to the media warehouse; transmits the management operations of the resource pool triggered by the management console to the resource pool, manages the MongoDB inspection device in the resource pool, and sends the middleware instance defined by the user in the tenant console to the resource pool;

[0149] A media warehouse creation module is used to create a media warehouse, wherein the media warehouse is used to store middleware instances;

[0150] A resource pool creation module is used to create a resource pool, wherein the resource pool is used to execute instructions issued by the central controller and download necessary media from the media warehouse to complete the deployment of the middleware instance;

[0151] A MongoDB inspection device creation module is used to create a MongoDB inspection device, wherein the MongoDB inspection device is used for inspection;

[0152] MongoDB instance creation component, used to create a MongoDB instance, including the following modules:

[0153] The basic instance creation module is used by users to enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, and configuration parameters. The deployment modes include stand-alone, replica set, and sharding.

[0154] The Kubernetes workload template module is used to submit the basic instance to the central controller. The central controller converts the basic instance into a Kubernetes workload template in YAML format, selects qualified nodes from the resource pool according to affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template.

[0155] The MongoDB instance creation module is used to send the final YAML-formatted Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance based on the YAML template;

[0156] A MongoDB instance list module is used to add a newly created MongoDB instance to the inspection list based on the MongoDB inspection device monitoring the newly created MongoDB instance. The MongoDB inspection device periodically polls the list of MongoDB instances to be inspected.

[0157] The MongoDB instance inspection component includes the following modules:

[0158] The inspection queue inclusion module is used to monitor the newly added MongoDB instances in real time based on the MongoDB inspection device deployed in the resource pool, and add the newly created MongoDB instances to the inspection list in time;

[0159] The inspection and maintenance module is used to perform corresponding operation and maintenance operations according to the status of the MongoDB instance to be inspected, and includes the following units:

[0160] Resource specification inspection unit, used for resource rule inspection. If the Pod of the MongoDB instance is not mounted with shared storage, the resource specification inspection is skipped. If it is found that the Kubernetes node resources where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, Kubernetes nodes that meet the specification requirements and node affinity rules are selected from other Kubernetes nodes, and rescheduling is triggered.

[0161] The connection number inspection unit is used to inspect the number of connections, detect the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, and monitor the trend of the number of connections. If an abnormal increase in the number of connections is found, an analysis report is generated based on the current connection status, an early warning is issued in time, and data backup is triggered to protect data, and the server pressure is relieved by expanding the capacity of the MongoDB instance;

[0162] Security inspection unit, used for security inspection, detecting the complexity of passwords, password change cycle, and issuing early warnings when alarm thresholds are triggered;

[0163] The data integrity inspection unit is used for data integrity inspection. It checks the status of the replica set by calling the status() function. When a replication delay problem is identified, it automatically repairs the replication error according to the error code, including solving network partitions and node failures.

[0164] The network status inspection unit is used to inspect the network status, obtain the access address, and detect whether there are problems with the network connection inside and outside the MongoDB cluster. If problems are detected in the network connection access inside and outside the MongoDB cluster, the repl-set.cnf file is backed up, and the hostIp and Port parameters of all Pods are obtained, the hostIp and Port recorded in the repl-set.cnf file are refreshed, and the accessibility of the MongoDB cluster and the observation service is reorganized.

[0165] The capacity expansion inspection unit is used for capacity expansion inspection. For MongoDB instances in replica set or sharding mode, it detects whether there are newly created Pods after the capacity expansion of the MongoDB instance. If it is detected that the MongoDB instance has newly created Pods after the capacity expansion, the repl-set.cnf file is backed up, the hostIP and Port of the newly created Pods after the capacity expansion are added to the repl-set.cnf file, and the MongoDB cluster is reorganized to observe the network connection availability inside and outside the MongoDB cluster.

[0166] The shrink inspection unit is used for shrink inspection. For MongoDB instances in replica set or sharding mode, it detects whether the MongoDB cluster has triggered shrinkage. If it is detected that the MongoDB cluster has triggered shrinkage, the repl-set.cnf file is backed up, and the hostIP and Port of the Pod that has been reduced by shrinkage are removed from the repl-set.cnf file. The MongoDB cluster is reorganized and the network connection availability inside and outside the MongoDB cluster is observed.

[0167] The Piaoyi inspection unit is used for Piaoyi inspection. For MongoDB instances in replica set or sharding mode, it checks whether there are Piaoyi Pods in the MongoDB cluster. If Piaoyi Pods are found in the MongoDB cluster, it backs up the repl-set.cnf file, modifies the hostIP and Port recorded in the repl-set.cnf file, reorganizes the MongoDB cluster, and observes the network connection availability inside and outside the MongoDB cluster.

[0168] The exit condition inspection unit is used for the exit condition inspection to detect whether there is a stopped MongoDB instance. If a stopped MongoDB instance is detected, the storage resources occupied by the MongoDB instance are released, the access route is cleaned up, and the MongoDB instance is removed from the inspection list.

[0169] An embodiment of the present invention further discloses an electronic device, which includes: a processor and a memory. The processor and the memory are connected, such as through a bus. Optionally, the electronic device may also include a transceiver. It should be noted that in actual applications, the transceiver is not limited to one, and the structure of the electronic device does not constitute a limitation on the embodiment of the present invention.

[0170] The processor may be a CPU central processing unit, a general processor, a DSP data signal processor, an ASIC application specific integrated circuit, an FPGA field programmable gate array or other programmable logic devices, transistor logic devices, hardware components or any combination thereof. It may implement or execute various exemplary logic blocks, modules and circuits described in conjunction with the disclosure of the present invention. The processor may also be a combination that implements computing functions, such as a combination of one or more microprocessors, a combination of a DSP and a microprocessor, etc.

[0171] The bus may include a path to transmit information between the above components. The bus may be a PCI peripheral component interconnect standard bus or an EISA extended industry standard architecture bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc.

[0172] The memory can be a ROM read-only memory or other types of static storage devices that can store static information and instructions, a RAM random access memory or other types of dynamic storage devices that can store information and instructions, or an EEPROM electrically erasable programmable read-only memory, a CD-ROM read-only disc or other optical disc storage, an optical disc storage (including compressed optical discs, laser discs, optical discs, digital versatile discs, Blu-ray discs, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited to this.

[0173] The memory is used to store the application code for executing the solution of the present invention, and the execution is controlled by the processor. The processor is used to execute the application code stored in the memory to implement the content shown in the method for deploying and inspecting MongoDB clusters based on Kubernetes disclosed in the above embodiment.

[0174] The electronic devices include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), and fixed terminals such as digital TVs, desktop computers, etc. It can also be a server, etc.

[0175] An embodiment of the present invention discloses a computer-readable storage medium, on which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer can execute the corresponding content of the method for deploying and inspecting a MongoDB cluster based on Kubernetes disclosed in the above embodiment.

[0176] The above are all preferred embodiments of the present invention, and are not intended to limit the protection scope of the present invention. Therefore, any equivalent changes made based on the structure, shape, and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for deploying and inspecting a MongoDB cluster based on Kubernetes, characterized in that: The following steps are involved: S1. Create and deploy a middleware management platform; The middleware management platform includes tenant console, management console, central controller, media warehouse, resource pool and MongoDB inspection device, where the resource pool is one or more kubernetes clusters; S2. Create a MongoDB instance. S201. Enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, and configuration parameters; Submit the basic instance to the central controller, which converts the basic instance into a Kubernetes workload template in YAML format, selects eligible nodes from the resource pool based on affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template; S202, sending the final yaml format Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance according to the yaml template; S203, when the MongoDB inspection device detects a newly created MongoDB instance, the MongoDB instance is added to the inspection list, and the MongoDB inspection device periodically polls the list of MongoDB instances to be inspected; S3 and MongoDB instance inspection; S301, adding to the inspection queue: the MongoDB inspection device deployed in the resource pool monitors the newly added MongoDB instance in real time and adds the newly created MongoDB instance to the inspection list in time; S302, inspecting the MongoDB instances in the instance list: performing corresponding operation and maintenance operations according to the status of the MongoDB instances to be inspected, including the following situations: a) Resource specification inspection: If the Pod of the MongoDB instance is not mounted with shared storage, the resource specification inspection is skipped. If it is found that the Kubernetes node resources where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, Kubernetes nodes that meet the specification requirements and node affinity rules are selected from other Kubernetes nodes, and rescheduling is triggered; b) Connection number inspection: Check the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, monitor the trend of the number of connections, and if an abnormal increase in the number of connections is found, generate an analysis report based on the current connection status, issue an early warning in a timely manner, trigger data backup to protect data, and relieve server pressure by expanding the capacity of the MongoDB instance; c) Security inspection: Check the complexity of passwords and password change cycles, and issue warnings in a timely manner when alarm thresholds are triggered; d) Data integrity inspection: Check the status of the replica set by calling the status() function. When a replication delay problem is identified, the replication error is automatically repaired according to the error code, including solving network partitions and node failures. e) Network status inspection: Get the access address. If there is a problem with the network connection access inside and outside the MongoDB cluster, back up the repl-set.cnf file, get the hostIp and Port parameters of all Pods, refresh the hostIp and Port recorded in the repl-set.cnf file, and reorganize the accessibility of the MongoDB cluster and observation service.

2. According to a method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 1, it is characterized in that: In S1: The tenant console is used to provide users with an entry point for creating middleware instances; The management console is used to interact with the central controller through an interface and provides an entry point for middleware listing management and resource pool management; The central controller is used for centralized management and control. It interacts with the media warehouse and resource pool through interfaces and uploads the media of the middleware instances listed on the management console to the media warehouse. Transmit the management operations of the resource pool triggered by the management console to the resource pool, manage the MongoDB inspection device in the resource pool, and send the middleware instances defined by the user in the tenant console to the resource pool; The media repository is used to store middleware instances; The resource pool is used to execute the instructions issued by the central controller and download the necessary media from the media warehouse to complete the deployment of the middleware instance; The MongoDB inspection device is used for inspection.

3. According to a method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 1, it is characterized in that: The deployment modes include stand-alone, replica set, and sharding.

4. According to a method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 3, it is characterized in that: The S302 further includes the following steps: f) Expansion inspection: For a MongoDB instance in replica set or sharding mode, if a new Pod is detected after the expansion of the MongoDB instance, back up the repl-set.cnf file, add the hostIP and Port of the new Pod after the expansion to the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

5. According to a method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 3, it is characterized in that: The S302 further includes the following steps: g) Shrinkage inspection: For MongoDB instances in replica set or sharding mode, if it is detected that the MongoDB cluster has triggered a shrinkage, back up the repl-set.cnf file, remove the hostIP and Port of the Pod that has been reduced from the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

6. A method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 3, characterized in that: The S302 further includes the following steps: h) Piaoyi inspection: For MongoDB instances in replica set or sharding mode, if Piaoyi Pod is detected in the MongoDB cluster, back up the repl-set.cnf file, modify the hostIP and Port recorded in the repl-set.cnf file, reorganize the MongoDB cluster, and observe the network connection availability inside and outside the MongoDB cluster.

7. A method for deploying and inspecting MongoDB clusters based on Kubernetes according to claim 3, characterized in that: The S302 further includes the following steps: i) Exit condition inspection: If a stopped MongoDB instance is detected, the storage resources occupied by the MongoDB instance are released, the access routes are cleaned up, and the MongoDB instance is removed from the inspection list.

8. A device for deploying and inspecting MongoDB clusters based on Kubernetes, characterized in that: include: The middleware management platform creates a deployment component, which is used to create and deploy the middleware management platform. The middleware management platform includes a tenant console, a management console, a central controller, a media warehouse, a resource pool, and a MongoDB inspection device, wherein the resource pool is one or more kubernetes clusters; Tenant console creation module, used to create tenant console creation; Management console creation module, used to create a management console; A central controller creation module is used to create a central controller; A media warehouse creation module is used to create a media warehouse; Resource pool creation module, used to create resource pools; MongoDB inspection device creation module, used to create MongoDB inspection devices; MongoDB instance creation component, used to create a MongoDB instance, including the following modules: The basic instance creation module is used by users to enter the tenant console, select the MongoDB component to create a new basic instance, and enter the following parameters: MongoDB version, resource pool, resource specifications, deployment mode, node label, and configuration parameters; The Kubernetes workload template module is used to submit the basic instance to the central controller. The central controller converts the basic instance into a Kubernetes workload template in YAML format, selects qualified nodes from the resource pool according to affinity rules and resource specifications, and fills in the node affinity in the Kubernetes workload template. The MongoDB instance creation module is used to send the final YAML-formatted Kubernetes workload template to the Kubernetes cluster, and the Kubernetes cluster creates a MongoDB instance based on the YAML template; A MongoDB instance list module is used to add a newly created MongoDB instance to the inspection list based on the MongoDB inspection device monitoring the newly created MongoDB instance. The MongoDB inspection device periodically polls the list of MongoDB instances to be inspected. The MongoDB instance inspection component includes the following modules: The inspection queue inclusion module is used to monitor the newly added MongoDB instances in real time based on the MongoDB inspection device deployed in the resource pool, and add the newly created MongoDB instances to the inspection list in time; The inspection and maintenance module is used to perform corresponding operation and maintenance operations according to the status of the MongoDB instance to be inspected, and includes the following units: Resource specification inspection unit, used for resource rule inspection. If the Pod of the MongoDB instance is not mounted with shared storage, the resource specification inspection is skipped. If it is found that the Kubernetes node resources where the Pod of the MongoDB instance is located no longer meet the Pod resource specification requirements, Kubernetes nodes that meet the specification requirements and node affinity rules are selected from other Kubernetes nodes, and rescheduling is triggered. The connection number inspection unit is used to inspect the number of connections, detect the number of connections of the current MongoDB instance, analyze the source IP of the connection and the corresponding number of connections, and monitor the trend of the number of connections. If an abnormal increase in the number of connections is found, an analysis report is generated based on the current connection status, an early warning is issued in time, and data backup is triggered to protect data, and the server pressure is relieved by expanding the capacity of the MongoDB instance; Security inspection unit, used for security inspection, detecting the complexity of passwords, password change cycle, and issuing early warnings when alarm thresholds are triggered; The data integrity inspection unit is used for data integrity inspection. It checks the status of the replica set by calling the status() function. When a replication delay problem is identified, it automatically repairs the replication error according to the error code, including solving network partitions and node failures. The network status inspection unit is used to inspect the network status and obtain the access address. If problems are detected in the network connection access inside and outside the MongoDB cluster, the repl-set.cnf file is backed up, and the hostIp and Port parameters of all Pods are obtained. The hostIp and Port recorded in the repl-set.cnf file are refreshed, and the accessibility of the MongoDB cluster and the observation service is reorganized.

9. An electronic device, characterized in that: It includes: One or more processors; Memory; One or more applications, wherein the one or more applications are stored in the memory and configured to be executed by the one or more processors, and the one or more applications are configured to: execute a method for deploying and inspecting MongoDB clusters based on Kubernetes according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for deploying and inspecting a MongoDB cluster based on Kubernetes as described in any one of claims 1 to 7 is implemented.

Images