Ambari-based Cluster Version Management System

Abstract

The present application discloses a cluster version management system based on Ambari, which relates to the field of big data cluster management. The system includes: a version management client based on Ambari and a version management background based on Ambari; the version management client is used to submit version information to the version management background and to pull the version information in the version management background for version control; the version management background is used to store the version information. The present application can support the multi-version management of software stacks and each service, can support version upgrade or version rollback of the versions of services in the software stack, and also supports version upgrade or rollback of the entire software stack, facilitating and flexible in version management under a big data cluster, and providing a good user experience.

Description

Technical Field

[0001] This application relates to the field of big data cluster management, and specifically relates to a cluster version management system based on Ambari. Background Art

[0002] Ambari is a Web-based Hadoop distributed cluster configuration management tool that can support the provisioning, management, and monitoring of Apache Hadoop clusters, support the RestAPI style of interaction, and is used for the creation and maintenance of big data clusters. Users can use Ambari to create various service components in a big data cluster, greatly simplifying the use of big data clusters. A big data cluster can contain many service components, each service has its own version number, all the services in the cluster form a software stack, and this software stack also has a version number. There are many version numbers for Ambari services.

[0003] However, the existing cluster version management function is not perfect and does not support version modification. The existing cluster version management system has the following problems:

[0004] 1. It can only support one default software stack version, and each service in the software stack can only have one version;

[0005] 2. It lacks version management for the software stack and each service, and cannot upgrade or roll back the version of the services in the software stack, nor can it upgrade or roll back the version of the entire software stack.

[0006] Application Content

[0007] To solve at least one of the problems mentioned in the above background art, this application provides a cluster version management system based on Ambari, which can support multi-version management of the software stack and each service, can support version upgrade or version rollback of the services in the software stack, and also supports version upgrade or version rollback of the entire software stack, facilitating and flexible version management for big data clusters, and providing a good user experience.

[0008] The specific technical solution provided by the embodiments of this application is as follows:

[0009] Provide a cluster version management system based on Ambari, including:

[0010] An Ambari-based version management client and an Ambari-based version management background;

[0011] The version management client is used to submit version information to the version management background and to pull version information from the version management background for version control;

[0012] The version management background is used to store the version information.

[0013] Furthermore, the version management client includes a version control client.

[0014] The version management background includes a version control server and a repository.

[0015] The version control client is used to interact with the version control server for the version information; the repository is used to store the version information for the version control server to pull.

[0016] Furthermore, the version management client further includes an Ambari-based server management node for processing Rest requests submitted to Ambari.

[0017] The Ambari-based cluster version management system further includes one or more Ambari-based client nodes for executing instructions sent by the server management node.

[0018] The client node is further used to receive the version information of the version management client and perform version synchronization.

[0019] Furthermore, the version information includes a software stack version number.

[0020] The version control client is further used to submit and save the software stack version number to the repository.

[0021] The version control client is further used to provide a software version list.

[0022] The software version list is used to display the software stack version number and the corresponding software stack version name for users to add, delete, change, and select software stack versions.

[0023] Furthermore, the software version list further includes one or more software service version lists.

[0024] Each software service version list corresponds to a software stack version number.

[0025] The software service version list is used to display the software service versions under the software stack version number.

[0026] Furthermore, the software service version list is further used to select the software service version for software service version control.

[0027] The software service version control includes at least one of version upgrade or version rollback.

[0028] The software service version control also includes at least one of a quick mode and a rolling mode;

[0029] The quick mode is to stop the service to be controlled and perform the version upgrade or version rollback on the service to be controlled;

[0030] The rolling mode is to keep the service to be controlled running normally and perform the version upgrade or version rollback on each part component of the service to be controlled in batches.

[0031] Further, the rolling mode further includes creating a current master node and a current slave node, and replacing the historical master node and the historical slave node according to the current master node and the current slave node.

[0032] Further, the version information further includes service configuration version information;

[0033] The version library is also used to store a service configuration version list;

[0034] The service configuration version list is used to display the service configuration version information of one or more service configurations and the configuration modification records corresponding to any of the service configurations, so that the user can roll back to any version of the service configuration according to the service configuration version list.

[0035] Further, the version management background further includes a version alarm definition file storage module;

[0036] The version alarm definition file storage module is used to store a preset alarm definition file;

[0037] The version management client further includes an alarm operation module;

[0038] The alarm definition file is also used to send an alarm definition to the alarm operation module;

[0039] The alarm operation module is used to perform a version alarm according to the alarm definition.

[0040] Further, the version management client further includes a replica snapshot module;

[0041] The replica snapshot module is used to take a snapshot of any node in the cluster version management system and generate a replica snapshot version file;

[0042] The version library is also used to store the replica snapshot version file.

[0043] The embodiments of the present application have the following beneficial effects:

[0044] A cluster version management system based on Ambari provided by an embodiment of the present application incorporates version information in multiple dimensions, such as software stack version numbers, software service version list information, service configuration version information, alarm definition information, and replica snapshot version information, into version management. Users can freely perform version upgrades or rollbacks, and at the same time, an alarm function for versions is added, which can give alarm prompts in a timely manner for potential version problems or compatibility problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0046] Figure 1 Shows the version control schematic diagram of the cluster version management system based on Ambari provided by the embodiment of the present application;

[0047] Figure 2 Shows the structural schematic diagram of the cluster version management system based on Ambari according to an embodiment of the present application;

[0048] Figure 3 Shows the schematic diagram of the software version list according to an embodiment of the present application;

[0049] Figure 4 Shows the schematic diagram of the HDFS rolling mode upgrade process according to an embodiment of the present application;

[0050] Figure 5 Shows the schematic diagram of the service configuration version list according to an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0051] To make the objectives, technical solutions, and advantages of the present application clearer, the following will clearly and completely describe the technical solutions in the embodiments of the present application with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some, rather than all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art without creative efforts based on the embodiments of the present application belong to the scope of protection of the present application.

[0052] It should be understood that in the description of the present application, unless otherwise clearly required by the context, the words such as "include" and "comprise" and similar words in the whole specification and claims should be interpreted as the meaning of inclusion rather than exclusion or exhaustion; that is, the meaning of "including but not limited to".

[0053] It should also be understood that the terms "first", "second", etc. are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In addition, in the description of this application, unless otherwise specified, the meaning of "a plurality" is two or more.

[0054] This application provides a cluster version management system 100 based on Ambari. Referring to Figure 1 , the cluster version management system 100 includes: an Ambari-based version management client 110 and an Ambari-based version management background 120. Among them, the version management client 110 is used to submit version information to the version management background 120 and to pull the version information in the version management background 120 for version control; the version management background 120 is used to store version information.

[0055] Specifically, Ambari is a Web-based Hadoop distributed cluster configuration management tool that can support the provisioning, management, and monitoring of Apache Hadoop clusters and also support the RestAPI-style interaction method. Among them, Hadoop is a distributed system infrastructure developed by the Apache Foundation and is mainly used for developing distributed programs.

[0056] Specifically, the version information may include at least one of a software stack version number, a software service version list information, a service configuration version information, an alarm definition information, and a replica snapshot version information. It should be noted that after the system is deployed, a default software stack version will be automatically created, which contains all default software services and corresponding service versions. Therefore, the version information includes the default software stack version information and the software service version information.

[0057] Specifically, the Ambari-based cluster version management system 100 stores all historical versions of each dimension of the system. Ambari-Server can query all historical versions from it and perform version control. Among them, Ambari-Server is the server management node in Ambari and is used to process the Rest requests submitted to Ambari. Version control may include version upgrade or version rollback.

[0058] In some embodiments, the Ambari-based cluster version management system 100 further includes one or more Ambari Agents. Here, the Ambari Agent is a client node based on Ambari, which is used to execute instructions sent by the Ambari Server and also used to receive version information from the version management client 110 and perform version synchronization. The client program is installed in both the Ambari Server node and the Ambari Agent node for version control. Version control may include version upgrade or version rollback.

[0059] In some embodiments, referring to Figure 2 , the version management client 110 may include a version control client 111, and the version management backend 120 includes a version control server 121 and a repository 122.

[0060] Among them, the version control client 111 is used to interact with the version control server 121 for version information; the repository 122 is used to store version information for the version control server 121 to pull.

[0061] Specifically, the version control client 111 may submit version information (information in the above-mentioned multiple dimensions) to the repository 122 for saving version information. When version control is required, the corresponding version can be pulled from the repository 122 for version restoration. The user can also actively submit the latest version to the repository 122 through the version control client 111.

[0062] In some embodiments, the version control client 111 is also used to submit and save the software stack version number to the repository 122. The version control client 111 is also used to provide a software version list. Among them, the software version list is used to display the software stack version number and the corresponding software stack version name for the user to add, delete, change, and select the software stack version.

[0063] Specifically, the installation script of the system contains configuration files for the version numbers of each software service and the software stack version number. During installation and deployment, the version control client 111 will submit and save the above version numbers to the repository 122 as the default (initial) software stack version / software service version. The version control client 111 will regularly scan for the release of new software versions and synchronize them to the repository 122.

[0064] Specifically, referring to Figure 3 , the software version list also includes one or more software service version lists. Figure 3It shows a schematic diagram of a software version list, displaying a software stack version list for multi-version management and a software service version list therein. Each software service version list corresponds to a software stack version number; the software service version list is used to display the software service versions under the software stack version number. The default system has one software stack version (where each service is the initial default version), and users can customize other software stack versions, and for the software service versions therein, users can freely select from all existing versions.

[0065] In some embodiments, the software service version list can also be used to select software service versions for software service version control; wherein, the software service version control includes at least one of version upgrade or version rollback. The software service version control also includes at least one of a fast mode and a rolling mode. The fast mode is to stop the service to be controlled and perform version upgrade or version rollback on the service to be controlled; the rolling mode is to keep the service to be controlled running normally and perform version upgrade or version rollback on each part component of the service to be controlled in batches.

[0066] Specifically, from Figure 3 the software service version list in, the software service that needs to be version-upgraded or version-rolled back can be selected to upgrade to a new version or roll back to a previous version. Both the upgrade and rollback of the software service support the fast mode and the rolling mode.

[0067] Taking HDFS as an example below, referring to Figure 4 , the rolling mode version upgrade is described as follows:

[0068] In some embodiments, the rolling mode further includes creating a current NameNode and a current DataNode, and replacing the historical NameNode and the historical DataNode according to the current NameNode and the current DataNode; wherein, the NameNode is the master node and the DataNode is the slave node.

[0069] Specifically, HDFS is a distributed file system that stores a large amount of file content in a distributed manner across multiple machines. The NameNode, as the main control component of HDFS, manages the namespace of the file system and maintains all files and directories; the DataNode, as the slave component of HDFS, receives commands sent from the NameNode and copies block data to another machine. When creating a new NameNode and one or more new DataNodes, the historical versions of the NameNode and DataNodes are retained; the new DataNodes are incorporated into the new NameNode, and the historical old-version DataNodes are deleted. This process is repeated until all DataNodes are upgraded, and finally the historical NameNode is deleted to complete the rolling version upgrade. It should be noted that the upgrade or rollback of the software stack means the upgrade or rollback of all software services, which is to upgrade or roll back the versions of all services at once.

[0070] In some embodiments, the repository 122 is also used to store a service configuration version list. The service configuration version list is used to display the service configuration version information of one or more service configurations and the configuration modification records corresponding to any service configuration, so that users can roll back to any version of the service configuration according to the service configuration version list.

[0071] Specifically, referring to Figure 5 , after the configuration of the software service is modified and saved, a version file of the service configuration is generated, and then a service configuration version list is formed. The service configuration version list records all the configuration modification records (i.e., version records) of a certain service configuration. Users can roll back to the corresponding historical version according to any configuration modification record, or of course, roll back to the current version.

[0072] In some embodiments, referring to Figure 2 , the version management backend 120 further includes a version alarm definition file storage module 123. The version alarm definition file storage module 123 is used to store a preset alarm definition file. The version management client 110 further includes an alarm operation module 112. The alarm definition file is also used to send an alarm definition to the alarm operation module 112, and the alarm operation module 112 is used to execute a version alarm according to the alarm definition.

[0073] Specifically, after multiple versions of software services are introduced, version incompatibility may occur between services due to dependencies. The version management background 120 has pre-set an alarm definition file, which will be dynamically updated as the service version increases. Each node will run an instance of these alarm files. For example, the alarm definition will be sent to run in the Ambari Server to detect version compatibility and other issues. Among them, the alarm definition file has pre-set multiple version alarms, and users can also customize and edit the alarms according to actual needs. Before the service is installed, version compatibility will be detected according to the alarm definition. If compatibility issues are found, a version alarm prompt will be given, indicating possible service operation problems caused by compatibility.

[0074] In some embodiments, referring to Figure 2 , the version management client 110 further includes a replica snapshot module 113. The replica snapshot module 113 is used to take a snapshot of any node in the cluster version management system 100 to generate a replica snapshot version file, and the version library 122 is also used to store the replica snapshot version file.

[0075] Specifically, the replica can store the running status of all nodes in the system. Through the replica, the platform can be quickly restored to a certain historical state as a whole. The replica snapshot module 113 can take a snapshot of each node to generate a replica snapshot version file and incorporate it into the supported version management functions. When it is necessary to restore the replica version, the historical replica snapshot version file can be pulled from the version library 122. Similarly, the historical snapshot can be restored through the replica snapshot module 113.

[0076] In this embodiment, a cluster version management system is provided based on the Ambari big data management platform. Version information in multiple dimensions such as software stack version numbers, software service version list information, service configuration version information, alarm definition information, and replica snapshot version information is incorporated into version management. Users can freely perform version upgrades or rollbacks, and at the same time, an additional version alarm function is added, which can give alarm prompts for potential version problems or compatibility problems in a timely manner.

[0077] Although the preferred embodiments in the embodiments of the present application have been described, those skilled in the art can make additional changes and modifications once they know the basic creative concept. Therefore, the appended claims are intended to be construed as including the preferred embodiments and all changes and modifications falling within the scope of the embodiments of the present application.

[0078] Obviously, those skilled in the art can make various changes and variations to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalent technologies, the present application is also intended to include these modifications and variations.

Claims

1. A cluster version management system based on Ambari, characterized in that Including: An Ambari-based version management client and an Ambari-based version management backend; The version management client is used to submit version information to the version management backend and to pull the version information in the version management backend for version control. The version information includes any historical version of software services in the cluster in multiple dimensions, service configuration version information, and software stack version numbers; The version management backend includes a version control server and a version repository. The version repository is used to store the version information for the version control server to pull and to store a service configuration version list. Among them, the service configuration version list is used to display the service configuration version information of one or more service configurations and the configuration modification records corresponding to any of the service configurations, so that users can roll back to any version of the service configuration according to the service configuration version list; The version management client includes a version control client, and the version control client is used to provide a software version list. The software version list also includes one or more software service version lists, and each software service version list corresponds to a software stack version number; The version control client is also used to display the software service versions under the software stack version number, and to perform version control of the software services in the cluster by selecting the software service versions; The version control client is also used to periodically scan for new software stack version numbers and submit and save them to the version repository.

2. The Ambari-based cluster version management system according to claim 1, characterized in that, The version control client is used to interact with the version control server for the version information.

3. The Ambari-based cluster version management system according to claim 1, characterized in that The version management client also includes an Ambari-based server management node for processing Rest requests submitted to Ambari; The Ambari-based cluster version management system also includes one or more Ambari-based client nodes for executing instructions sent by the server management node; The client node is also used to receive the version information of the version management client and perform version synchronization.

4. The Ambari-based cluster version management system according to claim 2, wherein The software version list is used to display the software stack version number and the corresponding software stack version name, so that users can add, delete, change, and select software stack versions.

5. The Ambari-based cluster version management system according to claim 4, characterized in that, The software service version list is also used to select the software service version for software service version control; The software service version control includes at least one of version upgrade or version rollback; The software service version control also includes at least one of a quick mode and a rolling mode; The quick mode is to stop the service to be controlled and perform the version upgrade or the version rollback on the service to be controlled; The rolling mode is to keep the service to be controlled running normally and perform the version upgrade or the version rollback on each part of the components in the service to be controlled in batches.

6. The Ambari-based cluster version management system according to claim 5, wherein The rolling mode also includes creating a current master node and a current slave node, and replacing the historical master node and the historical slave node according to the current master node and the current slave node.

7. The Ambari-based cluster version management system according to claim 2, wherein The version management backend also includes a version alarm definition file storage module; The version alarm definition file storage module is used to store a pre-set alarm definition file; The version management client further includes an alarm operation module; The alarm definition file is further used to send an alarm definition to the alarm operation module; The alarm operation module is used to execute a version alarm according to the alarm definition.

8. The Ambari-based cluster version management system according to claim 2, wherein The version management client further includes a replica snapshot module; The replica snapshot module is used to take a snapshot of any node in the cluster version management system to generate a replica snapshot version file; The version library is further used to store the replica snapshot version file.

Images