AI-driven configuration management

Abstract

Techniques for artificial intelligence-driven configuration management are described herein. In some embodiments, a machine learning process determines a feature set for multiple deployments of a software resource. Based on a change value in the feature set, the process clusters each of the multiple deployments into a cluster in a plurality of clusters. Each of the multiple clusters includes one or more nodes, and each of the one or more nodes corresponds to at least a subset of the values ​​of the feature set detected in at least one of the multiple deployments of the software resource. The process determines a representative node for each of the multiple clusters. Operations may be performed based on a representative node of at least one cluster.

Description

[0001] Statement of Interest; Related Applications; Incorporation by Reference

[0002] This application claims the benefit of U.S. Non-Provisional Patent Application No. 16 / 042,971, filed on July 23, 2018, which further claims the benefit of U.S. Provisional Patent Application No. 62 / 566,235, filed on September 29, 2017, which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to managing the configuration of deployed software resources. Summary of the invention

[0004] Complex software systems are often difficult to manage due to the large number of possible combinations of configuration settings. For example, different deployments of the same software resource may have different combinations of language settings, security settings, patch set updates (PSUs), and network settings, among other configuration settings. Failure to properly manage deployment configurations can expose the system to a variety of problems, including security vulnerabilities and performance degradation.

[0005] One approach to managing software configurations is to manually create a gold image. According to this approach, a system administrator selects the best configuration settings for a set of software deployments and creates a software image (the "golden" image) that contains those configuration settings. The software deployments are then compared to the gold image to identify configuration differences. This approach works well when the system administrator knows the best configuration for each software resource. However, in large and growing environments, the best configuration may differ between different deployments of the same software resource, and the current best configuration may not be the best configuration in the future. System administrators may have limited domain knowledge and bandwidth to determine which configuration settings are optimal and which are problematic. Therefore, this approach does not scale well to cloud-based applications and other large-scale systems.

[0006] The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise indicated, any approach described in this section should not be construed to qualify as prior art merely by virtue of its inclusion in this section. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] In the figures of the accompanying drawings, embodiments are illustrated by way of example and not limitation. It should be noted that references to "an embodiment" or "one embodiment" in this disclosure do not necessarily refer to the same embodiment, and they mean at least one. In the drawings:

[0008] Figure 1 illustrates an example system architecture for utilizing machine learning to drive configuration management in accordance with some embodiments;

[0009] Figure 2 illustrates an example set of operations for managing deployment of software resources according to some embodiments;

[0010] Figure 3 illustrates an example data flow graph for generating an annotated cluster set according to some embodiments;

[0011] Figure 4 illustrates an example set of operations for extracting a set of features for clustering deployments of software resources in accordance with some embodiments;

[0012] Figure 5 illustrates an example set of operations for clustering a set of deployments of software resources according to some embodiments;

[0013] Figure 6 illustrates an example set of operations for aggregating a set of clusters according to some embodiments;

[0014] Figure 7 illustrates an example interactive visualization of a set of clusters according to some embodiments;

[0015] Figure 8 illustrates an example interactive visualization for a selected cluster according to some embodiments;

[0016] Fig. 9 illustrates an example summary of a set of clusters according to some embodiments;

[0017] Fig.10 illustrates an example process for correcting drift based on a clustered set of deployments according to some embodiments; and

[0018] Fig.11 A block diagram illustrating a computer system according to some embodiments is shown. DETAILED DESCRIPTION

[0019] In the following description, for the purpose of explanation, many specific details are set forth in order to provide a thorough understanding. Some embodiments may be practiced without these specific details. Features described in one embodiment may be combined with features described in another embodiment. In some examples, well-known structures and devices are described with reference to block diagram form to avoid unnecessary confusion of the present invention.

[0020] 1. General Overview

[0021] 2. System Architecture

[0022] 3. AI-driven deployment clustering

[0023] 3.1 Feature set extraction

[0024] 3.2 Cluster Generation

[0025] 3.3 Clustering summary and representative nodes

[0026] 3.4 Clustering of different feature sets

[0027] 3.5 Scalable In-Memory Clustering Model

[0028] 4. Interactive interface for configuration management

[0029] 4.1 Clustering Visualization and Interface

[0030] 4.2 Hypothetical clustering and representative node changes

[0031] 4.3 Summary Visualization and Drill-Down Operation

[0032] 4.4 Drag and Drop Interface

[0033] 5. Configuration management operations

[0034] 5.1 Initialize a new node

[0035] 5.2 Update nodes and correct drift

[0036] 5.3 Golden Image Subscription

[0037] 5.4 Analysis Operation

[0038] 5.5 Troubleshooting Operations

[0039] 6. Computer Networks and Cloud Networks

[0040] 7. Microservice Application

[0041] 8. Hardware Overview

[0042] 9. Other Matters; Extension

[0043] 1. General Overview

[0044] Techniques for artificial intelligence-driven configuration management are described herein. In some embodiments, a machine learning process determines a feature set for multiple deployments of a software resource. Based on a change value in the feature set, the process clusters each of the multiple deployments into a cluster in a plurality of clusters. Each of the multiple clusters includes one or more nodes, and each of the one or more nodes corresponds to at least a subset of the values ​​of the feature set detected in at least one of the multiple deployments of the software resource. The process determines a representative node for each of the multiple clusters. Operations may be performed based on a representative node of at least one cluster.

[0045] 2. System Architecture

[0046] Figure 1 An example system architecture for utilizing machine learning to drive configuration management is illustrated in accordance with some embodiments. Figure 1 As shown, the system 100 includes targets 102a-i, collection agents 104a-j, analysis services 106, data repositories 118, and clients 128a-k. In some embodiments, the system 100 may include Figure 1 The components shown may be greater or less than the components shown. Figure 1 The components shown may be local to each other or remote from each other. Figure 1 The components shown can be implemented in software and / or hardware. Each component can be distributed across multiple applications and / or machines. Multiple components can be combined into one application and / or machine. Operations described with respect to one component can be performed by another component instead.

[0047] Targets 102a-i correspond to hardware and / or software components used to support the deployment of specific resources. Example hardware targets may include host machines, server devices, network adapters, and storage drivers. Example software targets may include listeners, load balancers, virtual machines, middleware, and application instances. In this context, a software resource or system may be a multi-layer software application that runs using various hardware and software components. For example, a multi-layer application may include a presentation layer for connecting to one or more user interfaces, an application layer for executing application-specific logic, and a data layer for storing and retrieving data. In other cases, a software resource may be an application or other software component within a single layer of a multi-layer application. For example, a software resource may correspond to a database management system executed in a data layer of a multi-layer application. In other embodiments, a software resource may be an independent application or other software component. For example, a software resource may be an independent desktop application. Therefore, the artificial intelligence-driven management technology described herein may be applied to a wide variety of software resources.

[0048] The collection agents 104a-j collect deployment parameters from the targets 102a-i and provide the deployment parameters to the analysis service 106 for subsequent processing. One or more of the collection agents 104a-j may be a daemon or background process that executes on the same host as the target. Additionally or alternatively, one or more of the collection agents 104a-j may be located on a remote host remote from the target, where the agent is configured to collect parameters for the target. Although one collection agent is depicted per target, the collection agent may collect parameters from multiple targets or from a single target depending on the implementation.

[0049] The deployment parameters collected may vary depending on your implementation. Example parameters include one or more of the following:

[0050] Configuration settings: These parameters identify the configuration of the deployment of the software resource. For example, these parameters may identify which patches have been applied to the deployment, security settings (such as firewall and open network port configuration), language settings (such as which language has been selected at boot time), operating system (OS) settings (such as the OS name and version on the host machine on which the software resource is configured to execute), and log settings (such as maximum log size and rate).

[0051] • Static attributes: These parameters identify characteristics of the deployment that do not change during runtime. For example, static attributes may encode information about the software binaries or code footprint used for the deployment.

[0052] Runtime attributes: These parameters identify characteristics of the deployment that are captured during runtime. For example, parameters can capture runtime performance attributes such as average / minimum / maximum CPU performance, memory bandwidth, active sessions, and I / O throughput. These parameters can also capture error reports, security vulnerabilities, log write throughput, and other information that is not statically available.

[0053] System Architecture: These parameters identify the structure and relationships of the components used to run the software resource. For example, the attributes may identify information about the target host(s) on which the software resource runs, such as geographic location, host name, and IP address. In another example, these parameters may capture topology / architecture information, such as dependencies between different targets supporting the deployment of the software resource. In yet another example, these parameters may identify a list of available services supported by the deployment of the software resource.

[0054] The analysis service 106 includes a set of applications and / or network services that can be called to perform clustering and management operations as further described herein. The analysis service 106 includes collection logic 108, feature extractor 110, cluster generator 112, cluster analyzer 114, and console 116. Each service implements a function or set of functions, as follows:

[0055] • Collection logic 108 interfaces with collection agents 104a-j to collect deployment parameters for multiple deployments of software resources.

[0056] A feature extractor 110 extracts features from deployment parameters for clustering deployments of software resources;

[0057] • A cluster generator 112 clusters deployment parameters based on values ​​of feature sets detected in the plurality of deployments of software resources.

[0058] Cluster analyzer 114 summarizes / annotates clusters based on their membership; and

[0059] The console 116 provides an interface through which one or more functions of the analysis service 106 can be invoked.

[0060] The set of functions summarized above can be implemented by a single service or by different services. Additionally or alternatively, depending on the specific implementation, the functions shown as being performed by a single service can be performed by different services. One or more services can be encapsulated into a microservice application and / or a cloud application, such as a software as a service (SaaS) application.

[0061] In an embodiment, the analysis service 106 is implemented on one or more digital devices. The term "digital device" generally refers to any hardware device including a processor. A digital device may refer to a physical device that executes an application or a virtual machine. Examples of digital devices include computers, tablet computers, laptops, desktops, netbooks, servers, web servers, network policy servers, proxy servers, general-purpose computers, function-specific hardware devices, hardware routers, hardware switches, hardware firewalls, hardware firewalls, hardware network address translators (NATs), hardware load balancers, mainframes, televisions, content receivers, set-top boxes, printers, mobile handsets, smart phones, personal digital assistants ("PDAs"), wireless receivers and / or transmitters, base stations, communication management equipment, routers, switches, controllers, access points, and / or client devices.

[0062] The data repository 118 includes volatile and / or non-volatile storage for storing data generated and / or used by the analysis service 106. The stored data sets include:

[0063] Collected attributes 120 , which include deployment parameters collected by the collection logic 108 ;

[0064] Feature vectors 122, which include feature vectors encoding different instances of the feature set detected by feature extractor 110;

[0065] The results of the clustering 124, which group the deployments / instances of the feature sets into clusters; and

[0066] • Interface objects 126, which include charts, graphs, annotations, summaries, and / or other elements to be presented to the user.

[0067] In some embodiments, data repository 118 is any type of storage unit and / or device for storing data (e.g., a file system, a database, a collection of tables, or any other storage mechanism). In addition, data repository 118 may include multiple different storage units and / or devices. Multiple different storage units and / or devices may or may not be of the same type or may or may not be located at the same physical site. In addition, data repository 118 may be implemented or executed on the same computing system as one or more other components of system 100 and / or on a completely independent computing system. Data repository 118 may be coupled to analysis service 106 via a direct connection or via a network communication.

[0068] Clients 128a-k represent one or more clients that can access analysis service 106 to perform clustering and management operations as further described herein. In this context, a "client" can be a human user, such as an administrator, a client program, or some other application instance. The client can be executed locally on the same host as analysis service 106, or can be executed on a different machine. If executed on a different machine, the client can communicate with analysis service 106 via one or more data communication protocols according to a client-server model, such as by submitting an HTTP request to call one or more services and receiving an HTTP response including the results generated by the one or more called services.

[0069] The console 116 may provide an interface to the clients 128a-k through which one or more of the provided services may be invoked. Examples of interfaces include a graphical user interface (GUI), a command line interface (CLI), an application programming interface (API), a tactile interface, and a voice command interface. Examples of user interface elements include check boxes, radio buttons, drop-down lists, list boxes, buttons, toggle switches, text fields, date and time selectors, command lines, sliders, pages, and tables.

[0070] Additional embodiments related to computer networks and microservice applications are described below in Section 6 entitled "Computer Networks and Cloud Networks" and Section 7 entitled "Microservice Applications", respectively.

[0071] 3. AI-driven deployment clustering

[0072] In some embodiments, analysis service 106 is configured to perform unsupervised cluster analysis on deployment parameters collected from multiple deployments of software resources. Clusters may be formed based on learned patterns in deployment parameters between different configurations of software resources. These patterns may be hidden or otherwise not so obvious, making it extremely expensive to determine the inferences to be made from the raw deployment data.

[0073] Figure 2 An example set of operations for managing deployment of software resources according to some embodiments is illustrated. Additional examples and embodiments of each of these operations are provided in further detail in subsections 3.1 to 3.4 below. The set of operations includes collecting parameters for multiple deployments of software resources (operation 202). For example, the collection logic 108 can collect configuration settings, static properties, runtime properties, and / or system architecture data from the deployment of a particular software application or system.

[0074] The set of operations also includes determining a feature set for the deployment of the software resource (operation 204). The feature set used to cluster the deployment may vary from implementation to implementation. In some embodiments, a combination of collected deployment parameters is used. For example, the feature set may include one or more parameters collected at operation 202, such as one or more configuration settings, static properties, runtime properties, and / or system architecture properties. As described in further detail in Section 3.4, different feature sets may be used to generate different clusters. In other embodiments, clustering may be performed on a single feature set.

[0075] The manner in which parameters are selected for a feature set may vary depending on the specific implementation. Parameters may be selected in an unsupervised or supervised manner. For example, parameters may be selected by an algorithm, such as a machine learning model that trains a correlation model using a supervised or unsupervised process. An example correlation-based machine learning model is described in U.S. Application No. 15 / 155,486, entitled “CORRELATION-BASED ANALYTIC FOR TIME-SERIESDATA,” the entire contents of which are incorporated herein by reference as if fully set forth herein. As another example, parameters may be automatically selected based on a set of filtering criteria, which may be predefined by default or selected by a user. Parameters that meet the filtering criteria are retained in the feature set, while parameters that do not meet the filtering criteria are removed. In yet another example, parameters may be curated by an external user / client. The organized parameter set may be stored in a list or similar data structure and matched with the collected parameters.

[0076] Reference again Figure 2 , the set of operations also includes generating clusters of nodes based on the changed values ​​of the feature set detected in the collected deployment parameters (operation 206). The values ​​in the feature set can be determined from the collected deployment parameters. In general, deployments are clustered based on similarity of values ​​of the feature set (such as configuration settings, runtime metrics, etc.). That is, a deployment is more likely to be assigned to the same cluster as another deployment with similar values ​​of the feature set than to another deployment with different values ​​of the feature set.

[0077] The set of operations also includes determining a representative node in each cluster (operation 208). The representative node of the cluster can be selected from an existing node, or can be constructed as a new node of the cluster. In some embodiments, the representative node corresponds to a pattern of the cluster. In this context, a pattern is a node with the most frequently occurring value combination in a feature set in a deployment assigned to a cluster.

[0078] Determining the pattern may involve further excluding certain features from the set of features that are not relevant to the intended purpose of the representative node. As an example, if a feature does not affect and / or is not affected by an upgrade or patch operation (which may be determined by looking at the features that are the subject of the upgrade or patch operation), then the feature may be excluded. As another example, if a feature is not relevant to the function being analyzed (which may be determined based on whether the value exists in some systems that have the function but not in other systems that do not have the function), then the feature may be excluded. The result is a pattern that determines a relevant subset of features.

[0079] In other embodiments, the representative node may be constructed or selected in some other manner, such as by calculating the mean of the values ​​in the feature set or selecting the last known configuration of the cluster, such as the latest version or date of an upgrade or patch present in the cluster. For example, the feature value of the representative node may be calculated as the average across deployments, which may or may not have been detected in one of the deployments assigned to the cluster.

[0080] Figure 3 An example data flow diagram for generating an annotated set of clusters according to some embodiments is illustrated. The feature extractor 110 receives a set of deployment parameters denoted as D for multiple deployments of a software resource. The data point set {x 1,1 , …, x i,j} may include parameter values ​​for different deployment parameters. For example, the values ​​may identify configuration settings, runtime metrics, topology information, etc. for the corresponding deployment. The feature extractor 110 also receives a feature set, denoted as F, which identifies features that should be used for clustering. Based on the feature set F and the deployment parameters, the feature extractor 110 creates a feature vector, denoted as FV, ​​for each software deployment. The cluster generator 112 receives the feature vector and generates a set of clusters, denoted as C. Each cluster in C includes one or more nodes that correspond to at least a subset of the values ​​in the feature set F. The cluster analyzer 114 determines a representative node for each cluster, which is identified as R in the data set.

[0081] 3.1 Feature set extraction

[0082] A feature set refers to a set of attributes or parameters that drive the clustering process. A feature set may include any combination of parameters collected from software deployments. Additionally or alternatively, a feature set may include attributes calculated from the collected deployment parameters (referred to herein as "calculated features"). For example, a feature set for clustering database instances may include the following three raw features:

[0083] CPU count, which identifies the number of CPUs available to the database instance.

[0084] Memory allocated to the database instance; and

[0085] The storage throughput of the database instance.

[0086] The feature set can also include the following two calculated features:

[0087] a parameter referred to herein as "count" that is assigned a value identifying the number of configuration settings that have been changed from the default setting; and

[0088] • A parameter referred to herein as "_count", which is assigned a value identifying the number of undocumented configuration settings utilized.

[0089] The above-described feature set is useful for clustering database instances based on performance metrics and configuration settings. However, depending on the specific implementation, the combination of features and the number of features used may vary.

[0090] In some cases, the feature set may include all collected deployment parameters (if any) in addition to the calculated features. However, many parameters may not be helpful for clustering. For example, if all deployments run on the same version of the software resource, or the administrator is not interested in using the version number as a distinguishing feature, then clustering based on the version number may not convey useful information. However, for other applications, clustering based on version numbers as well as other features may be useful, such as for comparing performance metrics between different versions of software resources. The feature extractor 110 can extract features for clustering and filter / remove unused parameters. As previously described, the feature extractor 110 can perform extraction based on a set of filtering criteria and / or through an algorithm based on injected domain knowledge, such as an organization's parameter set.

[0091] Figure 4 An example set of operations for extracting a set of features for clustering deployments of software resources according to some embodiments is illustrated. The set of operations includes identifying a set of parameters collected from a target software deployment (operation 402).

[0092] The set of operations also includes determining a feature set based on filtering criteria and / or organization (operation 404). Example filtering criteria may include cardinality of the parameter. For example, parameters that are common between all instances or have a cardinality exceeding a threshold may be removed. In the latter case, for example, if the features are common between more than one hundred deployed resources, more than 80% of deployed resources, or any other threshold, then these features may be removed. This filter will remove parameters that have little variation between different software deployments.

[0093] Additionally or alternatively, other filtering criteria may be used to remove parameters from the feature set. For example, the feature extractor 110 may detect and stem predefined parameters, such as version numbers and / or other irrelevant parameters.

[0094] In some embodiments, the parameter may be analyzed to determine whether it satisfies a filter. For example, the parameter may be compared to a patch information set to determine whether the parameter is the subject of an upgrade operation or a patch operation. If not, the parameter may be filtered out from the feature set.

[0095] Additionally or alternatively, parameters may be selected based on organization. For example, a system administrator or other user may provide a list of parameters of interest. Only features on the parameter list are then included in the feature set. As another example, a system administrator or other user may provide a list of parameters that should be excluded. Features on the list will be excluded from the feature set, while features not on the parameter list may or may not be included in the feature set. Feature organization allows domain knowledge to be injected into the process to customize the feature extraction process.

[0096] Additionally or alternatively, features within a feature set can be calculated features that are calculated based on one or more collected parameters. For example, a feature can be created by counting the number of non-default parameter settings (such as configuration values ​​that have been changed from a default state by a user). A higher value can reflect a greater number of changes from a default state. As another example, a feature can be created by counting the number of undocumented parameter sets. The value can be incremented for each customized patch or other add-on applied to a software deployment. In yet another example, performance metrics can be aggregated or otherwise combined.

[0097] The set of operations also includes creating a feature vector that includes the values ​​of the feature set detected in the collected parameters (operation 406). A unique combination of values ​​for a feature set is referred to herein as an "instance" of the feature set. In the example where the feature set is {CPU count, allocated memory, memory throughput, count, _count}, an example feature vector for an instance of the feature set may be {320, 8GB, 97%, 30, 25}. However, these values ​​may vary between different deployments and feature vectors.

[0098] The set of operations also includes rolling up common feature vectors (operation 408). For example, if two or more deployments share the same value for a feature set or a related subset of a feature set, then the feature vectors may be rolled up into one. A count value may be maintained that identifies how many deployments share the feature value encoded by the feature vector. Additionally or alternatively, a mapping between the feature vectors and the deployments represented by the feature vectors may be maintained.

[0099] 3.2 Cluster Generation

[0100] Once the cluster features are extracted, the cluster generator 112 runs a cluster analysis that produces clusters of target resources, such as software deployments. Figure 5 An example set of operations for clustering a set of deployments of software resources according to some embodiments is illustrated. The set of operations includes receiving a clustering feature set (operation 502). The feature set may be extracted as described in the previous section and provided to the cluster generator 112.

[0101] The operation set also includes dividing the feature vector into multiple clusters (operation 504). The clustering method used may vary depending on the implementation. Examples may include, but are not limited to, k-mode clustering and k-means clustering. For example, using k-mode clustering, the clustering process can randomly select k unique feature vectors as initial cluster centers. Then, the clustering process can calculate the distance between each feature vector and the clustering pattern, and assign the feature vector to the cluster whose center has the shortest distance to the feature vector. Then a new pattern can be selected for each cluster and compared with the previous pattern. If different, the object can be reassigned. Otherwise, clustering can stop.

[0102] The set of operations also includes determining an optimal number of clusters for software deployment (operation 506). In some embodiments, the optimal number of clusters is determined by performing operation 504 for multiple values ​​of k, where k is the total number of clusters. For example, the feature vector can be divided into two clusters, three clusters, and so on, up to a maximum threshold. The gap statistics can then be used to select the optimal k. According to this technique, one or more random data sets are clustered and compared to the results of clustering the actual data set. The number of clusters k is then selected to maximize the gap statistics, which is defined as:

[0103]

[0104] in represents the expectation from a random distribution with a sample size of n, and W k is the pooled within-cluster sum of squares around the cluster mode or mean.

[0105] The set of operations also includes retaining the cluster with the best number of clusters (operation 508). For example, if the best number of k is 3, then three clusters are retained as clusters generated by k-modes or k-means clustering.

[0106] As previously mentioned, feature vectors can be assigned to clusters based on distance. The distance function used can vary depending on the implementation. In some embodiments, Euclidean distance is used to assign feature vectors to clusters. In other cases, features can be associated with weights. For example, features that are considered to be more important differences can be given greater weights in distance calculations than less important features. In other cases, all features in the feature set can be weighted equally.

[0107] The result of the clustering operation is a set of clusters containing one or more nodes. Each node in the cluster corresponds to a feature vector storing the value of a feature set detected in one or more software deployments. For example, the feature vector {320, 8GB, 97%, 30, 25} can be an instance of the feature set {CPU count, allocated memory, memory throughput, count, _count}. This feature vector (which is an instance of the above feature set) may have been detected in a single deployment or in multiple deployments. In the latter case, assigning the feature vector to a cluster effectively assigns multiple deployments with corresponding feature values ​​to the cluster.

[0108] 3.3 Clustering summary and representative nodes

[0109] In some embodiments, cluster analyzer 114 is configured to generate a summary for each cluster. The summary may contain information about hidden patterns or other information about the cluster that may be useful for configuration management. For example, the summary may identify the number of different configurations of software deployments that have been assigned to the cluster, the size of the dispersion / differences between clusters, and / or common configuration settings among the clusters.

[0110] Additionally or alternatively, the summary can identify a representative node for the cluster. In the context of configuration settings, a representative node can be selected as a set of the most frequently detected configuration values ​​assigned to the deployment of the cluster. A representative node can be selected as a gold / recommended configuration level for other nodes assigned to the cluster. In other contexts, a representative node can identify the most common security vulnerabilities, errors, performance metrics, or other deployment parameters detected in software deployments. A representative node can allow a user to quickly understand and visualize configuration drift and / or other potential problems in a large number of software resource deployments.

[0111] Figure 6 An example set of operations for summarizing a set of clusters according to some embodiments is illustrated. The set of operations includes selecting a cluster (operation 602). The clusters may be analyzed in any order. For example, the cluster analyzer 114 may select the first cluster c1 from the data set C, or start with any other cluster in the data set.

[0112] The set of operations also includes determining different instances / feature vectors of the feature set within the cluster (operation 604). For example, in the context of a feature set comprising a combination of configuration settings, the operation may involve determining how many unique combinations of values ​​of the configuration settings there are in the cluster. The number corresponds to the number of unique configurations of the feature set for the software deployment in the cluster.

[0113] The set of operations also includes determining a representative node for the cluster based on the instances of the feature set in the cluster (operation 606). As previously described, the representative node can be selected as the node in the cluster corresponding to the largest number of software deployments. In other embodiments, the node can be constructed as a new node rather than selected from existing nodes. For example, the new node can be constructed by averaging the values ​​in the feature set between multiple nodes in the cluster.

[0114] The set of operations also includes generating a hierarchical representation of the cluster (operation 608). In some embodiments, the hierarchical representation is a graph calculated based on the differences of each non-representative node in the cluster. The graph can be recursively constructed to describe the structure of how the configuration is different from the representative node. The hierarchical graph can identify the complete path to each different configuration / state of the software deployment in the cluster. The hierarchical graph can also identify any common intermediate observed configurations / states. The graph can provide a convenient interface for up-levelling. By having intermediate states, the graph can be recursively collapsed or expanded based on the interests and preferences of the user.

[0115] Once the summary has been generated, the process determines whether there are any remaining clusters to be analyzed (operation 610). If so, the process is repeated for the next selected cluster. If there are no remaining clusters to be summarized, the process ends.

[0116] 3.4 Clustering of different feature sets

[0117] In some cases, software deployments may be clustered multiple times using different clustering feature sets. For example, a cluster set may be generated based on a feature set that includes various security parameters. Another cluster set may be generated using a feature set that includes performance features. Yet another cluster set may focus on patch set updates. Thus, different cluster sets may be consistent with different interests.

[0118] In some embodiments, feature sets can be grouped into categories, which can include user-defined categories and / or system-defined categories. Different categories can be associated with different feature sets. For example, performance categories can be defined to cluster target resources using performance attribute sets, which can be defined by system 100 or users. For example, performance attributes can include CPU utilization, network throughput and / or memory utilization. Additionally or alternatively, security categories can be defined as combinations including security parameters, such as patch levels, open ports and firewall settings. Additionally or alternatively, categories can be defined to analyze the various layers of multi-layer applications. For example, a presentation layer category can be defined as a combination of attributes associated with the presentation layer of a target deployment, such as web server host configuration settings and web server performance information. Similar categories can be defined for application layer, data layer and / or any other deployment parameter set.

[0119] When multiple categories are defined, the user can select the category on which clustering is to be performed. For example, if the user selects the performance category, a set of feature vectors can be created using a combination of performance attributes defined for that category. Clustering can then be run to group software deployments that perform similarly. The user can then select another category, such as security, to create a feature vector using a combination of security attributes. Clustering can then be run using the newly created feature vector to group software deployments with similar security settings. Additionally or alternatively, clustering can be run based on feature vectors created from other combinations of attributes. Clustering using different feature vectors can be performed sequentially or in parallel.

[0120] In the examples given above, software deployments can be assigned to different sets of clusters across different categories. For example, when clustering performance attributes, a software deployment can be assigned to one set of clusters that contains one or more other software deployments. When clustering different sets of attributes, a software deployment can be assigned to another set of clusters that contains different groups of software deployments.

[0121] In some embodiments, software deployment can be linked across different clustering sets. For example, hyperlinks and / or references can be maintained that allow users to navigate between different clustering sets to which software deployment is assigned. A user can select a link associated with a software deployment to jump to and / or view another clustering set. For example, a clustering set generated according to performance attributes and a clustering set generated according to security attributes can be presented to the user. A user can click or otherwise select a link representing a software deployment in a performance clustering set to view the location to which the software deployment is assigned in a security clustering set. The clustering set of interest can be annotated, amplified, or otherwise highlighted for easy identification. Therefore, a user can easily determine and visualize how a specific software deployment is consistent with other deployments along several feature set dimensions.

[0122] 3.5 Scalable In-Memory Clustering Model

[0123] One approach to modeling clustering in memory is to create data points representing individual target resources and run a clustering process on these data points. A data point for a given target resource may store or be linked to the configuration settings of the target resource. A feature vector may be created from the configuration settings of the target resource and compared to the cluster centroids to compute distances and select clusters for the data points. Since each data point is independently assigned to a cluster, this approach may involve maintaining a list of configuration settings for each target resource to be clustered, which may be difficult to scale. Large data centers and cloud environments may have tens of thousands of target resources. Therefore, the amount of memory overhead for maintaining a complete list of configuration settings may be significant.

[0124] An alternative approach to reducing memory usage is to replace the list of entity configurations with different configurations. In this approach, redundant configurations are discarded and only different configurations are kept in memory. The different configurations can then be mapped to a list of entities with those different configurations. Thus, the number of entity configurations can be significantly compressed, which can significantly reduce the memory usage of the clustering model.

[0125] The memory savings of replacing the entity configuration list with different configurations is illustrated in the following example. The following raw sample can be provided as input to the clustering model:

[0126] Entity 1: "param_a":1, "param_b":2, "param_c":3, "param_d":4

[0127] Entity 2: "param_a":1, "param_b":20, "param_c":30, "param_d":4

[0128] Entity 3: "param_a":1, "param_b":2, "param_c":3, "param_d":4

[0129] Entity 4: "param_a":10, "param_b":2, "param_c":3, "param_d":42

[0130] Entity 5: "param_a":1, "param_b":20, "param_c":30, "param_d":4

[0131] From the raw input above, three different configuration settings are detected, which can be represented as follows:

[0132] ·State 1: "param_a":1, "param_b":2, "param_c":3, "param_d":4, "entities":[1,3]

[0133] ·State 2: "param_a": 1, "param_b": 20, "param_c": 30, "param_d": 4, "entities": [2,5]

[0134] ·State 3: "param_a": 10, "param_b": 2, "param_c": 3, "param_d": 42, "entities": [4]

[0135] Each different configuration is mapped to one or more target resources that share the specified configuration settings. As can be seen, the list of five configuration settings in the original data is reduced to three different configuration settings. When there are hundreds or more inputs, the cost savings can be much greater.

[0136] Scalability can also be improved by enumerating states between parameters used for clustering that correspond to the parameters selected as part of the feature set. For example, clustering can be performed on the parameters "param_a" and "param_d" in the collected configuration settings listed above. Therefore, the space can be collapsed into the following two feature vector instances:

[0137] Vector 1: "param_a":1, "param_d":4

[0138] Vector 2: "param_a":10, "param_d":42

[0139] The different feature vector instances can then be clustered as previously described. Once clustering is complete, the states and entities can be assigned to the corresponding feature vectors. For example, state 1 and state 2 and the corresponding entities [1, 2, 3, 5] are assigned to vector 1. State 3 and the corresponding entity [4] are assigned to vector 2. Differences can be calculated based on param_b and param_c of entities associated with the same feature vector.

[0140] 4. Interactive interface for configuration management

[0141] One or more interactive interfaces may be generated based on the summarized clustering set. The interactive interface may allow a user to visualize, navigate, and otherwise access summary information about the clustering. Additionally or alternatively, the interactive interface may provide the user with options for performing one or more management operations based on the clustering results / summary. An example that may be initiated through the interactive interface provided herein is described in Section 5 entitled "Configuration Management".

[0142] Subsections 4.1 to 4.4 provide examples of interactive interfaces and visualizations that can be generated from the result set of clustering. Interactive interfaces and visualizations can facilitate configuration management in various ways. For example, an interactive interface can allow configuration drift between similar target resources to be quickly identified and efficiently corrected through intuitive visualizations. As another example, an interactive interface can allow security vulnerabilities and performance degradation to be quickly identified and resolved. The example interfaces and visualizations described below are provided for illustrative purposes only. The presentation and format of the interactive interfaces and visualizations may vary from implementation to implementation.

[0143] 4.1 Clustering Visualization and Interface

[0144] In some embodiments, the cluster visualization includes a graphical representation of a set of nodes and edges. The graphical representation can be generated based on a hierarchical graph generated for the results of clustering. The nodes in the cluster visualization can represent unique instances of one or more target resources and feature vectors. Edges connect different nodes that have been assigned to the same cluster. For example, Figure 7 An example interactive visualization of a set of clusters 700 is illustrated. Interactive visualization 700 displays Cluster 0, Cluster 1, Cluster 2, and Cluster 3. Each cluster includes a plurality of nodes or vertices corresponding to different instances of a cluster feature set.

[0145] In some embodiments, a representative node is selected for a cluster. For example, the most frequently occurring instance of a feature vector may be selected as a representative node, or a representative node may be constructed as previously described. The representative node may correspond to a golden image or recommended configuration for a deployment assigned to the cluster. For example, reference Figure 7 , the golden image representations of clusters 0-3 are nodes 702, 704, 706, and 708, respectively. The representative nodes can be visually highlighted for easy identification. For example, the representative nodes can be displayed in a color separate from the other nodes in the cluster. Additionally or alternatively, the representative nodes can be displayed and / or annotated in a larger size.

[0146] In some embodiments, the size of the nodes / vertices in the graph is displayed based on the number of software deployments having configuration settings and / or other values ​​corresponding to the feature set of the node. For example, the size of the node can be proportional to the logarithm of the number of software deployments. Thus, nodes corresponding to a greater number of software deployments can have a larger display size / area than nodes corresponding to fewer deployments. The representative node can have the largest display size of the nodes in the cluster.

[0147] In some embodiments, nodes belonging to the same cluster are connected by edges, which can be represented as lines on a graph. For example, the representative nodes of each cluster (nodes 702, 704, 706, and 708) are each connected to multiple other nodes representing different instances of feature vectors. These nodes can also be connected to other nodes by edges, thereby supporting hypothetical clusters, as described in Section 4.2.

[0148] Can be selected based on display resolution Figure 7 The edge length between the nodes in the cluster may be increased to fit the display. In other embodiments, the edge length may encode the distance between the vectors representing the two nodes. For example, the edge size may increase in proportion to the Euclidean distance between the two nodes. Thus, relatively similar nodes are displayed closer together than nodes that are relatively different relative to other nodes in the cluster.

[0149] In some embodiments, the nodes of clustering are interactive, thereby allowing users to quickly access information about deployment sets and initiate management operations. For example, a user can click or otherwise select a node, such as node 704, to view information about the deployment associated with the node. The example information that can be presented can include configuration settings or other feature values ​​represented by the node, the number of deployments assigned to the node, one or more geographical locations deployed in the node, and / or attributes that are common to the deployment associated with the node. As another example, when selecting a node, a list of target deployments and the collected configuration settings for each target deployment can be presented to the user. In yet another example, an available set of management operations can be provided to the user when selecting a node. Then, the user can select the management operations to be performed relative to the selected node. Example management operations are further described below in Section 5 entitled "Configuration Management".

[0150] Additionally or alternatively, the edge between two nodes in the graph can be interactive. For example, a user can select an edge to identify differences between different parts of the cluster. For example, referring to Figure 7 , selecting edge 710 can cause a comparison between different parameter values ​​to be displayed. As can be seen, different sub-clusters have different CPU counts, log buffer rates, and CPU allocations. Selecting an edge can allow a user to perform one or more operations with respect to the nodes and / or sub-clusters connected by the edge. For example, in some cases, the difference between two nodes can be reconciled by an update operation.

[0151] Additionally or alternatively, clusters and / or groups of nodes can be interactive. For example, a user can select one of clusters 0-3 to drill down and view more information about the cluster, or perform management operations with respect to the cluster. As another example, a user can select a subset of nodes in a cluster to view node relationships and perform management operations on the nodes. The interface can thus allow a user to perform operations at various levels of granularity. Performing management operations on large groups of nodes can provide efficient batch operations. At the individual node level, more customized operations can be performed.

[0152] 4.2 Hypothetical clustering and representative node changes

[0153] In some embodiments, the clustering interface can facilitate the identification and execution of operations on a hypothetical cluster. In this context, a hypothetical cluster refers to a sub-cluster within a cluster that shares common parameters (such as configuration settings). The parameters common to the hypothetical cluster may be different from the parameters of other nodes (such as nodes belonging to other sub-clusters) in the same cluster.

[0154] In certain embodiments, imaginary clustering can be a sub-clustering of a representative node connected to a clustering. For example, edge 710 connects representative node 706 to a corresponding sub-clustering. Although representative node 706 has been selected as a gold standard, sub-clustering supports the assumption of additional clustering. In the example shown, the nodes of sub-clustering share common CPU count, log buffer rate and CPU allocation. These parameters common to sub-clustering are different from the representative node of clustering and the nodes in other sub-clusters. Except for the sub-clusters connected by edge 710, other imaginary clusterings are also connected to node 704.

[0155] In some embodiments, each hypothetical cluster includes a node representing a sub-cluster, which is referred to herein as a hypothetical representative node. The hypothetical representative node is directly connected to the representative node of the entire cluster. For example, edge 710 connects the hypothetical representative node of an hypothetical cluster. The most common instance of the feature vector in the sub-cluster can be selected as the hypothetical representative node. Additional nodes in the sub-cluster can be directly connected to the hypothetical representative node instead of the representative node of the cluster.

[0156] In some embodiments, the imaginary clustering can be run in depth at multiple levels. For example, an imaginary representative node ("parent" node) can be connected to the representative node of the entire cluster. Another imaginary representative node ("child" node) can be connected to the imaginary representative node of the imaginary cluster. When a large number of nodes share common parameters with the parent node of the main imaginary cluster, this scenario may occur, but it can also be subdivided along other different parameters. Therefore, the imaginary clustering can facilitate the identification of common parameter groups between different target deployments at multiple granularity levels.

[0157] In some embodiments, the hypothetical cluster can be separated from the main cluster to form a new cluster. For example, the user can cut off the sub-cluster along edge 710 to create a new cluster from a subset of the nodes belonging to the sub-cluster. The center / mode of the subset of nodes can then be assigned as the representative node of the new cluster. The initially selected representative node can remain the representative node of the nodes that are not cut off.

[0158] In some embodiments, the interface allows the user to change the representative node of a cluster and / or a hypothetical cluster. For example, the user can right-click or otherwise select a node other than a representative node in the cluster. The user can then be presented with an option to set the node as a representative node. In response to the selection, the representative node can be changed to the selected node. The visualization can be updated to highlight the new representative node. Additionally or alternatively, one or more edges can be redrawn to connect to the new representative node instead of the previous representative node. This method may be useful when the optimal configuration is followed by relatively few deployments in the cluster, or when new nodes / deployments are introduced. In these cases, the automatically selected representative nodes can be updated to match the new nodes.

[0159] 4.3 Summary Visualization and Drill-Down Operations

[0160] In some embodiments, a user can drill down on individual clusters and / or nodes to view more detailed information. For example, a user can select one of clusters 0-3 to view a more detailed interactive visualization of that cluster, such as that given in the example below. Additionally or alternatively, a user can select individual nodes to identify which software deployments are associated with the node. For example, selecting a node can cause an identifier to be displayed for each software deployment that shares a configuration corresponding to the node.

[0161] Figure 8 An example interactive visualization 800 for a selected cluster is illustrated in accordance with some embodiments. The interactive visualization 800 shows how many deployments share a feature value corresponding to a node. For example, a representative node 802 indicates that the corresponding feature value was detected in 874 deployments, including 89% of the deployments in the cluster. The next largest node (node ​​804) indicates that the corresponding feature value was detected in 74 different deployments, including 8% of the software deployments.

[0162] In some embodiments, differences in node configuration are displayed next to the node or edge. Figure 8 , the summary displayed next to node 804 indicates that there are differences in the number of database files between the 74 deployments corresponding to that node and the 874 deployments corresponding to the representative node 802. The summary displayed next to node 806 indicates that there are differences in the number of database files and the parameter "_fix_control" between the 6 deployments corresponding to that node and the 874 deployments corresponding to the representative node 802. Similarly, summaries are presented next to each other node identifying the differences between the deployment corresponding to that node and the deployment corresponding to the representative node 802.

[0163] In addition to or instead of an interactive clustering graph, a summary graph may be presented to the user. For example, Fig. 9Example summary graphs 900 for a set of clusters according to some embodiments are illustrated. These graphs identify the number of deployments in each cluster, the dispersion within each cluster, and the feature values ​​at the cluster center. Each cluster can be labeled to highlight differences. For example, cluster 2 groups deployments that exhibit high and variable memory. Cluster 0 groups clusters with high values ​​of cpu_count, and cluster 3 groups clusters with small and variable memory. Depending on the specific implementation, the information displayed in the summary table may be different. One or more attributes displayed in the summary table may be exposed to and configured by the end user, allowing custom summaries to be generated.

[0164] In some embodiments, the drill-down operation may include re-clustering a subset of nodes based on different parameters. For example, clusters 0-3 may be initially clustered based on a first set of characteristics (such as a combination of performance parameters). A user may then select one of the clusters or a subset of nodes in a cluster and request that the nodes be re-clustered based on a different set of attributes (such as security attributes or host machine attributes). In response, the nodes of the selected cluster are re-clustered into one or more sub-clusters based on a new set of parameters.

[0165] Drill-down operations can be performed to visualize deployment patterns and manage deployment resources across various dimensions. For example, target deployments can be initially clustered based on performance attributes. Clusters corresponding to low-performing deployments can then be re-clustered based on geographic location and / or other attributes. This approach can visualize and resolve patterns in low-performing deployments. Users can similarly drill down on other dimensions, including any combination of configuration settings for deployed resources.

[0166] 4.4 Drag and Drop Interface

[0167] In some embodiments, interactive visualization supports a drag-and-drop interface to perform management operations. The drag-and-drop interface allows a user to click or otherwise select a graphical representation, such as an icon, of one or more nodes in a cluster. The user can then drag the graphical representation and drop the graphical representation of one or more nodes to another area of ​​the display. The user can then drop the graphical representation, such as by releasing a clicked button, to initiate one or more management operations on the node. Drag-and-drop can provide an intuitive interface for managing the configuration of a large number of objects.

[0168] In some embodiments, one or more management operations are triggered in response to a user dragging and dropping one or more nodes to another node. For example, a user can drag one or more nodes to a representative node or any other node in a cluster. In response, an update operation can be triggered to coordinate the dragged node with the representative node. The update operation can involve applying a patch and / or adjusting other configuration settings of the node to match the configuration settings of the representative node, as described in further detail below.

[0169] Additionally or alternatively, depending on the particular implementation, management operations may be triggered via other interfaces or automatically. For example, a user may trigger a management operation through a menu interface, a CLI, or through any other user interface. The interface may allow a user to initiate multiple operations in parallel on different deployments with minimal input, such as drag and drop or single clicks.

[0170] 5. Configuration Management

[0171] In some embodiments, management operations may be triggered automatically or on demand based on the results of the clustering process. Example management operations may include:

[0172] Initialize new nodes / deployment;

[0173] Update existing nodes / deployments

[0174] Correct drift within cluster nodes

[0175] Subscribe to the golden image for the node

[0176] Perform analysis; and

[0177] Perform troubleshooting.

[0178] Example embodiments of management operations are provided below in Sections 5.1 to 5.5.

[0179] As provided in the examples below, the functionality of a management operation can depend on one or more values ​​of a feature set of a representative node or other characteristics of a representative node, by causing information in the feature set to be displayed on an interface, for example, in association with the representative node or its cluster, or by using such information to drive target performance of a corrective operation, a maintenance operation, or some other operation relative to the representative node or its corresponding cluster. For example, any operation that requires a cluster to be treated as if it were a single node (e.g., for visualization purposes, to simplify software configuration complexity, or other purposes) can utilize one or more values ​​of a feature set of a representative node or any other characteristics of a representative node, even if those characteristics are not used for the purpose of generating the cluster. This information can be provided to a service that performs the operation.

[0180] 5.1 Initialize a new node

[0181] In some embodiments, the results of the clustering operation can be used to initialize new nodes. If a new deployment of software resources comes online, the new deployment can be assigned to a cluster based on initial parameters. Configuration settings can then be set to match the configuration settings of the representative node in the cluster.

[0182] In some embodiments, initialization of the new node is performed automatically. When a new deployment is detected for the first time, a feature vector can be formed. The feature vector can be used to assign the deployment to the cluster with the centroid with the shortest distance. The configuration settings of the representative node can then be determined and applied to the new node. For example, the new node may lack one or more patch set updates of the representative node. In this case, a patch process can be triggered on the new node to apply the patch set update. As another example, the security settings on the new node can be configured to match the representative node, such as by opening or closing network ports, configuring identity and access management policies, and / or updating firewall settings.

[0183] In other embodiments, the initialization of the new node can be completed by request. For example, the user can drag and drop the graphical representation of the new node to the representative node or any other node in the cluster. In response, one or more process sets can be started on the new node to match the configuration settings of the representative node with the initialization configuration settings.

[0184] 5.2 Update nodes and correct drift

[0185] In some embodiments, the results of the clustering operation can be used to update the nodes in the cluster. For example, a node or a group of nodes in a cluster can be updated automatically or by request to match the configuration settings of a representative node or any other node in the cluster. In response to receiving a request to update a node or set of nodes, one or more update processes can be instantiated. The update process can apply a patch to modify a software binary file, execute a script to modify initialization parameters, or otherwise update configuration settings, as described in the previous section. Updates to multiple nodes can be performed in parallel.

[0186] In some embodiments, an update operation may add a feature to a deployment that already existed in the representative node but was initially missing from the upgraded node. For example, an update operation may add a user-defined patch, script, or other custom feature to a deployment. The result set of clustering may therefore be used to update a set of resource components from which one or more deployments are run.

[0187] In some embodiments, update operations may be performed to correct for configuration drift within a cluster. For example, nodes belonging to the same cluster may be updated automatically or by request to match the configuration of a representative node in the cluster. Drift coordination may be applied to a single node in a cluster, a subset of nodes, or all nodes in a cluster. The system may provide a user with a time estimate for the operation. Updates may be scheduled and / or staggered to reduce downtime.

[0188] Fig.10An example process for correcting drift based on a clustered set of deployments according to some embodiments is illustrated. Initially, deployment parameters 1002 for multiple deployments are received. In response, a summary graph 1004 and an interactive visualization 1006 can be generated based on the above-mentioned techniques. In the example shown, 3584 database deployments are grouped into six clusters, but the number of deployments and clusters may vary from implementation to implementation. The interactive visualization 1006 provides the option of correcting drift within a cluster via a GUI window 1008. The user can select an option to initiate drift repair. If this option is selected, the database deployment will be updated to match the configuration settings of the golden / representative node.

[0189] The correction can be performed globally for all nodes in a cluster or a subset of nodes therein. In some embodiments, the correction can be performed on the basis of an imaginary cluster. As previously described, the imaginary cluster has an imaginary representative node that is different from the representative node of the entire cluster. The user can select an option to use the imaginary representative node to correct drift in a sub-cluster rather than a global representative node. This method allows the user to use different configuration settings to coordinate different sub-clusters based on how the nodes are organized into sub-clusters.

[0190] 5.3 Golden Image Subscription

[0191] In some embodiments, the results of the clustering operation can be used to generate a golden image subscription. For example, a representative node can be automatically or by a user selected as a golden image for one or more other nodes in the cluster. In response, a subscription data set is generated that maps the subscribed nodes to the golden image. Once subscribed, the node will follow changes to the representative node. Thus, an upgrade detected at the representative node can trigger an upgrade to the subscribed nodes / deployments.

[0192] In some embodiments, all nodes within a cluster subscribe to the representative node of the cluster. However, in other embodiments, different subsets of nodes within a cluster may subscribe to different golden images. For example, nodes belonging to a hypothetical cluster may subscribe to a hypothetical representative node of a sub-cluster, rather than a representative node of the entire cluster. In other cases, a user may select a golden image from any other existing node in a cluster, or may create a new golden image node with a new feature set.

[0193] 5.4 Analysis Operation

[0194] In some embodiments, analysis operations can be run on one or more nodes in a cluster to display analysis information about the nodes. Analysis operations can be used to gather insights about target deployments that are not so obvious outside the context of the result set of the cluster. As an example, aggregate performance metrics such as average CPU utilization and memory throughput can be presented to an end user for a set of nodes belonging to a cluster or sub-cluster. As another example, a dispersion rate that presents the variability of feature values ​​in a cluster or sub-cluster can be presented. Other analysis operations can also be performed based on one or more features of the nodes in the cluster.

[0195] In some embodiments, analysis operations may be performed to highlight differences between different clusters and / or sub-clusters. Displaying information about a small set of representative nodes may make it clearer to understand the different purposes of different clusters in the system, where a subset of information about the representative nodes of each cluster is displayed in order to understand these differences. Thus, the presentation of information about different clusters (or hypothetical clusters) may focus on the differences. Features shared by all representative nodes (or hypothetical representative nodes) may be excluded from the presentation of information.

[0196] In some embodiments, the analysis information can be presented to the user to facilitate management operations. As an example, in response to a user selecting a node in a cluster, the interactive interface can display information about how the node has drifted from the representative node. For example, the information may include a list of configuration settings that are different from the representative node, such as differences in patch levels and security settings. Additionally or alternatively, the information may include a cost estimate for reconciling the node with the representative node. The cost estimate can be determined based on the steps required to update the configuration settings of the node to match the representative node. The cost estimate can include an estimated time to perform the step, which can help the user schedule management operations at the appropriate time.

[0197] 5.5 Troubleshooting Operations

[0198] It is often easier to troubleshoot a problem with one node than with an entire cluster. If the user is able to determine through visualization of the clusters and a representative node that a problem exists with a particular cluster, the user can attempt to troubleshoot the representative node, track configuration changes made to the representative node, and, once troubleshooting is complete, apply all such changes to the entire cluster. For example, problems can be detected based on frequently occurring error reports in the cluster, frequently appearing error logs associated with the cluster, poor performance metrics for the cluster, or based on metrics that indicate a problem displayed simultaneously with the cluster. The user can leverage the information presented about the cluster to isolate and resolve the root cause of the problem.

[0199] In some embodiments, the interactive interface allows the user to take corrective actions when attempting to resolve problems such as performance degradation or security violations. For example, a user can shut down one or more nodes in a cluster to prevent further security violations. Once the problem on the representative node has been resolved, the user can bring the node back online. Any configuration changes made on the representative node can be propagated to other nodes before the other nodes are back online. For example, nodes belonging to the cluster can apply any patches, security settings updates, or other configuration changes made to the representative node.

[0200] 6. Computer Networks and Cloud Networks

[0201] In some embodiments, a computer network provides connections between a collection of nodes. The nodes can be local and / or remote from each other. The nodes are connected by a set of links. Examples of links include coaxial cables, unshielded twisted cables, copper cables, optical fibers, and virtual links.

[0202] A subset of nodes implements a computer network. Examples of such nodes include switches, routers, firewalls, and network address translators (NATs). Another subset of nodes uses a computer network. Such nodes (also referred to as "hosts") can execute client processes and / or server processes. A client process makes a request for a computing service (such as the execution of a particular application and / or the storage of a particular amount of data). The server process responds by performing the requested service and / or returning corresponding data.

[0203] A computer network may be a physical network, comprising physical nodes connected by physical links. A physical node is any digital device. A physical node may be a function-specific hardware device, such as a hardware switch, a hardware router, a hardware firewall, and a hardware NAT. Additionally or alternatively, a physical node may be a general-purpose machine configured to execute various virtual machines and / or applications that execute corresponding functions. A physical link is a physical medium that connects two or more physical nodes. Examples of links include coaxial cables, unshielded twisted cables, copper cables, and optical fibers.

[0204] A computer network may be an overlay network. An overlay network is a logical network implemented on top of another network (such as a physical network). Each node in the overlay network corresponds to a corresponding node in the underlying network. Therefore, each node in the overlay network is associated with both an overlay address (addressing the overlay node) and an underlying address (addressing the underlying node that implements the overlay node). An overlay node may be a digital device and / or a software process (such as a virtual machine, an application instance, or a thread). The link connecting the overlay nodes is implemented as a tunnel through the underlying network. The overlay nodes at either end of the tunnel treat the underlying multi-hop path between them as a single logical link. Tunneling is performed by encapsulation and decapsulation.

[0205] In an embodiment, the client may be located locally and / or remote from the computer network. The client may access the computer network through other computer networks, such as a private network or the Internet. The client may transmit the request to the computer network using a communication protocol, such as a hypertext transfer protocol (HTTP). The request is transmitted through an interface, such as a client interface, such as a web browser, a program interface, or an application programming interface (API).

[0206] In an embodiment, a computer network provides a connection between a client and a network resource. The network resource includes hardware and / or software configured to execute a server process. Examples of network resources include processors, data storage devices, virtual machines, containers, and / or software applications. Network resources are shared between multiple clients. Clients request computing services from a computer network independently of each other. Network resources are dynamically allocated to requests and / or clients on demand. The network resources allocated to each request and / or client can be expanded or reduced based on, for example, (a) computing services requested by a specific client, (b) aggregate computing services requested by a specific tenant, and / or (c) the requested aggregate computing services of the computer network. Such a computer network may be referred to as a "cloud network."

[0207] In an embodiment, a service provider provides a cloud network to one or more end users. The cloud network can implement various service models, including but not limited to software as a service (SaaS), platform as a service (PaaS), and infrastructure as a service (IaaS). In SaaS, a service provider provides the end user with the ability to use the service provider's applications that are being executed on network resources. In PaaS, a service provider provides the end user with the ability to deploy customized applications on network resources. Customized applications can be created using programming languages, libraries, services, and tools supported by the service provider. In IaaS, a service provider provides the end user with the ability to supply processing, storage, network, and other basic computing resources provided by network resources. Any arbitrary application, including an operating system, can be deployed on network resources.

[0208] In an embodiment, a computer network can implement various deployment models, including but not limited to private cloud, public cloud and hybrid cloud. In a private cloud, network resources are supplied to a specific group of one or more entities (the term "entity" as used herein refers to an enterprise, organization, individual or other entity) for exclusive use. Network resources can be local to and / or away from the premises of a specific entity group. In a public cloud, cloud resources are supplied to multiple entities (also referred to as "tenants" or "customers") that are independent of each other. A computer network and its network resources are accessed by clients corresponding to different tenants. Such a computer network may be referred to as a "multi-tenant computer network". Several tenants may use the same specific network resources at different times and / or at the same time. Network resources may be local to and / or away from the premises of a tenant. In a hybrid cloud, a computer network includes a private cloud and a public cloud. The interface between a private cloud and a public cloud allows portability of data and applications. Data stored at a private cloud and data stored at a public cloud can be exchanged through an interface. Applications implemented at a private cloud and applications implemented at a public cloud may have dependencies on each other. Calls from an application at the private cloud to an application at the public cloud (and vice versa) may be performed through the interface.

[0209] In an embodiment, the tenants of a multi-tenant computer network are independent of each other. For example, the business or operation of one tenant can be separated from the business or operation of another tenant. Different tenants may have different network requirements for the computer network. Examples of network requirements include processing speed, data storage capacity, security requirements, performance requirements, throughput requirements, latency requirements, elasticity requirements, quality of service (QoS) requirements, tenant isolation and / or consistency. The same computer network may need to implement different network requirements required by different tenants.

[0210] In some embodiments, in a multi-tenant computer network, tenant isolation is implemented to ensure that applications and / or data of different tenants are not shared with each other. Various tenant isolation methods can be used.

[0211] In an embodiment, each tenant is associated with a tenant ID. Each network resource of the multi-tenant computer network is marked with a tenant ID. Only when the tenant and the specific network resource are associated with the same tenant ID, the tenant is allowed to access the specific network resource.

[0212] In an embodiment, each tenant is associated with a tenant ID. Each application implemented by the computer network is tagged with the tenant ID. Additionally or alternatively, each data structure and / or data set stored by the computer network is tagged with the tenant ID. A tenant is allowed to access a specific application, data structure, and / or data set only if the tenant and the specific application, data structure, and / or data set are associated with the same tenant ID.

[0213] As an example, each database implemented by a multi-tenant computer network can be marked with a tenant ID. Only the tenant associated with the corresponding tenant ID can access the data of a particular database. As another example, each entry in a database implemented by a multi-tenant computer network can be marked with a tenant ID. Only the tenant associated with the corresponding tenant ID can access the data of a particular entry. However, a database can be shared by multiple tenants.

[0214] In an embodiment, the subscription list indicates which tenants have access to which applications. For each application, a list of tenant IDs of tenants authorized to access the application is stored. Only when the tenant ID of a tenant is included in the subscription list corresponding to a specific application, the tenant is allowed to access the specific application.

[0215] In an embodiment, network resources corresponding to different tenants (such as digital devices, virtual machines, application instances, and threads) are isolated to tenant-specific overlay networks maintained by a multi-tenant computer network. As an example, a data packet from any source device in a tenant overlay network can be sent only to other devices within the same tenant overlay network. Encapsulation tunnels are used to prohibit any transmission from a source device on a tenant overlay network to devices in other tenant overlay networks. Specifically, a data packet received from a source device is encapsulated in an outer data packet. The outer data packet is sent from a first encapsulation tunnel endpoint (communicating with a source device in a tenant overlay network) to a second encapsulation tunnel endpoint (communicating with a destination device in a tenant overlay network). The second encapsulation tunnel endpoint decapsulates the outer data packet to obtain the original data packet sent by the source device. The original data packet is sent from the second encapsulation tunnel endpoint to a destination device in the same specific overlay network.

[0216] 7. Microservice Application

[0217] According to some embodiments, the technology described herein is implemented with a microservices architecture. Microservices in this context refer to software logic designed to be independently deployable, which has endpoints that can be logically coupled to other microservices to build various applications. Applications built using microservices are different from monolithic applications, which are designed as a single fixed unit and typically include a single logical executable file. With microservices applications, different microservices can be independently deployed as separate executable files. Microservices can communicate using Hypertext Transfer Protocol (HTTP) messages and / or via API endpoints according to other communication protocols. Microservices can be managed and updated separately, written in different languages, and executed independently of other microservices.

[0218] Microservices provide flexibility in managing and building applications. Different applications can be built by connecting different sets of microservices without changing the source code of the microservices. Therefore, microservices act as logical building blocks, and logical building blocks can be arranged in various ways to build different applications. Microservices can provide monitoring services that notify the microservice manager (such as If-This-Then-That (IFTTT), Zapier or Oracle Self-Service Automation (OSSA)) when a trigger event in a trigger event set exposed to the microservice manager occurs. Microservices exposed to applications can alternatively or additionally provide action services that perform actions in applications based on data received from the microservice manager (by passing values, connecting actions to other triggers and / or data passed from other actions in the microservice manager, controllable and configurable via the microservice manager). Microservice triggers and / or actions can be chained together to form recipes for actions that occur in optional different applications, which are otherwise unaware of each other or have no control or dependency on each other. These managed applications can be authenticated or plugged into the microservice manager, for example, using application credentials supplied by a user to the manager, without requiring re-authentication each time the managed application is used alone or in conjunction with other applications.

[0219] In some embodiments, microservices can be connected via a GUI. For example, microservices can be displayed as logical blocks within a window, frame, or other element of a GUI. A user can drag and drop a microservice into an area of ​​the GUI for building an application. A user can connect the output of one microservice to the input of another microservice using a directed arrow or any other GUI element. The application builder can run validation tests to confirm that the output and input are compatible (e.g., by checking data types, size limits, etc.).

[0220] trigger

[0221] According to some embodiments, the above techniques may be encapsulated into a microservice. In other words, a microservice may trigger a notification (entered into a microservice manager for optional use by other inserted applications referred to herein as "target" microservices) based on the above techniques and / or may be represented as a GUI block and connected to one or more other microservices. The triggering condition may include an absolute or relative threshold of a value, and / or an absolute or relative threshold of the amount of data to be analyzed or the duration of the data, such that whenever an inserted microservice application detects that a threshold has been crossed, a trigger to the microservice manager occurs. For example, a user may request a trigger from the microservice manager when a microservice application detects that a value has crossed a triggering threshold.

[0222] In one embodiment, the trigger, when satisfied, may output data for consumption by the target microservice. In another embodiment, the trigger, when satisfied, outputs a binary value indicating that the trigger has been satisfied, or outputs the field name or other context information that satisfies the trigger condition. Additionally or alternatively, the target microservice may be connected to one or more other microservices so that an alert is input to the other microservices. The other microservices may perform response actions based on the above techniques, including but not limited to deploying additional resources, adjusting system configuration, and / or generating a GUI.

[0223] action

[0224] In some embodiments, the inserted microservice application can expose an action to the microservice manager. The exposed action can receive the identification of the data or data object or the location of the data as input, which causes the data to be moved to the data cloud.

[0225] In some embodiments, the exposed action may receive as input a request to increase or decrease an existing alarm threshold. The input may identify an existing in-application alarm threshold and whether to increase, decrease, or delete the threshold. Additionally or alternatively, the input may request the microservice application to create a new in-application alarm threshold. In-application alarms may trigger an alarm to the user when logging into the application, or may use a default or user-selected alarm mechanism available within the microservice application itself, rather than triggering an alarm to the user through other applications plugged into the microservice manager.

[0226] In some embodiments, the microservice application can generate and provide output based on inputs that identify, locate, or provide historical data and define the extent or scope of the requested output. The action, when triggered, causes the microservice application to provide, store, or display output, for example, as a data model or as aggregated data describing the data model.

[0227] 8. Hardware Overview

[0228] According to one embodiment, the technology described herein is implemented by one or more special-purpose computing devices. The special-purpose computing device can be hard-wired to perform the present technology, or can include a digital electronic device that is permanently programmed to perform the present technology, such as one or more application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or network processing units (NPUs), or can include one or more general-purpose hardware processors that are programmed to perform the present technology according to program instructions in firmware, memory, other storage devices, or combinations. Such special-purpose computing devices can also combine customized hard-wired logic, ASICs, FPGAs, or NPUs with customized programming to implement the present technology. The special-purpose computing device can be a desktop computer system, a portable computer system, a handheld device, a networking device, or any other device that combines hard-wiring and / or program logic to implement the technology.

[0229] For example, Fig.11 1 is a block diagram illustrating a computer system 1100 upon which embodiments of the present invention may be implemented. Computer system 1100 includes a bus 1102 or other communication mechanism for communicating information and a hardware processor 1104 coupled with bus 1102 for processing information. Hardware processor 1104 may be, for example, a general purpose microprocessor.

[0230] The computer system 1100 also includes a main memory 1106, such as a random access memory (RAM) or other dynamic storage device, coupled to the bus 1102 for storing information and instructions to be executed by the processor 1104. The main memory 1106 may also be used to store temporary variables or other intermediate information during execution of instructions to be executed by the processor 1104. When such instructions are stored in a non-transitory storage medium accessible to the processor 1104, such instructions cause the computer system 1100 to become a special-purpose machine customized to perform the operations specified in the instructions.

[0231] Computer system 1100 also includes a read only memory (ROM) 1108 or other static storage device coupled to bus 1102 for storing static information and instructions for processor 1104. A storage device 1110, such as a magnetic or optical disk, is provided and coupled to bus 1102 for storing information and instructions.

[0232] The computer system 1100 may be coupled via the bus 1102 to a display 1112, such as a cathode ray tube (CRT), for displaying information to a computer user. An input device 1114, including alphanumeric and other keys, is coupled to the bus 1102 for communicating information and command selections to the processor 1104. Another type of user input device is a cursor control 1116, such as a mouse, trackball, or cursor direction keys, for communicating direction information and command selections to the processor 1104 and for controlling cursor movement on the display 1112. Such input devices typically have two degrees of freedom in two axes, a first axis (e.g., x) and a second axis (e.g., y), to allow the device to specify a position in a plane.

[0233] The computer system 1100 may implement the techniques described herein using custom hardwired logic, one or more ASICs or FPGAs, firmware, and / or program logic that, in combination with the computer system, makes the computer system 1100 a special purpose machine or programs the computer system 1100 to be a special purpose machine. According to one embodiment, the techniques herein are performed by the computer system 1100 in response to the processor 1104 executing one or more sequences of one or more instructions contained in the main memory 1106. These instructions may be read into the main memory 1106 from another storage medium, such as the storage device 1110. Execution of the sequences of instructions contained in the main memory 1106 causes the processor 1104 to perform the process steps described herein. In alternative embodiments, hardwired circuitry may be used in place of or in combination with software instructions.

[0234] The term "storage medium" as used herein refers to any non-transitory medium that stores data and / or instructions that cause a machine to operate in a particular manner. Such storage media may include non-volatile media and / or volatile media. Non-volatile media include, for example, optical or magnetic disks, such as storage device 1110. Volatile media include dynamic memory, such as main memory 1106. Common forms of storage media include, for example, floppy disks, flexible disks, hard disks, solid-state drives, tapes or any other magnetic data storage media, CD-ROMs, any other optical data storage media, any physical media with hole patterns, RAM, PROM and EPROM, FLASH-EPROM, NVRAM, any other memory chip or cassette, content addressable memory (CAM) and ternary content addressable memory (TCAM).

[0235] Storage media are distinct from but may be used in conjunction with transmission media. Transmission media participate in the transfer of information between storage media. For example, transmission media include coaxial cables, copper wire, and optical fiber, including the wires that comprise bus 1102. Transmission media may also take the form of acoustic or light waves, such as those generated during radio wave and infrared data communications.

[0236] Various forms of media may be involved in carrying one or more sequences of one or more instructions to the processor 1104 for execution. For example, the instructions may initially be carried on a disk or solid-state drive of a remote computer. The remote computer can load the instructions into its dynamic memory and send the instructions over a telephone line using a modem. A modem local to the computer system 1100 can receive the data on the telephone line and use an infrared transmitter to convert the data to an infrared signal. An infrared detector can receive the data carried in the infrared signal and appropriate circuitry can place the data on the bus 1102. The bus 1102 carries the data to the main memory 1106, from which the processor 1104 retrieves and executes the instructions. The instructions received by the main memory 1106 may optionally be stored on the storage device 1110 before or after execution by the processor 1104.

[0237] Computer system 1100 also includes a communication interface 1118 coupled to bus 1102. Communication interface 1118 provides bidirectional data communication coupled to network link 1120, wherein network link 1120 is connected to local network 1122. For example, communication interface 1118 can be an integrated services digital network (ISDN) card, a cable modem, a satellite modem, or a modem that provides a data communication connection to a telephone line of a corresponding type. As another example, communication interface 1118 can be a LAN card that provides a data communication connection to a compatible local area network (LAN). Wireless links can also be implemented. In any such implementation, communication interface 1118 sends and receives electrical signals, electromagnetic signals, or optical signals that carry digital data streams representing various types of information.

[0238] The network link 1120 typically provides data communication to other data devices through one or more networks. For example, the network link 1120 can provide a connection to a host computer 1124 or to data equipment operated by an Internet Service Provider (ISP) 1126 through a local network 1122. The ISP 1126, in turn, provides data communication services through the global packet data communication network now commonly referred to as the "Internet" 1128. Both the local network 1122 and the Internet 1128 use electrical, electromagnetic, or optical signals that carry digital data streams. The signals through the various networks and the signals on the network link 1120 and through the communication interface 1118 are example forms of transmission media that carry the digital data to or from the computer system 1100.

[0239] Computer system 1100 can send messages and receive data, including program code, through the network(s), network link 1120, and communication interface 1118. In the Internet example, server 1130 can transmit the requested code for an application program through Internet 1128, ISP 1126, local network 1122, and communication interface 1118.

[0240] The received code may be executed by processor 1104 as it is received, and / or stored in storage device 1110 or other non-volatile storage for later execution.

[0241] 9. Other Matters; Extension

[0242] Embodiments are directed to a system having one or more devices including a hardware processor and configured to perform any of the operations described herein and / or in any of the following claims.

[0243] In an embodiment, a non-transitory computer-readable storage medium comprises instructions that, when executed by one or more hardware processors, cause performance of any of the operations described herein and / or any of the claims.

[0244] According to some embodiments, any combination of the features and functions described herein may be used. In the foregoing description, embodiments have been described with reference to many specific details that vary depending on the implementation. Therefore, the description and drawings should be considered illustrative rather than restrictive. The sole and exclusive indicator of the scope of the present invention and what the applicant intends to be the scope of the present invention is the literal and equivalent range of the set of claims issued from this application, in the specific form in which such claims are issued, including any subsequent corrections.

Claims

1. A method for managing the configuration of software resources, the method comprising: determining a set of characteristics for a plurality of deployments of a software resource; clustering each of the plurality of deployments into a cluster in the plurality of clusters based on a change value in the feature set; wherein each cluster of the plurality of clusters comprises one or more nodes; wherein at least a first cluster of the plurality of clusters comprises a plurality of nodes; wherein at least a subset of the plurality of nodes corresponds to different configuration settings among deployments of software resources that have been clustered into the first cluster; wherein each of the one or more nodes corresponds to at least a subset of values ​​of a feature set detected in at least one of the plurality of deployments of the software resource; determining a representative node for each cluster of the plurality of clusters; as well as For at least a first deployment that has been clustered into a first cluster and has a first set of configuration settings that is different from a second set of configuration settings corresponding to a representative node of the first cluster, the first deployment is coordinated with the representative node of the first cluster by updating the first set of configuration settings for at least the first deployment to match the second set of configuration settings corresponding to the representative node of the first cluster. 2 . The method of claim 1 , wherein the representative node of the first cluster corresponds to a specific set of values ​​of a feature set that is most frequently detected in deployments belonging to the first cluster. 3 . The method of claim 1 , wherein the representative node of the first cluster corresponds to configuration settings recommended for deployments belonging to the first cluster among the plurality of deployments of software resources.

4. The method as described in any one of claims 1-2 further includes generating a hierarchical graph for each cluster based on different values ​​of the feature set of each cluster.

5. The method of any one of claims 1-2, further comprising generating an interactive visualization for the plurality of clusters; wherein the interactive visualization for the clusters comprises a visual representation of the plurality of nodes belonging to a first cluster and edges connecting the plurality of nodes; wherein at least one visual aspect of each of the plurality of nodes is based on how many of the plurality of deployments have a value of a feature set that matches a value of a feature set corresponding to the node.

6. The method of claim 5 further comprises receiving, through the interactive visualization, a selection of a subset of nodes belonging to the first cluster; generating a new cluster for the subset of nodes in response to the selection; and determining a new representative node for the new cluster.

7. A method as claimed in claim 5, wherein the interactive visualization allows selection of individual nodes in the first cluster to be coordinated with the representative node, or coordination of groups of nodes in the first cluster with the representative node of the first cluster; wherein the interactive visualization allows the user to drag and drop to reconfigure nodes that have drifted from the representative node of the first cluster.

8. A method as described in any one of claims 1-2 or 6-7, wherein the multiple clusters are a first set of clusters, and the method also includes generating multiple different sets of clusters; wherein each corresponding set of clusters is generated using a different corresponding set of features associated with the multiple deployments.

9. A method as described in any one of claims 1-2 or 6-7, wherein the coordination is performed in response to a selection; wherein the coordination is performed at least in part based on one or more values ​​of a specific feature set of representative nodes of the first cluster or at least in part based on other characteristics of the representative nodes of the first cluster.

10. The method according to any one of claims 1-2 or 6-7, further comprising: The nodes within the first cluster are displayed along with information about how the nodes differ from a representative node of the first cluster.

11. One or more non-transitory computer-readable media storing instructions which, when executed by one or more hardware processors, cause the method of any one of claims 1 to 10 to be performed.

12. A system comprising one or more hardware processors and one or more non-transitory computer-readable media storing instructions that, when executed by the one or more hardware processors, cause the method of any one of claims 1-10 to be performed.

13. An apparatus comprising means for performing the method according to any one of claims 1-10.

Images