System and method for identifying and resolving performance problems of automated components

Through the K-mean clustering algorithm and machine learning model, the monitoring problem of automation component performance problems is solved, the accurate grouping and performance ranking of components are achieved, and the accuracy and efficiency of performance analysis are improved.

CN118891618APending Publication Date: 2024-11-01MICROSOFT TECHNOLOGY LICENSING LLC
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202280093245.6
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Priority Date
2022-03-29
Filing Date
2022-12-30
Publication Date
2024-11-01

AI Technical Summary

Technical Problem

The prior art is difficult to effectively monitor and solve the performance problems of automation components, especially when processing large amounts of data, and traditional methods can lead to inaccurate grouping and performance analysis.

Method used

By applying the K-mean clustering algorithm and initializing the clustering center of mass set using the context rule set, the automated components are divided into groups, combining the machine learning classification model to determine the importance value of performance characteristics, and calculate the performance score of each component, thereby performing Accurate ranking and performance problem identification.

Benefits of technology

Accurate identification and ranking of performance issues of automation components is achieved, the accuracy and efficiency of performance analysis is improved, and components with poor performance can be effectively identified and corresponding measures are taken to improve performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118891618A_ABST
    Figure CN118891618A_ABST
Patent Text Reader

Abstract

A system and method for identifying and resolving performance issues of automated components is described. Automation components are segmented into groups by applying a K-means clustering algorithm to the automation components based on segmentation feature values respectively associated with the automation components, where an initial set of centroids for the K-means clustering algorithm is selected by applying a set of context rules to the automation components. Then, for each group, a performance ranking is generated based at least on a set of performance feature values associated with each automation component in the group and a feature importance value for each performance feature. Feature importance values are determined by training a machine learning-based classification model to classify the automation components into each group, wherein the training is performed based on respective performance feature values of the automation components and respective groups assigned thereto.
Need to check novelty before this filing date? Find Prior Art

Description

Background Art

[0001] In various applications, it may be considered desirable to monitor the performance of an automation component so that performance issues associated with the performance may be identified and resolved. For example, it may be considered desirable to monitor and evaluate the performance of an automation component relative to other components within a population to help identify performance issues thereof. Summary of the invention

[0002] This summary is intended to introduce a series of concepts in a simplified form, which will be further described in the detailed description below. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter.

[0003] A method, system, and apparatus for identifying and resolving performance issues of automation components are described. In one implementation, for each of a plurality of automation components, a set of segmentation feature values ​​corresponding to a set of segmentation features and a set of performance feature values ​​corresponding to a set of performance features are received. The plurality of automation components are segmented into groups by applying a clustering algorithm to the plurality of automation components based on the segmentation feature values ​​respectively associated with the plurality of automation components. For each group of automation components, a ranking of the automation components of the group is generated based at least on the sets of performance feature values ​​respectively associated with the automation components of the group.

[0004] In yet another example implementation, a plurality of automation components are segmented into groups by applying a K-means clustering algorithm to the plurality of automation components based on segmentation feature values ​​associated with the plurality of automation components. According to the implementation, applying the K-means clustering algorithm includes initializing a set of cluster centroids used in applying the K-means clustering algorithm by applying a set of contextual rules to the plurality of automation components. This approach differs from conventional K-means clustering methods in which an initial set of cluster centroids is randomly selected.

[0005] In another example implementation, the performance of the automation components within each of the groups is ranked by determining a feature importance value for each of the performance features. The feature importance values ​​are determined by training a machine learning (ML) based classification model to predict the groups segmented by applying the above-mentioned K-means clustering algorithm (or any other clustering algorithm suitable for grouping automation components into groups). The training is performed based on the respective performance feature values ​​of the automation components and the respective groups assigned to the automation components. For each automation component, a performance score for the automation component is calculated based on the performance feature values ​​of the automation component and the feature importance value of each performance feature. In an embodiment, the score for a particular entity is a weighted sum of the performance feature values ​​and the feature importances determined by the classification algorithm. For each group of automation components, a ranking of the automation components of the group is generated based on the respective performance scores.

[0006] More features and advantages of the embodiments and the structure and operation of various embodiments will be described in detail below with reference to the accompanying drawings. It should be noted that the claimed subject matter is not limited to the specific examples described herein. This article presents such embodiments only for illustrative purposes. Based on the teachings contained herein, additional embodiments will be apparent to those skilled in the relevant art(s). BRIEF DESCRIPTION OF THE DRAWINGS

[0007] The accompanying drawings, which are incorporated herein and constitute a part of the specification, illustrate the embodiments and, together with the description, further serve to explain the principles of the embodiments and to enable those skilled in the relevant art to make and use the embodiments.

[0008] Figure 1 is a block diagram of a system that may be used to identify and resolve performance issues of automation components according to an example embodiment.

[0009] Figure 2 is a block diagram of a segmentation and assessment component that may be used to identify and resolve performance issues of automation components according to an example embodiment.

[0010] Figure 3 is a flow diagram of a process for identifying and resolving performance issues of automation components according to an example embodiment.

[0011] Figure 4 is a block diagram of a clustering component according to an example embodiment.

[0012] Figure 5 is a flow diagram of a process for initializing a set of cluster centroids according to an example embodiment.

[0013] Fig. 6A and Figure 6Bis a flow diagram of a process for segmenting a plurality of automation components into groups according to an example embodiment.

[0014] Figure 7 is a flow chart of a process for ranking performance of automation components in a group of automation components.

[0015] Figure 8 is a block diagram of a ranking component according to an example embodiment.

[0016] Fig. 9 is a block diagram of an example computer system that can be used to implement an embodiment.

[0017] Embodiments will now be described with reference to the accompanying drawings. In the accompanying drawings, the same reference numerals represent the same or functionally similar elements. In addition, the leftmost digit of a reference number identifies the drawing in which the reference number first appears. DETAILED DESCRIPTION

[0018] I. Introduction

[0019] The following detailed description discloses many example embodiments. The scope of this patent application is not limited to the disclosed embodiments, but also covers group combinations of the disclosed embodiments and modifications to the disclosed embodiments.

[0020] References in the specification to "one embodiment," "an embodiment," "an example embodiment," etc. indicate that the described embodiment may include a particular feature, structure, or characteristic, but not every embodiment may include the particular feature, structure, or characteristic. Furthermore, these phrases do not necessarily refer to the same embodiment. Furthermore, when a particular feature, structure, or characteristic is described in conjunction with an embodiment, it should be understood that those skilled in the art may affect such feature, structure, or characteristic in conjunction with other embodiments (whether or not explicitly described).

[0021] In the discussion, unless otherwise stated, adjectives such as "substantially" and "about" that modify conditions or relational characteristics of one or more features of the embodiments of the present disclosure should be understood to mean that the condition or characteristic is defined within an acceptable tolerance range for operation of the embodiment for its intended application.

[0022] If the performance of an operation is described herein as being "based on" one or more factors, it should be understood that the performance of the operation may be based only on such factor(s), or may be based on such factor(s) and one or more additional factors. Therefore, as used herein, the term "based on" should be understood to be equivalent to the term "based at least on".

[0023] A number of exemplary embodiments are now described. Any section / subsection headings provided herein are not intended to be limiting. Embodiments are described in this document, and any type of embodiment may be included under any section / subsection. In addition, the embodiments disclosed in any section / subsection may be combined in any manner with any other embodiments described in the same section / subsection and / or in different sections / subsections.

[0024] II. Example Systems and Methods for Identifying and Resolving Performance Issues

[0025] Population analysis can be performed to identify performance issues of automation components relative to their population. Automation components can include hardware components (e.g., computing devices in a data center or network, or components thereof), software components (e.g., resource meters, applications, operating systems, or services in a cloud computing service), or any other type of automation component whose performance can be monitored and compared to similar automation components (e.g., vehicles in a fleet, telecommunications equipment in a telecommunications network, cooling equipment (e.g., fans) in a data center).

[0026] As used herein, the term "population" refers to a group of automation components (e.g., a group of automation components segmented using a clustering algorithm as further described below). For a single automation component, performance issues can be identified relative to its population or across multiple (e.g., some or all) automation components in a population.

[0027] There are various methods for determining groups in population analysis. For example, expert-based and / or pairwise comparison-based methods can be used in multi-criteria decision making to filter components based on similarity. However, these methods have difficulties in processing large amounts of data. In a machine learning (ML) environment, clustering algorithms can be used to group components into clusters. For example, the K-means clustering algorithm is a clustering algorithm that generates mathematically feasible clusters from randomly selected initial cluster centroids. However, this clustering algorithm does not take into account the context of the observations to be clustered, and therefore may produce clusters that are irrelevant to the application. For example, in the context of population analysis, clustering of results may lead to inaccurate grouping of groups and analysis of components within groups.

[0028] In one aspect of the present disclosure, systems and methods utilize data collected for automation components to segment automation components into groups. In an embodiment, the automation components are segmented into groups by applying a (e.g., supervised or unsupervised) clustering algorithm to the automation components. In a non-limiting example embodiment, a K-means clustering algorithm is applied to the automation components to segment the automation components into groups. The application of the K-means clustering algorithm may include initializing a set of cluster centroids used in applying the K-means clustering algorithm by applying a set of contextual rules to the automation components. The set of contextual rules specifies a non-random method for initially selecting a set of cluster centroids. In this way, the systems and methods described herein have a greater chance of creating groups based on the results of the K-means clustering algorithm, which makes logical sense for ranking the performance of the automation components, as further described below. In the described K-means clustering algorithm, the cluster centroids are selected based on a set of contextual rules adjusted to the problem, rather than randomly selected.

[0029] Contextual rules can be based on segmentation features (e.g., location of automation components, type of storage or memory of automation components, type of resources associated with automation components (e.g., computing devices, virtual machines, etc.), type of services performed by automation components, etc.). For example, contextual rules according to an embodiment suggest grouping automation components based on geographic regions and types of services performed by automation components. In an example embodiment, segmentation feature values ​​are extracted from data collected for automation components. The system described herein utilizes segmentation features to select an initial set of cluster centroids, thereby improving segmentation of automation components into groups. In addition, such a system can process large data sets from different dimensions across a variety of automation components.

[0030] Once a population is defined, its components can be evaluated with respect to various performance criteria; however, depending on the implementation, not all performance criteria may affect the performance of a component in the same way. In traditional approaches, all performance criteria or a subset of performance criteria are provided without considering the importance of each performance criterion.

[0031] According to an embodiment, a single performance score is calculated for each automation component. To calculate the performance score, a classification model is trained using the group segmented by the above-mentioned clustering algorithm (or the group segmented by another clustering algorithm suitable for segmenting the automation components into groups) as the target group. The classification algorithm seeks to predict the number of clusters based on performance characteristics. In an embodiment, the segmentation is based on the segmentation characteristics, and the performance characteristics are used to predict the group. The feature importance value of the performance characteristics of the automation component is determined by training an ML-based classification model (e.g., a supervised learning classifier) ​​based on the corresponding performance characteristic values ​​of the automation component and the group to which such automation components have been assigned. For example, but not limited to, the performance characteristics may include the occurrence, frequency, or duration of interruptions or anomalies detected in the operation of the automation component, the measurement of the usage of the automation component, the measurement of the power consumed by the automation component, the measurement of the usage of the computing, network and / or storage resources used by the automation component, the measurement of the delay of the automation component, the measurement of the throughput of the automation component, and / or the measurement of other performance of the automation component, which is described elsewhere herein and / or understood by (multiple) technicians in the relevant fields who benefit from this disclosure.

[0032] The system described herein calculates a performance score for an automation component based on performance feature values ​​for the automation component and a feature importance value for each performance feature. In an embodiment, the feature importance value of the performance feature determines the weight of the associated performance feature value to the performance score of the automation component. By training a machine learning-based classification model to determine the feature importance value for each performance feature, the information value of the performance score for the automation component can be enhanced by ensuring that performance features of relatively higher importance contribute more to the performance score than features of relatively lower importance. According to an embodiment, a single performance score is determined for each automation component based on its performance feature value and feature importance value. For example, in a non-limiting example embodiment, a single score for each automation component is calculated by summing a weighted product of the feature importance value and the performance feature value of the automation component. In an embodiment, automation components in a group are ranked with respect to each other based on the corresponding performance scores.

[0033] The embodiments and techniques described herein group automation components and rank the performance of these components within these groups, thereby enabling better identification of performance issues of automation components by ranking the performance with respect to correctly identified peer groups. In this context, components with poor performance can be identified and mitigation measures can be taken to improve performance. For example, performance issues can be identified and / or resolved based on the ranking of automation components. Depending on the implementation, embodiments of the present disclosure can (e.g., automatically or semi-automatically) perform actions based on the ranking of automation components to resolve performance issues and / or generate commands to external systems to resolve performance issues. In an embodiment, the characteristics of one or more highly ranked automation components (e.g., segmentation feature values ​​and / or performance feature values) can be compared with the characteristics of one or more low-ranked automation components to determine actions to improve the performance of low-ranked automation components. For example, the system described herein can determine performance targets for one or more performance features of automation components within a group based on the automation components that rank high (e.g., perform best) in the group and generate an alert that notifies the user of one or more areas for improving the performance of automation components that rank low (e.g., perform poorly).

[0034] In addition, the embodiments and techniques described herein can improve the functionality of one or more computers. For example, in an embodiment, a system segments computer hardware and / or software entities into groups. The system is configured to identify one or more underperforming computer hardware and / or software entities within the group. The system is also configured to automatically perform one or more actions to improve the performance of the one or more identified underperforming entities. In this manner, the systems and methods described herein can improve the functionality of one or more computers.

[0035] In an embodiment, the identification and resolution of performance issues in an automation component may be accomplished in various ways. For example, according to an example embodiment, Figure 1 1 is a block diagram of a system 100 that can be used to identify and resolve performance issues of automation components. Depending on the implementation, the system 100 can be implemented on a single computing device or can be implemented across multiple computing devices. Fig. 9 1 is used to describe non-limiting examples of computing devices that may be used to implement system 100 .

[0036] like Figure 1As shown in FIG. 1 , system 100 includes a data collection and pre-processing component 102, a segmentation and evaluation component 104, automation components 106A-106N (collectively referred to herein as “automation components 106”), and a user interface 108. Data collection and pre-processing component 102 is configured to collect data 116A-116N (collectively referred to herein as “data 116”) from or about respective automation components 106 and prepare the data for use by segmentation and evaluation component 104. Figure 1 As shown in , the data collection and pre-processing component 102 includes a data collector 110, a feature identifier 112, and a pre-processing component 114. The data collector 110 is configured to collect data 116 corresponding to the automation component 106 and generate a collection data set 118. In an embodiment, the data 116 is stored in one or more databases. In this context, the data collector 110 retrieves the data 116 from the one or more databases to generate the collection data set 118. In another embodiment, the data collector 110 includes a monitoring system that is configured to collect the data 116 by monitoring the automation component 106. In this context, the data collector 110 can store the data 116 in a database ( Figure 1 ) to generate a collection data set 118.

[0037] The feature identifier 112 is configured to receive the collection data set 118 and extract feature values ​​120 for each of the automation components 106. Figure 1 As shown in , feature identifier 112 may receive collection data set 118 from data collector 110; however, in some embodiments, feature identifier 112 may receive collection data set 118 from a database (for simplicity, Figure 1 1 (not shown in the figure), retrieves the collected data set 118. The feature values ​​120 may include segmentation feature values ​​and / or performance feature values ​​corresponding to corresponding automation components in the automation components 106. According to an embodiment, the feature identifier 112 is an ML model trained to identify the feature values ​​120 from the collected data set 118. For example, the ML model according to an embodiment is configured to automatically extract segmentation features from the collected data set 118.

[0038] The preprocessing component 114 is configured to apply various preprocessing steps to the collected data set 118 and / or the feature values ​​120. For example, the preprocessing component 114 can remove outliers from the collected data set 118 and / or the feature values ​​120, encode features based on the feature values ​​120, normalize the data, and / or perform other preprocessing operations to prepare the collected data set 118 and / or the feature values ​​120 for segmentation and evaluation by the segmentation and evaluation component 104. Figure 1As shown in , the pre-processing component 114 generates a plurality of automation component descriptors 122, segmentation feature values ​​124, and performance feature values ​​126 representing the automation components 106. The plurality of automation component descriptors 122 may include information associated with the automation components 106, such as identifiers, geographic addresses, device addresses, and / or other information for identifying and / or distinguishing each of the automation components 106. In an embodiment, the pre-processing component 114 automatically performs the pre-processing steps. In an embodiment, the pre-processing component 114 stores the plurality of automation component descriptors 122, segmentation feature values ​​124, and / or performance feature values ​​126 in a database, which for brevity is shown in FIG. Figure 1 For example, pre-processing component 114 may store multiple automation component descriptors 122, segmentation feature values ​​124, and / or performance feature values ​​126 in a database external to system 100, in a memory internal to a computing device implementing one or more components of system 100, or in a working memory of an application implementing a component of system 100.

[0039] The segmentation and evaluation component 104 is configured to identify and resolve performance issues with the automation component 106. For example, as described below with reference to Figure 2 As described, the segmentation and evaluation component 104 can segment the plurality of automation components 106 into groups by applying a clustering algorithm based on the segmentation feature values ​​124, generating a ranking of the automation components for each group based at least on the corresponding performance feature values ​​126, and / or performing an action 132 based on the ranking of at least one automation component. The action 132 can include generating an alert, generating one or more recommended actions for resolving a performance problem associated with at least one automation component 106, initiating one or more automated processes for resolving a performance problem associated with at least one automation component 106, and / or any other type of action associated with identifying and / or resolving a performance problem of an automation component. In a non-limiting example, the automation component 106 is a hardware and / or software component in a computing environment. In this context, the actions 132 may include modifying the configuration of at least one automation component 106, updating and / or patching software and / or firmware, performing code analysis operations with respect to one or more automation components 106, performing virus scanning operations with respect to one or more automation components 106, allocating one or more additional computing resources to at least one automation component 106, performing load balancing operations, and / or any other type of action associated with identifying and / or resolving performance issues with hardware and / or software components in a computing environment. Depending on the embodiment, the system 100 may be configured to perform any of the above actions, and alternatively may transmit commands to a system external to the system 100 to perform the action. Figure 1As shown in FIG. 1 , the segmentation and evaluation component 104 receives the automation component descriptor 122, the segmentation feature value 124, and the performance feature value 126 from the pre-processing component 114. Alternatively, the segmentation and evaluation component 104 may be obtained from the pre-processing component 114 for simplicity. Figure 1 One or more of the automation component descriptor 122 , the segmentation characteristic value 124 , and / or the performance characteristic value 126 are obtained from a database (not shown).

[0040] According to the embodiment, and as will be referred to below Figure 5 As discussed, the segmentation and evaluation component 104 is configured to apply a K-means clustering algorithm to group the automation components 106, including initializing a set of cluster centroids based on contextual rules 128. The contextual rules 128 may include and / or be based on a subset of segmentation features corresponding to the segmentation feature values ​​124. Figure 1 As shown in , the segmentation and evaluation component 104 can receive contextual rules 128 from the user interface 108 as a form of user input; however, alternatively or additionally, one or more contextual rules 128 can be predetermined and / or automatically determined (e.g., by the pre-processing component 114 or by the segmentation and evaluation component 104).

[0041] like Figure 1 As shown in , the system 100 may include a user interface 108 configured to enable a user to interact with the segmentation and evaluation component 104. For example, the user interface 108 may enable a user to select one or more contextual rules 128 to apply a clustering algorithm of the segmentation and evaluation component 104. In an embodiment, the user interface 108 receives the segmentation feature values ​​124 and enables the user to select one or more contextual rules 128 based on the segmentation feature values ​​124. The user may utilize the user interface 108 to otherwise transmit user input 130 to the segmentation and evaluation component 104 to modify the operation of the segmentation and evaluation component 104. For example, as will be described below with reference to Figure 2 As discussed, a user may transmit user input 130 to change the feature importance value of a given performance feature (eg, from a positive value to a corresponding negative value).

[0042] Note that the segmentation and evaluation component 104 can be implemented in various ways to perform its functions. For example, according to an example embodiment, Figure 2 is a block diagram of a segmentation and assessment component 200 that can be used to identify and resolve performance issues of automation components. The segmentation and assessment component 200 is Figure 1 Another embodiment of the segmentation and evaluation component 104 of Figure 1 The segmentation and evaluation component 200 is described based on the system 100. Based on the following description, more structural and operational examples will be clear to those skilled in the relevant art(s).

[0043] The segmentation and evaluation component 200 may be implemented on a single computing device or across multiple computing devices. The segmentation and evaluation component 200 receives the automation component descriptor 122, the segmentation feature value 124, the performance feature value 126, the contextual rule 128, and the user input 130. Figure 2 As shown in FIG. 1 , the segmentation and evaluation component 200 includes a clustering component 202, a ranking component 204, and a ranking interpreter 206. The clustering component 202 is configured to segment the plurality of automation components 106 into groups by applying a clustering algorithm (e.g., a K-means clustering algorithm) to the plurality of automation components 106 based on the segmentation feature values ​​124 respectively associated with the automation components 106. For example, the clustering component 202 generates an automation component group 208 (hereinafter referred to as “group 208”). As will be described below, Figure 4 , Fig. 6A and Figure 6B As discussed, the group 208 may include a selected set of cluster centroids. Additionally, an initial seed (eg, an initial set of cluster centroids) may be selected for the clustering algorithm based on the contextual rules 128 .

[0044] The ranking component 204 is configured to: for each group 208, rank each automation component of the group based on at least the performance characteristic value 126 respectively associated with the automation component. For example, the ranking component 204 generates a ranking set 210 for the automation components of each group 208 based on at least the performance characteristic value 126 respectively associated with the automation component. In an embodiment, the ranking set 210 may include a ranking of one of the groups 208, a corresponding ranking of each of the groups 208, or a corresponding ranking of a subset of the groups 208. As described below with respect to Figure 7 and Figure 8 As discussed, a performance score can be calculated for each automation component. In this context, the ranking component 204 is configured to generate a ranking set 210 based on the calculated performance scores. In an embodiment, the impact of one or more performance feature values ​​on the performance score can be automatically changed (e.g., by the ranking component 204) and / or manually changed (e.g., by user input 130).

[0045] The ranking interpreter 206 is configured to perform actions 132 based on the ranking of the at least one automation component. As discussed above, the actions 132 may include generating an alert, generating one or more suggested actions for resolving a performance issue associated with the at least one automation component 106, initiating one or more automated processes for resolving a performance issue associated with the at least one automation component 106, and / or any other type of action associated with identifying and / or resolving performance issues of automation components. For example, the ranking interpreter 206 may perform an alert by transmitting an alert to the Figure 1 The user interface 108 of the embodiment of the present invention is used to perform action 132. In an embodiment, action 132 includes comparing at least one characteristic of one or more top-ranked automation components with at least one characteristic of one or more bottom-ranked automation components. In this way, characteristics that contribute to improving the performance of the automation components can be determined.

[0046] In an embodiment, Figure 2 The segmentation and evaluation component 200 can operate in various ways. For example, according to an example embodiment, Figure 3 300 is a flowchart of a process for identifying and resolving performance issues of an automation component. In an embodiment, the segmentation and evaluation component 200 can be operated to perform one or all steps of the flowchart 300. The flowchart 300 is as follows: Figure 1 System 100 and Figure 2 Based on the following description, further structural and operational embodiments will be apparent to those skilled in the relevant art(s). Please note that not all steps of the flowchart 300 need to be performed in all embodiments.

[0047] Flowchart 300 begins at step 302. In step 302, for each of a plurality of automation components, a corresponding segmentation feature value set and a corresponding performance feature value set are received. For example, Figure 2 The segmentation and evaluation component 200 is configured to: for each automation component in the automation component 106, from Figure 1 The pre-processing component 114 receives the segmentation characteristic values ​​124 and the performance characteristic values ​​126 (as described by the automation component descriptor 122). In an embodiment, the segmentation and evaluation component 200 can retrieve the automation component descriptor 122, the segmentation characteristic values ​​124, and / or the performance characteristic values ​​126 from a database.

[0048] In step 304, the plurality of automation components are segmented into groups by applying a clustering algorithm to the plurality of automation components based on the segmentation feature values ​​respectively associated with the automation components. For example, the clustering component 202 is configured to segment the plurality of automation components 106 into the groups 202 by applying the clustering algorithm to the plurality of automation components 106 based on the segmentation feature values ​​124. In an embodiment, the clustering algorithm is a K-means clustering algorithm. Figure 5 As discussed, the clustering component 202 can select an initial set of cluster centroids based on the contextual rules 128 .

[0049] At step 306, for each group of automation components, a ranking of the automation components of the group is generated based at least on the set of performance characteristic values ​​respectively associated with the automation component group. For example, the ranking component 204 is configured to generate a ranking set 210 for each group 208 based at least on the performance characteristic values ​​126 respectively associated with the group. In an embodiment, the ranking set 210 can include a corresponding ranking for one of the groups 208, a subset of the groups 208, or all of the groups 208.

[0050] In step 308, an action is performed based on the ranking of the at least one automation component. For example, the ranking interpreter 206 is configured to perform the action 132 based on at least one ranking of the ranking set 210. As discussed above, the action 132 can include generating an alert, generating one or more recommended actions for resolving a performance issue associated with the at least one automation component 106, initiating one or more automated processes for resolving a performance issue associated with the at least one automation component 106, and / or any other type of action associated with identifying and / or resolving a performance issue of an automation component.

[0051] In an embodiment, a user may modify or control one or more steps, for example, via the user interface 108. For example, a user may select a subset of the segmentation feature values ​​124 upon which the application of the clustering algorithm in step 304 will be based. In another example, a user may select a subset of the performance feature values ​​126 upon which the ranking component 204 will generate the ranking set 210.

[0052] III. Example Systems and Methods for Clustering

[0053] The systems and methods described herein segment a plurality of automation components into groups by applying a clustering algorithm to the plurality of automation components based on segmentation feature values ​​associated with the automation components. Figure 2 The clustering component 202 can be implemented in various ways to apply a clustering algorithm to segment the plurality of automation components 106. For example, in an example embodiment, the clustering component 202 is configured to apply a K-means clustering algorithm to segment the plurality of automation components 106. In this example, the clustering component 202 can be implemented in various ways to apply a clustering algorithm to segment the plurality of automation components 106. Figure 1 The context rule 128 initializes the cluster centroids of the K-means clustering algorithm.

[0054] Figure 2 The clustering component 202 can be implemented in various ways to apply the K-means algorithm to multiple automation components. For example, Figure 4 is a block diagram of a clustering component 400 according to an example embodiment. The clustering component 400 is Figure 2 Another embodiment of the clustering component 202 of FIG. Figure 1The clustering component 400 is described with reference to the system 100. Further structural and operational examples will be apparent to those skilled in the relevant art(s) based on the following description.

[0055] like Figure 4 As shown in , the clustering component 400 includes an initial cluster centroid selector 402, a cluster generator 404, and a cluster quality assessment and selection component 406. In an embodiment, the clustering component 400 can be configured to interface with a data repository 414. The data repository 414 can store cluster sets 416, cluster quality measures 418, and / or any other information associated with the operation of the clustering component 400 described herein (e.g., data associated with automation components, segmentation features, user preferences, previous cluster centroids, etc.). The data repository 414 can be implemented on one or more memory devices (e.g., volatile or non-volatile memory devices) accessible to the clustering component 400, and can be internal or external to the computing device on which the clustering component 400 is executed. According to an embodiment, the cluster sets 416 and cluster quality measures 418 are stored in the data repository 414 as data structures. The following is further described in detail. Figure 4 Subcomponents of the clustering component 400 shown in .

[0056] The initial cluster centroid selector 402 is configured to select an initial cluster centroid set 420. For example, Figure 4 As shown in , the initial cluster centroid selector receives the automation component descriptor 122, the segmentation feature value 124, and the contextual rule 128. By using the contextual rule 128 to select the initial cluster centroid set 420, the initial cluster centroid selector 402 increases the chance that the groups 208 generated by the clustering component 400 make logical sense to rank the performance of the automation components. In a non-limiting example, no additional parameters or degrees of freedom are introduced in the initial cluster centroid selector 402 or in the subsequent clustering process. In this context, the replicability of the cluster assignments is improved, thereby reducing the differences between different runs of the clustering component 400 without introducing additional parameters in the clustering process.

[0057] The cluster generator 404 is configured to generate a set of clusters 422 and a cluster quality measure 424. Figure 4 As shown in FIG. 4 , the cluster generator 404 includes an algorithm executor 408, a profile calculator 410, and a cluster limit evaluator 412. The algorithm executor 408 is configured to apply a clustering algorithm (e.g., a K-means clustering algorithm) starting from an initial cluster centroid set 420. For example, according to the following description of Fig. 6AIn the discussed embodiment, the algorithm executor 408 is configured to apply a K-means clustering algorithm to determine a set of clusters that meet a convergence criterion. For example, in the context of a K-means clustering algorithm, the convergence criterion includes a K-means cost function, which is the sum of the distances from the automation components to their assigned cluster centroids. In this example, the application executor 408 is configured to minimize the K-means cost function value to determine a suitable set of clusters 422. Figure 4 As shown in , algorithm executor 408 may store set of clusters 422 in data repository 414 as set of clusters 416 .

[0058] In an embodiment, the cluster generator 404 may be configured to generate (e.g., via the algorithm executor 408) multiple cluster sets corresponding to different numbers of clusters. In this case, the cluster quality of each of the different numbers of clusters is evaluated. However, the K-means cost function is affected by the number of clusters. Generally, the greater the number of clusters, the smaller the K-means cost function value. To this end, the cluster generator 404 may be configured to use another measure to evaluate the quality between different cluster sets. For example, Figure 4 As shown in FIG. 4 , cluster generator 404 includes a silhouette calculator 410. Silhouette calculator 410 is configured to determine cluster quality measure 424 by calculating an average silhouette coefficient based on segmentation feature values ​​124 of automation components 106 and cluster set 422. According to an embodiment, the calculation of the silhouette coefficient is based on a complete pairwise distance matrix of all automation components. For example, as described below with respect to Fig. 6A As discussed, the silhouette calculator 410 calculates a silhouette score for each automation component by measuring how similar the automation component is to its assigned cluster compared to other clusters in the cluster set 422. In this context, a high value indicates that the automation component matches its assigned cluster well and matches its neighboring clusters poorly. The average silhouette coefficient is calculated as the average of the silhouette scores across multiple automation components 106. Figure 4 As shown in , the silhouette calculator generates a cluster quality measure 424 representing the calculated average silhouette coefficient and stores the cluster quality measure 424 in the data repository 414 as part of the cluster quality measures 418 .

[0059] As mentioned above, the cluster generator 404 can iterate over different numbers of clusters. In this context, the cluster limit evaluator 412 is configured to determine whether the cluster generator 404 has iterated over a certain number of cluster centroids. For example, as described below with respect to Figure 6B As discussed, the algorithm executor 408 and the profile calculator 410 may generate a cluster set 422 and a cluster quality measure 424 for each of a plurality of cluster centroids within a certain range. The initial number of cluster centroids may be predefined, as determined by Figure 1The number of cluster centroids may be determined automatically by the system 100 (e.g., via the pre-processing component 114) or selected by the user (e.g., via the user interface 108). In addition, the pre-defined minimum number of cluster centroids may be defined based on a default number (e.g., two clusters), automatically defined by the system 100, or defined by the user via the user interface 108. For each iteration, the cluster constraint evaluator 412 determines whether the number of cluster centroids in the cluster set 422 is equal to the pre-defined minimum number of cluster centroids. If so, the cluster generator 404 completes the clustering process. Otherwise, the cluster constraint evaluator reduces the number of cluster centroids in the cluster set 422 by 1, and the algorithm executor 408 and the profile calculator 410 repeat their respective processes for the reduced number of cluster centroids. Reference will be made to the following. Figure 6B A non-limiting example of a process for reducing the number of cluster centroids in the set of clusters 422 is described below.

[0060] The cluster quality assessment and selection component 406 is configured to select a cluster set having a highest cluster quality measure from the cluster set 416. For example, the cluster quality assessment and selection component 406 determines the maximum average silhouette coefficient from the cluster quality measures 418 and selects the set of clusters in the cluster set 416 corresponding to the maximum average silhouette coefficient. Figure 4 As shown in , the cluster quality assessment and selection component 406 obtains (eg, by retrieving) a set of clusters 416 and a cluster quality measure 418 from a data repository 414 .

[0061] In an embodiment, the initial cluster centroid set 420 may be selected in various ways. For example, according to an example embodiment, Figure 5 is a flowchart 500 of a process for initializing a set of cluster centroids. In an embodiment, the flowchart 500 is Figure 3 In an embodiment, the initial cluster centroid selector 402 may be operable to perform one or all of the steps of the flowchart 500. The flowchart 500 is as follows: Figure 1 System 100 and Figure 4 Based on the following description, further structural and operational embodiments will be apparent to those skilled in the relevant art(s). Please note that not all steps of flowchart 500 need to be performed in all embodiments.

[0062] Flowchart 500 begins at step 502. In step 502, a subset of segmentation features is identified. For example, a subset of segmentation features may be identified from the segmentation features corresponding to segmentation feature value 124. In an embodiment, the subset of segmentation features may be composed of Figure 1 The pre-processing component 114 identifies Figure 4 The initial cluster centroid selector 402 identifies the initial cluster centroid selector 402, which can be selected by the user via Figure 1 The user interface 108 of the system 100 may be selected (e.g., selected as user input), may be retrieved from a configuration file, etc. via a component of the system 100 (e.g., the pre-processing component 114 or the initial cluster centroid selector 402). Alternatively, a component of the system 100 (e.g., the user interface 108, the pre-processing component 114, or the initial cluster centroid selector 402) may be pre-programmed to use a specific subset of segmentation features.

[0063] In step 504, a set of contextual rules is generated based on at least the subset of segmentation features. For example, the contextual rules 128 are generated based on at least the subset of segmentation features identified in step 502. In an embodiment, the contextual rules 128 may be generated by the user interface 108 (e.g., generated as user input), by the pre-processing component 114, or by the initial cluster centroid selector 402. According to an embodiment, the contextual rules 128 may be stored in a memory (e.g., as a configuration file).

[0064] In step 506, the set of cluster centroids is initialized based on the set of contextual rules. For example, the initial cluster centroid selector 402 selects the initial set of cluster centroids 420 based on the contextual rules 128. In an embodiment, the initial cluster centroid selector 402 may receive the contextual rules 128 from the user interface 108 (e.g., as user input), or may retrieve the contextual rules 128 from a configuration file, etc. Step 506 may include selecting a number of cluster centroids. The number of cluster centroids to be selected may be predefined for the clustering component 400, determined by Figure 1 The processing may be determined automatically by a component of the system 100 (eg, the pre-processing component 114) or indicated by a user (eg, via the user interface 108).

[0065] In an embodiment, the group 208 may be generated in various ways. For example, according to an example embodiment, Fig. 6A and Figure 6B is a flow diagram of a process 600 for segmenting multiple automation components. Figure 6B yes Fig. 6A In an embodiment, process 600 is Figure 3 A subset of step 304 of flowchart 300, and Figure 5 In an embodiment, the cluster generator 404 and / or the cluster quality assessment and selection component 406 may be operable to perform one or all steps of the process 600. Figure 1 System 100 and Figure 4The clustering component 400 is described as follows. Further structural and operational embodiments will be apparent to those skilled in the relevant art(s) based on the following description. Please note that not all steps of the process 600 need to be performed in all embodiments.

[0066] like Fig. 6A As shown in , process 600 starts at step 602. In step 602, the K-means clustering algorithm is applied until it is determined whether the convergence criteria have been met for the set of cluster centroids. For example, Figure 4 The algorithm executor 408 may apply the K-means clustering algorithm to the plurality of automation components 106 until a convergence criterion is met. According to an embodiment, the convergence criterion is met when the K-means cost function value is minimized. Step 602 is performed on the cluster set based on the initial cluster centroid set 420. Step 602 includes sub-steps 604, 606, and 608.

[0067] In sub-step 604, for each automation component in the automation components, a nearest centroid in the set of cluster centroids is determined based on the corresponding set of segmentation feature values ​​of the automation component. Figure 4 The algorithm executor 408 may apply the K-means clustering algorithm to the plurality of automation components 106 until a convergence criterion is met. According to an embodiment, the convergence criterion is met when the K-means cost function value is minimized. Step 602 is performed on the cluster set based on the initial cluster centroid set 420. Step 602 includes sub-steps 604, 606, and 608.

[0068] In sub-step 604, for each automation component, the nearest centroid in the set of cluster centroids is determined based on the corresponding set of segmentation feature values ​​of the automation component. Figure 4 The algorithm executor 408 is configured to determine, for each of the plurality of automation components 106 , a nearest centroid of the initial set of cluster centroids 420 based on the segmentation feature values ​​124 corresponding to the automation component.

[0069] In sub-step 606, for each of the automation components, the automation component is assigned to the determined nearest centroid. Figure 4 The algorithm executor 408 is configured to, for each of the plurality of automation components 106 , assign the automation component to the closest centroid determined in step 604 .

[0070] In sub-step 608, for each cluster centroid in the set of cluster centroids, the cluster centroid is updated based on which automation components are assigned to the cluster centroid. Figure 4The algorithm executor 408 is configured to: for each cluster centroid of the initial cluster centroid set 420, update the cluster centroid based on which automation components of the plurality of automation components 106 are assigned to the cluster centroid. In an embodiment, sub-steps 604, 606, and 608 are repeated until the convergence criteria for the cluster centroid set is met, as described above with respect to step 602. In an embodiment, the algorithm executor 408 generates a cluster set 422, wherein the cluster set 422 is a cluster set having cluster centroids that meet the convergence criteria. As described above with respect to Figure 4 As described above, algorithm executor 408 may store set of clusters 422 in data repository 414 .

[0071] In step 610, an average silhouette coefficient is calculated based on a plurality of automated components and a set of cluster centroids. For example, Figure 4 The silhouette calculator 410 calculates an average silhouette coefficient based on the segmentation feature values ​​124 of the automation components 106 and the set of clusters 422. According to an embodiment, the silhouette calculator 410 calculates a silhouette score for each automation component in the plurality of automation components 106 by measuring how similar the automation component is to its assigned cluster compared to other clusters in the set of clusters 422. The silhouette calculator 410 calculates an average of the silhouette scores of the plurality of automation components 106 to determine an average silhouette coefficient. In an embodiment, the silhouette calculator 410 generates a cluster quality measure 424 based on the average silhouette coefficient.

[0072] In step 612, the average silhouette coefficient is stored in an average silhouette coefficient set. For example, Figure 4 The profile calculator 410 stores the cluster quality measures 424 in the cluster quality measures 418 of the data repository 414. According to an embodiment, the cluster quality measures 418 are keys of a data structure (eg, a Python dictionary), each key having a corresponding cluster set in the cluster set 416 as a value.

[0073] like Figure 6B As shown in , process 600 continues with step 614. In step 614, the number of cluster centroids in the set of cluster centroids is reduced, and steps 602, 610, and 612 are repeated until the number of cluster centroids in the set of cluster centroids equals a predefined minimum number of cluster centroids. For example, Figure 4The cluster limit evaluator 412 is configured to determine whether the number of cluster centroids in the cluster set 422 is equal to a predefined minimum number of cluster centroids. If so, the process 600 proceeds to step 622. Otherwise, the cluster limit evaluator 412 reduces the number of cluster centroids in the cluster set 422, and the process 600 performs steps 602, 610, and 612 with respect to the reduced cluster set 422. In an embodiment, the predefined minimum number of cluster centroids may be a default number (e.g., two clusters), a number automatically determined by the system 100 (e.g., via Figure 1 The number of pre-processing components 114), or a number defined by a user (e.g., via the user interface 108). Figure 6B As shown in , step 614 includes sub-steps 616 , 618 , and 620 .

[0074] In sub-step 616, the closest cluster centroid pair is determined from among the cluster centroid set. For example, Figure 4 The cluster constraint evaluator 412 is configured to determine the nearest pair of cluster centroids from the set of clusters 422 .

[0075] In sub-step 618, the determined nearest cluster centroid pairs are merged to simplify the cluster centroid set. Figure 4 The cluster constraint evaluator 412 is configured to merge the closest pairs of cluster centroids determined in sub-step 616 to simplify the set of cluster centroids.

[0076] In sub-step 620, the K-means clustering algorithm is repeatedly applied to the plurality of automation components, the average silhouette coefficient is calculated, and the average silhouette coefficient is stored. Figure 4 The algorithm executor 408 is configured to receive the simplified set of cluster centroids generated by the cluster restriction evaluator 412 in sub-step 616. In this context, the algorithm executor 408 performs step 602 (including sub-steps 604, 606, and 608) with respect to the simplified set of cluster centroids. In addition, the profile calculator 410 performs steps 610 and 612 with respect to the simplified set of cluster centroids. In an embodiment, step 614 (including sub-steps 616, 618, and 620) is repeated until the number of cluster centroids in the cluster set 422 is equal to a predefined minimum number of cluster centroids.

[0077] In step 622, the maximum average silhouette coefficient is determined from the set of average silhouette coefficients. For example, Figure 4 The cluster quality assessment and selection component 406 is configured to obtain a cluster quality measure 418 (eg, from the data repository 414 ) and determine a maximum average silhouette coefficient from the cluster quality measure 418 .

[0078] In step 624, the cluster centroid set corresponding to the maximum average silhouette coefficient is selected to generate a selected cluster centroid set. Figure 4 The cluster quality assessment and selection component 406 is configured to select the set of cluster centroids corresponding to the maximum average silhouette coefficient determined in step 624 from the set of clusters 416 to generate the group 208. According to an embodiment, the set of clusters 416 is a data structure indexed by the corresponding cluster quality measures 418. For example, the data structure (e.g., a Python dictionary) includes the cluster quality measures 418 as keys, and each key has a corresponding set of clusters in the set of clusters 416 as a value.

[0079] As described herein, the automation components may be segmented into groups by applying a clustering algorithm to the automation components based on segmentation feature values ​​respectively associated with the automation components. In an embodiment, the automation components may be segmented into groups according to Algorithm 1 shown below in this document:

[0080]

[0081]

[0082]

[0083] Continue to refer Figure 4 The clustering component 400, Figure 5 500, and Fig. 6A and Figure 6B The process 600 of Algorithm 1 is used to describe Algorithm 1. For Algorithm 1, E = {e1, e2 ... e n} is a set of n automation components (eg, the plurality of automation components 106, as described by the automation component descriptor 122). Each automation component e∈E is represented by a corresponding F={f1, f2 . . . f m} and the corresponding denoted by (as discussed below with respect to Algorithm 2), where F is a set of m segmentation feature values ​​(e.g., segmentation feature value 124), and P is a set of z performance feature values ​​(e.g., performance feature value 126). B = {b1, b2, ..., b r} is a context rule set (e.g., context rule 128).

[0084] Initialize K cluster centroids, μ = {μc1, μc2, …, μc}, by applying B to E and considering (e.g., only considering) F (Algorithm 1, Line 1). k}.

[0085] In Algorithm 1, line 2, an empty dictionary clusters_quality is created.

[0086] After initializing the set of cluster centroids in line 1 of Algorithm 1 and creating the clusters_quality dictionary in line 2, the K-means clustering algorithm is applied to E in lines 4-12 of Algorithm 1, while also considering (e.g., only considering) F. Algorithm 1 loops in lines 4-12, but the convergence criteria (e.g., the K-means cost function value is minimized) have not yet been met.

[0087] Specifically, Algorithm 1 loops through each automation component in E in lines 5-8 and assigns each automation component to a cluster c i Specifically, the cluster centroid in μ that is closest to the automation component is determined (e.g., based on the segmentation feature value corresponding to the automation component in F) (Algorithm 1, line 6), and the automation component is assigned to the cluster C = {c1, c2, ..., c k} (Algorithm 1, line 7).

[0088] Furthermore, Algorithm 1 loops for each cluster in C and recomputes the cluster centroids in μ in Lines 9-11.

[0089] As mentioned above about Figure 4 As discussed in the contour calculator 410 of , since the K-means cost function is affected by the number of clusters in C, an additional measure is used to evaluate the quality of C. To determine the quality of C, Algorithm 1 calculates and stores the average contour coefficient of C at lines 13 and 14. In particular, a contour score is determined for each automation component in E, and the average of the contour scores on E (e.g., the average contour coefficient) is calculated (Algorithm 1 line 13). For example, in an embodiment, the average contour coefficient is calculated according to Formula 1 as follows:

[0090]

[0091] In Formula 1, sil(e) is the silhouette score of the automation component e in E, and n is the number of automation components in E. In line 13 of Algorithm 1, the average silhouette coefficient of C is stored in the variable avg SC middle.

[0092] After calculating the average silhouette coefficient in line 13 of Algorithm 1, the average silhouette coefficient and C are stored in clusters_quality. SC as a key.

[0093] As indicated in row 3, Algorithm 1 loops until K equals 1 or another limit (e.g., as described above with respect to Figure 4 The clustering restriction evaluator 412 and Figure 6BTo proceed to the next iteration of the loop in lines 3-20, Algorithm 1 decreases K.

[0094] Specifically, the two clusters with the closest centroids are merged (Algorithm 1, line 15), and K is reduced by 1 (Algorithm 1, line 16).

[0095] Initialize μ by computing the average of the assigned automation components of E (Algorithm 18, line 18), and perform a loop for each cluster in C (Algorithm 1, lines 17-19).

[0096] After reducing K and initializing μ, Algorithm 1 repeats the loop of Lines 3-20 until K equals 1 or its constraints are satisfied.

[0097] After completing the loop of Algorithm 1, Lines 3-20, the clustering quality is evaluated and a cluster set is selected in Algorithm 1, Lines 21-23, as described below.

[0098] To determine which average silhouette coefficient is the highest, the max function is applied to the keys of the clusters_quality dictionary, and the result is stored in the variable max SC (Algorithm 1, line 21). max SC is used as a key to obtain the cluster set with the maximum average silhouette coefficient, and the cluster set is stored as C (Algorithm 1, line 22).

[0099] After completing Algorithm 1, Fig. 6A and Figure 6B The process 600 of is also completed, and the embodiment can use the cluster set C returned by Algorithm 1 to evaluate and rank the performance of the automation components within the clusters.

[0100] IV. Example Systems and Methods for Ranking

[0101] The systems and methods described herein rank automation components within a group based at least on a set of performance characteristic values ​​associated with each of the automation components. Figure 2 The ranking component 204 can be configured in various ways to rank the automation components in the group 208. For example, according to an example embodiment, Figure 7 is a flowchart 700 of a process for ranking components. Flowchart 700 is Figure 3 In an embodiment, the ranking component 204 can be operated to perform one or all steps of the flowchart 700. For the purpose of illustration, Figure 8 Describes Figure 7 700. According to an example embodiment, Figure 8A block diagram of a ranking component 800 is shown. The ranking component 800 is Figure 2 Another embodiment of the ranking component 204. Figure 8 As shown in FIG. 8 , the ranking component 800 includes a machine learning-based classification model 802 (hereinafter “model 802”), a performance score calculator 804, and a ranking generator 806. The flowchart 700 and the ranking component 800 are described as follows. Note that in some embodiments, the steps of the flowchart 700 may be performed in the same manner as described in the example of FIG. Figure 7 In addition, not all steps of flowchart 700 need to be performed in all embodiments. Based on the following description, more structural and operational embodiments will be clear to those skilled in the relevant art(s).

[0102] Flowchart 700 begins with step 702. In an embodiment, step 702 is performed at Figure 6B The process 600 is then performed after step 624. In step 702, a feature importance value for each of the set of performance features is determined by training an ML-based classification model. The ML-based classification model predicts a specific group in the group based on the performance feature value of the automation component. For example, the model 802 is trained using the performance feature value 126 of the automation component 106 (as described by the automation component descriptor 122), where the group 208 is the target attribute. The training of the model 802 generates a feature importance value 808 for each performance feature in the set of performance features. The feature importance values ​​808 can be used to rank the automation components, to indicate the impact of the performance feature on the performance of the automation components, to distinguish between automation component attributes in different groups of the group 208, to process related features, and / or to otherwise explain and / or analyze the performance of one or more automation components. Figure 8 As shown in , model 802 can be configured to receive user input 130. According to an embodiment, user input 130 changes the feature importance value of a given performance feature (e.g., from a positive value to a corresponding negative value). Depending on the implementation, model 802 can be a gradient boosted decision tree model, a regularized regression model, a random forest model, or other types of ML-based classification models. Figure 8 As shown in , the feature importance values ​​generated by the training model 802 can be provided as outputs of the ranking component 800. For example, the feature importance values ​​808 can be transmitted to Figure 1 's user interface 108 to enable a user to review feature importance values ​​808 (eg, to be used to generate user input 130).

[0103] At step 704, for each of the automation components, a performance score of the automation component is calculated based on the performance feature value of the automation component and the feature importance value of each performance feature. For example, the performance score calculator 804 is configured to calculate the performance score 810 for each of the plurality of automation components 106 based on the performance feature value 126 and the feature importance value 808. According to an embodiment, the performance score calculator 804 calculates the performance score of the automation component by multiplying each performance feature value of the automation component by the corresponding feature importance value, and dividing the result by the sum of the feature importance values.

[0104] In step 706, for each group of each automation component, a ranking of the automation components of the group is generated based on the corresponding performance score. For example, the ranking generator 806 is configured to generate the ranking set 210 by generating, for each group, a ranking of the automation components of the group based on the corresponding performance score 810. The ranking set 210 can include a ranking for each of the plurality of automation components 106, for an automation component, for automation components of a group, and / or for automation components of a subset of the group 208.

[0105] As described herein, based on the performance feature values ​​of the automation components and the feature importance values ​​of the performance features, the automation components can be ranked within the group. In an embodiment, the automation components can be ranked according to Algorithm 2 shown below:

[0106]

[0107]

[0108] Continue to refer Figure 7 Flowchart 700 and Figure 8 Algorithm 2 is described using the ranking component of . As discussed above with respect to Algorithm 1, E is a set of n automation components (eg, the plurality of automation components 106 ), C is a set of K clusters, and P is a set of z performance feature values ​​(eg, the performance feature values ​​126 ).

[0109] For each performance feature corresponding to the corresponding performance feature value P, calculate the feature importance value PI = {pi1, pi2 ... pi z}(Algorithm 1, line 1). In Algorithm 2, a gradient boosted decision tree model is trained to generate PI.

[0110] After calculating PI, Algorithm 2 loops through each automation component of E (lines 2-4) and calculates the performance score s of the automation component in line 3 to generate a set of performance scores S = {s1, s2…s nFor example, in an embodiment, the performance score of the automation component is calculated according to Formula 2 as follows:

[0111]

[0112] In Formula 2, s is calculated by summing the products of each performance characteristic value of the automation component and the corresponding characteristic importance value and dividing the result by the sum of the characteristic importance values.

[0113] After completing Algorithm 2, Figure 7 The flowchart 700 of is also completed, and an embodiment can use the feature importance value PI and / or the performance score S returned by Algorithm 2 to identify and / or solve performance problems in automation components. For example, according to an embodiment, a recommendation is made for the automation component based on the best performing automation component in the same cluster. In this context, the recommendation can include an indicator that the automation component is lower / higher than the average of all automation components in the cluster, lower / higher than the average of a subset of automation components in the cluster (e.g., the best performers), and / or other indicators associated with the performance and / or ranking of the automation component.

[0114] V. Further Example Embodiments and Advantages

[0115] As described above, the systems and devices can be configured in various ways to identify and resolve performance issues. Example embodiments are described with respect to automated components (such as hardware components and software components); however, other types of automated components can be monitored and evaluated to identify and / or resolve performance issues. For example, the system 100 can be configured to monitor automated components (such as telecommunications equipment in a telecommunications network, vehicles in a fleet of vehicles, cooling units in a data center, etc.) to identify and / or resolve performance issues in one or more of these automated components. In addition, the embodiments and techniques described herein can be applied to business entities (including partners, suppliers, customers, or any other entity actively involved in the business), and their performance can be evaluated over time. Furthermore, in embodiments, the systems or methods described herein can be implemented in an analytical service, such as a customer relationship management (CRM) tool (e.g., Microsoft application, CRM applications, Freshsales TM CRM software, and CRM applications, etc.), analytical services associated with cloud computing services, retail analytics software provided by software vendors (e.g. Oracle, SAP、Celerant Technology、 etc.) and so on.

[0116] The systems and methods described herein are described with respect to a K-means clustering algorithm. In particular, embodiments enable selection of an initial set of cluster centroids for the K-means clustering algorithm based on contextual rules. As described elsewhere herein, contextual rules can be used to identify a subset of segmentation features to generate an initial set of cluster centroids. In this way, groups created based on the results of the K-means clustering algorithm have the potential to make logical sense for the performance ranking of automation components. In addition, it is contemplated herein that the system can be configured to use a clustering algorithm other than the K-means clustering algorithm (e.g., a hierarchical clustering algorithm, a database scanning algorithm, or other types of algorithms suitable for segmenting multiple automation components into groups). In this case, the contextual rules can be inputs to initialize the clustering algorithm.

[0117] The embodiments described herein (e.g., as described above with respect to Figure 4 The contour calculator 410 and Figure 6A-6B Process 600 discussed above includes calculating the average silhouette coefficient of iteratively smaller sets of cluster centroids until a minimum set (e.g., based on a predefined minimum number of cluster centroids), and then selecting the set of cluster centroids with the best (e.g., highest) average silhouette coefficient. In this way, the system according to an embodiment selects a set of centroid clusters that provide good (e.g., best) clustering while avoiding increasing the set of cluster centroids to an undesirable degree in the name of obtaining good (e.g., best) clustering. However, depending on the implementation, embodiments of the present disclosure may use another measure to improve the selection of the set of cluster centroids in addition to or in lieu of the calculation of the average silhouette coefficient.

[0118] In addition, according to the described embodiments and techniques, based on other operations, functions, actions, etc. (including the initialization, completion, and / or execution of operations, functions, actions, etc.), any component of the system, data collection and preprocessing component, segmentation and evaluation component, clustering component, ranking component, ranking interpreter, and / or user interface and its functions can be activated for operation / execution.

[0119] In some example embodiments, one or more operations of the flowcharts described herein may not be performed. In addition, operations may be performed in addition to or in place of the operations of the flowcharts described herein. In addition, in some example embodiments, one or more operations of the flowcharts described herein may be performed out of order, in an alternating order, or partially (or completely) simultaneously with each other or other operations.

[0120] The further example embodiments and advantages described in this section may apply to any embodiments disclosed in this section or any other part of the disclosure.

[0121] The embodiments described herein and / or any further systems, subsystems, devices, and / or components disclosed herein may be implemented with hardware (e.g., hardware logic / circuitry) or any combination of hardware and software (computer program code configured to be executed in one or more processors or processing devices) and / or firmware.

[0122] VI. Example Computer System Implementation

[0123] System 100, data collection and preprocessing component 102, segmentation and evaluation component 104, automation component 106, user interface 108, data collector 110, feature identifier 112, preprocessing component 114, segmentation and evaluation component 200, clustering component 202, ranking component 204, ranking interpreter 206, flowchart 300, clustering component 400, initial cluster centroid selector 402, cluster generator 404, cluster quality assessment and selection component 406, algorithm executor 408, profile calculator 410, cluster restriction evaluator 412, data The repository 414, the cluster set 416, the cluster quality measure 418, the flowchart 500, the process 600, the flowchart 700, the ranking component 800, the model 802, the performance score calculator 804, and / or the ranking generator 806 can be implemented in hardware or any combination of hardware and software and / or firmware (including being implemented as a computer program code that is configured to be executed in one or more processors and stored in a computer-readable storage medium), or as a hardware logic / circuitry system (such as being framed as a system on a chip (SoC)). The SoC may include an integrated circuit chip that includes one or more processors (e.g., a microcontroller, a microprocessor, a digital signal processor (DSP), etc.), a memory, one or more communication interfaces and / or more circuits and / or embedded firmware to perform its functions.

[0124] Fig. 9 An exemplary implementation of a computer system 900 (herein "system 900") in which embodiments may be implemented is depicted. For example, system 900 may be used to implement system 100, data collection and preprocessing component 102, segmentation and evaluation component 104, automation component 106, user interface 108, data collector 110, feature identifier 112, and / or preprocessing component 114, as described above with reference to Figure 1 The system 900 can also be used to implement the segmentation and evaluation component 200, the clustering component 202, the ranking component 204, and / or the ranking interpreter 206, as described above with reference to Figure 2The system 900 can also be used to implement the clustering component 400, the initial cluster centroid selector 402, the cluster generator 404, the cluster quality assessment and selection component 406, the algorithm executor 408, the profile calculator 410, the cluster restriction evaluator 412, the data repository 414, the cluster set 416, and the cluster quality measure 418, as described above with reference to Figure 4 The system 900 can also be used to implement the ranking component 800, the model 802, the performance score calculator 804, and / or the ranking generator, as described above with reference to Figure 8 The system 900 can also be used to implement Figure 3 and / or Figure 5-Figure 7 Any step of any flowchart in , as described above. The description of system 900 provided herein is for illustrative purposes only and is not intended to be limiting. The embodiments may be implemented in other types of computer systems, as known to those skilled in the relevant art.

[0125] like Fig. 9 As shown in , system 900 includes one or more processors (referred to as processor units 902), system memory 904, and bus 906, which couples various system components including system memory 904 to processor unit 902. Processor unit 902 is an electrical and / or optical circuit that is implemented as a central processing unit (CPU), microcontroller, microprocessor, and / or other physical hardware processor circuit in one or more physical hardware circuit device elements and / or integrated circuit devices (semiconductor material chips or dies). Processor unit 902 can execute program codes stored in computer-readable media, such as program codes of operating system 930, application program 932, other programs 934, etc. Bus 906 represents one or more of several types of bus structures, including memory bus or memory controller, peripheral bus, accelerated graphics port, and processor or local bus using any of a variety of bus architectures. System memory 904 includes read-only memory (ROM) 908 and random access memory (RAM) 910. Basic input / output system 912 (BIOS) is stored in ROM 908.

[0126] The system 900 also has one or more of the following drives: a hard disk drive 914 for reading from and writing to a hard disk, a disk drive 916 for reading from or writing to a removable disk 918, and an optical drive 920 for reading from or writing to a removable optical disk 922 (such as a CDROM, DVD ROM, or other optical media). The hard disk drive 914, the disk drive 916, and the optical drive 920 are connected to the bus 906 via a hard disk drive interface 924, a disk drive interface 926, and an optical drive interface 928, respectively. The drive and its associated computer-readable media provide non-volatile storage of computer-readable instructions, data structures, program modules, and other data for the computer. Although a hard disk, a removable disk, and a removable optical disk are described, other types of hardware-based computer-readable storage media may also be used to store data, such as flash memory cards and drives (e.g., solid-state drives (SSDs)), digital video disks, RAM, ROM, and other hardware storage media.

[0127] Several program modules or components may be stored on a hard disk, a magnetic disk, an optical disk, a ROM, or a RAM. These program modules include an operating system 930, one or more application programs 932, other program modules 934, and program data 936. According to various embodiments, the program modules may include computer program logic that can be executed by the processing unit 902 to perform data collection and preprocessing component 102, segmentation and evaluation component 104, automation component 106, user interface 108, data collector 110, feature identifier 112, preprocessing component 114, segmentation and evaluation component 200, clustering component 202, ranking component 204, ranking interpreter 206, flowchart 300, clustering component 400, initialization component 400, and the like. The invention also provides for the present invention to provide a method for performing a clustering operation of the present invention in accordance with the present invention. The invention also provides for the present invention to provide a method for performing a clustering operation of the present invention in accordance with the present invention.

[0128] A user may enter commands and information into the system 900 through input devices such as a keyboard 938 and a pointing device 940. Other input devices (not shown) may include a microphone, a joystick, a game controller, a satellite dish, a scanner, a touch screen and / or touch pad, a voice identification system for receiving voice input, a gesture identification system for receiving gesture input, etc. These and other input devices are typically connected to the processor unit 902 through a serial port interface 942 coupled to the bus 906, but may also be linked through other interface connections, such as a parallel port, a game port, or a universal serial bus (USB).

[0129] Display screen 944 is also connected to bus 906 via an interface, such as a video adapter 946. Display screen 944 may be external to system 900 or included in system 900. Display screen 944 may display information and may also serve as a user interface to receive user commands and / or other information (e.g., through touch, finger gestures, virtual keyboard, etc.). For example, display screen 944 may implement Figure 1 The user interface 108 is configured to display Figure 1 segmentation features and / or performance features, segmentation feature values ​​124, performance feature values ​​126, and / or context rules 128, Figure 2 groups 208 and / or ranked sets 210, Figure 8 The feature importance value 808 and / or performance score 810, and / or other information associated with the identification and / or resolution of performance issues. In addition to the display screen 944, the system 900 may also include other peripheral output devices (not shown), such as speakers and printers.

[0130] The system 900 is connected to a network 948 (e.g., the Internet) through an adapter or network interface 950, a modem 952, or other means for establishing communications over the network. The modem 952 (which may be internal or external) may be connected to the bus 906 (e.g., the Internet) via a serial port interface 942. Fig. 9 ), or may be connected to bus 906 using another interface type, including a parallel interface.

[0131] As used herein, the terms "computer program medium", "computer readable medium", and "computer readable storage medium" are used to refer to physical hardware media (such as a hard disk associated with hard disk drive 914, removable disk 918, removable optical disk 922), other physical hardware media (such as RAM, ROM, flash memory card, digital video disk, zip disk, MEM, nanotechnology-based memory devices), and other types of physical / tangible hardware storage media. Such computer readable storage media are distinct from and do not overlap with communication media (excluding communication media). Communication media embody computer readable instructions, data structures, program modules, or other data in a modulated data signal (such as a carrier wave). The term "modulated data signal" refers to a signal whose one or more characteristics are set or changed to encode information in the signal. By way of example and not limitation, communication media include wireless media (such as acoustic, RF, infrared, and other wireless media) and wired media. Embodiments are also directed to such communication media, which are separate and non-overlapping from the embodiments directed to computer readable storage media.

[0132] As described above, computer programs and modules (including application programs 932 and other programs 934) can be stored on a hard disk, disk, optical disk, ROM, RAM, or other hardware storage medium. Such computer programs can also be received via network interface 950, serial port interface 942, or any other interface type. When executed or loaded by an application, such computer programs enable system 900 to implement the features of the embodiments described herein. Therefore, such computer programs represent the controller of system 900.

[0133] Embodiments are also directed to computer program products that include computer code or instructions stored on any computer readable medium. Such computer program products include hard drives, optical drives, memory device packages, portable memory sticks, memory cards, and other types of physical storage hardware. According to various embodiments, program modules may include computer program logic that is executable by processing unit 902 to perform data collection and preprocessing component 102, segmentation and evaluation component 104, automation component 106, user interface 108, data collector 110, feature identifier 112, and / or preprocessing component 114 (as described above with reference to Figure 1 ), the segmentation and evaluation component 200, the clustering component 202, the ranking component 204, and / or the ranking interpreter 206 (as described above with reference to Figure 2 ), clustering component 400, initial cluster centroid selector 402, cluster generator 404, cluster quality assessment and selection component 406, algorithm executor 408, profile calculator 410, cluster restriction evaluator 412 (as described above with reference to Figure 4), ranking component 800, model 802, performance score calculator 804, and / or ranking generator 806 (as described above with reference to Figure 8 The program module may also include computer program logic that, when executed by the processing unit 902, causes the processing unit 902 to perform the above-described Figure 3 and / or Question 5- Figure 7 Any step in any flowchart.

[0134] VII. Additional Exemplary Embodiments

[0135] In an embodiment, a system includes one or more processors and one or more memory devices. The one or more memory devices store program code to be executed by the one or more processors. The program code includes a clustering component and a ranking component. The clustering component is configured to, for each of a plurality of automation components, receive a corresponding set of segmentation feature values ​​corresponding to a set of segmentation features and a corresponding set of performance feature values ​​corresponding to a set of performance features. The clustering component is also configured to segment the plurality of automation components into groups by applying a K-means clustering algorithm to the plurality of automation components based on the segmentation feature values ​​associated with the automation components. The clustering component is also configured to apply the K-means clustering algorithm by initializing a set of cluster centroids used when applying the K-means clustering algorithm. The initialization includes applying a set of contextual rules to the plurality of automation components. The ranking component is configured to rank the performance of the automation components within each group by determining a feature importance value for each performance feature. The ranking component is configured to determine the feature importance value for each performance feature by training a machine learning-based classification model that predicts a specific group based on the performance feature values ​​of the automation components. For each automation component, the ranking component is configured to calculate a performance score of the automation component based on a performance feature value of the automation component and a feature importance value of each performance feature. For each group of automation components, the ranking component is configured to generate a ranking of the automation components of the group based on their respective performance scores.

[0136] In an embodiment, the clustering component is configured to initialize a set of cluster centroids used when applying a K-means clustering algorithm by identifying a subset of segmentation features. A set of contextual rules is generated based on at least the subset of segmentation features. The set of cluster centroids is initialized based on the set of contextual rules.

[0137] In an embodiment, the clustering component is configured to segment the plurality of automation components into groups by applying a K-means clustering algorithm until it is determined that a set of cluster centroids meets a convergence criterion by performing the following operations: for each automation component, determining a nearest centroid of a set of cluster centroids based on a corresponding set of segmentation feature values ​​of the automation component, and assigning the automation component to the determined nearest centroid; and for each cluster centroid of the set of cluster centroids, updating the cluster centroid based on which automation components have been assigned to the cluster centroid. The clustering component is also configured to calculate an average silhouette coefficient based on the plurality of automation components and the set of cluster centroids, and store the average silhouette coefficient in the set of average silhouette coefficients. The clustering component is also configured to perform the following operations until the number of cluster centroids in the set of cluster centroids equals a predefined minimum number of cluster centroids: determining the nearest cluster centroid pairs from among the set of cluster centroids; merging the determined nearest cluster centroid pairs to simplify the set of cluster centroids; and repeating the applying of the K-means clustering algorithm to the plurality of automation components, the calculating of the average silhouette coefficient, and the storing of the average silhouette coefficient. The clustering component is further configured to determine a maximum average silhouette coefficient from the set of average silhouette coefficients and select a set of cluster centroids corresponding to the maximum average silhouette coefficient to generate a selected set of cluster centroids.

[0138] In an embodiment, the ranking component is configured to, for each automation component, calculate a performance score for the automation component by adding the product of each performance feature value of the automation component and the corresponding feature importance value and dividing the result by the sum of the feature importance values.

[0139] In an embodiment, the program code further comprises an interface. The interface is configured to enable a user to change the feature importance value of a given one of the performance features from a positive value to a corresponding negative value.

[0140] In an embodiment, the program code further comprises a ranking interpreter configured to perform one or more of the following operations based on the ranking of at least one automation component: generate an alert; generate one or more recommended actions for resolving performance issues associated with at least one automation component; or initiate one or more automated processes for resolving performance issues associated with at least one automation component.

[0141] In an embodiment, the plurality of automation components include hardware or software components in a computing environment. The program code also includes a ranking interpreter. The ranking interpreter is configured to perform one or more of the following operations based on the ranking of at least one automation component: modify the configuration of at least one automation component; update or patch software or firmware; perform a code analysis operation; perform a virus scanning operation; allocate one or more additional computing resources to at least one automation component; or perform a load balancing operation.

[0142] In an embodiment, the plurality of automated components include resource meters for a cloud computing service, computing equipment in a data center or network, vehicles in a fleet, telecommunications equipment in a telecommunications network, or cooling equipment in a data center.

[0143] In an embodiment, the machine learning based classification model is a gradient boosting decision tree model.

[0144] In an embodiment, a computer-implemented method for identifying and resolving performance issues of automation components is performed. The computer-implemented method includes: for each of a plurality of automation components, receiving a corresponding set of segmentation feature values ​​corresponding to a set of segmentation features and a corresponding set of performance feature values ​​corresponding to a set of performance features. The plurality of automation components are segmented into groups by applying a K-means clustering algorithm to the plurality of automation components based on the segmentation feature values ​​associated with the automation components. A set of cluster centroids used in applying the K-means clustering algorithm is initialized by applying a set of contextual rules to the plurality of automation components. For each group of automation components, a ranking of the automation components of the group is generated based at least on the sets of performance feature values ​​respectively associated with the automation components.

[0145] In an embodiment, a set of cluster centroids used when applying a K-means clustering algorithm is initialized by identifying a subset of segmentation features. A set of contextual rules is generated based on at least the subset of segmentation features. The set of cluster centroids is initialized based on the set of contextual rules.

[0146] In an embodiment, the plurality of automation components are further segmented into groups by applying a K-means clustering algorithm, including performing the following operations until it is determined that the set of cluster centroids has met a convergence criterion: for each automation component, determining a nearest centroid of a set of cluster centroids based on a corresponding set of segmentation feature values ​​of the automation component, and assigning the automation component to the determined nearest centroid; and for each cluster centroid of the set of cluster centroids, updating the cluster centroid based on which automation components are assigned to the cluster centroid. An average silhouette coefficient is calculated based on the plurality of automation components and the set of cluster centroids. The average silhouette coefficient is stored in a set of average silhouette coefficients. Performing the following operations until the number of cluster centroids in the set of cluster centroids equals a predefined minimum number of cluster centroids: determining the nearest cluster centroid pairs from the set of cluster centroids; merging the determined nearest cluster centroid pairs to simplify the set of cluster centroids; and repeatedly applying the K-means clustering algorithm to the plurality of automation components, calculating the average silhouette coefficient, and storing the average silhouette coefficient. Determining a maximum average silhouette coefficient from the set of average silhouette coefficients. The set of cluster centroids corresponding to the maximum average silhouette coefficient is selected to generate a selected set of cluster centroids.

[0147] In an embodiment, the plurality of automated components includes a resource meter for a cloud computing service, computing equipment in a data center or network, vehicles in a fleet, telecommunications equipment in a telecommunications network, or cooling equipment in a data center.

[0148] In an embodiment, a computer-implemented method for identifying and resolving performance issues of automation components is performed. The computer-implemented method includes, for each of a plurality of automation components, receiving a corresponding set of segmentation feature values ​​corresponding to a set of segmentation features and a corresponding set of performance feature values ​​corresponding to a set of performance features. The plurality of automation components are segmented into groups by applying a clustering algorithm to the plurality of automation components based on the segmentation feature values ​​respectively associated with the automation components. The performance of the automation components in each group is ranked by determining a feature importance value for each performance feature by training a machine learning-based classification model, the classification model predicting a particular group based on the performance feature values ​​of the automation components. For each automation component, a performance score of the automation component is calculated based on the performance feature values ​​of the automation component and the feature importance value of each performance feature. For each group of automation components, a ranking of the automation components of the group is generated based on their corresponding performance scores.

[0149] In an embodiment, the performance score of the automation component is calculated by adding the product of each performance feature value of the automation component and the corresponding feature importance value and dividing the result by the sum of the feature importance values.

[0150] In an embodiment, an interface is provided. The interface is configured to enable a user to change the feature importance value of a given one of the performance features from a positive value to a corresponding negative value.

[0151] In an embodiment, one or more of the following operations are performed based on the ranking of at least one automation component: generating an alert; generating one or more recommended actions for resolving performance issues associated with at least one automation component; or initiating one or more automated processes for resolving performance issues associated with at least one automation component.

[0152] In an embodiment, the plurality of automation components include hardware or software components in a computing environment. Based on the ranking of at least one automation component, one or more of the following operations are performed: modifying the configuration of at least one automation component; updating or patching software or firmware; performing a code analysis operation; performing a virus scanning operation; assigning one or more additional computing resources to at least one automation component; or performing a load balancing operation.

[0153] In an embodiment, the plurality of automated components include resource meters for a cloud computing service, computing equipment in a data center or network, vehicles in a fleet, telecommunications equipment in a telecommunications network, or cooling equipment in a data center.

[0154] In an embodiment, the machine learning based classification model is a gradient boosting decision tree model.

[0155] VIII. Conclusion

[0156] Although various embodiments have been described above, it should be understood that they are presented by way of example only, and not limitation. Those skilled in the relevant art will appreciate that various changes in form and detail may be made therein without departing from the spirit and scope of the embodiments. Therefore, the breadth and scope of the embodiments should not be limited by any of the above-described exemplary embodiments, but should be limited only in accordance with the following claims and their equivalents.

Claims

1. A system comprising: one or more processors; as well as one or more memory devices storing program code to be executed by the one or more processors, the program code comprising: A clustering component, wherein the clustering component is configured to: For each automation component of the plurality of automation components, receiving a respective set of segmentation feature values ​​corresponding to the set of segmentation features; and segmenting the plurality of automation components into groups by applying a K-means clustering algorithm to the plurality of automation components based on the segmentation feature values ​​associated with the plurality of automation components, wherein applying the K-means clustering algorithm comprises initializing a set of cluster centroids used in applying the K-means clustering algorithm by applying a set of contextual rules to the plurality of automation components; and A ranking component, wherein the ranking component is configured to: receiving, for each automation component of the plurality of automation components, a respective set of performance characteristic values ​​corresponding to the set of performance characteristics; and The performance of the automation components within each of the groups is ranked by: determining a feature importance value for each of the performance features by training a machine learning-based classification model that predicts a particular group in the groups based on the performance feature values ​​of the automation components; calculating, for each of the automation components, a performance score for the automation component based on the performance feature value of the automation component and the feature importance value of each of the performance features; and For each group of the automation components, a ranking of the automation components of the group is generated based on the respective performance scores of the automation components of the group.

2. The system of claim 1, wherein the clustering component is configured to initialize the set of cluster centroids used in applying the K-means clustering algorithm by: identifying a subset of the segmentation features; generating the set of contextual rules based at least on the subset of the segmentation features; and The set of cluster centroids is initialized based on the set of contextual rules.

3. The system of claim 1 , wherein the clustering component is configured to segment the plurality of automation components into the groups by: Applying the K-means clustering algorithm until a convergence criterion is determined to have been met for the set of cluster centroids by performing the following operations: For each of the automation components: determining a nearest centroid of the set of cluster centroids based on the corresponding set of segmentation feature values ​​of the automation component; as well as assigning the automation component to the determined closest centroid; as well as for each cluster centroid of the set of cluster centroids, updating the cluster centroid based on which automation components have been assigned to the cluster centroid; calculating an average silhouette coefficient based on the plurality of automated components and the set of cluster centroids; storing the average silhouette coefficient in an average silhouette coefficient set; The following operations are performed until the number of cluster centroids in the cluster centroid set is equal to a predefined minimum number of cluster centroids: Determining the nearest cluster centroid pair from among the set of cluster centroids; merging the determined nearest cluster centroid pairs to simplify the set of cluster centroids; as well as Repeating the steps of applying the K-means clustering algorithm to the plurality of automation components, calculating the average silhouette coefficient, and storing the average silhouette coefficient; determining a maximum average silhouette coefficient from the set of average silhouette coefficients; as well as The set of cluster centroids corresponding to the maximum average silhouette coefficient is selected to generate a selected set of cluster centroids.

4. The system of claim 1 , wherein the ranking component is configured to calculate, for each of the automation components, the performance score for the automation component by multiplying and summing each performance feature value of the automation component by the corresponding feature importance value, and dividing the result by the sum of the feature importance values.

5. The system of claim 4, wherein the program code further comprises an interface configured to enable a user to change the feature importance value for a given one of the performance features from a positive value to a corresponding negative value.

6. The system of claim 1 , wherein the program code further comprises a ranking interpreter configured to perform one or more of the following based on the ranking of at least one of the automation components: Generate alerts; generating one or more suggested actions for resolving a performance issue associated with the at least one of the automation components; or One or more automated processes are initiated for resolving a performance issue associated with the at least one of the automated components.

7. The system of claim 1, wherein: The plurality of automation components include hardware or software components in a computing environment; and The program code also includes a ranking interpreter configured to perform one or more of the following based on the ranking of at least one of the automation components: modifying a configuration of said at least one of said automation components; Update or patch software or firmware; Perform code analysis operations; Perform virus scanning operations; allocating one or more additional computing resources to the at least one of the automation components; or Performs load balancing operations.

8. The system of claim 1, wherein the plurality of automation components comprises: Resource meter for cloud computing services; Computing equipment in a data center or network; vehicles in a convoy; Telecommunications equipment in telecommunications networks; or Cooling equipment in a data center.

9. The system of claim 1, wherein the machine learning based classification model is a gradient boosted decision tree model.

10. A computer-implemented method for identifying and resolving performance issues of an automation component, comprising: For each automation component of the plurality of automation components, receiving a corresponding set of segmentation feature values ​​corresponding to the set of segmentation features and a corresponding set of performance feature values ​​corresponding to the set of performance features; segmenting the plurality of automation components into groups by applying a clustering algorithm to the plurality of automation components based on the segmentation feature values ​​respectively associated with the plurality of automation components; as well as The performance of the automation components within each of the groups is ranked by: determining a feature importance value for each of the performance features by training a machine learning-based classification model that predicts a particular group in the groups based on the performance feature values ​​of the automation components; For each of the automation components, calculating a performance score for the automation component based on the performance feature value of the automation component and the feature importance value of each of the performance features; as well as For each group of automation components, a ranking of the automation components of the group is generated based on the respective performance scores of the automation components of the group.

11. The computer-implemented method of claim 10, wherein said calculating said performance score for said automation component comprises: Each performance characteristic value of the automation component is multiplied by the corresponding characteristic importance value and the sum of the products is calculated, and the result is divided by the sum of the characteristic importance values.

12. The computer-implemented method of claim 11 , further comprising: An interface is provided, the interface being configured to enable a user to change the feature importance value for a given one of the performance features from a positive value to a corresponding negative value.

13. The computer-implemented method of claim 10, further comprising: Based on the ranking of at least one of the automation components, one or more of the following are performed: Generate alerts; generating one or more recommended actions for resolving a performance issue associated with at least one of the automation components; or One or more automated processes are initiated for resolving a performance issue associated with the at least one of the automation components.

14. The computer-implemented method of claim 10, wherein the plurality of automation components comprise hardware or software components in a computing environment, the computer-implemented method further comprising: Based on the ranking of at least one of the automation components, one or more of the following are performed: modifying a configuration of said at least one of said automation components; Update or patch software or firmware; Perform code analysis operations; Perform virus scanning operations; allocating one or more additional computing resources to the at least one of the automation components; or Performs load balancing operations.

15. A computer-readable storage medium having programmed instructions encoded thereon, the programmed instructions being executable by one or more processors to perform the method of any one of claims 10-14.

Citation Information

Cited By

  • Tool performance evaluation method and device

    CN121119441A