Detection of network service performance degradation based on user interaction group metrics

Abstract

Apparatus, systems, and techniques for identifying the causes of performance degradation in a web-based service. In at least one embodiment, the causes of performance degradation are identified by comparing performance metrics associated with a first set of user interactions with the web-based service and performance metrics associated with a second set of user interactions with the web-based service.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Patent Application No. 16 / 987,252, filed August 6, 2020, entitled “Performance Analysis,” the entire contents of which are incorporated herein by reference and used for all purposes. Technical Field

[0003] According to the various new technologies described herein, at least one embodiment relates to a processor or computer system for detecting and diagnosing one or more causes of performance degradation in web-based services. Background Technology

[0004] Techniques for automatically detecting and diagnosing performance degradation in web-based services can consume significant computing resources and may be inaccurate. Improvements can be made to the accuracy of performance degradation detection and diagnosis, as well as the amount of computing resources used. Attached Figure Description

[0005] Figure 1 The illustration depicts a time-series analysis of a network-based service according to at least one embodiment;

[0006] Figure 2 Metric resampling according to at least one embodiment is illustrated;

[0007] Figure 3 Transition detection according to at least one embodiment is illustrated;

[0008] Figure 4 The illustration depicts an example process for time-series analysis of a network-based service according to at least one embodiment;

[0009] Figure 5 An example process for subcontext isolation according to at least one embodiment is shown;

[0010] Figure 6 An example process for sub-environment isolation according to at least one embodiment is shown;

[0011] Figure 7 An example visualization of sub-environment and sub-context isolation according to at least one embodiment is shown;

[0012] Figure 8 Another example visualization of sub-environment and sub-context isolation according to at least one embodiment is shown;

[0013] Figure 9 Another example visualization of sub-environment and sub-context isolation according to at least one embodiment is shown;

[0014] Figure 10 The illustration shows an example directed graph visualization of sub-environment and sub-context isolation according to at least one embodiment;

[0015] Figure 11 A distributed system according to at least one embodiment is shown;

[0016] Figure 12 An exemplary data center according to at least one embodiment is shown;

[0017] Figure 13 A client-server network according to at least one embodiment is shown;

[0018] Figure 14 A computer network according to at least one embodiment is shown;

[0019] Figure 15A A networked computer system according to at least one embodiment is shown;

[0020] Figure 15B A networked computer system according to at least one embodiment is shown;

[0021] Figure 15C A networked computer system according to at least one embodiment is shown;

[0022] Figure 16 The illustration shows one or more components of a system environment according to at least one embodiment, in which the service can be provided as a third-party network service;

[0023] Figure 17 A cloud computing environment according to at least one embodiment is shown;

[0024] Figure 18 This illustrates a set of functional abstraction layers provided by a cloud computing environment according to at least one embodiment;

[0025] Figure 19 A supercomputer at the chip level is illustrated according to at least one embodiment;

[0026] Figure 20 A supercomputer at the rack module level is shown according to at least one embodiment;

[0027] Figure 21 A supercomputer at the rack level is shown according to at least one embodiment;

[0028] Figure 22 A supercomputer at the entire system level is illustrated according to at least one embodiment;

[0029] Figure 23AThe inference and / or training logic according to at least one embodiment is illustrated;

[0030] Figure 23B The inference and / or training logic according to at least one embodiment is illustrated;

[0031] Figure 24 The training and deployment of a neural network according to at least one embodiment are illustrated;

[0032] Figure 25 The architecture of a network system according to at least one embodiment is shown;

[0033] Figure 26 The architecture of a network system according to at least one embodiment is shown;

[0034] Figure 27 A control plane protocol stack according to at least one embodiment is shown;

[0035] Figure 28 A user plane protocol stack according to at least one embodiment is shown;

[0036] Figure 29 The components of a core network according to at least one embodiment are shown;

[0037] Figure 30 Components of a system supporting Network Function Virtualization (NFV) according to at least one embodiment are shown;

[0038] Figure 31 A processing system according to at least one embodiment is shown;

[0039] Figure 32 A computer system according to at least one embodiment is shown;

[0040] Figure 33 A system according to at least one embodiment is shown;

[0041] Figure 34 An exemplary integrated circuit according to at least one embodiment is shown;

[0042] Figure 35 A computing system according to at least one embodiment is shown;

[0043] Figure 36 An APU according to at least one embodiment is shown;

[0044] Figure 37 A CPU according to at least one embodiment is shown;

[0045] Figure 38 An exemplary accelerator integration slice according to at least one embodiment is shown;

[0046] Figure 39A and 39B An exemplary graphics processor according to at least one embodiment is shown;

[0047] Figure 40A A graphics core according to at least one embodiment is shown;

[0048] Figure 40B A GPGPU according to at least one embodiment is shown;

[0049] Figure 41A A parallel processor according to at least one embodiment is shown;

[0050] Figure 41B A processing cluster according to at least one embodiment is shown;

[0051] Figure 41C A graphics multiprocessor according to at least one embodiment is shown;

[0052] Figure 42 A software stack of a programming platform according to at least one embodiment is shown;

[0053] Figure 43 The illustration shows an embodiment according to at least one of the embodiments. Figure 42 The CUDA implementation of the software stack;

[0054] Figure 44 The illustration shows an embodiment according to at least one of the embodiments. Figure 42 The ROCm implementation of the software stack;

[0055] Figure 45 The illustration shows an embodiment according to at least one of the embodiments. Figure 42 The OpenCL implementation of the software stack;

[0056] Figure 46 Software supported by a programming platform according to at least one embodiment is shown; and

[0057] Figure 47 A method for using at least one embodiment is shown. Figures 42-45 Compiled code executed on the programming platform. Detailed Implementation

[0058] In the following description, numerous specific details are set forth to provide a more thorough understanding of at least one embodiment. However, it will be apparent to those skilled in the art that the inventive concept can be practiced without one or more of these specific details.

[0059] Figure 1A time-series analysis of a web-based service according to at least one embodiment is illustrated. In at least one embodiment, in example 100 of the depicted web-based service 102, multiple servers are included. In at least one embodiment, the servers host one or more virtual machines. In at least one embodiment, the virtual machines execute one or more applications that provide the web-based service to one or more user devices.

[0060] In at least one embodiment, a user session corresponds to one or more interactions between a user's device and the web-based service 102. In at least one embodiment, the interaction may include, but is not limited to, requesting data, providing data, performing a requested action, performing a planned or unrequested action, connecting or disconnecting. For example, in at least one embodiment, the interaction includes presenting video frames related to a video game hosted and streamed to the user's device by the web-based service 102. In at least one embodiment, the interaction includes requesting or performing an action or performing an action related to computerized gameplay. In at least one embodiment, user interaction includes a user session related to gameplay or some other user session related to the web-based service.

[0061] In at least one embodiment, performance degradation of the network-based service 102 is automatically detected, and the cause of the degradation is identified through automatic analysis. In at least one embodiment, the degradation includes changes in performance characteristics. In at least one embodiment, the degradation includes unexpected negative changes in performance characteristics, the cause of which is unknown.

[0062] In at least one embodiment, the automatic degradation analysis is based on comparative analysis with groups of interactions with network-based service 102. In at least one embodiment, the comparative analysis includes comparing changes in degradation performance metrics for two or more groups of user interactions. In at least one embodiment, the comparative analysis further includes comparing changes in the proportion of user interactions in the corresponding group to changes in user interactions in other groups or to the total number of user interactions. In at least one embodiment, the user interaction groups are based on attributes associated with the group. For example, in at least one embodiment, the groups are based on version number categories of attributes, such that user interactions associated with “v1.0” of the application are placed in one group, and user interactions associated with “v2.0” of the application are placed in another group. In at least one embodiment, the comparative analysis is based on comparing these corresponding groups. In at least one embodiment, attribute categories such as version numbers are referred to as sub-environments, and the values ​​of attributes within that category are referred to as sub-contexts or sub-contexts of sub-environments. For example, in at least one embodiment, user interactions such as user sessions are grouped according to version number sub-environments, such as user interactions associated with the “v1.0” sub-context being placed in one group, and user interactions associated with the “v2.0” sub-context being placed in another group.

[0063] In at least one embodiment, time series of metrics are collected to monitor the performance of the network-based service 102. For example, in at least one embodiment, the metrics include values ​​indicating system performance, which may include, but are not limited to, measurements such as requests processed per second, number of active sessions, number of inactive sessions, central processing unit (“CPU”) utilization, memory utilization, bandwidth utilization, etc. In at least one embodiment, the time series of metrics includes a sequence of such values ​​collected over time, and thus represents the value of the corresponding metric over time.

[0064] In at least one embodiment, the network-based service 102 collects time-series metrics 104. For example, in at least one embodiment, the network-based service 102 periodically counts inactive user sessions and records the corresponding values ​​in an array or other storage structure suitable for maintaining time-series data.

[0065] In at least one embodiment, changes in the operational characteristics of the network-based service 102 are identified via the techniques described herein, through time-series analysis of metrics and other data. In at least one embodiment, the operational characteristics are performance characteristics, or characteristics indicating application functionality. In at least one embodiment, changes in the operational characteristics of the network-based service 102 are reflected as shifts in the time series of metrics. In at least one embodiment, the shifts include statistically significant changes in values ​​within the time series. In at least one embodiment, such changes indicate performance failures or degradation.

[0066] In at least one embodiment, the analytical techniques described herein are used to identify transitions in multiple time series. For example, in at least one embodiment, a web-based service 102 collects a large number of different time series related to various performance characteristics. In at least one embodiment, the web-based service 102 analyzes these time series to detect transitions in one of these time series. In at least one embodiment, the web-based service 102 identifies transitions that might be difficult, impossible, or impractical to detect by other means.

[0067] In at least one embodiment, the network-based service 102 has multiple attributes, such as characteristics, traits, features, or qualities. In at least one embodiment, the attributes are one-dimensional or multi-dimensional and can be represented as scalars, vectors, or arrays of numerical or text values. In at least one embodiment, examples of attributes associated with the network-based service 102 are the instance type 106 and software version 108 attributes. In at least one embodiment, instance type 106 refers to a category of virtual machine instances and can be represented as a vector describing how many of each category are operational at a given time. In at least one embodiment, software version 108 refers to the revision number of an application running on the network-based service 102 and is similarly represented as a vector describing how many instances of each revision are running or running at a given time. The attributes of the network-based service 102 change over time, for example, in response to a new application being installed or some other configuration change made to the network-based service 102, which in turn may cause a change in the time series 104 of the metric. In at least one embodiment, an isolation and drill-down process is used to identify the attributes whose changes cause said changes, embodiments of which are described herein. In at least one embodiment, the network-based service 102 has a large number of attributes, many of which can change independently over time, making the identification of attributes associated with metric changes difficult, impossible, or impractical. For example, in at least one embodiment, the number of “full instances” and “half instances” of virtual machines both increase during the time period associated with the change in the time series 104 of the metric, while the application with software version 108 decreases for version “v1.1” but increases for version “v2.0”. In at least one embodiment, in addition to those attributes described, there are many other attributes, each of which can change independently of the others. In at least one embodiment, isolation and drill-down processes as described herein are used to identify from the large number of attributes those that may be associated with the root cause of the metric change.

[0068] Figure 2The illustration depicts metric resampling according to at least one embodiment. In at least one embodiment, example 200 of the time series 202 of the metric includes metric values ​​sampled periodically over a period of time, such as once per hour over a period of several days. In at least one embodiment, the time series 202 exhibits a periodic or cyclical trend, which may be due to the natural fluctuations in demand for the system over time. For example, in at least one embodiment, peak usage of the network-based service 102 occurs in the evening. In at least one embodiment, even considering factors such as... Figure 2 The cyclical pattern described in the text also shows that resampling techniques have been used to facilitate the identification of transition points.

[0069] In at least one embodiment, resampling is performed by extracting values ​​from a portion of the time series and randomly assigning those values ​​to a number of buckets. For example, in at least one embodiment, the time series includes samples collected at periodic intervals throughout the day. In at least one embodiment, resampling involves randomly reassigning those values ​​to one of a plurality of buckets. In at least one embodiment, twenty-four buckets are used per day, but each bucket may contain samples collected at any time point during the day, so these buckets do not necessarily correspond to several hours of the day. In at least one embodiment, values ​​from the time series 202 of the metric are assigned to one and only one bucket within the resampled time series. In at least one embodiment, each bucket includes 1 / N samples after resampling, where N represents how many buckets are used per day. In at least one embodiment, N = 24. In at least one embodiment, N is chosen such that normal periodic fluctuations in the time series 202 of the metric are removed or reduced, balanced by the increased processing time that may be associated with an increased value of N. In at least one embodiment, a larger value of N may improve the ability to detect shifts in the overall mean of the metric of time series 202.

[0070] In at least one embodiment, the average value assigned to each bucket is calculated. In at least one embodiment, the average values ​​of each bucket collectively constitute the time series 204 of the resampled metric. In at least one embodiment, the resampled time series 204 can be as follows: Figure 2 The plot is shown below, where each intraday value corresponds to the average value of the corresponding bucket.

[0071] Figure 3 The illustration depicts transition detection according to at least one embodiment. In at least one embodiment, in example 300, the time series 304 of the resampled metric is, for example... Figure 2 The resampled time series 204 depicted in the figure was analyzed to identify one or more transition points.

[0072] In at least one embodiment, the t-test is used to identify the transition point. In at least one embodiment, Welch's t-test is used according to the equation:

[0073]

[0074] In at least one embodiment, t is used in conjunction with degrees of freedom, such as calculated based on multiple samples on each side, to produce a p-value corresponding to the estimated probability that the null hypothesis is true. In at least one embodiment, the null hypothesis is that each part of the time series has an equal population mean, while the alternative hypothesis is that each part does not have an equal population mean.

[0075] In at least one embodiment, a resampled time series is taken and split into two parts at a given index position. For each such position, in at least one embodiment, a t-test is performed and the t-statistic and p-value are recorded, and a transition point is identified by locating the position where its t-statistic has the largest absolute value. In at least one embodiment, the position with the largest absolute value and whose p-value indicates statistical significance is considered a transition point. In at least one embodiment, such positions are considered those where their p-value is below a threshold, such as below 0.001.

[0076] Figure 4 An example process for time series analysis of a network-based service according to at least one embodiment is illustrated. In at least one embodiment, one or more transition points in the time series of a metric are identified and isolation analysis is used to identify the potential cause of each transition. In at least one embodiment, the purpose of isolation analysis is to isolate the most likely cause of the transition. In at least one embodiment, isolation analysis is based on the assumption that transitions in the time series can be explained by correlated changes proportional to the subcontext.

[0077] Although example process 400 is depicted as a series of operations, it should be understood that in embodiments, the described operations may be varied in various ways, and some operations may be omitted, reordered, or performed in parallel with other operations unless explicitly stated or logically implied in order, such as when the input of one operation depends on the output of another operation.

[0078] Figure 4 The described operations can be performed by, for example Figure 1 The system, such as the network-based service 102 depicted, includes at least one processor and a memory containing instructions that, in response to execution by the at least one processor, cause the system to perform the depicted operations.

[0079] In 402, in at least one embodiment, transition points in the time series are identified. In at least one embodiment, the system uses information about... Figure 2-3The described techniques are used to identify the transition points. In at least one embodiment, the transition points are identified as targets for isolation and drill-down analysis based on statistical properties associated with them.

[0080] In at least one embodiment, for a given identification transition point, isolation

[0081] At 404, in at least one embodiment, sub-context and sub-environment data associated with the identified transition point are obtained. In at least one embodiment, the sub-context corresponds to the value of an attribute or trait, and the sub-environment corresponds to the category or type of said attribute or trait. For example, in at least one embodiment, "instance type" corresponds to a sub-environment, and "full instance" or "half instance" corresponds to a sub-context.

[0082] In at least one embodiment, sub-context and sub-environment data are obtained within a time period associated with the transition. In at least one embodiment, this includes data prior to the transition point undergoing isolation and drill-down analysis, as well as data after the transition point.

[0083] In 406, in at least one embodiment, for each sub-context within a given sub-environment, the computational complexity value is calculated as a function of the variation in mean and proportion. In at least one embodiment, the given sub-environment (e.g., instance type) has a one-to-many relationship with the sub-context, such as "half an instance" and "complete an instance". In at least one embodiment, the computational complexity is calculated based on the following:

[0084] sA=s[after mean]*s[after proportion]

[0085] sB=s[before mean]*s[before proportion]

[0086]

[0087] s[complexity]=sAB*s[after proportion]

[0088] In at least one embodiment, if sAB is not a number, it can be set to 1.0, for example, when sB equals 0. In at least one embodiment, the approximate scalar equivalent of before / after is calculated by multiplying the mean by a scale. In at least one embodiment, if the scalar equals zero, such as in the case of introducing a subcontext (e.g., a new software version), the previous scalar is set to non-numeric (“NaN”).

[0089] In at least one embodiment, the combined scalar, such as sAB shown above, is calculated as the ratio of change between the two scalars. In at least one embodiment, this ratio is multiplied by the proportion of the "after" component to bias complexity towards a higher level for the "newer" subcontext. This emphasizes the context that increases proportionally, rather than the subcontext that decreases proportionally. For example, in at least one embodiment, the system generates a more intuitive result by labeling the new version v2.0 as responsible for the transition rather than labeling "version downgraded to v1.0" as the cause of the transition.

[0090] In at least one embodiment, if the ratio becomes zero, such as when sB is zero due to the introduction of a new context, the scalar can be set to 1.0 so that the new context (without a priori ratio) can be identified as the root cause of the transition.

[0091] At 408, in at least one embodiment, subcontext isolation is performed. In at least one embodiment, the system uses subcontext isolation to identify attributes or traits whose changes are estimated to be the cause of the change.

[0092] In at least one embodiment, subcontext isolation includes limiting subcontexts to potential causes of transition. In at least one embodiment, one or more filtering criteria are applied. In at least one embodiment, a subcontext is eliminated as a potential cause when the proportion is below a threshold level. For example, in at least one embodiment, subcontexts associated with less than 5% of sessions during the relevant time period may be filtered out and not considered. In at least one embodiment, subcontexts that do not have a value on or before the transition date are excluded from consideration.

[0093] In at least one embodiment, a subcontext is included as a potential cause when the proportion of the subcontext is above a threshold level and there is data near the corresponding transition time.

[0094] In at least one embodiment, the complexity metric described above is converted into an impact factor. In at least one embodiment, if the mean or scale of the subcontext does not change, its impact is zero. Alternatively, in at least one embodiment, the impact factor of the subcontext is initialized to the absolute value of the subcontext complexity.

[0095] In at least one embodiment, logic is then executed to determine whether the subcontext change is consistent with or inconsistent with the transition. In at least one embodiment, when the transition is an increasing transition, a subcontext whose value is decreasing may be disqualified. In at least one embodiment, a subcontext is marked as disqualified by setting the impact factor to zero.

[0096] In at least one embodiment, a given sub-environment may contain multiple sub-contexts, and each sub-context may have its own scale vector and mean vector, where the vectors refer to the direction and amount of change in each sub-context. The mean of each sub-context may increase or decrease, and such increase or decrease may be below, above, or across the transformation mean. Each sub-context may also increase or decrease in its overall scale. In at least one embodiment, a sub-context is identified as a potential cause of a transformation based on its relative movement with respect to the mean and scale of the transformation. In at least one embodiment, an influence factor associated with a sub-context is set to zero, and if the vector of a sub-context is not aligned with the vector of the corresponding transformation, the sub-context is deemed unqualified.

[0097] In at least one embodiment, the impact factor associated with each sub-context is used to identify potential root causes of the shift. In at least one embodiment, a sub-context is considered a root cause if its impact factor percentage is greater than a threshold. For example, in at least one embodiment, a sub-context is considered a root cause if its impact factor is greater than 20% of the impact factor attributable to the shift. In at least one embodiment, a sub-context can also be considered a root cause if it is a new sub-context and its percentage is greater than the threshold amount.

[0098] In 410, in at least one embodiment, subenvironment isolation is performed. In at least one embodiment, the system is associated with a number of possible subenvironments, and the system performs subenvironment isolation to identify which subenvironment is most relevant to the transition.

[0099] In at least one embodiment, subenvironment isolation includes the analysis of one or more subenvironments associated with the system. In at least one embodiment, some subenvironments are excluded from the analysis based on various criteria. In at least one embodiment, subenvironments are excluded based on the information gain associated with the movement of associated subcontexts. For example, in at least one embodiment, a subenvironment is excluded if all its subcontexts are overwhelmingly increasing or decreasing because the information gain associated with the subenvironment is low when the corresponding movements of its associated subcontexts align.

[0100] In at least one embodiment, the average absolute complexity is calculated across all sub-contexts within the environment. In at least one embodiment, this includes sub-contexts not identified as the root cause. In at least one embodiment, the average absolute complexity represents the amount of movement within a sub-context. In at least one embodiment, the sub-context with the highest average absolute complexity is selected.

[0101] Figure 5 An example process for subcontext isolation according to at least one embodiment is shown.

[0102] Although example process 500 is depicted as a series of operations, it should be understood that in embodiments, the described operations may be modified in various ways, and some operations may be omitted, reordered, or performed in parallel with other operations unless explicitly stated or logically implied in order, such as when the input of one operation depends on the output of another operation.

[0103] Figure 5 The described operations can be performed by, for example Figure 1 The system, such as the network-based service 102 depicted, includes at least one processor and a memory containing instructions that, in response to execution by the at least one processor, cause the system to perform the depicted operations.

[0104] In 502, in at least one embodiment, the influence factor of the subcontext is initialized based on the complexity value associated with the subcontext, or zero if the mean and proportion of the subcontext have not changed relative to other subcontexts within the subcontext.

[0105] In 504, in at least one embodiment, the influence factor is adjusted based on the relative change in the mean of the sub-context. In at least one embodiment, the change vector of the sub-context is compared with other sub-contexts in the associated sub-environment, wherein the influence is adjusted upward when the vector has a larger magnitude or is in a different direction compared to the change vectors of other sub-contexts in the sub-environment. In at least one embodiment, the influence is adjusted downward when the vector has a similar magnitude or a similar direction to other change vectors.

[0106] In 506, in at least one embodiment, the influence factor is adjusted based on the relative change in the proportion of the sub-contexts. In at least one embodiment, the influence is adjusted upwards when the proportion of one sub-context increases relative to other sub-contexts, and downwards when the proportion decreases.

[0107] In at least one embodiment, as shown in element 508, an influence factor is calculated for each subcontext.

[0108] In 510, in at least one embodiment, one or more sub-contexts are selected as potential causes of the transition based on a calculated impact factor.

[0109] Figure 6 An example process for sub-environment isolation according to at least one embodiment is shown.

[0110] Although the example process 600 is depicted as a series of operations, it should be understood that, in embodiments, the described operations may be modified in various ways, and some operations may be omitted, reordered, or performed in parallel with other operations, unless explicitly stated or logically implied as such, for example when the input of one operation depends on the output of another operation.

[0111] Figure 6 The described operations can be performed by, for example Figure 1 The system, such as the network-based service 102 depicted, is used to execute the described operation. The system includes at least one processor and a memory containing instructions that, in response to execution by the at least one processor, cause the system to perform the described operation.

[0112] In 602, in at least one embodiment, sub-environments are identified for analysis. In at least one embodiment, the identification includes filtering sub-environments based on one or more criteria. In at least one embodiment, the criteria include the availability of data related to the sub-environments surrounding the transition.

[0113] In 604, in at least one embodiment, information gain in a sub-environment is analyzed. In at least one embodiment, the system analyzes information gain by comparing relative changes in its sub-environments. In at least one embodiment, a sub-environment is determined to have relatively high information gain when the changes in size or orientation of one or more sub-contexts in the sub-environment are significantly different from those of most other sub-contexts in the sub-environment. In at least one embodiment, a sub-environment is determined to have relatively low information gain when the sub-contexts of the sub-environment do not change significantly in size or orientation, or when most of its sub-contexts change in similar size and orientation.

[0114] In 606, in at least one embodiment, a subenvironment is rejected as potentially related to the transition if the subenvironment has low information gain.

[0115] In 608, in at least one embodiment, for a non-rejected sub-environment, the complexity value is computed across the sub-contexts within the sub-environment.

[0116] In at least one embodiment, as shown in operation 610, each identified sub-context is analyzed based on information gain, and if the information gain is appropriately high, a complexity metric is computed for its associated sub-context.

[0117] In 612, in at least one embodiment, the sub-environment with the highest associated complexity is selected as the potential cause of the transition. In at least one embodiment, the sub-environment is selected based on its average absolute complexity, which is calculated based on the complexity values ​​associated with each of its sub-contexts. In at least one embodiment, the sub-environment with the highest average absolute complexity is selected.

[0118] Figure 7 An example visualization of sub-environment and sub-context isolation according to at least one embodiment is shown. In at least one embodiment, Figure 700 is used to visualize sub-environment and sub-context isolation. In at least one embodiment, Figure 700 shows that half of the instances constitute 80% of the instance-type sub-environment, and full instances constitute 20%, and that the average value 708 has increased for all instances 702. Furthermore, the average value associated with half of the instances 704 has increased, while the average value associated with full instances 706 has not changed. The relative proportions 710 of the half and full instances have not changed, being 20% ​​and 30%, respectively. In at least one embodiment, this sub-environment is associated with high information gain due to the change in the average value associated with half of the instances, as indicated by arrow 704.

[0119] Figure 8 Another example visualization of sub-environment and sub-context isolation according to at least one embodiment is shown. In at least one embodiment, Figure 800 is used to visualize sub-environment and sub-context isolation. In at least one embodiment, Figure 800 shows that half of the instances 804 and the complete instances 806 constitute 80% and 20% of the instance-type sub-environments, respectively, compared to all instances 802. Furthermore, Figure 800 shows that these respective proportions do not change. Regarding the average value, Figure 800 shows that the average value of metric 808 changes by a similar size and direction for both half of the instances and the complete instances. In at least one embodiment, this results in the determination that the sub-environment has low information gain.

[0120] Figure 9 Another example visualization of subenvironment and subcontext isolation according to at least one embodiment is shown. In at least one embodiment, Figure 900 is used to visualize subenvironment and subcontext isolation. In at least one embodiment, Figure 900 shows the changes in the mean and proportion of subcontexts within a subenvironment in relation to a version number. Furthermore, Figure 900 shows that the mean of metric 908 decreases for all versions, as indicated by arrow 902, and that the proportion of version "v2.0" has increased from 20% to 40%. Although the changes in the mean 908 associated with versions 2.0 and v1.1 are shown, the changes in elements 904 and 906 in Figure 900 are relatively small, but because versions v1.1 and v2.0 change in opposite directions, Figure 900 can be considered to have a relatively high information gain.

[0121] Figure 10 The illustration depicts an example directed graph visualization of sub-environment and sub-context isolation and drill-down according to at least one embodiment. In at least one embodiment, the system generates a graph similar to... Figure 10The visualizations depicted in the document are intended to facilitate the identification and understanding of one or more reasons for the shift in metrics.

[0122] In at least one embodiment, element 1002 of visualization 1000 depicts degradation in metric M. In at least one embodiment, element 1002 is linked to element 1004 describing a sub-environment that, based on the isolation process described herein, has been identified as a potential cause of the degradation in metric M. In at least one embodiment, as depicted by arrows 1010a, b leading to element 1016, a change in the "half-instance" sub-context within the instance type sub-environment has been identified as a potential cause of the degradation.

[0123] In at least one embodiment, the drill-down process identifies user category sub-environments, as shown in element 1006, which include “existing” user category sub-contexts that have been identified as potential causes of the degradation based on the isolation process described herein. Similarly, the application version sub-environment described in element 1008 includes a sub-context of “v2.1”, which has also been identified as a potential cause of the degradation in metric M based on the isolation process described herein.

[0124] In at least one embodiment, visualization 1000 depicts the correlation between tagged sub-environments and sub-contexts. For example, in at least one embodiment, visualization 1000 depicts a degradation in metric M that might be associated with half of the instances of the application running version 2.1 for existing users. In at least one embodiment, the relationships between sub-contexts are depicted as arrows 1010, 1012, and 1014. For example, arrow 1104 correlates the statistics in element 1018 related to sessions running on half of the instances for existing users with the statistics in element 2020 related to sessions running on half of the instances for existing users running version 2.1.

[0125] In at least one embodiment, the processor includes one or more circuits configured to compare a performance metric of the web-based service in response to a first group of users interacting with the web-based service with one or more performance metrics of the web-based service in response to a second group of users interacting with the web-based service.

[0126] In at least one embodiment, the one or more circuits are configured to determine that the performance of the network-based service has degraded by at least generating a resampled time series and by at least randomly reassigning points of the time series of one or more performance metrics of the network-based service to buckets of the resampled time series. In at least one embodiment, the one or more circuits are also configured to identify transition points in the resampled time series at least in part based on statistical comparisons of segments of the resampled time series.

[0127] In at least one embodiment, the one or more circuits are configured to compare the rate of change of one or more performance metrics of a network-based service in response to a first set of user interactions with the rate of change of one or more performance metrics of a network-based service in response to a second set of user interactions.

[0128] In at least one embodiment, the one or more circuits are configured to compare the proportion of a first group of user interactions with the proportion of a second group of user interactions.

[0129] In at least one embodiment, a first set of user interactions is associated with a first attribute in an attribute category, and a second set of user interactions is associated with a second attribute in an attribute category.

[0130] In at least one embodiment, the one or more circuits are configured to determine, at least in part, the attributes associated with the first set of user interactions as possible causes of performance degradation of the network-based service, based on information metrics obtained by comparing one or more performance metrics of the first set of user interactions with one or more performance metrics of the second set of user interactions.

[0131] In at least one embodiment, the one or more circuits are configured to at least partially compare user interaction groups based on recursion, wherein each recursive level is based at least partially on an attribute category that is different from the attribute category in earlier recursive levels.

[0132] In at least one embodiment, user interaction includes the use of web-based services by a client device associated with the user.

[0133] Servers and data centers

[0134] The following figures illustrate, but are not limited to, systems based on exemplary network servers and data centers that can be used to implement at least one embodiment.

[0135] Figure 11 A distributed system 1100 according to at least one embodiment is illustrated. In at least one embodiment, the distributed system 1100 includes one or more client computing devices 1102, 1104, 1106, and 1108 configured to execute and operate client applications, such as web browsers, proprietary clients, and / or variations thereof, on one or more networks 1110. In at least one embodiment, a server 1112 may be communicatively coupled to remote client computing devices 1102, 1104, 1106, and 1108 via network 1110.

[0136] In at least one embodiment, server 1112 may be adapted to run one or more services or software applications, such as services and applications that manage session activity for single sign-on (SSO) access across multiple data centers. In at least one embodiment, server 1112 may also provide other services or software applications, which may include non-virtual and virtual environments. In at least one embodiment, these services may be provided as web-based services or cloud services or under a Software as a Service (SaaS) model to users of client computing devices 1102, 1104, 1106, and / or 1108. In at least one embodiment, users operating client computing devices 1102, 1104, 1106, and / or 1108 may, in turn, utilize one or more client applications to interact with server 1112 to utilize the services provided by these components.

[0137] In at least one embodiment, software components 1118, 1120, and 1122 of system 1100 are implemented on server 1112. In at least one embodiment, one or more components of system 1100 and / or the services provided by these components may also be implemented by one or more client computing devices 1102, 1104, 1106, and / or 1108. In at least one embodiment, a user operating a client computing device can then utilize one or more client applications to use the services provided by these components. In at least one embodiment, these components may be implemented using hardware, firmware, software, or a combination thereof. It should be understood that various different system configurations are possible and may differ from the distributed system 1100. Therefore, Figure 11 The embodiments shown are examples of distributed systems for implementing the systems of the embodiments, and are not intended to be limiting.

[0138] In at least one embodiment, client computing devices 1102, 1104, 1106, and / or 1108 may include different types of computing systems. In at least one embodiment, the client computing device may include a portable handheld device (e.g., Cellular phone Computing tablets, personal digital assistants (PDAs), or wearable devices (e.g., Google) Head-mounted display), running software (such as Microsoft Windows) And / or various mobile operating systems (such as iOS, Windows Phone, Android, BlackBerry 10, Palm OS, and / or variants thereof). In at least one embodiment, the device may support different applications, such as various Internet-related applications, email, short message service (SMS) applications, and may use various other communication protocols. In at least one embodiment, the client computing device may also include a general-purpose personal computer, which, by way of example, includes various versions of Microsoft... Apple Personal computers and / or laptops running Linux operating systems. In at least one embodiment, the client computing device can be running various commercially available operating systems. The client computing device may be a workstation computer operating system similar to UNIX, including but not limited to various GNU / Linux operating systems such as Google Chrome OS. In at least one embodiment, the client computing device may further include electronic devices capable of communicating over one or more networks, such as thin client computers, internet-enabled gaming systems (e.g., with or without networks 1110). Gesture input devices include Microsoft Xbox game consoles and / or personal messaging devices. Despite Figure 11 The distributed system 1100 is shown as having four client computing devices, but can support any number of client computing devices. Other devices (such as devices with sensors) can interact with the server 1112.

[0139] In at least one embodiment, network 1110 in distributed system 1100 can be any type of network capable of supporting data communication using any of the various available protocols, including but not limited to TCP / IP (Transmission Control Protocol / Internet Protocol), SNA (System Network Architecture), IPX (Internet Packet Switching), AppleTalk, and / or variations thereof. In at least one embodiment, network 1110 can be a local area network (LAN), an Ethernet-based network, Token Ring, a wide area network, the Internet, a virtual network, a virtual private network (VPN), an intranet, an extranet, a public switched telephone network (PSTN), an infrared network, or a wireless network (e.g., in the IEEE 802.11 protocol suite). Networks operating under any of the wireless protocols (and / or any other wireless protocols), and / or any combination of these and / or other networks.

[0140] In at least one embodiment, server 1112 may consist of one or more general-purpose computers, dedicated server computers (including, by way of example, PC (personal computer) servers, etc. Servers (including mid-range servers, mainframe computers, rack servers, etc.), server farms, server clusters, or any other suitable arrangement and / or combination thereof. In at least one embodiment, server 1112 may include one or more virtual machines running a virtual operating system or other computing architectures involving virtualization. In at least one embodiment, one or more flexible pools of logical storage devices may be virtualized to maintain virtual storage devices for the server. In at least one embodiment, the virtual network may be controlled by server 1112 using software-defined networking. In at least one embodiment, server 1112 may be adapted to run one or more services or software applications.

[0141] In at least one embodiment, server 1112 can run any operating system, and any commercially available server operating system. In at least one embodiment, server 1112 can also run any of a variety of additional server applications and / or mid-level applications, including HTTP (Hypertext Transfer Protocol) servers, FTP (File Transfer Protocol) servers, CGI (Common Gateway Interface) servers, etc. Servers, database servers, and / or variations thereof. In at least one embodiment, exemplary database servers include, but are not limited to, those commercially available from Oracle, Microsoft, Sybase, IBM (International Business Machines), and / or variations thereof.

[0142] In at least one embodiment, server 1112 may include one or more applications for analyzing and merging data feeds and / or event updates received from users of client computing devices 1102, 1104, 1106, and 1108. In at least one embodiment, data feeds and / or event updates may include, but are not limited to, data received from one or more third-party information sources and continuous data streams. feed, Updates or real-time updates may include real-time events related to sensor data applications, financial quotes, network performance measurement tools (e.g., network monitoring and business management applications), clickstream analysis tools, vehicle traffic monitoring, and / or their changes. In at least one embodiment, server 1112 may also include one or more applications for displaying data feeds and / or real-time events via one or more display devices of client computing devices 1102, 1104, 1106, and 1108.

[0143] In at least one embodiment, the distributed system 1100 may further include one or more databases 1114 and 1116. In at least one embodiment, the databases may provide mechanisms for storing information such as user interaction information, usage pattern information, adaptation rule information, and other information. In at least one embodiment, databases 1114 and 1116 may reside in various locations. In at least one embodiment, one or more of databases 1114 and 1116 may reside on a non-transitory storage medium local to server 1112 (and / or within server 1112). In at least one embodiment, databases 1114 and 1116 may be located remotely from server 1112 and communicate with server 1112 via a web-based connection or a dedicated connection. In at least one embodiment, databases 1114 and 1116 may reside in a storage area network (SAN). In at least one embodiment, any necessary files for performing functions belonging to server 1112 may be appropriately stored locally on server 1112 and / or remotely. In at least one embodiment, databases 1114 and 1116 may include relational databases, such as databases adapted to store, update, and retrieve data in response to SQL-formatted commands.

[0144] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to perform the tasks described. In at least one embodiment, the network-based service includes a distributed system 1100.

[0145] Figure 12 An exemplary data center 1200 according to at least one embodiment is shown. In at least one embodiment, the data center 1200 includes, but is not limited to, a data center infrastructure layer 1210, a framework layer 1220, a software layer 1230, and an application layer 1240.

[0146] Figure 12 An exemplary data center 1200 according to at least one embodiment is shown. In at least one embodiment, the data center 1200 includes, but is not limited to, a data center infrastructure layer 1210, a framework layer 1220, a software layer 1230, and an application layer 1240.

[0147] In at least one embodiment, such as Figure 12As shown, the data center infrastructure layer 1210 may include a resource coordinator 1212, grouped computing resources 1214, and node computing resources (“nodes CR”) 1216(1)-1216(N), where “N” represents any complete positive integer. In at least one embodiment, nodes CR 1216(1)-1216(N) may include, but are not limited to, any number of central processing units (“CPUs”) or other processors (including accelerators, field-programmable gate arrays (“FPGAs”), graphics processors, etc.), memory devices (e.g., dynamic read-only memory), storage devices (e.g., solid-state drives or disk drives), network input / output (“NW I / O”) devices, network switches, virtual machines (“VMs”), power modules, and cooling modules, etc. In at least one embodiment, one or more nodes CR 1216(1)-1216(N) may be servers having one or more of the aforementioned computing resources.

[0148] In at least one embodiment, the grouped computing resources 1214 may include individual groups (not shown) of node CRs housed in one or more racks, or a plurality of racks (also not shown) housed in data centers in various geographic locations. The individual groups of node CRs within the grouped computing resources 1214 may include computing, networking, memory, or storage resources that can be configured or allocated to support groups of one or more workloads. In at least one embodiment, several node CRs, including CPUs or processors, may be grouped within one or more racks to provide computing resources to support one or more workloads. In at least one embodiment, one or more racks may also include any number of power modules, cooling modules, and network switches, in any combination.

[0149] In at least one embodiment, resource coordinator 1212 may be configured or otherwise control one or more nodes CR1216(1)-1216(N) and / or grouped computing resources 1214. In at least one embodiment, resource coordinator 1212 may include a software-designed infrastructure (“SDI”) management entity for data center 1200. In at least one embodiment, resource coordinator 1212 may include hardware, software, or some combination thereof. In at least one embodiment, such as Figure 12As shown, the framework layer 1220 includes, but is not limited to, a job scheduler 1232, a configuration manager 1234, a resource manager 1236, and a distributed file system 1238. In at least one embodiment, the framework layer 1220 may include a framework of software 1252 supporting the software layer 1230 and / or one or more applications 1242 supporting the application layer 1240. In at least one embodiment, the software 1252 or application 1242 may respectively include web-based service software or applications, such as services or applications provided by Amazon Web Services, Google Cloud, and Microsoft Azure. In at least one embodiment, the framework layer 1220 may be, but is not limited to, a free and open-source software web application framework, such as Apache Spark™ (hereinafter referred to as "Spark") which can utilize the distributed file system 1238 for large-scale data processing (e.g., "big data"). In at least one embodiment, the job scheduler 1232 may include a Spark driver to facilitate the scheduling of workloads supported by the various layers of the data center 1200. In at least one embodiment, configuration manager 1234 may be able to configure different layers, such as software layer 1230 and framework layer 1220 including Spark and distributed file system 1238 for supporting large-scale data processing. In at least one embodiment, resource manager 1236 is able to manage cluster or group computing resources mapped to or allocated to support distributed file system 1238 and job scheduler 1232. In at least one embodiment, cluster or group computing resources may include grouped computing resources 1214 on data center infrastructure layer 1210. In at least one embodiment, resource manager 1236 may coordinate with resource coordinator 1212 to manage these mapped or allocated computing resources.

[0150] In at least one embodiment, the software 1252 included in the software layer 1230 may include software used by at least a portion of nodes CR1216(1)-1216(N), grouped computing resources 1214, and / or the distributed file system 1238 of the framework layer 1220. One or more types of software may include, but are not limited to, Internet web page search software, email virus scanning software, database software, and streaming video content software.

[0151] In at least one embodiment, one or more applications 1242 included in application layer 1240 may include one or more types of applications used by at least a portion of nodes CR1216(1)-1216(N), grouped computing resources 1214, and / or the distributed file system 1238 of framework layer 1220. One or more types of applications may include, but are not limited to, CUDA applications, 5G network applications, artificial intelligence applications, data center applications, and / or variations thereof.

[0152] In at least one embodiment, any of the configuration manager 1234, resource manager 1236, and resource coordinator 1212 can implement any number and type of self-modification actions based on any amount and type of data acquired in any technically feasible manner. In at least one embodiment, self-modification actions can mitigate potentially poor configuration decisions by data center operators of data center 1200 and can prevent underutilization and / or poor performance of the data center.

[0153] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0154] Figure 13 A client-server network 1304, formed by a plurality of interconnected network server computers 1302, is illustrated according to at least one embodiment. In at least one embodiment, each network server computer 1302 stores data accessible to other network server computers 1302 and client computers 1306 and networks 1308 linked to the wide area network 1304. In at least one embodiment, the configuration of the client-server network 1304 may change over time when client computers 1306 and one or more networks 1308 connect and disconnect from the network 1304, and when one or more backbone server computers 1302 are added to or removed from the network 1304. In at least one embodiment, the client-server network includes client computers 1306 and networks 1308 when they are connected to network server computers 1302. In at least one embodiment, the term "computer" includes any device or machine capable of accepting data, applying prescribed processes to the data, and providing the results of those processes.

[0155] In at least one embodiment, the client-server network 1304 stores information accessible to the network server computer 1302, the remote network 1308, and the client computer 1306. In at least one embodiment, the network server computer 1302 is formed from a mainframe computer, a minicomputer, and / or a microcomputer, each having one or more processors. In at least one embodiment, the server computer 1302 is linked together via wired and / or wireless transmission media (such as wires, fiber optic cables) and / or microwave transmission media, satellite transmission media, or other conductive, optical, or electromagnetic wave transmission media. In at least one embodiment, the client computer 1306 accesses the network server computer 1302 via similar wired or wireless transmission media. In at least one embodiment, the client computer 1306 can be linked to the client-server network 1304 using a modem and a standard telephone communication network. In at least one embodiment, alternative carrier systems (such as cable and satellite communication systems) can also be used to link to the client-server network 1304. In at least one embodiment, other private or time-sharing carrier systems can be used. In at least one embodiment, the network 1304 is a global information network, such as the Internet. In at least one embodiment, the network is a private intranet using protocols similar to the Internet but with added security measures and restricted access controls. In at least one embodiment, network 1304 is a private or semi-private network using proprietary communication protocols.

[0156] In at least one embodiment, client computer 1306 is any end-user computer, and may also be a mainframe computer, minicomputer, or microcomputer with one or more microprocessors. In at least one embodiment, server computer 1302 may sometimes be used as a client computer to access another server computer 1302. In at least one embodiment, remote network 1308 may be a local area network (LAN), a network added to a wide area network via an independent service provider (ISP) for the Internet, or another group of computers interconnected via wired or wireless transmission media with fixed or time-varying configurations. In at least one embodiment, client computer 1306 may independently or via remote network 1308 be linked to and access network 1304.

[0157] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0158] Figure 14 A computer network 1408 connecting one or more computing machines is illustrated according to at least one embodiment. In at least one embodiment, network 1408 can be any type of electrically connected group of computers, including, for example, the Internet, intranet, local area network (LAN), wide area network (WAN), or an interconnection combination of these network types. In at least one embodiment, the connection within network 1408 can be a remote modem, Ethernet (IEEE 802.3), Token Ring (IEEE 802.5), Fiber Distributed Data Link Interface (FDDI), Asynchronous Transfer Mode (ATM), or any other communication protocol. In at least one embodiment, the computing devices linked to the network can be desktop computers, servers, portable, handheld, set-top boxes, personal digital assistants (PDAs), terminals, or any other desired type or configuration. In at least one embodiment, network-connected devices can vary widely in terms of processing power, internal memory, and other performance characteristics, depending on their functionality. In at least one embodiment, communication within the network, as well as communication to or from computing devices connected to the network, can be wired or wireless. In at least one embodiment, network 1408 may at least partially comprise the global public Internet, which typically connects multiple users according to the Transmission Control Protocol / Internet Protocol (TCP / IP) specification in a client-server model. In at least one embodiment, the client-server network is the dominant model for communication between two computers. In at least one embodiment, a client computer (“client”) issues one or more commands to a server computer (“server”). In at least one embodiment, the server fulfills client commands by accessing available network resources and returning information to the client in accordance with the client commands. In at least one embodiment, client computer systems and network resources residing on the network server are assigned network addresses for identification during communication between network elements. In at least one embodiment, communication from other network-connected systems to the server will include the network address of the relevant server / network resource as part of the communication, such that the appropriate destination of the data / request is identified as the recipient. In at least one embodiment, when network 1408 comprises the global Internet, the network address is an IP address in TCP / IP format, which can at least partially route data to email accounts, websites, or other Internet tools residing on the server. In at least one embodiment, information and services residing on a web server can be made available to a client computer's web browser via a domain name (e.g., www.site.com) (which maps to the IP address of the web server).

[0159] In at least one embodiment, multiple clients 1402, 1404, and 1406 are connected to network 1408 via respective communication links. In at least one embodiment, each of these clients can access network 1408 via any desired form of communication, such as via dial-up modem connection, cable link, digital subscriber line (DSL), wireless or satellite link, or any other form of communication. In at least one embodiment, each client can communicate using any machine compatible with network 1408 (e.g., personal computer (PC), workstation, dedicated terminal, personal data assistant (PDA), or other similar device). In at least one embodiment, clients 1402, 1404, and 1406 may or may not be located in the same geographical area.

[0160] In at least one embodiment, multiple servers 1410, 1412, and 1414 are connected to network 1418 to serve clients communicating with network 1418. In at least one embodiment, each server is typically a powerful computer or device that manages network resources and responds to client commands. In at least one embodiment, the server includes computer-readable data storage media, such as hard disk drives and RAM memory, that store program instructions and data. In at least one embodiment, servers 1410, 1412, and 1414 run applications that respond to client commands. In at least one embodiment, server 1410 may run a web server application for responding to client requests for HTML pages, and may also run a mail server application for receiving and routing emails. In at least one embodiment, other applications, such as an FTP server or media server for streaming audio / video data to clients, may also run on server 1410. In at least one embodiment, different servers may be dedicated to performing different tasks. In at least one embodiment, server 1410 may be a dedicated web server for managing website-related resources for different users, while server 1412 may be dedicated to providing email management. In at least one embodiment, the other servers may be dedicated to a combination of two or more services typically available or provided over a network, such as media (audio, video, etc.), File Transfer Protocol (FTP), or other services. In at least one embodiment, each server may be located in the same or different location as the other servers. In at least one embodiment, multiple servers may exist to perform mirroring tasks for users, thereby mitigating congestion or minimizing traffic directed to and from a single server. In at least one embodiment, servers 1410, 1412, and 1414 are under the control of a web hosting provider that maintains and delivers third-party content over network 1418.

[0161] In at least one embodiment, the web hosting provider delivers services to two different types of clients. In at least one embodiment, one type, which may be referred to as a browser, requests content, such as web pages, email messages, video clips, etc., from servers 1410, 1412, and 1414. In at least one embodiment, a second type (which may be referred to as a user) hires the web hosting provider to maintain network resources (such as websites) and make them available to the browser. In at least one embodiment, the user contracts with the web hosting provider to make memory space, processor capacity, and communication bandwidth available to the network resources they desire, according to the amount of server resources the user expects to utilize.

[0162] In at least one embodiment, in order for a web hosting provider to serve both clients, the application managing network resources hosted on the server must be properly configured. In at least one embodiment, the program configuration process involves defining a set of parameters that at least partially control the application's response to browser requests and also at least partially define the server resources available to a particular user.

[0163] In one embodiment, intranet server 1416 communicates with network 1408 via a communication link. In at least one embodiment, intranet server 1416 communicates with server manager 1418. In at least one embodiment, server manager 1418 includes a database of application configuration parameters used by servers 1410, 1412, and 1414. In at least one embodiment, a user modifies database 1420 via intranet 1416, and server manager 1418 interacts with servers 1410, 1412, and 1414 to modify application parameters such that they match the contents of the database. In at least one embodiment, a user logs into intranet 1416 by connecting to intranet 1416 via computer 1402 and entering authentication information such as a username and password.

[0164] In at least one embodiment, when a user wishes to log in to a new service or modify an existing service, the intranet server 1416 authenticates the user and provides the user with an interactive screen display / control panel that allows the user access to configuration parameters for a specific application. In at least one embodiment, multiple modifiable text boxes describing aspects of the user's website or other network resources are presented to the user. In at least one embodiment, if the user desires to increase the storage space reserved for their website on the server, a field is provided where the user specifies the desired storage space. In at least one embodiment, in response to receiving this information, the intranet server 1416 updates the database 1420. In at least one embodiment, the server manager 1418 forwards the information to the appropriate server and uses the new parameters during application operation. In at least one embodiment, the intranet server 1416 is configured to provide the user with access to configuration parameters of network resources (e.g., web pages, email, FTP sites, media sites, etc.) that the user has contracted with a web hosting service provider.

[0165] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0166] Figure 15A A networked computer system 1500A according to at least one embodiment is illustrated. In at least one embodiment, the networked computer system 1500A includes a plurality of nodes or personal computers (“PCs”) 1502, 1518, 1520. In at least one embodiment, the personal computer or node 1502 includes a processor 1514, memory 1516, a camera 1504, a microphone 1506, a mouse 1508, a speaker 1510, and a monitor 1512. In at least one embodiment, PCs 1502, 1518, 1520 may each run one or more desktop servers, such as those on an internal network within a given company, or may be servers on a general network not limited to a specific environment. In at least one embodiment, each PC node in the network has one server, such that each PC node in the network represents a specific network server with a specific network URL address. In at least one embodiment, each server defaults to a default webpage for the user of that server, and the default webpage itself may contain embedded URLs pointing to further subpages for that user on that server, or to pages on other servers on the network or on other servers.

[0167] In at least one embodiment, nodes 1502, 1518, 1520 and other nodes of the network are interconnected via medium 1522. In at least one embodiment, medium 1522 may be a communication channel such as Integrated Services Digital Network (“ISDN”). In at least one embodiment, the individual nodes of the networked computer system may be connected via various communication media, including a local area network (“LAN”), a simple old-fashioned telephone line (“POTS”) (sometimes referred to as the Public Switched Telephone Network (“PSTN”)), and / or variations thereof. In at least one embodiment, the individual nodes of the network may also constitute users of computer systems interconnected via a network such as the Internet. In at least one embodiment, each server on the network (running from a specific node of the network at a given instance) has a unique address or identifier within the network, which may be specified according to a URL.

[0168] In at least one embodiment, multiple multipoint conferencing units (“MCUs”) can therefore be used to transmit data to and from various nodes or “endpoints” of the conferencing system. In at least one embodiment, in addition to various other communication media (such as nodes connected via the Internet), the nodes and / or MCUs may be interconnected via ISDN links or through a local area network (“LAN”). In at least one embodiment, the nodes of the conferencing system may typically be directly connected to a communication medium (such as a LAN) or connected via an MCU, and the conferencing system may include other nodes or components, such as routers, servers, and / or variations thereof.

[0169] In at least one embodiment, processor 1514 is a general-purpose programmable processor. In at least one embodiment, the processor of a node in the networked computer system 1500A may also be a dedicated video processor. In at least one embodiment, the different peripheral devices and components of the node (such as those of node 1502) may be different from those of other nodes. In at least one embodiment, nodes 1518 and 1520 may be configured to be the same as or different from node 1502. In at least one embodiment, the node may be implemented on any suitable computer system other than a PC system.

[0170] Figure 15BA networked computer system 1500B according to at least one embodiment is illustrated. In at least one embodiment, system 1500B illustrates a network (such as LAN 1524) that can be used to interconnect various nodes that can communicate with each other. In at least one embodiment, multiple nodes, such as PC nodes 1526, 1528, and 1530, are attached to LAN 1524. In at least one embodiment, nodes may also be connected to the LAN via a network server or other means. In at least one embodiment, system 1500B includes other types of nodes or elements, such as routers, servers, and nodes.

[0171] Figure 15C A networked computer system 1500C is illustrated according to at least one embodiment. In at least one embodiment, system 1500C illustrates a WWW system with communication across a backbone communication network (such as the Internet 1532), the backbone communication network being used to interconnect various nodes of the network. In at least one embodiment, the WWW is a set of protocols operating on top of the Internet and allowing a graphical interface system to operate on it to access information via the Internet. In at least one embodiment, the Internet 1532 attached to the WWW consists of multiple nodes, such as PCs 1540, 1542, and 1544. In at least one embodiment, nodes interface with other nodes of the WWW via WWW HTTP servers (such as servers 1534 and 1536). In at least one embodiment, PC 1544 may be a PC forming a node of network 1532, and PC 1544 itself runs its server 1536, although for illustrative purposes... Figure 15C PC 1544 and server 1536 are shown separately.

[0172] In at least one embodiment, the WWW is a distributed type of application characterized by WWW HTTP, the WWW protocol, which runs on top of the Internet's Transmission Control Protocol / Internet Protocol (“TCP / IP”). In at least one embodiment, the WWW can therefore be characterized by a set of protocols running on the Internet (i.e., HTTP) as its “backbone”.

[0173] In at least one embodiment, the web browser is an application running on a node of a network in a WWW-compatible network system, allowing users of a particular server or node to view such information and thus allowing users to search for graphics and text-based documents linked together using hypertext links embedded in documents or files available from servers that understand HTTP. In at least one embodiment, when a user uses another server on a network such as the Internet to retrieve a given webpage from a first server associated with a first node, the retrieved document may have different hypertext links embedded therein, and a local copy of the page created locally by the user is also retrieved. In at least one embodiment, when a user clicks a hypertext link, locally stored information associated with the selected hypertext link is generally sufficient to allow the user's machine to open a connection over the Internet to the server indicated by the hypertext link.

[0174] In at least one embodiment, more than one user may be coupled to each HTTP server, for example, via a LAN (such as LAN 1538, as shown with respect to WWW HTTP server 1534). In at least one embodiment, system 1500C may also include other types of nodes or elements. In at least one embodiment, the WWW HTTP server is an application running on a machine such as a PC. In at least one embodiment, each user can be considered to have a unique "server," as shown with respect to PC 1544. In at least one embodiment, a server can be considered to be a server such as WWW HTTP server 1534 that provides access to the network for a LAN or multiple nodes or multiple LANs. In at least one embodiment, there are multiple users, each user having a desktop PC or a node on the network, each desktop PC potentially setting up a server for its users. In at least one embodiment, each server is associated with a specific network address or URL that, when accessed, provides a default webpage for that user at that specific network address or URL. In at least one embodiment, the webpage may contain further links (embedded URLs) pointing to further subpages for that user on that server, or to other servers on the network or to pages on other servers on the network.

[0175] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0176] Cloud computing and services

[0177] The following figures illustrate, but are not limited to, exemplary cloud-based systems that can be used to implement at least one embodiment.

[0178] In at least one embodiment, cloud computing is a computing style in which dynamically scalable and typically virtualized resources are provided as a service over the Internet. In at least one embodiment, users do not need knowledge of, expertise in, or control over the technical infrastructure supporting them, which may be referred to as "in the cloud." In at least one embodiment, cloud computing consolidates infrastructure into services, Platform as a Service (PaaS), Software as a Service (SaaS), and other variations with common Internet-dependent themes to meet users' computing needs. In at least one embodiment, a typical cloud deployment (such as in a private cloud (e.g., an enterprise network)) or a data center (DC) in a public cloud (e.g., the Internet) may consist of thousands of servers (or alternatively, VMs), hundreds of Ethernet, Fibre Channel, or Fibre Channel over Ethernet (FCoE) ports, switching and storage infrastructure, etc. In at least one embodiment, the cloud may also consist of network service infrastructure such as IPsec VPN hubs, firewalls, load balancers, wide area network (WAN) optimizers, etc. In at least one embodiment, remote subscribers can securely access cloud applications and services via a VPN tunnel (such as an IPsec VPN tunnel).

[0179] In at least one embodiment, cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g., networks, servers, storage devices, applications, and services) that can be quickly configured and released with minimal management effort or service provider interaction.

[0180] In at least one embodiment, cloud computing is characterized by on-demand self-service, where consumers can automatically and unilaterally provision computing power, such as server time and network storage, as needed, without human interaction with each service provider. In at least one embodiment, cloud computing is characterized by broad network access, where capabilities are available on the network and accessed via standard mechanisms that facilitate use by heterogeneous thin or thick client platforms (e.g., mobile phones, laptops, and PDAs). In at least one embodiment, cloud computing is characterized by resource pooling, where a provider's computing resources are pooled to serve multiple consumers using a multi-tenant model, where different physical and virtual resources are dynamically signed and reallocated based on consumer demand. In at least one embodiment, there is a sense of location independence, as consumers typically have no control or knowledge of the exact location of the provided resources, but may be able to specify the location at a higher level of abstraction (e.g., country, state, or data center). In at least one embodiment, examples of resources include storage, processing, memory, network bandwidth, and virtual machines. In at least one embodiment, cloud computing is characterized by rapid elasticity, where capabilities can be rapidly and elastically provisioned (in some cases automatically) to scale down quickly and be released quickly to scale up quickly. In at least one embodiment, for consumers, the available supply capacity generally appears unlimited and can be purchased at any time and in any quantity. In at least one embodiment, cloud computing is characterized by a measured service, wherein the cloud system automatically controls and optimizes resource usage by leveraging metering capabilities at some level of abstraction suitable for the service type (e.g., storage, processing, bandwidth, and active user accounts). In at least one embodiment, resource usage can be monitored, controlled, and reported, thereby providing transparency to both the service provider and the consumer.

[0181] In at least one embodiment, cloud computing may be associated with various services. In at least one embodiment, cloud Software as a Service (SaaS) may refer to the ability provided to consumers as a service that uses applications from a provider running on cloud infrastructure. In at least one embodiment, applications may be accessed from different client devices via a thin client interface such as a web browser (e.g., web-based email). In at least one embodiment, consumers do not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, storage, or even the capabilities of individual applications, with possible exceptions of limited user-specific application configuration settings.

[0182] In at least one embodiment, Cloud Platform as a Service (PaaS) can refer to a service in which the ability to provide consumers with the capability to deploy consumer-created or acquired applications onto cloud infrastructure, these applications being created using programming languages ​​and tools supported by the provider. In at least one embodiment, the consumer does not manage or control the underlying cloud infrastructure, including networks, servers, operating systems, or storage, but has control over the deployed applications and, possibly, the configuration of the application hosting environment.

[0183] In at least one embodiment, cloud infrastructure as a service (IaaS) can refer to a service in which the capabilities provided to consumers are processing, storage, networking, and other basic computing resources that consumers can deploy and run, including operating systems and applications. In at least one embodiment, consumers do not manage or control the underlying cloud infrastructure, but instead have control over the operating system, storage, deployed applications, and possibly limited control over selected networking components (e.g., host firewalls).

[0184] In at least one embodiment, cloud computing can be deployed in different ways. In at least one embodiment, a private cloud can refer to cloud infrastructure that is operated solely by an organization. In at least one embodiment, a private cloud can be managed by an organization or a third party and can exist on-site or off-site. In at least one embodiment, a community cloud can refer to cloud infrastructure shared by several organizations and supporting a specific community with shared concerns (e.g., mission, security requirements, policies, and compliance considerations). In at least one embodiment, a community cloud can be managed by an organization or a third party and can exist on-site or off-site. In at least one embodiment, a public cloud can refer to cloud infrastructure that is available to the general public or a large industry group and is owned by an organization providing cloud services. In at least one embodiment, a hybrid cloud can refer to cloud infrastructure that is composed of two or more clouds (private, community, or public) that remain a single entity but are bound together by standardization or proprietary technologies that enable data and application portability (e.g., cloud bursting for load balancing between clouds). In at least one embodiment, the cloud computing environment is service-oriented, focusing on statelessness, loose coupling, modularity, and semantic interoperability.

[0185] Figure 16The diagram illustrates one or more components of a system environment 1600 according to at least one embodiment, wherein services can be provided as third-party network services. In at least one embodiment, the third-party network may be referred to as a cloud, cloud network, cloud computing network, and / or variations thereof. In at least one embodiment, system environment 1600 includes one or more client computing devices 1604, 1606, and 1608, which can be used by users to interact with a third-party network infrastructure system 1602 that provides third-party network services (which may be referred to as cloud computing services). In at least one embodiment, third-party network infrastructure system 1602 may include one or more computers and / or servers.

[0186] It should be understood that Figure 16 The third-party network infrastructure system 1602 described herein may have components other than those described. Furthermore, Figure 16 An embodiment of a third-party network infrastructure system is described. In at least one embodiment, the third-party network infrastructure system 1602 may have a greater than Figure 16 The more or fewer components depicted may be combined into two or more components, or may have different component configurations or arrangements.

[0187] In at least one embodiment, client computing devices 1604, 1606, and 1608 may be configured to operate client applications, such as web browsers, which may be used by a user of the client computing devices to interact with a third-party network infrastructure system 1602 to use proprietary client applications or other applications that provide services provided by the third-party network infrastructure system 1602. Although the exemplary system environment 1600 is shown as having three client computing devices, any number of client computing devices can be supported. In at least one embodiment, other devices, such as devices with sensors, may interact with the third-party network infrastructure system 1602. In at least one embodiment, one or more networks 1610 may facilitate communication and data exchange between client computing devices 1604, 1606, and 1608 and the third-party network infrastructure system 1602.

[0188] In at least one embodiment, the services provided by the third-party network infrastructure system 1602 may include hosting services available on demand to users of the third-party network infrastructure system. In at least one embodiment, various services may also be provided, including but not limited to online data storage and backup solutions, web-based email services, hosted office suites and document collaboration services, database management and processing, managed technical support services, and / or variations thereof. In at least one embodiment, the services provided by the third-party network infrastructure system can be dynamically expanded to meet the needs of its users.

[0189] In at least one embodiment, a specific instantiation of a service provided by the third-party network infrastructure system 1602 may be referred to as a "service instance". In at least one embodiment, generally, any service available to a user from the third-party network service provider system via a communication network (such as the Internet) is referred to as a "third-party network service". In at least one embodiment, in a public third-party network environment, the servers and systems constituting the third-party network service provider system are different from the servers and systems on the customer's own premises. In at least one embodiment, the third-party network service provider system may host applications, and users may subscribe to and use applications on demand via a communication network (such as the Internet).

[0190] In at least one embodiment, services within a third-party computer network infrastructure may include protected computer network access to storage, hosted databases, hosted web servers, software applications, or other services provided to users by a third-party network provider. In at least one embodiment, services may include password-protected access to remote storage devices on a third-party network via the Internet. In at least one embodiment, services may include a web-based hosted relational database and a scripting language middleware engine for private use by networking developers. In at least one embodiment, services may include access to email software applications hosted on a website hosted by a third-party network provider.

[0191] In at least one embodiment, the third-party network infrastructure system 1602 may include a suite of application, middleware, and database service providers delivered to clients in a self-service, subscription-based, elastically scalable, reliable, highly available, and secure manner. In at least one embodiment, the third-party network infrastructure system 1602 may also provide “big data” related computing and analytics services. In at least one embodiment, the term “big data” is generally used to refer to extremely large datasets that can be stored and manipulated by analysts and researchers to visualize large amounts of data, detect trends, and / or otherwise interact with the data. In at least one embodiment, big data and related applications may be hosted and / or manipulated by the infrastructure system at many levels and at different scales. In at least one embodiment, dozens, hundreds, or thousands of processors linked in parallel may act on such data to present the data or simulate external forces on the data or what it represents. In at least one embodiment, these datasets may involve structured data (such as structured data organized in a database or otherwise according to a structured model) and / or unstructured data (e.g., emails, images, data blobs (binary large objects), web pages, complex event processing). In at least one embodiment, by leveraging the ability of the embodiment to focus more (or less) computing resources relatively quickly on the target, third-party network infrastructure systems can be better used to perform tasks on large datasets based on the needs of enterprises, government agencies, research organizations, private individuals, groups of like-minded individuals or organizations, or other entities.

[0192] In at least one embodiment, the third-party network infrastructure system 1602 can be adapted to automatically provide, manage, and track customer subscriptions to services provided by the third-party network infrastructure system 1602. In at least one embodiment, the third-party network infrastructure system 1602 can provide third-party network services via different deployment models. In at least one embodiment, services can be provided under a public third-party network model, wherein the third-party network infrastructure system 1602 is owned by an organization selling third-party network services, and the services are available to the general public or various industry enterprises. In at least one embodiment, services can be provided under a private third-party network model, in which the third-party network infrastructure system 1602 operates only for a single organization and can provide services to one or more entities within that organization. In at least one embodiment, third-party network services can also be provided under a community third-party network model, wherein the third-party network infrastructure system 1602 and the services provided by the third-party network infrastructure system 1602 are shared by several organizations in the relevant community. In at least one embodiment, third-party network services can also be provided under a hybrid third-party network model, which is a combination of two or more different models.

[0193] In at least one embodiment, the services provided by the third-party network infrastructure system 1602 may include one or more services offered under the Software as a Service (SaaS) category, Platform as a Service (PaaS) category, Infrastructure as a Service (IaaS) category, or other service categories that include hybrid services. In at least one embodiment, a customer may subscribe to one or more services provided by the third-party network infrastructure system 1602 via a subscription order. In at least one embodiment, the third-party network infrastructure system 1602 then performs processing to provide the services in the customer's subscription order.

[0194] In at least one embodiment, the services provided by the third-party network infrastructure system 1602 may include, but are not limited to, application services, platform services, and infrastructure services. In at least one embodiment, the application services may be provided by the third-party network infrastructure system via a SaaS platform. In at least one embodiment, the SaaS platform may be configured to provide third-party network services belonging to the SaaS category. In at least one embodiment, the SaaS platform may provide the ability to build and deliver a suite of on-demand applications on an integrated development and deployment platform. In at least one embodiment, the SaaS platform may manage and control the underlying software and infrastructure used to provide SaaS services. In at least one embodiment, by utilizing the services provided by the SaaS platform, customers can utilize applications running on the third-party network infrastructure system. In at least one embodiment, customers can obtain application services without needing to purchase separate licenses and support. In at least one embodiment, a variety of different SaaS services may be provided. In at least one embodiment, examples include, but are not limited to, services that provide solutions for sales performance management, enterprise integration, and business agility for large organizations.

[0195] In at least one embodiment, the platform service may be provided by a third-party network infrastructure system 1602 via a PaaS platform. In at least one embodiment, the PaaS platform may be configured to provide third-party network services belonging to the PaaS category. In at least one embodiment, examples of platform services may include, but are not limited to, services that enable organizations to merge existing applications on a shared public architecture, and the ability to build new applications utilizing shared services provided by the platform. In at least one embodiment, the PaaS platform may manage and control the underlying software and infrastructure used to provide the PaaS service. In at least one embodiment, customers can access the PaaS service provided by the third-party network infrastructure system 1602 without the need for customers to purchase separate licenses and support.

[0196] In at least one embodiment, by leveraging services provided by a PaaS platform, customers can employ programming languages ​​and tools supported by a third-party network infrastructure system and also control the deployed services. In at least one embodiment, the platform services provided by the third-party network infrastructure system may include database third-party network services, middleware third-party network services, and third-party network services. In at least one embodiment, the database third-party network service may support a shared service deployment model that enables organizations to aggregate database resources and provide database-as-a-service to customers in the form of a database third-party network. In at least one embodiment, within the third-party network infrastructure system, the middleware third-party network service can provide customers with a platform to develop and deploy different business applications, and the third-party network service can provide customers with a platform to deploy applications.

[0197] In at least one embodiment, various infrastructure services may be provided by an IaaS platform within a third-party network infrastructure system. In at least one embodiment, infrastructure services facilitate the management and control of underlying computing resources (such as storage, networking, and other basic computing resources) by customers utilizing services provided by SaaS and PaaS platforms.

[0198] In at least one embodiment, the third-party network infrastructure system 1602 may further include infrastructure resources 1630 for providing resources for offering various services to customers of the third-party network infrastructure system. In at least one embodiment, infrastructure resources 1630 may include a pre-integrated and optimized combination of hardware (such as servers, storage, and networking resources) for performing services provided by PaaS platforms and SaaS platforms, as well as other resources.

[0199] In at least one embodiment, resources in the third-party network infrastructure system 1602 can be shared by multiple users and dynamically reallocated as needed. In at least one embodiment, resources can be allocated to users in different time zones. In at least one embodiment, the third-party network infrastructure system 1602 can enable a first group of users in a first time zone to utilize the resources of the third-party network infrastructure system for a specified number of hours, and subsequently enable the reallocation of the same resources to another group of users located in a different time zone, thereby maximizing resource utilization.

[0200] In at least one embodiment, multiple internal shared services 1632, shared by different components or modules of the third-party network infrastructure system 1602, may be provided to enable services provided by the third-party network infrastructure system 1602. In at least one embodiment, these internal shared services may include, but are not limited to, security and identity services, integration services, enterprise library services, enterprise manager services, virus scanning and whitelisting services, high availability, backup and recovery services, services for enabling third-party network support, email services, notification services, file transfer services, and / or variations thereof.

[0201] In at least one embodiment, the third-party network infrastructure system 1602 can provide comprehensive management of third-party network services (e.g., SaaS, PaaS, and IaaS services) within the third-party network infrastructure system. In at least one embodiment, the third-party network management functionality may include the ability to provision, manage, and track customer subscriptions received by the third-party network infrastructure system 1602 and / or variations thereof.

[0202] In at least one embodiment, such as Figure 16 As shown, third-party network management functions can be provided by one or more modules, such as order management module 1620, order coordination module 1622, order supply module 1624, order management and monitoring module 1626, and identity management module 1628. In at least one embodiment, these modules may include or be provided using one or more computers and / or servers, which may be general-purpose computers, dedicated server computers, server farms, server clusters, or any other suitable arrangement and / or combination.

[0203] In at least one embodiment, in step 1634, a customer using a client device (such as client computing device 1604, 1606, or 1608) interacts with the third-party network infrastructure system 1602 by requesting one or more services provided by the third-party network infrastructure system 1602 and placing an order for a subscription to one or more services provided by the third-party network infrastructure system 1602. In at least one embodiment, the customer may access a third-party network user interface (UI), such as third-party network UI 1612, third-party network UI 1614, and / or third-party network UI 1616, and place orders via these UIs. In at least one embodiment, order information received by the third-party network infrastructure system 1602 in response to a customer placing an order may include information identifying the customer and the one or more services provided by the third-party network infrastructure system 1602 that the customer wishes to subscribe to.

[0204] In at least one embodiment, in step 1636, the order information received from the customer may be stored in the order database 1618. In at least one embodiment, if this is a new order, a new record may be created for the order. In at least one embodiment, the order database 1618 may be one of several databases operated by a third-party network infrastructure system 1618 and in conjunction with other system components.

[0205] In at least one embodiment, in step 1638, the order information can be forwarded to the order management module 1620, which can be configured to perform billing and accounting functions related to the order, such as verifying the order and, after verification, booking an order.

[0206] In at least one embodiment, in step 1640, information about the order may be transmitted to an order coordination module 1622, which is configured to coordinate the provision of services and resources for orders placed by customers. In at least one embodiment, the order coordination module 1622 may use the services of the order provisioning module 1624 for provisioning. In at least one embodiment, the order coordination module 1622 enables the management of business processes associated with each order and applies business logic to determine whether an order should continue to be provisioned.

[0207] In at least one embodiment, in step 1642, upon receiving a new subscription order, the order coordination module 1622 sends a request to the order provisioning module 1624 to allocate resources and configure the resources required to satisfy the subscription order. In at least one embodiment, the order provisioning module 1624 implements resource allocation for the service ordered by the customer. In at least one embodiment, the order provisioning module 1624 provides an abstraction level between third-party network services provided by the third-party network infrastructure system 1600 and the physical implementation layer used to supply resources for providing the requested service. In at least one embodiment, this allows the order coordination module 1622 to be isolated from implementation details, such as whether services and resources are actually provisioned in real-time or pre-provisioned and allocated / assigned only upon request.

[0208] In at least one embodiment, in step 1644, once the service and resources are provided, a notification instructing the subscribing customer that the requested service is now ready for use can be sent. In at least one embodiment, information (e.g., a link) can be sent to the customer, enabling the customer to begin using the requested service.

[0209] In at least one embodiment, in step 1646, the customer's subscription order can be managed and tracked by the order management and monitoring module 1626. In at least one embodiment, the order management and monitoring module 1626 can be configured to collect usage statistics regarding customer use of the subscription service. In at least one embodiment, statistics can be collected for storage usage, data transfer volume, number of users, and the amount and / or changes in system power-on and power-off times.

[0210] In at least one embodiment, the third-party network infrastructure system 1600 may include an identity management module 1628 configured to provide identity services, such as access management and authorization services within the third-party network infrastructure system 1600. In at least one embodiment, the identity management module 1628 may control information about customers who wish to utilize services provided by the third-party network infrastructure system 1602. In at least one embodiment, such information may include information authenticating the identity of such customers and information describing which actions those customers are authorized to perform relative to various system resources (e.g., files, directories, applications, communication ports, memory segments, etc.). In at least one embodiment, the identity management module 1628 may also include managing descriptive information about each customer, as well as information about how and by whom that descriptive information can be accessed and modified.

[0211] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0212] Figure 17 A cloud computing environment 1702 according to at least one embodiment is illustrated. In at least one embodiment, the cloud computing environment 1702 includes one or more computer systems / servers 1704, with computing devices such as personal digital assistants (PDAs) or cellular phones 1706A, desktop computers 1706B, laptop computers 1706C, and / or automotive computer systems 1706N communicating with the one or more computer systems / servers 1704. In at least one embodiment, this allows infrastructure, platforms, and / or software to be provided as services from the cloud computing environment 1702 so that each client does not need to maintain such resources individually. It should be understood that... Figure 17The types of computing devices 1706A-N shown are intended to be illustrative only, and the cloud computing environment 1702 can communicate with any type of computerized device via any type of network and / or network / addressable connectivity (e.g., using a web browser).

[0213] In at least one embodiment, the computer system / server 1704, which may be represented as a cloud computing node, may operate with many other general-purpose or special-purpose computing system environments or configurations. Examples of computing systems, environments, and / or configurations suitable for use with the computer system / server 1704 in at least one embodiment include, but are not limited to, personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics, network PCs, minicomputer systems, mainframe computer systems, and distributed cloud computing environments that include any of the aforementioned systems or devices, and / or variations thereof.

[0214] In at least one embodiment, the computer system / server 1704 can be described in the general context of computer system executable instructions (such as program modules) executed by the computer system. In at least one embodiment, the program module includes routines, programs, objects, components, logic, data structures, etc., that perform a specific task or implement a specific abstract data type. In at least one embodiment, the exemplary computer system / server 1704 can be practiced in a distributed cloud computing environment, where the task is performed by a remote processing device linked via a communication network. In at least one embodiment, in a distributed cloud computing environment, the program module may reside in both local and remote computer system storage media, including memory storage devices.

[0215] Figure 18 The cloud computing environment 1702 (illustrated according to at least one embodiment) is shown. Figure 17 This provides a set of functional abstractions. It should be understood beforehand. Figure 18 The components, layers, and functions shown are intended to be illustrative only, and may vary.

[0216] In at least one embodiment, the hardware and software layer 1802 includes hardware and software components. Examples of hardware components in at least one embodiment include mainframes, servers based on various RISC (Reduced Instruction Set Computer) architectures, various computing systems, supercomputing systems, storage devices, networks, networking components, and / or variations thereof. Examples of software components in at least one embodiment include network application server software, various application server software, various database software, and / or variations thereof.

[0217] In at least one embodiment, the virtualization layer 1804 provides an abstraction layer from which exemplary virtual entities such as virtual servers, virtual storage, virtual networks (including virtual private networks), virtual applications, virtual clients, and / or variations thereof can be provided.

[0218] In at least one embodiment, the management layer 1806 provides various functionalities. In at least one embodiment, resource provisioning provides the dynamic acquisition of computing resources and other resources for performing tasks within the cloud computing environment. In at least one embodiment, metering provides usage tracking of resources within the cloud computing environment, and billing or invoicing for the consumption of these resources. In at least one embodiment, resources may include application software licenses. In at least one embodiment, security provides authentication for users and tasks, and protection for data and other resources. In at least one embodiment, the user interface provides access to the cloud computing environment for both users and system administrators. In at least one embodiment, service level management provides the allocation and management of cloud computing resources to meet required service levels. In at least one embodiment, service level agreement (SLA) management provides the pre-deployment and acquisition of cloud computing resources, anticipating future demand for those resources according to the SLA.

[0219] In at least one embodiment, workload layer 1808 provides functionality that leverages a cloud computing environment. Examples of workloads and functionalities that may be provided from this layer in at least one embodiment include: mapping and navigation, software development and management, educational services, data analysis and processing, transaction processing, and service delivery.

[0220] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0221] Supercomputing

[0222] The following figures illustrate, but are not limited to, exemplary supercomputer-based systems that can be used to implement at least one embodiment.

[0223] In at least one embodiment, a supercomputer can refer to a hardware system exhibiting significant parallelism and comprising at least one chip, wherein the chips in the system are interconnected via a network and housed in a hierarchically organized enclosure. In at least one embodiment, a large hardware system filling a server room with several racks is a special case of a supercomputer, each rack containing several board / rack modules, each board / rack module containing several chips all interconnected by a scalable network. In at least one embodiment, a single rack of such a large hardware system is another example of a supercomputer. In at least one embodiment, a single chip exhibiting significant parallelism and comprising several hardware components can also be considered a supercomputer, because as feature sizes may decrease, the number of hardware components that can be incorporated into a single chip may also increase.

[0224] Figure 19 A chip-level supercomputer according to at least one embodiment is illustrated. In at least one embodiment, the main computation is performed within a finite state machine (1904) referred to as a thread unit, inside an FPGA or ASIC chip. In at least one embodiment, a task and synchronization network (1902) connects the finite state machine and is used to dispatch threads and perform operations in the correct order. In at least one embodiment, a memory network (1906, 1910) is used to access a multi-level partitioned on-chip cache hierarchy (1908, 1912). In at least one embodiment, a memory controller (1916) and an off-chip memory network (1914) are used to access off-chip memory. In at least one embodiment, an I / O controller (1918) is used for cross-chip communication when the design is not suitable for a single logic chip.

[0225] Figure 20 A supercomputer at the rack module level is illustrated according to at least one embodiment. In at least one embodiment, within the rack module, there are multiple FPGA or ASIC chips (2002) connected to one or more DRAM cells (2004) constituting the main accelerator memory. In at least one embodiment, each FPGA / ASIC chip is connected to its adjacent FPGA / ASIC chip using a wide on-board bus with differential high-speed signaling (2006). In at least one embodiment, each FPGA / ASIC chip is also connected to at least one high-speed serial communication cable.

[0226] Figure 21 A rack-level supercomputer according to at least one embodiment is shown. Figure 22 A supercomputer at the entire system level is illustrated according to at least one embodiment. In at least one embodiment, see [link to at least one embodiment]. Figure 21 and Figure 22A scalable, potentially incomplete, hypercube network is implemented between rack modules within a rack and across the entire system rack using high-speed serial optical or copper cables (2102, 2202). In at least one embodiment, one of the FPGA / ASIC chips in the accelerator is connected to a host system (2204) via a PCI-Express connection. In at least one embodiment, the host system includes a host microprocessor (2208) on which the software portion of the application runs, and a memory consisting of one or more host memory DRAM cells (2206) consistent with the memory on the accelerator. In at least one embodiment, the host system may be a separate module on one of the racks or may be integrated with one of the modules of the supercomputer. In at least one embodiment, a circular topology of cube connections provides communication links to create a hypercube network for a large supercomputer. In at least one embodiment, a group of FPGA / ASIC chips on a rack module may act as a single hypercube node, increasing the total number of external links per group compared to a single chip. In at least one embodiment, a group comprises chips A, B, C, and D on a rack module having an internal wide differential bus connecting A, B, C, and D in a circular organization. In at least one embodiment, there are 12 serial communication cables connecting the rack module to the outside world. In at least one embodiment, chip A on the rack module is connected to serial communication cables 0, 1, and 2. In at least one embodiment, chip B is connected to cables 3, 4, and 5. In at least one embodiment, chip C is connected to cables 6, 7, and 8. In at least one embodiment, chip D is connected to cables 9, 10, and 11. In at least one embodiment, the entire group {A, B, C, D} constituting the rack module can form a hypercube node within a supercomputer system, containing up to 2^12 = 4096 rack modules (21384 FPGA / ASIC chips). In at least one embodiment, in order for chip A to send a message outward on link 4 of group {A, B, C, D}, the message must first be routed to chip B using an on-board differential wide bus connection. In at least one embodiment, messages arriving on link 4 from group {A, B, C, D} (i.e., arriving at B) to chip A must also first be routed to the correct destination chip (A) within group {A, B, C, D}. In at least one embodiment, parallel supercomputer systems of other sizes can also be implemented.

[0227] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10The embodiments described in the figures are implemented to achieve the desired results.

[0228] AI

[0229] The following figures illustrate, but are not limited to, exemplary artificial intelligence-based systems that can be used to implement at least one embodiment.

[0230] Figure 23A Inference and / or training logic 2315 is illustrated for performing inference and / or training operations associated with one or more embodiments. The following is in conjunction with... Figure 23A and / or Figure 23B Provide details about reasoning and / or training logic 2315.

[0231] In at least one embodiment, the inference and / or training logic 2315 may include, but is not limited to, code and / or data storage 2301 for storing forward and / or output weights and / or input / output data, and / or other parameters for configuring neurons or layers of a neural network being trained and / or used for inference in aspects of one or more embodiments. In at least one embodiment, the training logic 2315 may include or be coupled to the code and / or data storage 2301 for storing graph code or other software to control timing and / or sequence, wherein weight and / or other parameter information is loaded to configure the logic, including integer and / or floating-point units (collectively, arithmetic logic units (ALUs)). In at least one embodiment, code (such as graph code) loads weight or other parameter information into the processor ALU based on the architecture of the neural network to which such code corresponds. In at least one embodiment, the code and / or data storage 2301 stores weight parameters and / or input / output data for each layer of a neural network that is trained or used in conjunction with one or more embodiments during forward propagation of input / output data and / or weight parameters during training and / or inference using aspects of one or more embodiments. In at least one embodiment, any portion of the code and / or data storage 2301 may be included together with other on-chip or off-chip data storage devices, including the processor's L1, L2, or L3 cache memory or system memory.

[0232] In at least one embodiment, any portion of the code and / or data storage 2301 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 2301 may be a cache memory, dynamic random-addressable memory (“DRAM”), static random-addressable memory (“SRAM”), non-volatile memory (e.g., flash memory), or other storage devices. In at least one embodiment, for example, the choice of whether the code and / or data storage 2301 is internal or external to the processor, or the choice to include DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0233] In at least one embodiment, the inference and / or training logic 2315 may include, but is not limited to, code and / or data storage 2305 for storing backpropagation and / or output weights and / or input / output data corresponding to neurons or layers of a neural network trained and / or used for inference in one or more aspects of the embodiments. In at least one embodiment, the code and / or data storage 2305 stores weight parameters and / or input / output data for each layer of the neural network, which is trained or used in conjunction with one or more embodiments during backpropagation of input / output data and / or weight parameters during training and / or inference in one or more aspects of the embodiments. In at least one embodiment, the training logic 2315 may include or be coupled to the code and / or data storage 2305 to store graph code or other software to control timing and / or sequencing, wherein weight and / or other parameter information is loaded to configure logic, including integer and / or floating-point units (collectively referred to as Arithmetic Logic Units (ALUs)).

[0234] In at least one embodiment, code (such as graph code) causes the architecture of the neural network corresponding to such code to load weights or other parameter information into the processor ALU. In at least one embodiment, any portion of the code and / or data storage 2305 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. In at least one embodiment, any portion of the code and / or data storage 2305 may be internal or external to one or more processors or other hardware logic devices or circuits. In at least one embodiment, the code and / or data storage 2305 may be cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage devices. In at least one embodiment, for example, the choice of whether the code and / or data storage 2305 is internal or external to the processor, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0235] In at least one embodiment, code and / or data storage 2301 and code and / or data storage 2305 may be separate storage structures. In at least one embodiment, code and / or data storage 2301 and code and / or data storage 2305 may be combined storage structures. In at least one embodiment, code and / or data storage 2301 and code and / or data storage 2305 may be partially combined and partially separated. In at least one embodiment, any portion of code and / or data storage 2301 and code and / or data storage 2305 may be included together with other on-chip or off-chip data storage (including the processor's L1, L2, or L3 cache or system memory).

[0236] In at least one embodiment, the inference and / or training logic 2315 may include, but is not limited to, one or more arithmetic logic units (“ALUs”) 2310, including integer and / or floating-point units, for performing logical and / or mathematical operations at least in part based on or instructed by training and / or inference code (e.g., graph code), the results of which may produce activations (e.g., output values ​​from layers or neurons within a neural network) stored in activation storage 2320, which is a function of input / output and / or weight parameter data stored in code and / or data storage 2301 and / or code and / or data storage 2305. In at least one embodiment, activations stored in activation memory 2320 are generated based on linear algebra and / or matrix-based mathematics performed by ALU 2310 in response to execution instructions or other code, wherein weight values ​​stored in code and / or data storage 2305 and / or data storage 2301 are used as operands along with other values ​​(such as bias values, gradient information, momentum values, or other parameters or hyperparameters), any or all of which may be stored in code and / or data storage 2305 or code and / or data storage 2301 or in another memory on or off the chip.

[0237] In at least one embodiment, one or more ALUs 2310 are included within one or more processors or other hardware logic devices or circuits, while in another embodiment, one or more ALUs 2310 may be external to the processor or other hardware logic devices or circuits (e.g., coprocessors) that use them. In at least one embodiment, ALUs 2310 may be included within an execution unit of a processor or otherwise within an ALU library accessible by the execution unit of the processor, which may be within the same processor or distributed among different types of processors (e.g., central processing unit, graphics processing unit, fixed-function unit, etc.). In at least one embodiment, code and / or data storage 2301, code and / or data storage 2305, and activation storage 2320 may share a processor or other hardware logic device or circuit, while in another embodiment, they may be in different processors or other hardware logic devices or circuits, or in some combination of the same and different processors or other hardware logic devices or circuits. In at least one embodiment, any portion of the activation storage 2320 may be included together with other on-chip or off-chip data storage, including the processor's L1, L2, or L3 cache or system memory. Furthermore, inference and / or training code may be stored together with other code accessible to the processor or other hardware logic or circuitry and acquired and / or processed using the processor's fetch, decode, schedule, execute, retire, and / or other logic circuitry.

[0238] In at least one embodiment, the active memory 2320 may be a cache memory, DRAM, SRAM, non-volatile memory (e.g., flash memory), or other storage device. In at least one embodiment, the active memory 2320 may be wholly or partially located within or outside one or more processors or other logic circuits. In at least one embodiment, for example, the choice of whether the active memory 2320 is inside or outside the processor, or includes DRAM, SRAM, flash memory, or some other storage type, may depend on the available on-chip storage relative to off-chip storage, the latency requirements of the training and / or inference functions being performed, the batch size of the data used in the inference and / or training of the neural network, or some combination of these factors.

[0239] In at least one embodiment, Figure 23A The inference and / or training logic 2315 shown can be used in conjunction with an application-specific integrated circuit (“ASIC”), such as those from Google. Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, a "Lake Crest" processor. In at least one embodiment, Figure 23A The inference and / or training logic 2315 shown can be used in conjunction with central processing unit (“CPU”) hardware, graphics processing unit (“GPU”) hardware or other hardware such as field programmable gate array (“FPGA”)

[0240] Figure 23B Inference and / or training logic 2315 according to at least one embodiment is illustrated. In at least one embodiment, the inference and / or training logic 2315 may include, but is not limited to, hardware logic in which computational resources are dedicated or otherwise exclusively used in conjunction with weight values ​​or other information corresponding to one or more neuron layers within a neural network. In at least one embodiment, Figure 23B The inference and / or training logic 2315 shown can be combined with an application-specific integrated circuit (ASIC) (such as those from Google). Processing unit, from Graphcore TM Inference processing unit (IPU), or from Intel Corporation (For example, "Lake Crest") processors are used. In at least one embodiment, Figure 23BThe inference and / or training logic 2315 shown can be used in conjunction with central processing unit (CPU) hardware, graphics processing unit (GPU) hardware, or other hardware such as a field-programmable gate array (FPGA). In at least one embodiment, the inference and / or training logic 2315 includes, but is not limited to, code and / or data storage 2301 and code and / or data storage 2305, which can be used to store code (e.g., graph code), weight values, and / or other information, including bias values, gradient information, momentum values, and / or other parameter or hyperparameter information. Figure 23B In at least one embodiment described herein, each of code and / or data storage 2301 and code and / or data storage 2305 is associated with a dedicated computing resource (e.g., computing hardware 2302 and computing hardware 2306). In at least one embodiment, each of computing hardware 2302 and computing hardware 2306 includes one or more ALUs that perform mathematical functions (such as linear algebraic functions) on the information stored in code and / or data storage 2301 and code and / or data storage 2305, respectively, and the results are stored in active storage 2320.

[0241] In at least one embodiment, each code and / or data storage 2301 and 2305, and the corresponding computing hardware 2302 and 2306, respectively correspond to different layers of the neural network, such that the activation result from one storage / computation pair 2301 / 2302 of the code and / or data storage 2301 and computing hardware 2302 is provided as input to the next storage / computation pair 2305 / 2306 of the code and / or data storage 2305 and computing hardware 2306, in order to mirror the conceptual organization of the neural network. In at least one embodiment, each of the storage / computation pairs 2301 / 2302 and 2305 / 2306 may correspond to more than one neural network layer. In at least one embodiment, additional storage / computation pairs (not shown) following or paralleling the storage / computation pairs 2301 / 2302 and 2305 / 2306 may be included in the inference and / or training logic 2315.

[0242] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0243] Figure 24The training and deployment of a deep neural network according to at least one embodiment are illustrated. In at least one embodiment, an untrained neural network 2406 is trained using a training dataset 2402. In at least one embodiment, the training framework 2404 is a PyTorch framework, while in other embodiments, the training framework 2404 is TensorFlow, Boost, Caffe, Microsoft Cognitive Toolkit / CNTK, MXNet, Chainer, Keras, Deeplearning4j, or other training frameworks. In at least one embodiment, the training framework 2404 trains the untrained neural network 2406 and enables it to be trained using the processing resources described herein to generate a trained neural network 2408. In at least one embodiment, the weights may be randomly selected or selected by pre-training using a deep belief network. In at least one embodiment, training may be performed in a supervised, partially supervised, or unsupervised manner.

[0244] In at least one embodiment, supervised learning is used to train an untrained neural network 2406, wherein the training dataset 2402 includes inputs paired with desired outputs for input, or wherein the training dataset 2402 includes inputs with known outputs, and the outputs of the neural network 2406 are manually graded. In at least one embodiment, the untrained neural network 2406 is trained in a supervised manner, and inputs from the training dataset 2402 are processed, and the resulting outputs are compared with a set of expected or desired outputs. In at least one embodiment, the error is then backpropagated through the untrained neural network 2406. In at least one embodiment, a training framework 2404 adjusts the weights controlling the untrained neural network 2406. In at least one embodiment, the training framework 2404 includes tools for monitoring how well the untrained neural network 2406 converges toward a model (such as a trained neural network 2408) adapted to generate correct answers (such as results 2414) based on input data (such as a new dataset 2412). In at least one embodiment, the training framework 2404 repeatedly trains the untrained neural network 2406 while using a loss function and tuning algorithms (such as stochastic gradient descent) to adjust the weights to refine the output of the untrained neural network 2406. In at least one embodiment, the training framework 2404 trains the untrained neural network 2406 until the untrained neural network 2406 achieves the desired accuracy. In at least one embodiment, the trained neural network 2408 can then be deployed to implement any number of machine learning operations.

[0245] In at least one embodiment, unsupervised learning is used to train an untrained neural network 2406, wherein the untrained neural network 2406 attempts to train itself using unlabeled data. In at least one embodiment, the unsupervised learning training dataset 2402 will include input data without any associated output data or "ground truth" data. In at least one embodiment, the untrained neural network 2406 can learn groupings within the training dataset 2402 and can determine how each input relates to the untrained dataset 2402. In at least one embodiment, unsupervised training can be used to generate self-organizing maps in the trained neural network 2408 capable of performing operations useful in reducing the dimensionality of the new dataset 2412. In at least one embodiment, unsupervised training can also be used to perform anomaly detection, which allows the identification of data points in the new dataset 2412 that deviate from the normal patterns of the new dataset 2412.

[0246] In at least one embodiment, semi-supervised learning can be used, which is a technique in which a mixture of labeled and unlabeled data is included in the training dataset 2402. In at least one embodiment, the training framework 2404 can be used to perform incremental learning, such as through transfer learning techniques. In at least one embodiment, incremental learning enables the trained neural network 2408 to adapt to a new dataset 2412 without forgetting the knowledge injected into the trained neural network 1408 during initial training.

[0247] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0248] 5G network

[0249] The following figures illustrate, but are not limited to, exemplary 5G-based systems that can be used to implement at least one embodiment.

[0250] Figure 25An architecture of a system 2500 for a network according to at least one embodiment is illustrated. In at least one embodiment, system 2500 is shown as including user equipment (UE) 2502 and UE 2504. In at least one embodiment, UE 2502 and 2504 are shown as smartphones (e.g., handheld touchscreen mobile computing devices capable of connecting to one or more cellular networks), but may also include any mobile or non-mobile computing device, such as a personal digital assistant (PDA), pager, laptop computer, desktop computer, wireless handheld device, or any computing device including a wireless communication interface.

[0251] In at least one embodiment, either UE 2502 or UE 2504 may include an Internet of Things (IoT) UE, which may include a network access layer designed for low-power IoT applications utilizing short-lived UE connections. In at least one embodiment, the IoT UE may utilize technologies such as machine-to-machine (M2M) or machine-type communication (MTC) for exchanging data with an MTC server or device via a Public Land Mobile Network (PLMN), Proximity-Based Service (ProSe), or Device-to-Device (D2D) communication, sensor networks, or the IoT network. In at least one embodiment, the M2M or MTC data exchange may be machine-initiated data exchange. In at least one embodiment, the IoT network describes interconnected IoT UEs, which may include uniquely identifiable embedded computing devices (within the Internet infrastructure) with short-lived connections. In at least one embodiment, the IoT UE may execute background applications (e.g., keep-alive messages, state updates, etc.) to facilitate connectivity to the IoT network.

[0252] In at least one embodiment, UE 2502 and UE 2504 may be configured to connect (e.g., communicatively coupled) to a radio access network (RAN) 2516. In at least one embodiment, RAN 2516 may be, for example, an evolved Universal Mobile Telecommunications System (UMTS) Terrestrial Radio Access Network (E-UTRAN), a NextGen RAN (NG RAN), or some other type of RAN. In at least one embodiment, UE 2502 and UE 2504 utilize connection 2512 and connection 2514, respectively, each connection including a physical communication interface or layer. In at least one embodiment, connections 2512 and 2514 are shown as air interfaces for implementing communication coupling and may be consistent with cellular communication protocols such as the Global System for Mobile Communications (GSM) protocol, Code Division Multiple Access (CDMA) network protocol, Push-to-Talk (PTT) protocol, Cellular PTT (POC) protocol, Universal Mobile Telecommunications System (UMTS) protocol, 3GPP Long Term Evolution (LTE) protocol, 5G protocol, New Radio (NR) protocol, and variants thereof.

[0253] In at least one embodiment, UEs 2502 and 2504 may also directly exchange communication data via ProSe interface 2506. In at least one embodiment, ProSe interface 2506 may alternatively be referred to as a sidelink interface, which includes one or more logical channels, including but not limited to the Physical Sidelink Control Channel (PSCCH), Physical Sidelink Shared Channel (PSSCH), Physical Sidelink Discovery Channel (PSDCH), and Physical Sidelink Broadcast Channel (PSBCH).

[0254] In at least one embodiment, UE 2504 is shown configured to access access point (AP) 2510 via connection 2508. In at least one embodiment, connection 2508 may include a local wireless connection, such as a connection consistent with any IEEE 802.11 protocol, wherein AP 2510 will include Wireless Fidelity. Router. In at least one embodiment, AP 2510 is shown as connected to the Internet but not to the core network of a wireless system.

[0255] In at least one embodiment, RAN 2516 may include one or more access nodes enabling connectivity between 2512 and 2514. In at least one embodiment, these access nodes (ANs) may be referred to as base stations (BS), NodeBs, evolved NodeBs (eNBs), next-generation NodeBs (gNBs), RAN nodes, etc., and may include ground stations (e.g., ground access points) or satellite stations providing coverage within a geographic area (e.g., a cell). In at least one embodiment, RAN 2516 may include one or more RAN nodes (e.g., macro RAN node 2518) for providing macrocell coverage and one or more RAN nodes (e.g., low-power (LP) RAN node 2520) for providing femtocells or picocells (e.g., cells with smaller coverage areas, smaller user capacity, or higher bandwidth compared to macrocells).

[0256] In at least one embodiment, either RAN node 2518 or 2520 may terminate the air interface protocol and may be the first contact point for UEs 2502 and 2504. In at least one embodiment, either RAN node 2518 or 2520 may implement various logical functions of RAN 2516, including but not limited to Radio Network Controller (RNC) functions such as radio bearer management, uplink and downlink dynamic radio resource management, and data packet scheduling and mobility management.

[0257] In at least one embodiment, UE 2502 and UE 2504 may be configured to communicate with each other or with either RAN node 2518 and RAN node 2520 via a multi-carrier communication channel using orthogonal frequency division multiplexing (OFDM) communication signals, according to various communication technologies such as, but not limited to, orthogonal frequency division multiple access (OFDMA) communication technology (e.g., for downlink communication) or single-carrier frequency division multiple access (SC-FDMA) communication technology (e.g., for uplink and ProSe or sidelink communication), and / or variations thereof. In at least one embodiment, the OFDM signal may include multiple orthogonal subcarriers.

[0258] In at least one embodiment, the downlink resource grid can be used for downlink transmissions from either RAN nodes 2518 and 2520 to UEs 2502 and 2504, while uplink transmissions can utilize similar techniques. In at least one embodiment, the grid can be a time-frequency grid, referred to as a resource grid or time-frequency resource grid, which represents the physical resources in the downlink within each time slot. In at least one embodiment, this time-frequency plane representation is a common practice in OFDM systems, making it intuitive for radio resource allocation. In at least one embodiment, each column and each row of the resource grid corresponds to an OFDM symbol and an OFDM subcarrier, respectively. In at least one embodiment, the duration of the resource grid in the time domain corresponds to a time slot in a radio frame. In at least one embodiment, the minimum time-frequency unit in the resource grid is represented as a resource element. In at least one embodiment, each resource grid comprises multiple resource blocks that describe the mapping of certain physical channels to resource elements. In at least one embodiment, each resource block comprises a set of resource elements. In at least one embodiment, in the frequency domain, this can represent the minimum number of resources that can currently be allocated. In at least one embodiment, there are several different physical downlink channels that use such resource blocks for transmission.

[0259] In at least one embodiment, the Physical Downlink Shared Channel (PDSCH) can carry user data and higher-layer signaling to UEs 2502 and 2504. In at least one embodiment, the Physical Downlink Control Channel (PDCCH) can carry information such as transmission format and resource allocation related to the PDSCH channel. In at least one embodiment, it can also notify UEs 2502 and 2504 of transmission format, resource allocation, and HARQ (Hybrid Automatic Repeat Request) information related to the uplink shared channel. In at least one embodiment, typically, downlink scheduling (allocating control and shared channel resource blocks to UE 2502 within the cell) can be performed at either RAN node 2518 or 2520 based on channel quality information fed back from either UE 2502 or 2504. In at least one embodiment, downlink resource allocation information can be transmitted on the PDCCH used for (e.g., allocated to) each of UEs 2502 and 2504.

[0260] In at least one embodiment, the PDCCH can use Control Channel Elements (CCEs) to transmit control information. In at least one embodiment, PDCCH complex-valued symbols can first be organized into quadruplets before being mapped to resource elements, and then permuted using a sub-block interleaver for rate matching. In at least one embodiment, one or more of these CCEs can be used to transmit each PDCCH, where each CCE can correspond to nine sets of four physical resource elements referred to as resource element groups (REGs). In at least one embodiment, four Quadrature Phase Shift Keying (QPSK) symbols can be mapped to each REG. In at least one embodiment, depending on the size of the downlink control information (DCI) and channel conditions, one or more CCEs can be used to transmit the PDCCH. In at least one embodiment, there can be four or more different PDCCH formats (e.g., aggregation levels, L = 1, 2, 4, or 8) with different numbers of CCEs as defined in LTE.

[0261] In at least one embodiment, the Enhanced Physical Downlink Control Channel (EPDCCH) using PDSCH resources can be used for control information transmission. In at least one embodiment, one or more Enhanced Control Channel Elements (ECCEs) can be used to transmit the EPDCCH. In at least one embodiment, each ECCE can correspond to nine sets of four physical resource elements referred to as Enhanced Resource Element Groups (EREGs). In at least one embodiment, the ECCE can have a different number of EREGs in some cases.

[0262] In at least one embodiment, RAN 2516 is shown communicatively coupled to core network (CN) 2538 via S1 interface 2522. In at least one embodiment, CN 2538 may be an evolved packet core (EPC) network, a NextGen packet core (NPC) network, or some other type of CN. In at least one embodiment, S1 interface 2522 is divided into two parts: S1-U interface 2526, which carries service data between RAN nodes 2518 and 2520 and serving gateway (S-GW) 2530; and S1-Mobility Management Entity (MME) interface 2524, which is the signaling interface between RAN nodes 2518 and 2520 and MME 2528.

[0263] In at least one embodiment, CN 2538 includes MME 2528, S-GW 2530, Packet Data Network (PDN) Gateway (P-GW) 2534, and Home Subscriber Server (HSS) 2532. In at least one embodiment, MME 2528 may functionally resemble the control plane of a legacy General Packet Radio Service (GPRS) Support Node (SGSN). In at least one embodiment, MME 2528 may manage mobility aspects of access, such as gateway selection and tracking area list management. In at least one embodiment, HSS 2532 may include a database for network users, including subscription-related information to support network entities in handling communication sessions. In at least one embodiment, CN 2538 may include one or more HSS 2532s, depending on the number of mobile users, device capacity, network organization, etc. In at least one embodiment, HSS 2532 may provide support for routing / roaming, authentication, authorization, naming / addressing resolution, location dependencies, etc.

[0264] In at least one embodiment, the S-GW 2530 can terminate the S1 interface 2522 toward RAN 2516 and route data packets between RAN 2516 and CN 2538. In at least one embodiment, the S-GW 2530 can be a local mobility anchor point for inter-RAN node handover and can also provide an anchor point for inter-3GPP mobility. In at least one embodiment, other responsibilities may include lawful interception, charging, and some policy enforcement.

[0265] In at least one embodiment, P-GW 2534 can terminate the SGi interface toward the PDN. In at least one embodiment, P-GW 2534 can route data packets between EPC network 2538 and external networks (such as networks including application server 2540 (or application function (AF))) via Internet Protocol (IP) interface 2542. In at least one embodiment, application server 2540 can be an element that provides applications using IP bearer resources using a core network (e.g., UMTS Packet Service (PS) domain, LTE PS data service, etc.). In at least one embodiment, P-GW 2534 is shown communicatively coupled to application server 2540 via IP communication interface 2542. In at least one embodiment, application server 2540 can also be configured to support one or more communication services (e.g., Voice over Internet Protocol (VoIP) sessions, PTT sessions, group communication sessions, social networking services, etc.) of UEs 2502 and 2504 via CN 2538.

[0266] In at least one embodiment, P-GW 2534 may also be a node for policy enforcement and charging data collection. In at least one embodiment, Policy and Charging Enforcement Function (PCRF) 2536 is the policy and charging control element of CN 2538. In at least one embodiment, in a non-roaming scenario, a single PCRF may exist in the Home Public Land Mobile Network (HPLMN) associated with the UE's Internet Protocol Connectivity Access Network (IP-CAN) session. In at least one embodiment, in a roaming scenario with local traffic breaches, two PCRFs may exist associated with the UE's IP-CAN session: the Home PCRF (H-PCRF) within the HPLMMN and the Visited PCRF (V-PCRF) within the Visited Public Land Mobile Network (VPLMN). In at least one embodiment, PCRF 2536 may be communicatively coupled to application server 2540 via P-GW 2534. In at least one embodiment, application server 2540 may signal PCRF 2536 to indicate new service flows and select appropriate Quality of Service (QoS) and charging parameters. In at least one embodiment, PCRF 2536 can supply this rule to a Policy and Charging Enforcement Function (PCEF) (not shown) with an appropriate Service Flow Template (TFT) and identifier for a QoS Class (QCI), which is initiated by the QoS and charging specified by the application server 2540.

[0267] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0268] Figure 26 The architecture of a system 2600 of a network according to some embodiments is shown. In at least one embodiment, the system 2600 is shown to include a UE 2602, a 5G access node or RAN node (shown as (R)AN node 2608), a user plane function (shown as UPF 2604), a data network (DN 2606), which may be, for example, operator services, Internet access or third-party services, and a 5G core network (5GC) (shown as CN 2610).

[0269] In at least one embodiment, CN 2610 includes authentication server functionality (AUSF 2614); core access and mobility management functionality (AMF 2612); session management functionality (SMF 2618); network exposure functionality (NEF 2616); policy control functionality (PCF 2622); network function (NF) repository functionality (NRF 2620); unified data management (UDM 2624); and application functionality (AF 2626). In at least one embodiment, CN 2610 may also include other elements not shown, such as structured data storage network functionality (SDSF), unstructured data storage network functionality (UDSF), and variations thereof.

[0270] In at least one embodiment, the UPF 2604 can act as an anchor point for mobility within and between RATs, an external PDU session point interconnecting to the DN2606, and a branch point supporting multi-homed PDU sessions. In at least one embodiment, the UPF 2604 can also perform packet routing and forwarding, packet inspection, user plane portion enforcement of policy rules, lawful packet interception (UP collection), service usage reporting, performing QoS processing for the user plane (e.g., packet filtering, gating, UL / DL rate enforcement), performing uplink service verification (e.g., SDF-to-QoS flow mapping), transport-level packet marking in uplink and downlink, and downlink packet buffering and downlink data notification triggering. In at least one embodiment, the UPF 2604 may include an uplink classifier for supporting the routing of service flows to the data network. In at least one embodiment, the DN 2606 may represent various network operator services, Internet access, or third-party services.

[0271] In at least one embodiment, AUSF 2614 can store data for authentication of UE 2602 and handle authentication-related functions. In at least one embodiment, AUSF 2614 can facilitate a common authentication framework for various access types.

[0272] In at least one embodiment, AMF 2612 can be responsible for registration management (e.g., for registering UE 2602, etc.), connection management, reachability management, mobility management, and lawful interception of AMF-related events, as well as access authentication and authorization. In at least one embodiment, AMF 2612 can provide SM message transmission for SMF 2618 and act as a transparent proxy for routing SM messages. In at least one embodiment, AMF 2612 can also provide UE 2602 with SMS functionality (SMSF) (…). Figure 26 Transmission of Short Message Service (SMS) messages between (not shown). In at least one embodiment, AMF 2612 may act as a Security Anchoring Function (SEA), which may include interaction with AUSF 2614 and UE 2602 and receiving an intermediate key established as a result of the UE 2602 authentication process. In at least one embodiment, in the case of using USIM-based authentication, AMF 2612 may retrieve security material from AUSF 2614. In at least one embodiment, AMF 2612 may also include a Security Context Management (SCM) function, which receives from the SEA a key it uses to derive the access network-specific key. Furthermore, in at least one embodiment, AMF 2612 may be the termination point (N2 reference point) of the RAN CP interface, the termination point of NAS (NI) signaling, and perform NAS encryption and integrity protection.

[0273] In at least one embodiment, AMF 2612 can also support NAS signaling with UE 2602 via the N3 Interoperability Function (IWF) interface. In at least one embodiment, the N3IWF can be used to provide access to untrusted entities. In at least one embodiment, the N3IWF can be the termination point of the N2 and N3 interfaces of the control plane and user plane, respectively, thus enabling the processing of N2 signaling from SMF and AMF for PDU sessions and QoS, encapsulation / decapsulation of IPSec and N3 tunnel packets, marking N3 user plane packets in the uplink, and implementing QoS corresponding to the N3 packet marking, taking into account the QoS requirements associated with such marking received via N2. In at least one embodiment, the N3IWF can also relay uplink and downlink control plane NAS (NI) signaling between UE 2602 and AMF 2612, and relay uplink and downlink user plane packets between UE 2602 and UPF 2604. In at least one embodiment, the N3IWF also provides a mechanism for establishing an IPsec tunnel with the UE 2602.

[0274] In at least one embodiment, the SMF 2618 may be responsible for session management (e.g., session establishment, modification, and release, including tunnel maintenance between the UPF and AN nodes); UE IP address allocation and management (including optional authorization); selection and control of UP functions; configuring traffic redirection at the UPF to route traffic to the appropriate destination; interface termination towards policy control functions; policy enforcement and QoS control portions; lawful interception (for SM events and interfaces to the LI system); termination of the SM portion of NAS messages; downlink data notification; initiator of AN-specific SM information, which is sent to the AN via the AMF on N2; and determination of the SSC mode of the session. In at least one embodiment, the SMF 2618 may include the following roaming functions: handling local implementation to apply QoS SLAB (VPLMN); charge data collection and charge interface (VPLMN); lawful interception (for SM events in the VPLMN and interface to the LI system); and supporting interaction with external DNs to transmit signaling for PDU session authorization / authentication performed by the external DNs.

[0275] In at least one embodiment, NEF 2616 can provide means for securely exposing services and capabilities provided to third parties by 3GPP network functions, internal exposure / re-exposure, application functions (e.g., AF 2626), edge computing or fog computing systems, etc. In at least one embodiment, NEF 2616 can authenticate, authorize, and / or throttle AFs. In at least one embodiment, NEF 2616 can also translate information exchanged with AF 2626 and information exchanged with internal network functions. In at least one embodiment, NEF 2616 can translate between AF service identifiers and internal 5GC information. In at least one embodiment, NEF 2616 can also receive information from other network functions (NFs) based on the ability to expose other network functions. In at least one embodiment, this information can be stored as structured data at NEF 2616 or stored at a data storage NF using a standardized interface. In at least one embodiment, the stored information can then be re-exposed by NEF 2616 to other NFs and AFs, and / or used for other purposes, such as analysis.

[0276] In at least one embodiment, the NRF 2620 may support service discovery functionality, receiving NF discovery requests from NF instances and providing information about discovered NF instances to the NF instances. In at least one embodiment, the NRF 2620 also maintains information about available NF instances and the services they support.

[0277] In at least one embodiment, the PCF 2622 may provide policy rules to control plane functions for enforcement, and may also support a unified policy framework for managing network behavior. In at least one embodiment, the PCF 2622 may also implement a front-end (FE) for accessing subscription information related to policy decisions in the UDR of the UDM 2624.

[0278] In at least one embodiment, UDM 2624 can process subscription-related information to support network entities in handling communication sessions and can store subscription data of UE 2602. In at least one embodiment, UDM 2624 can include two parts: an application FE and a user data repository (UDR). In at least one embodiment, UDM can include a UDM FE responsible for handling credentials, location management, subscription management, etc. In at least one embodiment, several different front-ends can serve the same user in different transactions. In at least one embodiment, UDM-FE accesses sub-subscription information stored in UDR and performs authentication credential processing; user identification processing; access authorization; registration / mobility management; and subscription management. In at least one embodiment, UDR can interact with PCF 2622. In at least one embodiment, UDM 2624 can also support SMS management, where SMS-FE implements similar application logic as described above.

[0279] In at least one embodiment, AF 2626 can provide application effects on service routing, access to Network Capability Exposure (NCE), and interaction with a policy framework for policy control. In at least one embodiment, NCE can be a mechanism allowing 5GC and AF 2626 to provide information to each other via NEF 2616, which can be used for edge computing implementations. In at least one embodiment, network operators and third-party services can be hosted near the attached access point of UE 2602 to achieve efficient service delivery by reducing end-to-end latency and load on the transport network. In at least one embodiment, for edge computing implementations, 5GC can select UPF 2604 close to UE 2602 and perform service bootstrapping from UPF 2604 to DN 2606 via the N6 interface. In at least one embodiment, this can be based on UE subscription data, UE location, and information provided by AF 2626. In at least one embodiment, AF 2626 can influence UPF (re)selection and service routing. In at least one embodiment, based on operator deployment, when AF 2626 is considered a trusted entity, the network operator may allow AF 2626 to interact directly with the relevant NF.

[0280] In at least one embodiment, CN 2610 may include an SMSF, which may be responsible for SMS subscription checks and authentication, and relay SM messages to / from UE 2602 to / from other entities, such as SMS-GMSC / IWMSC / SMS routers. In at least one embodiment, SMS may also interact with AMF 2612 and UDM 2624 for a notification process that UE 2602 is available for SMS delivery (e.g., setting a UE unreachable flag and notifying UDM 2624 when UE 2602 is available for SMS).

[0281] In at least one embodiment, system 2600 may include the following service-based interfaces: Namf: a service-based interface presented by AMF; Nsmf: a service-based interface presented by SMF; Nnef: a service-based interface presented by NEF; Npcf: a service-based interface presented by PCF; Nudm: a service-based interface presented by UDM; Naf: a service-based interface presented by AF; Nnrf: a service-based interface presented by NRF; and Nausf: a service-based interface presented by AUSF.

[0282] In at least one embodiment, system 2600 may include the following reference points: N1: a reference point between the UE and the AMF; N2: a reference point between the (R)AN and the AMF; N3: a reference point between the (R)AN and the UPF; N4: a reference point between the SMF and the UPF; and N6: a reference point between the UPF and the data network. In at least one embodiment, there may be more reference points and / or service-based interfaces between NF services in the NF; however, for clarity, these interfaces and reference points have been omitted. In at least one embodiment, the NS reference point may be between the PCF and the AF; the N7 reference point may be between the PCF and the SMF; the N11 reference point may be between the AMF and the SMF, and so on. In at least one embodiment, CN 2610 may include an Nx interface, which is an inter-CN interface between the MME and the AMF 2612 to enable interoperability between CN 2610 and CN 7226.

[0283] In at least one embodiment, system 2600 may include a plurality of RAN nodes (such as (R)AN nodes 2608), wherein an Xn interface is defined between two or more (R)AN nodes 2608 (e.g., gNBs) connected to 5GC 410, between (R)AN nodes 2608 (e.g., gNBs) and eNBs (e.g., macro RAN nodes) connected to CN 2610, and / or between two eNBs connected to CN 2610.

[0284] In at least one embodiment, the Xn interface may include an Xn user plane (Xn-U) interface and an Xn control plane (Xn-C) interface. In at least one embodiment, Xn-U may provide unguaranteed delivery of user plane PDUs and support / provide data forwarding and flow control functions. In at least one embodiment, Xn-C may provide management and error handling functions, functions for managing the Xn-C interface, and mobility support for UE 2602 in connected mode (e.g., CM-CONNECTED), including functions for managing UE mobility for connected modes between one or more (R)AN nodes 2608. In at least one embodiment, mobility support may include context transfer from the old (source) serving (R)AN node 2608 to the new (target) serving (R)AN node 2608; and control of user plane tunneling between the old (source) serving (R)AN node 2608 and the new (target) serving (R)AN node 2608.

[0285] In at least one embodiment, the Xn-U protocol stack may include a transport network layer built on top of the Internet Protocol (IP) transport layer and a GTP-U layer on top of UDP and / or one or more IP layers for carrying user plane PDUs. In at least one embodiment, the Xn-C protocol stack may include an application layer signaling protocol (referred to as the Xn Application Protocol (Xn-AP)) and a transport network layer built on top of the SCTP layer. In at least one embodiment, the SCTP layer may be on top of the IP layers. In at least one embodiment, the SCTP layer provides guaranteed delivery of application layer messages. In at least one embodiment, point-to-point transmission is used to deliver signaling PDUs in the transport IP layer. In at least one embodiment, the Xn-U protocol stack and / or the Xn-C protocol stack may be the same as or similar to the user plane and / or control plane protocol stacks shown and described herein.

[0286] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0287] Figure 27 This is an illustration of a control plane protocol stack according to some embodiments. In at least one embodiment, control plane 2700 is shown as a communication protocol stack between UE 2502 (or alternatively, UE 2504), RAN 2516, and MME 2528.

[0288] In at least one embodiment, PHY layer 2702 can transmit or receive information used by MAC layer 2704 through one or more air interfaces. In at least one embodiment, PHY layer 2702 can also perform link adaptive or adaptive modulation and coding (AMC), power control, cell search (e.g., for initial synchronization and handover purposes), and other measurements used by higher layers (e.g., RRC layer 2710). In at least one embodiment, PHY layer 2702 can further perform error detection, forward error correction (FEC) encoding / decoding of the transport channel, modulation / demodulation of the physical channel, interleaving, rate matching, mapping to the physical channel, and multiple-input multiple-output (MIMO) antenna processing.

[0289] In at least one embodiment, MAC layer 2704 can perform mapping between logical channels and transport channels, multiplexing MAC service data units (SDUs) from one or more logical channels onto a transport block (TB) to be delivered to the PHY via the transport channel, demultiplexing MAC SDUs from the transport block (TB) delivered from the PHY via the transport channel onto one or more logical channels, multiplexing MAC SDUs onto the TB, scheduling information reporting, error correction via hybrid automatic repeat request (HARD), and logical channel prioritization.

[0290] In at least one embodiment, the RLC layer 2706 can operate in multiple operating modes, including: Transparent Mode (TM), Unacknowledged Mode (UM), and Acknowledged Mode (AM). In at least one embodiment, the RLC layer 2706 can perform the transmission of upper-layer protocol data units (PDUs), error correction via Automatic Repeat Request (ARQ) for AM data transmission, and the concatenation, segmentation, and reassembly of RLC SDUs for UM and AM data transmission. In at least one embodiment, the RLC layer 2706 can also perform resegmentation of RLC data PDUs for AM data transmission, reordering of RLC data PDUs for UM and AM data transmission, detection of duplicate data for UM and AM data transmission, discarding of RLC SDUs for UM and AM data transmission, detection of protocol errors in AM data transmission, and performance of RLC reconstruction.

[0291] In at least one embodiment, the PDCP layer 2708 can perform header compression and decompression of IP data, maintain PDCP sequence numbers (SNs), perform intra-sequence delivery of higher-layer PDUs when reconstructing lower layers, eliminate duplication of lower-layer SDUs when reconstructing lower layers for radio bearers mapped on RLC AM, encrypt and decrypt control plane data, perform integrity protection and integrity verification of control plane data, discard data based on control timers, and perform security operations (e.g., encryption, decryption, integrity protection, integrity verification, etc.).

[0292] In at least one embodiment, the main services and functions of the RRC layer 2710 may include broadcasting system information (e.g., included in a Master Information Block (MIB) or System Information Block (SIB) associated with the Non-Access Stratum (NAS), broadcasting system information associated with the Access Stratum (AS), paging, establishment, maintenance, and release of RRC connections between the UE and the E-UTRAN (e.g., RRC connection paging, RRC connection establishment, RRC connection modification, and RRC connection release), establishment, configuration, maintenance, and release of point-to-point radio bearers, including security functions for key management, inter-Radio Access Technology (RAT) mobility, and measurement configuration for UE measurement reporting. In at least one embodiment, the MIB and SIB may include one or more Information Elements (IEs), each of which may include a separate data field or data structure.

[0293] In at least one embodiment, UE 2502 and RAN 2516 may use a Uu interface (e.g., LTE-Uu interface) to exchange control plane data via a protocol stack including PHY layer 2702, MAC layer 2704, RLC layer 2706, PDCP layer 2708 and RRC layer 2710.

[0294] In at least one embodiment, a Non-Access Stratum (NAS) protocol (NAS protocol 2712) forms the highest layer of the control plane between UE 2502 and MME 2528. In at least one embodiment, NAS protocol 2712 supports the mobility and session management procedures of UE 2502 to establish and maintain an IP connection between UE 2502 and P-GW 2534.

[0295] In at least one embodiment, the Si Application Protocol (Si-AP) layer (Si-AP layer 2722) may support the functionality of the Si interface and include basic procedures (EP). In at least one embodiment, the EP is the interaction unit between RAN 2516 and CN 2528. In at least one embodiment, Si-AP layer services may include two groups: UE-associated services and non-UE-associated services. In at least one embodiment, these services perform functions, including but not limited to: E-UTRAN Radio Access Bearer (E-RAB) management, UE capability indication, mobility, NAS signaling transmission, RAN Information Management (RIM), and configuration transfer.

[0296] In at least one embodiment, the Flow Control Transmission Protocol (SCTP) layer (or alternatively, the Flow Control Transmission Protocol / Internet Protocol (SCTP / IP) layer) (SCTP layer 2720) may be partially based on the IP protocol supported by IP layer 2718 to ensure reliable delivery of signaling messages between RAN 2516 and MME 2528. In at least one embodiment, L2 layer 2716 and L1 layer 2714 may refer to the communication links (e.g., wired or wireless) used by the RAN node and MME to exchange information.

[0297] In at least one embodiment, RAN 2516 and one or more MMEs 2528 can utilize the S1-MME interface to exchange control plane data via a protocol stack including L1 layer 2714, L2 layer 2716, IP layer 2718, SCTP layer 2725 and Si-AP layer 2722.

[0298] Figure 28 This is an illustration of a user plane protocol stack according to at least one embodiment. In at least one embodiment, user plane 2800 is shown as a communication protocol stack between UE 2502, RAN 2516, S-GW 2530, and P-GW 2534. In at least one embodiment, user plane 2800 may utilize the same protocol layer as control plane 2700. In at least one embodiment, for example, UE 2502 and RAN 2516 may utilize a Uu interface (e.g., LTE-Uu interface) to exchange user plane data via a protocol stack including PHY layer 2702, MAC layer 2704, RLC layer 2706, and PDCP layer 2708.

[0299] In at least one embodiment, the General Packet Radio Service (GPRS) Tunneling Protocol (GTP-U) layer (GTP-U layer 2804) for the user plane can be used to carry user data within the GPRS core network and between the radio access network and the core network. In at least one embodiment, the transmitted user data can be packets of any format, such as IPv4, IPv6, or PPP. In at least one embodiment, the UDP and IP Security (UDP / IP) layer (UDP / IP layer 2802) can provide checksums for data integrity, port numbers for addressing different functions at the source and destination, and encryption and authentication of selected data streams. In at least one embodiment, the RAN 2516 and S-GW 2530 can utilize the S1-U interface to exchange user plane data via a protocol stack including L1 layer 2714, L2 layer 2716, UDP / IP layer 2802, and GTP-U layer 2804. In at least one embodiment, the S-GW 2530 and P-GW 2534 can utilize the S5 / S8a interface to exchange user plane data via a protocol stack including L1 layer 2714, L2 layer 2716, UDP / IP layer 2802, and GTP-U layer 2804. In at least one embodiment, as described above... Figure 27 The NAS protocol discussed here supports the mobility and session management process of UE 2502 to establish and maintain an IP connection between UE 2502 and P-GW 2534.

[0300] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0301] Figure 29A component 2900 of a core network according to at least one embodiment is illustrated. In at least one embodiment, components of CN 2538 may be implemented in a physical node or a separate physical node, the separate physical node including components for reading and executing instructions from a machine-readable medium or a computer-readable medium (e.g., a non-transitory machine-readable storage medium). In at least one embodiment, network function virtualization (NFV) is used to virtualize any or all of the aforementioned network node functions via executable instructions stored in one or more computer-readable storage media (described further in detail below). In at least one embodiment, a logical instantiation of CN 2538 may be referred to as network slice 2902 (e.g., network slice 2902 is shown as including HSS 2532, MME 2528, and S-GW 2530). In at least one embodiment, a logical instantiation of a portion of CN 2538 may be referred to as network subslice 2904 (e.g., network subslice 2904 is shown as including P-GW 2534 and PCRF 2536).

[0302] In at least one embodiment, the NFV architecture and infrastructure can be used to virtualize one or more network functions onto physical resources comprising a combination of industry-standard server hardware, storage hardware, or switches, which may alternatively be performed by dedicated hardware. In at least one embodiment, the NFV system can be used to perform virtual or reconfigurable implementations of one or more EPC components / functions.

[0303] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0304] Figure 30 This is a block diagram illustrating the components of a system 3000 for supporting Network Functions Virtualization (NFV) according to at least one embodiment. In at least one embodiment, the system 3000 is shown to include a virtualization infrastructure manager (shown as VIM 3002), a network functions virtualization infrastructure (shown as NFVI 3004), a VNF manager (shown as VNFM 3006), virtualized network functions (shown as VNF 3008), a component manager (shown as EM 3010), an NFV coordinator (shown as NFVO 3012), and a network manager (shown as NM 3014).

[0305] In at least one embodiment, VIM 3002 manages the resources of NFVI 3004. In at least one embodiment, NFVI 3004 may include physical or virtual resources and applications (including hypervisors) for executing system 3000. In at least one embodiment, VIM 3002 may utilize NFVI 3004 to manage the lifecycle of virtual resources (e.g., the creation, maintenance, and teardown of virtual machines (VMs) associated with one or more physical resources), track VM instances, track performance, fault and security of VM instances and associated physical resources, and expose VM instances and associated physical resources to other management systems.

[0306] In at least one embodiment, VNFM 3006 can manage VNF 3008. In at least one embodiment, VNF 3008 can be used to perform EPC components / functions. In at least one embodiment, VNFM 3006 can manage the lifecycle of VNF 3008 and track the performance, faults, and security of the virtual aspects of VNF 3008. In at least one embodiment, EM 3010 can track the performance, faults, and security of the functional aspects of VNF 3008. In at least one embodiment, tracking data from VNFM 3006 and EM 3010 may include, for example, performance measurement (PM) data used by VIM 3002 or NFVI 3004. In at least one embodiment, both VNFM 3006 and EM 3010 can scale up / down the number of VNFs in system 3000.

[0307] In at least one embodiment, NFVO 3012 can coordinate, authorize, release, and occupy resources of NFVI 3004 to provide requested services (e.g., to perform EPC functions, components, or slices). In at least one embodiment, NM 3014 can provide an end-user function package responsible for managing a network that may include network elements having VNFs, non-virtualized network functions, or both (management of VNFs may occur via EM 3010).

[0308] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0309] Computer-based systems

[0310] The following figures present, but are not limited to, exemplary computer-based systems that can be used to implement at least one embodiment.

[0311] Figure 31 A processing system 3100 according to at least one embodiment is illustrated. In at least one embodiment, the system 3100 includes one or more processors 3102 and one or more graphics processors 3108, and may be a single-processor desktop system, a multi-processor workstation system, or a server system having a large number of processors 3102 or processor cores 3107. In at least one embodiment, the processing system 3100 is a processing platform incorporated within a system-on-a-chip (SoC) integrated circuit for use in mobile, handheld, or embedded devices.

[0312] In at least one embodiment, the processing system 3100 may include or be integrated into a server-based gaming platform, including a game console, mobile game console, handheld game console, or online game console, which are game and media consoles. In at least one embodiment, the processing system 3100 is a mobile phone, smartphone, tablet computing device, or mobile internet device. In at least one embodiment, the processing system 3100 may also include components coupled to or integrated into a wearable device, such as a smartwatch wearable device, smart glasses device, augmented reality device, or virtual reality device. In at least one embodiment, the processing system 3100 is a television or set-top box device having one or more processors 3102 and a graphical interface generated by one or more graphics processors 3108.

[0313] In at least one embodiment, each of the one or more processors 3102 includes one or more processor cores 3107 for processing instructions that, when executed, perform operations against the system and user software. In at least one embodiment, each of the one or more processor cores 3107 is configured to process a particular instruction set 3109. In at least one embodiment, the instruction set 3109 may facilitate Complex Instruction Set Computing (CISC), Reduced Instruction Set Computing (RISC), or computation via Very Long Instruction Word (VLIW). In at least one embodiment, the plurality of processor cores 3107 may each process a different instruction set 3109, which may include instructions that facilitate the emulation of other instruction sets. In at least one embodiment, the processor cores 3107 may also include other processing devices, such as digital signal processors (DSPs).

[0314] In at least one embodiment, processor 3102 includes cache memory 3104. In at least one embodiment, processor 3102 may have a single internal cache or multiple levels of internal caches. In at least one embodiment, the cache memory is shared among various components of processor 3102. In at least one embodiment, processor 3102 also uses an external cache (e.g., a Level 3 (L3) cache or a last-level cache (LLC)) (not shown), which can be shared among processor cores 3107 using known cache coherence techniques. In at least one embodiment, processor 3102 further includes a register file 3106, which may include different types of registers (e.g., integer registers, floating-point registers, status registers, and instruction pointer registers) for storing different types of data. In at least one embodiment, register file 3106 may include general-purpose registers or other registers.

[0315] In at least one embodiment, one or more processors 3102 are coupled to one or more interface buses 3110 to transmit communication signals, such as address, data, or control signals, between the processors 3102 and other components in the system 3100. In at least one embodiment, the interface bus 3110 may be a processor bus, such as a version of the Direct Media Interface (DMI) bus. In at least one embodiment, the interface bus 3110 is not limited to the DMI bus and may include one or more peripheral component interconnect buses (e.g., PCI, PCI Express), memory buses, or other types of interface buses. In at least one embodiment, the processor 3102 includes an integrated memory controller 3116 and a platform controller hub 3130. In at least one embodiment, the memory controller 3116 facilitates communication between storage devices and other components of the processing system 3100, while the platform controller hub (PCH) 3130 provides connectivity to input / output (I / O) devices via a local I / O bus.

[0316] In at least one embodiment, memory device 3120 may be a dynamic random access memory (DRAM) device, a static random access memory (SRAM) device, a flash memory device, a phase-change memory device, or a device with suitable performance for use as processor memory. In at least one embodiment, memory device 3120 may be used as system memory of processing system 3100 to store data 3122 and instructions 3121 for use when one or more processors 3102 execute applications or processes. In at least one embodiment, memory controller 3116 is also coupled to an optional external graphics processor 3112, which may communicate with one or more graphics processors 3108 of processor 3102 to perform graph and media operations. In at least one embodiment, display device 3111 may be connected to processor 3102. In at least one embodiment, display device 3111 may include one or more internal display devices, such as those in mobile electronic devices or portable computer devices, or external display devices connected via a display interface (e.g., DisplayPort). In at least one embodiment, the display device 3111 may include a head-mounted display (HMD), such as a stereoscopic display device for virtual reality (VR) or augmented reality (AR) applications.

[0317] In at least one embodiment, the platform controller hub 3130 enables peripheral devices to connect to the storage device 3120 and the processor 3102 via a high-speed I / O bus. In at least one embodiment, the I / O peripheral devices include, but are not limited to, an audio controller 3146, a network controller 3134, a firmware interface 3128, a wireless transceiver 3126, a touch sensor 3125, and a data storage device 3124 (e.g., a hard disk drive, flash memory, etc.). In at least one embodiment, the data storage device 3124 may be connected via a memory interface (e.g., SATA) or via a peripheral bus, such as a peripheral component interconnect bus (e.g., PCI, PCIe). In at least one embodiment, the touch sensor 3125 may include a touchscreen sensor, a pressure sensor, or a fingerprint sensor. In at least one embodiment, the wireless transceiver 3126 may be a Wi-Fi transceiver, a Bluetooth transceiver, or a mobile network transceiver, such as a 3G, 4G, or LTE transceiver. In at least one embodiment, the firmware interface 3128 enables communication with the system firmware and may be, for example, a Unified Extensible Firmware Interface (UEFI). In at least one embodiment, network controller 3134 may enable network connectivity to a wired network. In at least one embodiment, a high-performance network controller (not shown) is coupled to interface bus 3110. In at least one embodiment, audio controller 3146 is a multi-channel high-definition audio controller. In at least one embodiment, processing system 3100 includes an optional legacy I / O controller 3140 for coupling legacy (e.g., Personal System 2 (PS / 2)) devices to processing system 3100. In at least one embodiment, platform controller hub 3130 may also be connected to one or more Universal Serial Bus (USB) controllers 3142 that connect input devices, such as keyboard and mouse combinations 3143, cameras 3144, or other USB input devices.

[0318] In at least one embodiment, instances of the memory controller 3116 and platform controller hub 3130 may be integrated into a discrete external graphics processor, such as external graphics processor 3112. In at least one embodiment, the platform controller hub 3130 and / or the memory controller 3116 may be external to one or more processors 3102. For example, in at least one embodiment, the processing system 3100 may include an external memory controller 3116 and a platform controller hub 3130, which may be configured as a memory controller hub and a peripheral controller hub in a system chipset communicating with the processor 3102.

[0319] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0320] Figure 32 A computer system 3200 according to at least one embodiment is illustrated. In at least one embodiment, the computer system 3200 may be a system having interconnected devices and components, a System-on-a-Chip (SoC), or some combination thereof. In at least one embodiment, the computer system 3200 is formed by a processor 3202, which may include execution units for executing instructions. In at least one embodiment, the computer system 3200 may include, but is not limited to, components such as the processor 3202, which employs execution units including logic to execute algorithms for process data. In at least one embodiment, the computer system 3200 may include a processor, such as one available from Intel Corporation of Santa Clara, California. Processor family, Xeon™ XScale™ and / or StrongARM™ Core TM or Nervana TM A microprocessor may be used, although other systems (including PCs, engineering workstations, set-top boxes, etc.) with other microprocessors may also be used. In at least one embodiment, computer system 3200 may execute a version of the Windows operating system available from Microsoft Corporation of Redmond, Washington, although other operating systems (such as UNIX and Linux), embedded software, and / or graphical user interfaces may also be used.

[0321] In at least one embodiment, the computer system 3200 can be used in other devices, such as handheld devices and embedded applications. Some examples of handheld devices include cellular phones, Internet Protocol (IP) devices, digital cameras, personal digital assistants (“PDAs”), and handheld PCs. In at least one embodiment, the embedded application may include a microcontroller, a digital signal processor (“DSP”), a system-on-a-chip (SoC), a network computer (“NetPC”), a set-top box, a network hub, a wide area network (“WAN”) switch, or any other system capable of executing one or more instructions according to at least one embodiment.

[0322] In at least one embodiment, the computer system 3200 may include, but is not limited to, a processor 3202, which may include, but is not limited to, one or more execution units 3208 configured to execute a Computational Unified Device Architecture (“CUDA”). (Developed by NVIDIA Corporation, Santa Clara, California) In at least one embodiment, the CUDA program is at least a part of a software application written in the CUDA programming language. In at least one embodiment, the computer system 3200 is a single-processor desktop or server system. In at least one embodiment, the computer system 3200 may be a multiprocessor system. In at least one embodiment, the processor 3202 may include, but is not limited to, a CISC microprocessor, a RISC microprocessor, a VLIW microprocessor, a processor implementing instruction set combinations, or any other processor device, such as a digital signal processor. In at least one embodiment, the processor 3202 may be coupled to a processor bus 3210, which may transmit data signals between the processor 3202 and other components in the computer system 3200.

[0323] In at least one embodiment, processor 3202 may include, but is not limited to, a Level 1 (“L1”) internal cache memory (“cache”) 3204. In at least one embodiment, processor 3202 may have a single internal cache or multiple levels of internal cache. In at least one embodiment, the cache memory may reside external to processor 3202. In at least one embodiment, processor 3202 may include a combination of internal and external caches. In at least one embodiment, register file 3206 may store different types of data in various registers, including but not limited to integer registers, floating-point registers, status registers, and instruction pointer registers.

[0324] In at least one embodiment, an execution unit 3208, including but not limited to logic for performing integer and floating-point operations, is also located within the processor 3202. The processor 3202 may also include a microcode (“ucode”) read-only memory (“ROM”) for storing microcode of certain macro instructions. In at least one embodiment, the execution unit 3208 may include logic for processing a packaged instruction set 3209. In at least one embodiment, by including the packaged instruction set 3209 in the instruction set of the general-purpose processor 3202, along with the associated circuitry for executing the instructions, packaged data in the general-purpose processor 3202 can be used to perform operations used by many multimedia applications. In at least one embodiment, many multimedia applications can be executed more quickly and efficiently by using the full width of the processor’s data bus to perform operations on the packaged data, which may eliminate the need to transfer smaller data units on the processor’s data bus to perform one or more operations on a data element at a time.

[0325] In at least one embodiment, the execution unit 3208 may also be used in a microcontroller, embedded processor, graphics device, DSP, and other types of logic circuitry. In at least one embodiment, the computer system 3200 may include, but is not limited to, the memory 3220. In at least one embodiment, the memory 3220 may be implemented as a DRAM device, an SRAM device, a flash memory device, or other storage device. The memory 3220 may store instructions 3219 and / or data 3221 represented by data signals that can be executed by the processor 3202.

[0326] In at least one embodiment, the system logic chip may be coupled to processor bus 3210 and memory 3220. In at least one embodiment, the system logic chip may include, but is not limited to, a memory controller hub (“MCH”) 3216, and processor 3202 may communicate with MCH 3216 via processor bus 3210. In at least one embodiment, MCH 3216 may provide a high-bandwidth memory path 3218 to memory 3220 for instruction and data storage, as well as for storage of graphics commands, data, and textures. In at least one embodiment, MCH 3216 may initiate data signals between processor 3202, memory 3220, and other components in computer system 3200, and bridge data signals between processor bus 3210, memory 3220, and system I / O 3222. In at least one embodiment, the system logic chip may provide a graphics port for coupling to a graphics controller. In at least one embodiment, MCH 3216 may be coupled to memory 3220 via high-bandwidth memory path 3218, and graphics / video card 3212 may be coupled to MCH 3216 via Accelerated Graphics Port (“AGP”) interconnect 3214.

[0327] In at least one embodiment, computer system 3200 may use system I / O 3222 as a proprietary hub interface bus to couple MCH 3216 to I / O controller hub (“ICH”) 3230. In at least one embodiment, ICH 3230 may provide direct connectivity to certain I / O devices via a local I / O bus. In at least one embodiment, the local I / O bus may include, but is not limited to, a high-speed I / O bus for connecting peripheral devices to memory 3220, chipset, and processor 3202. Examples may include, but are not limited to, an audio controller 3229, a firmware hub (“Flash BIOS”) 3228, a wireless transceiver 3226, data storage 3224, a conventional I / O controller 3223 and keyboard interface including user input 3225, a serial expansion port 3277 (e.g., USB), and a network controller 3234. Data storage 3224 may include a hard disk drive, floppy disk drive, CD-ROM device, flash memory device, or other mass storage device.

[0328] In at least one embodiment, Figure 32 A system comprising interconnected hardware devices or "chips" is shown. In at least one embodiment, Figure 32 An exemplary SoC can be shown. In at least one embodiment, Figure 32The devices shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, one or more components of system 3200 are interconnected using a compute fast link (CXL) interconnect.

[0329] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0330] Figure 33 A system 3300 according to at least one embodiment is illustrated. In at least one embodiment, system 3300 is an electronic device utilizing processor 3310. In at least one embodiment, system 3300 may be, for example, but not limited to, a laptop computer, tower server, rack server, blade server, desktop computer, tablet computer, mobile device, telephone, embedded computer, or any other suitable electronic device.

[0331] In at least one embodiment, system 3300 may include, but is not limited to, processor 3310 communicatively coupled to any suitable number or type of components, peripherals, modules, or devices. In at least one embodiment, processor 3310 is coupled using a bus or interface, such as I... 2 C-bus, System Management Bus (“SMBus”), Low Pin Count (LPC) bus, Serial Peripheral Interface (“SPI”), High Definition Audio (“HDA”) bus, Serial Advanced Technology Accessory (“SATA”) bus, USB (versions 1, 2, and 3) or Universal Asynchronous Receiver / Transmitter (“UART”) bus. In at least one embodiment, Figure 33 A system is illustrated, comprising interconnected hardware devices or "chips". In at least one embodiment, Figure 33 An exemplary SoC can be shown. In at least one embodiment, Figure 33 The device shown can be interconnected with proprietary interconnects, standardized interconnects (e.g., PCIe), or some combination thereof. In at least one embodiment, Figure 33 One or more components are interconnected using Computational Fast Link (CXL) interconnects.

[0332] In at least one embodiment, Figure 33This may include a display 3324, a touchscreen 3325, a touchpad 3330, a near-field communication unit (“NFC”) 3345, a sensor hub 3340, a thermal sensor 3346, a fast chipset (“EC”) 3335, a trusted platform module (“TPM”) 3338, a BIOS / firmware / flash memory (“BIOS, FW Flash”) 3322, a DSP 3360, a solid-state drive (“SSD”) or hard disk drive (“HDD”) 3320, a wireless local area network unit (“WLAN”) 3350, a Bluetooth unit 3352, a wireless wide area network unit (“WWAN”) 3356, a global positioning system (GPS) 3355, a camera (“USB 3.0 camera”) 3354 (e.g., a USB 3.0 camera), or a low-power double data rate (“LPDDR”) memory unit (“LPDDR3”) 3315 implemented in accordance with the LPDDR3 standard. These components may each be implemented in any suitable manner.

[0333] In at least one embodiment, other components may be communicatively coupled to processor 3310 via the components discussed above. In at least one embodiment, accelerometer 3341, ambient light sensor (“ALS”) 3342, compass 3343, and gyroscope 3344 may be communicatively coupled to sensor hub 3340. In at least one embodiment, thermal sensor 3339, fan 3337, keyboard 3346, and touchpad 3330 may be communicatively coupled to EC 3335. In at least one embodiment, speaker 3363, earphone 3364, and microphone (“mic”) 3365 may be communicatively coupled to audio unit (“audio codec and Class D amplifier”) 3364, which in turn may be communicatively coupled to DSP 3360. In at least one embodiment, audio unit 3364 may include, but is not limited to, audio encoder / decoder (“codec”) and Class D amplifier. In at least one embodiment, SIM card (“SIM”) 3357 may be communicatively coupled to WWAN unit 3356. In at least one embodiment, components such as WLAN unit 3350, Bluetooth unit 3352, and WWAN unit 3356 can be implemented as next-generation form factor (NGFF).

[0334] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0335] Figure 34 An exemplary integrated circuit 3400 according to at least one embodiment is illustrated. In at least one embodiment, the exemplary integrated circuit 3400 is a SoC (System-on-a-Chip) that can be fabricated using one or more IP cores. In at least one embodiment, the integrated circuit 3400 includes one or more application processors 3405 (e.g., CPUs), at least one graphics processor 3410, and may additionally include an image processor 3415 and / or a video processor 3420, any of which may be a modular IP core. In at least one embodiment, the integrated circuit 3400 includes peripheral or bus logic, which includes a USB controller 3425, a UART controller 3430, an SPI / SDIO controller 3435, and an I... 2 S / I 2 C controller 3440. In at least one embodiment, integrated circuit 3400 may include display device 3445 coupled to one or more of high-definition multimedia interface (HDMI) controller 3450 and mobile industrial processor interface (MIPI) display interface 3455. In at least one embodiment, storage may be provided by flash memory subsystem 3460, including flash memory and flash memory controller. In at least one embodiment, a memory interface may be provided via memory controller 3465 for accessing SDRAM or SRAM memory devices. In at least one embodiment, some integrated circuits also include embedded security engine 3470.

[0336] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0337] Figure 35A computing system 3500 according to at least one embodiment is illustrated. In at least one embodiment, the computing system 3500 includes a processing subsystem 3501 having one or more processors 3502 and a system memory 3504 communicating via an interconnect path that may include a memory hub 3505. In at least one embodiment, the memory hub 3505 may be a separate component within a chipset assembly or may be integrated within one or more processors 3502. In at least one embodiment, the memory hub 3505 is coupled to an I / O subsystem 3511 via a communication link 3506. In at least one embodiment, the I / O subsystem 3511 includes an I / O hub 3507 that enables the computing system 3500 to receive input from one or more input devices 3508. In at least one embodiment, the I / O hub 3507 may enable a display controller, included in one or more processors 3502, for providing output to one or more display devices 3510A. In at least one embodiment, one or more display devices 3510A coupled to the I / O hub 3507 may include local, internal, or embedded display devices.

[0338] In at least one embodiment, the processing subsystem 3501 includes one or more parallel processors 3512 coupled to a memory hub 3505 via a bus or other communication link 3513. In at least one embodiment, the communication link 3513 may be one of many standards-based communication link technologies or protocols, such as, but not limited to, PCIe, or may be a vendor-specific communication interface or communication architecture. In at least one embodiment, the one or more parallel processors 3512 form a compute-intensive parallel or vector processing system that may include a large number of processing cores and / or processing clusters, such as a multi-core integrated (MIC) processor. In at least one embodiment, the one or more parallel processors 3512 form a graphics processing subsystem capable of outputting pixels to one or more display devices 3510A coupled via an I / O hub 3507. In at least one embodiment, the one or more parallel processors 3512 may also include a display controller and a display interface (not shown) to enable direct connection to one or more display devices 3510B.

[0339] In at least one embodiment, system storage unit 3514 may be connected to I / O hub 3507 to provide a storage mechanism for computing system 3500. In at least one embodiment, I / O switch 3516 may be used to provide an interface mechanism to enable connectivity between I / O hub 3507 and other components, such as network adapter 3518 and / or wireless network adapter 3519 that may be integrated into the platform, and various other devices that may be added via one or more additional devices 3520. In at least one embodiment, network adapter 3518 may be an Ethernet adapter or another wired network adapter. In at least one embodiment, wireless network adapter 3519 may include one or more Wi-Fi, Bluetooth, NFC, or other network devices comprising one or more radios.

[0340] In at least one embodiment, the computing system 3500 may include other components not explicitly shown, including USB or other port connections, optical storage drives, video capture devices and / or variations thereof, and may also be connected to the I / O hub 3507. In at least one embodiment, for Figure 35 The communication paths that interconnect the various components can be implemented using any suitable protocol, such as PCI (Peripheral Component Interconnect) based protocols (e.g., PCIe), or other bus or point-to-point communication interfaces and / or protocols (e.g., NVLink high-speed interconnect or interconnect protocols).

[0341] In at least one embodiment, one or more parallel processors 3512 include circuitry optimized for graphics and video processing (e.g., including video output circuitry) and constitute a graphics processing unit (GPU). In at least one embodiment, one or more parallel processors 3512 include circuitry optimized for general-purpose processing. In at least one embodiment, components of the computing system 3500 may be integrated with one or more other system elements on a single integrated circuit. For example, in at least one embodiment, one or more parallel processors 3512, memory hub 3505, processor 3502, and I / O hub 3507 may be integrated into a system-on-a-chip (SoC) integrated circuit. In at least one embodiment, components of the computing system 3500 may be integrated into a single package to form a system-in-package (SIP) configuration. In at least one embodiment, at least a portion of the components of the computing system 3500 may be integrated into a multi-chip module (MCM) that can interconnect with other MCMs to a modular computing system. In at least one embodiment, the I / O subsystem 3511 and display device 3510B are omitted from the computing system 3500.

[0342] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0343] Processing system

[0344] The following figures illustrate, but are not limited to, exemplary processing systems that can be used to implement at least one embodiment.

[0345] Figure 36 An accelerated processing unit (“APU”) 3600 according to at least one embodiment is illustrated. In at least one embodiment, the APU 3600 was developed by AMD Inc. of Santa Clara, California. In at least one embodiment, the APU 3600 can be configured to execute applications, such as CUDA programs. In at least one embodiment, the APU 3600 includes, but is not limited to, a core complex 3610, a graphics complex 3640, an architecture 3660, an I / O interface 3670, a memory controller 3680, a display controller 3692, and a multimedia engine 3694. In at least one embodiment, the APU 3600 can be, but is not limited to, any combination of any number of core complexes 3610, any number of graphics complexes 3640, any number of display controllers 3692, and any number of multimedia engines 3694. For illustrative purposes, multiple instances of similar objects are indicated herein by reference numerals, wherein the reference numerals identify the object, and the numbers in parentheses identify the desired instances.

[0346] In at least one embodiment, the core complex 3610 is a CPU, the graphics complex 3640 is a GPU, and the APU 3600 is a processing unit that is not limited to 3610 and 3640 integrated onto a single chip. In at least one embodiment, some tasks may be assigned to the core complex 3610, while other tasks may be assigned to the graphics complex 3640. In at least one embodiment, the core complex 3610 is configured to execute master control software associated with the APU 3600, such as an operating system. In at least one embodiment, the core complex 3610 is the master processor of the APU 3600, which controls and coordinates the operation of other processors. In at least one embodiment, the core complex 3610 issues commands to control the operation of the graphics complex 3640. In at least one embodiment, the core complex 3610 may be configured to execute host executable code derived from CUDA source code, and the graphics complex 3640 may be configured to execute device executable code derived from CUDA source code.

[0347] In at least one embodiment, the core complex 3610 includes, but is not limited to, cores 3620(1)-3620(4) and L3 cache 3630. In at least one embodiment, the core complex 3610 may include, but is not limited to, any combination of any number of cores 3620 and any number and type of cache. In at least one embodiment, the cores 3620 are configured to execute instructions of a specific instruction set architecture (“ISA”). In at least one embodiment, each core 3620 is a CPU core.

[0348] In at least one embodiment, each core 3620 includes, but is not limited to, a fetch / decode unit 3622, an integer execution engine 3624, a floating-point execution engine 3626, and an L2 cache 3628. In at least one embodiment, the fetch / decode unit 3622 fetches instructions, decodes these instructions, generates micro-operations, and dispatches individual micro-instructions to the integer execution engine 3624 and the floating-point execution engine 3626. In at least one embodiment, the fetch / decode unit 3622 may simultaneously dispatch one micro-instruction to the integer execution engine 3624 and another micro-instruction to the floating-point execution engine 3626. In at least one embodiment, the integer execution engine 3624 performs operations not limited to integer and memory operations. In at least one embodiment, the floating-point engine 3626 performs operations not limited to floating-point and vector operations. In at least one embodiment, the fetch-decode unit 3622 dispatches micro-instructions to a single execution engine, which replaces both the integer execution engine 3624 and the floating-point execution engine 3626.

[0349] In at least one embodiment, each core 3620(i) can access an L2 cache 3628(i) included in core 3620(i), where i is an integer representing a specific instance of core 3620. In at least one embodiment, each core 3620 included in core complex 3610(j) is connected to other cores 3620 included in core complex 3610(j) via an L3 cache 3630(j) included in core complex 3610(j), where j is an integer representing a specific instance of core complex 3610. In at least one embodiment, a core 3620 included in core complex 3610(j) can access all L3 caches 3630(j) included in core complex 3610(j), where j is an integer representing a specific instance of core complex 3610. In at least one embodiment, the L3 cache 3630 may include, but is not limited to, any number of slices.

[0350] In at least one embodiment, the graphics complex 3640 can be configured to perform computational operations in a highly parallel manner. In at least one embodiment, the graphics complex 3640 is configured to perform graphics pipeline operations, such as drawing commands, pixel operations, geometric calculations, and other operations associated with rendering an image to a display. In at least one embodiment, the graphics complex 3640 is configured to perform graphics-independent operations. In at least one embodiment, the graphics complex 3640 is configured to perform both graphics-related and graphics-independent operations.

[0351] In at least one embodiment, the graphics complex 3640 includes, but is not limited to, any number of computing units 3650 and an L2 cache 3642. In at least one embodiment, the computing units 3650 share the L2 cache 3642. In at least one embodiment, the L2 cache 3642 is partitioned. In at least one embodiment, the graphics complex 3640 includes, but is not limited to, any number of computing units 3650 and any number (including zero) and type of cache. In at least one embodiment, the graphics complex 3640 includes, but is not limited to, any number of dedicated graphics hardware.

[0352] In at least one embodiment, each computing unit 3650 includes, but is not limited to, any number of SIMD units 3652 and shared memory 3654. In at least one embodiment, each SIMD unit 3652 implements a SIMD architecture and is configured to execute operations in parallel. In at least one embodiment, each computing unit 3650 may execute any number of thread blocks, but each thread block executes on a single computing unit 3650. In at least one embodiment, a thread block includes, but is not limited to, any number of execution threads. In at least one embodiment, a workgroup is a thread block. In at least one embodiment, each SIMD unit 3652 executes a different warp. In at least one embodiment, a warp is a group of threads (e.g., 16 threads), where each thread in the warp belongs to a single thread block and is configured to process different datasets based on a single instruction set. In at least one embodiment, prediction can be used to disable one or more threads in a warp. In at least one embodiment, a channel is a thread. In at least one embodiment, a work item is a thread. In at least one embodiment, a wavefront is a warp. In at least one embodiment, different wavefronts in a thread block can be synchronized together and communicate via shared memory 3654.

[0353] In at least one embodiment, architecture 3660 is a system interconnect that facilitates data and control transfers across core complex 3610, graphics complex 3640, I / O interface 3670, memory controller 3680, display controller 3692, and multimedia engine 3694. In at least one embodiment, in addition to or instead of architecture 3660, APU 3600 may also include, but is not limited to, any number and type of system interconnects that facilitate data and control transfers across any number and type of components that may be directly or indirectly linked, either internally or externally to APU 3600. In at least one embodiment, I / O interface 3670 represents any number and type of I / O interface (e.g., PCI, PCI-Extended (“PCI-X”), PCIe, Gigabit Ethernet (“GBE”), USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3670. In at least one embodiment, the peripheral device coupled to the I / O interface 3670 may include, but is not limited to, a keyboard, mouse, printer, scanner, joystick or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0354] In at least one embodiment, the display controller AMD92 displays images on one or more display devices (e.g., liquid crystal display (LCD) devices). In at least one embodiment, the multimedia engine 240 includes, but is not limited to, any number and type of multimedia-related circuitry, such as video decoders, video encoders, image signal processors, etc. In at least one embodiment, the memory controller 3680 facilitates data transfer between the APU 3600 and the unified system memory 3690. In at least one embodiment, the core complex 3610 and the graphics complex 3640 share the unified system memory 3690.

[0355] In at least one embodiment, the APU 3600 implements a memory subsystem, including but not limited to any number and type of memory controllers 3680 and memory devices (e.g., shared memory 3654) that can be dedicated to a single component or shared among multiple components. In at least one embodiment, the APU 3600 implements a cache subsystem, including but not limited to one or more cache memories (e.g., L2 cache 3728, L3 cache 3630, and L2 cache 3642), each cache memory being component-private or shared among any number of components (e.g., core 3620, core complex 3610, SIMD unit 3652, compute unit 3650, and graphics complex 3640).

[0356] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0357] Figure 37A CPU 3700 according to at least one embodiment is illustrated. In at least one embodiment, the CPU 3700 was developed by AMD Inc. of Santa Clara, California. In at least one embodiment, the CPU 3700 can be configured to execute an application. In at least one embodiment, the CPU 3700 is configured to execute host control software, such as an operating system. In at least one embodiment, the CPU 3700 issues commands to control the operation of an external GPU (not shown). In at least one embodiment, the CPU 3700 can be configured to execute host executable code derived from CUDA source code, and the external GPU can be configured to execute device executable code derived from such CUDA source code. In at least one embodiment, the CPU 3700 includes, but is not limited to, any number of core complexes 3710, architectures 3760, I / O interfaces 3770, and memory controllers AMAD80.

[0358] In at least one embodiment, the core complex 3710 includes, but is not limited to, cores 3720(1)-3720(4) and L3 cache 3730. In at least one embodiment, the core complex 3710 may include, but is not limited to, any combination of any number of cores 3720 and any number and type of cache. In at least one embodiment, the cores 3720 are configured to execute instructions of a specific ISA. In at least one embodiment, each core 3720 is a CPU core.

[0359] In at least one embodiment, each core 3720 includes, but is not limited to, a fetch / decode unit 3722, an integer execution engine 3724, a floating-point execution engine 3726, and an L2 cache 3728. In at least one embodiment, the fetch / decode unit 3722 fetches instructions, decodes these instructions, generates micro-operations, and dispatches individual micro-instructions to the integer execution engine 3724 and the floating-point execution engine 3726. In at least one embodiment, the fetch / decode unit 3722 may simultaneously dispatch one micro-instruction to the integer execution engine 3724 and another micro-instruction to the floating-point execution engine 3726. In at least one embodiment, the integer execution engine 3724 performs operations not limited to integer and memory operations. In at least one embodiment, the floating-point engine 3726 performs operations not limited to floating-point and vector operations. In at least one embodiment, the fetch-decode unit 3722 dispatches micro-instructions to a single execution engine, which replaces both the integer execution engine 3724 and the floating-point execution engine 3726.

[0360] In at least one embodiment, each core 3720(i) can access an L2 cache 3728(i) included in core 3720(i), where i is an integer representing a specific instance of core 3720. In at least one embodiment, each core 3720 included in core complex 3710(j) is connected to other cores 3720 in core complex 3710(j) via an L3 cache 3730(j) included in core complex 3710(j), where j is an integer representing a specific instance of core complex 3710. In at least one embodiment, a core 3720 included in core complex 3710(j) can access all L3 caches 3730(j) included in core complex 3710(j), where j is an integer representing a specific instance of core complex 3710. In at least one embodiment, the L3 cache 3730 may include, but is not limited to, any number of slices.

[0361] In at least one embodiment, structure 3760 is a system interconnect that facilitates data and control transfers across core complexes 3710(1)-3710(N) (where N is a positive integer), I / O interface 3770, and memory controller 3780. In at least one embodiment, in addition to or instead of structure 3760, CPU 3700 may also include, but is not limited to, any number and type of system interconnects that facilitate data and control transfers across any number and type of components that may be directly or indirectly linked, either inside or outside CPU 3700. In at least one embodiment, I / O interface 3770 represents any number and type of I / O interfaces (e.g., PCI, PCI-X, PCIe, GBE, USB, etc.). In at least one embodiment, various types of peripheral devices are coupled to I / O interface 3770. In at least one embodiment, peripheral devices coupled to I / O interface 3770 may include, but are not limited to, displays, keyboards, mice, printers, scanners, joysticks or other types of game controllers, media recording devices, external storage devices, network interface cards, etc.

[0362] In at least one embodiment, memory controller 3780 facilitates data transfer between CPU 3700 and system memory 3790. In at least one embodiment, core complex 3710 and graphics complex 3740 share system memory 3790. In at least one embodiment, CPU 3700 implements a memory subsystem, which includes, but is not limited to, any number and type of memory controllers 3780 and memory devices that may be dedicated to a component or shared among multiple components. In at least one embodiment, CPU 3700 implements a cache subsystem, which includes, but is not limited to, one or more cache memories (e.g., L2 cache 3728 and L3 cache 3730), each cache memory may be component-private or shared among any number of components (e.g., core 3720 and core complex 3710).

[0363] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0364] Figure 38 An exemplary accelerator integration slice 3890 according to at least one embodiment is illustrated. As used herein, a "slice" includes a designated portion of the processing resources of an accelerator integrated circuit. In at least one embodiment, the accelerator integrated circuit provides cache management, memory access, environment management, and interrupt management services for multiple graphics processing engines among multiple graphics acceleration modules. Each graphics processing engine may comprise a separate GPU. Optionally, the graphics processing engine may include different types of graphics processing engines within the GPU, such as graphics execution units, media processing engines (e.g., video encoders / decoders), samplers, and blit engines. In at least one embodiment, a graphics acceleration module may be a GPU having multiple graphics processing engines. In at least one embodiment, the graphics processing engines may be individual GPUs integrated on a general-purpose package, line card, or chip.

[0365] The application's effective address space 3882 within system memory 3814 stores process element 3883. In one embodiment, process element 3883 is stored in response to a GPU call 3881 from an application 3880 executing on processor 3807. Process element 3883 contains the processing state of the corresponding application 3880. A job descriptor (WD) 3884 contained in process element 3883 may be a single job requested by the application or may contain pointers to job queues. In at least one embodiment, WD 3884 is a pointer to a job request queue in the application's effective address space 3882.

[0366] The graphics acceleration module 3846 and / or individual graphics processing engines may be shared by all or some processes in the system. In at least one embodiment, infrastructure may be included for establishing a processing state and sending the WD 3884 to the graphics acceleration module 3846 to begin operation in a virtualized environment.

[0367] In at least one embodiment, a dedicated process programming model is used for implementation. In this model, a single process owns the graphics acceleration module 3846 or an individual graphics processing engine. Since the graphics acceleration module 3846 is owned by a single process, the hypervisor initializes the accelerator integrated circuit for the owned partition, and the operating system initializes the accelerator integrated circuit for the owned partition when the graphics acceleration module 3846 is allocated.

[0368] During operation, the WD fetch unit 3891 in the accelerator integrated slice 3890 fetches the next WD 3884, which includes instructions for the work to be performed by one or more graphics processing engines of the graphics acceleration module 3846. Data from the WD 3884 can be stored in register 3845 and used by the memory management unit (MMU) 3839, interrupt management circuitry 3847, and / or environment management circuitry 3848, as shown. For example, one embodiment of the MMU 3839 includes segment / page roaming circuitry for accessing segment / page tables 3886 within the OS virtual address space 3885. The interrupt management circuitry 3847 can handle interrupt events (INT) 3892 received from the graphics acceleration module 3846. When performing graph operations, the effective address 3893 generated by the graphics processing engine is translated into an actual address by the MMU 3839.

[0369] In one embodiment, the same register set 3845 is copied for each graphics processing engine and / or graphics acceleration module 3846 and can be initialized by the hypervisor or operating system. Each of these copied registers can be included in the accelerator integration slice 3890. Exemplary registers that can be initialized by the hypervisor are shown in Table 1.

[0370] Table 1 – Registers for Supervisor Initialization

[0371]

[0372]

[0373] Table 2 shows exemplary registers that can be initialized by the operating system.

[0374] Table 2 – Operating System Initialization Registers

[0375] 1 Process and thread identification 2 Valid Address (EA) Environment Save / Restore Pointer 3 Virtual Address (VA) accelerator utilization record pointer 4 Virtual address (VA) stores segment table pointers 5 mask of authority 6 Job descriptor

[0376] In one embodiment, each WD 3884 is specific to a particular graphics acceleration module 3846 and / or a particular graphics processing engine. It contains all the information required for the graphics processing engine to perform its work or to do its job, or it may be a pointer to a memory location where the application has established a command queue for the work to be done.

[0377] Figure 39A and 39B An exemplary graphics processor according to at least one embodiment herein is illustrated. In at least one embodiment, any exemplary graphics processor may be manufactured using one or more IP cores. In addition to the illustrations, other logic and circuitry may be included in at least one embodiment, including additional graphics processor / cores, peripheral interface controllers, or general-purpose processor cores. In at least one embodiment, the exemplary graphics processor is used within a System-on-a-Chip (SoC).

[0378] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0379] Figure 39A An exemplary graphics processor 3910 of a SoC integrated circuit according to at least one embodiment is shown, which can be manufactured using one or more IP cores. Figure 39B An additional example graphics processor 3940 of a SoC integrated circuit according to at least one embodiment is shown, which can be manufactured using one or more IP cores. In at least one embodiment, Figure 39A The graphics processor 3910 is a low-power graphics processor core. In at least one embodiment, Figure 39BThe graphics processor 3940 is a higher-performance graphics processor core. In at least one embodiment, the graphics processors 3910 and 3940 may be variations of the graphics processor 1510 of FIG. 15.

[0380] In at least one embodiment, the graphics processor 3910 includes a vertex processor 3905 and one or more fragment processors 3915A-3915N (e.g., 3915A, 3915B, 3915C, 3915D to 3915N-1 and 3915N). In at least one embodiment, the graphics processor 3910 can execute different shader programs via separate logic, such that the vertex processor 3905 is optimized to perform operations for the vertex shader program, while one or more fragment processors 3915A-3915N perform fragment (e.g., pixel) shading operations for fragments or pixels or shader programs. In at least one embodiment, the vertex processor 3905 performs the vertex processing stage of the 3D graphics pipeline and generates primitive and vertex data. In at least one embodiment, the fragment processors 3915A-3915N use the primitive and vertex data generated by the vertex processor 3905 to generate framebuffers for display on a display device. In at least one embodiment, the fragment processors 3915A-3915N are optimized to execute fragment shader programs as provided in the OpenGL API, which can be used to perform operations similar to those of pixel shader programs provided in the Direct 3D API.

[0381] In at least one embodiment, the graphics processor 3910 additionally includes one or more MMUs 3920A-3920B, caches 3925A-3925B, and circuit interconnects 3930A-3930B. In at least one embodiment, one or more MMUs 3920A-3920B provide a virtual-to-physical address mapping for the graphics processor 3910, including for the vertex processor 3905 and / or fragment processors 3915A-3915N, which can reference vertex or image / texture data stored in memory, in addition to the vertex or image / texture data stored in one or more caches 3925A-3925B. In at least one embodiment, one or more MMUs 3920A-3920B can be synchronized with other MMUs within the system, including one or more MMUs associated with one or more application processors 1505, graphics processors 1515, and / or video processors 1520 of FIG. 15, such that each processor 1505-1520 can participate in a shared or unified virtual memory system. In at least one embodiment, one or more circuit interconnects 3930A-3930B enable the graphics processor 3910 to connect to other IP cores within the SoC via the SoC's internal bus or via a direct connection.

[0382] In at least one embodiment, the graphics processor 3940 includes Figure 39A The graphics processor 3910 includes one or more MMUs 3920A-3920B, caches 3925A-3925B, and circuit interconnects 3930A-3930B. In at least one embodiment, the graphics processor 3940 includes one or more shader cores 3955A-3955N (e.g., 3955A, 3955B, 3955C, 3955D, 3955E, 3955F, to 3955N-1 and 3955N) that provide a unified shader core architecture, wherein a single core or type of core can execute all types of programmable shader code, including shader program code for implementing vertex shaders, fragment shaders, and / or compute shaders. In at least one embodiment, the number of shader cores may vary. In at least one embodiment, the graphics processor 3940 includes an inter-core task manager 3945 that acts as a thread dispatcher to assign execution threads to one or more shader cores 3955A-3955N and a tile unit 3958 to accelerate tile-based rendering operations, wherein rendering operations of a scene are subdivided in image space, for example to take advantage of local spatial consistency within the scene or to optimize the use of internal caches.

[0383] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0384] Figure 40A A graphics core 4000 according to at least one embodiment is shown. In at least one embodiment, the graphics core 4000 may include... Figure 34 The graphics processor 3410 is located within it. In at least one embodiment, the graphics core 4000 may be... Figure 39BThe graphics core 4000 uses a unified shader core 3955A-3955N. In at least one embodiment, the graphics core 4000 includes a shared instruction cache 4002, texture units 4018, and cache / shared memory 4020, which are common to execution resources within the graphics core 4000. In at least one embodiment, the graphics core 4000 may include multiple slices 4001A-4001N or partitions of each core, and the graphics processor may include multiple instances of the graphics core 4000. Slices 4001A-4001N may include supporting logic, including local instruction caches 4004A-4004N, thread schedulers 4006A-4006N, thread dispatchers 4008A-4008N, and a set of registers 4010A-4010N. In at least one embodiment, slices 4001A-4001N may include a set of additional functional units (AFU) 4012A-4012N, floating-point units (FPU) 4014A-4014N, integer arithmetic logic units (ALU) 4016A-4016N, address calculation units (ACU) 4013A-4013N, double-precision floating-point units (DPFPU) 4015A-4015N, and matrix processing units (MPU) 4017A-4017N.

[0385] In one embodiment, the FPU 4014A-4014N can perform single-precision (32-bit) and half-precision (16-bit) floating-point operations, while the DPFPU 4015A-4015N can perform double-precision (64-bit) floating-point operations. In at least one embodiment, the ALU 4016A-4016N can perform variable-precision integer operations with 8-bit, 16-bit, and 32-bit precision, and can be configured for mixed-precision operations. In at least one embodiment, the MPU 4017A-4017N can also be configured for mixed-precision matrix operations, including half-precision floating-point operations and 8-bit integer operations. In at least one embodiment, the MPU 4017A-4017N can perform various matrix operations to accelerate CUDA programs, including enabling support for accelerated General Matrix-to-Matrix Multiplication (GEMM). In at least one embodiment, the AFU 4012A-4012N can perform additional logical operations not supported by floating-point or integer units, including trigonometric operations (e.g., Sine, Cosine, etc.).

[0386] Figure 40BA general-purpose graphics processing unit (GPGPU) 4030 is illustrated in at least one embodiment. In at least one embodiment, the GPGPU 4030 is highly parallel and suitable for deployment on a multi-chip module. In at least one embodiment, the GPGPU 4030 can be configured to enable highly parallel computational operations to be performed by a GPU array. In at least one embodiment, the GPGPU 4030 can be directly linked to other instances of the GPGPU 4030 to create a multi-GPU cluster to improve execution time for CUDA programs. In at least one embodiment, the GPGPU 4030 includes a host interface 4032 for connection to a host processor. In at least one embodiment, the host interface 4032 is a PCIe interface. In at least one embodiment, the host interface 4032 can be a vendor-specific communication interface or communication structure. In at least one embodiment, the GPGPU 4030 receives commands from the host processor and uses a global scheduler 4034 to assign execution threads associated with those commands to a set of compute clusters 4036A-4036H. In at least one embodiment, computing clusters 4036A-4036H share a cache memory 4038. In at least one embodiment, cache memory 4038 can be used as an advanced cache of the cache memory within computing clusters 4036A-4036H.

[0387] In at least one embodiment, the GPGPU 4030 includes memory 4044A-4044B coupled to computing clusters 4036A-4036H via a set of memory controllers 4042A-4042B. In at least one embodiment, memory 4044A-4044B may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory.

[0388] In at least one embodiment, computing clusters 4036A-4036H each include a set of graphics cores, such as Figure 40A The graphics core 4000 may include various types of integer and floating-point logic units, capable of performing computational operations at various precisions, including computations suitable for CUDA programs. For example, in at least one embodiment, at least a subset of the floating-point units in each computing cluster 4036A-4036H may be configured to perform 16-bit or 32-bit floating-point operations, while different subsets of the floating-point units may be configured to perform 64-bit floating-point operations.

[0389] In at least one embodiment, multiple instances of the GPGPU 4030 can be configured to operate as a compute cluster. In at least one embodiment, the compute clusters 4036A-4036H can implement any technically feasible communication technology for synchronization and data exchange. In at least one embodiment, the multiple instances of the GPGPU 4030 communicate via a host interface 4032. In at least one embodiment, the GPGPU 4030 includes an I / O hub 4039 that couples the GPGPU 4030 to a GPU link 4040, enabling direct connection to other instances of the GPGPU 4030. In at least one embodiment, the GPU link 4040 is coupled to a dedicated GPU-to-GPU bridge, enabling communication and synchronization among the multiple instances of the GPGPU 4030. In at least one embodiment, the GPU link 4040 is coupled to a high-speed interconnect for sending and receiving data to and from other GPGPUs or parallel processors. In at least one embodiment, the multiple instances of the GPGPU 4030 reside in a separate data processing system and communicate via a network device accessible via the host interface 4032. In at least one embodiment, the GPU link 4040 may be configured to connect to a host processor, supplementing or replacing the host interface 4032. In at least one embodiment, the GPGPU 4030 may be configured to execute CUDA programs.

[0390] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0391] Figure 41A A parallel processor 4100 according to at least one embodiment is shown. In at least one embodiment, various components of the parallel processor 4100 may be implemented using one or more integrated circuit devices, such as programmable processors, application-specific integrated circuits (ASICs), or FPGAs.

[0392] In at least one embodiment, the parallel processor 4100 includes a parallel processing unit 4102. In at least one embodiment, the parallel processing unit 4102 includes an I / O unit 4104 that enables communication with other devices, including other instances of the parallel processing unit 4102. In at least one embodiment, the I / O unit 4104 can be directly connected to other devices. In at least one embodiment, the I / O unit 4104 is connected to other devices using a hub or switch interface (e.g., memory hub 1605). In at least one embodiment, the connection between the memory hub 1605 and the I / O unit 4104 forms a communication link. In at least one embodiment, the I / O unit 4104 is connected to a host interface 4106 and a memory crossbar switch 4116, wherein the host interface 4106 receives commands for performing processing operations, and the memory crossbar switch 4116 receives commands for performing memory operations.

[0393] In at least one embodiment, when host interface 4106 receives a command buffer via I / O unit 4104, host interface 4106 can direct work operations to execute those commands to front end 4108. In at least one embodiment, front end 4108 is coupled to scheduler 4110, which is configured to assign commands or other work items to processing array 4112. In at least one embodiment, scheduler 4110 ensures that processing array 4112 is correctly configured and in an active state before assigning tasks to processing array 4112. In at least one embodiment, scheduler 4110 is implemented via firmware logic executed on a microcontroller. In at least one embodiment, the microcontroller-implemented scheduler 4110 can be configured to perform complex scheduling and work assignment operations at both coarse and fine granular levels, enabling fast preemption and environment switching of threads executing on processing array 4112. In at least one embodiment, host software can demonstrate workloads scheduled on processing array 4112 via one of multiple graphics processing doorbells. In at least one embodiment, the workload can then be automatically distributed on the processing array 4112 by the scheduler 4110 logic within the microcontroller, which includes the scheduler 4110.

[0394] In at least one embodiment, the processing array 4112 may include up to "N" processing clusters (e.g., clusters 4114A, 4114B to 4114N). In at least one embodiment, each cluster 4114A-4114N of the processing array 4112 may execute a large number of concurrent threads. In at least one embodiment, the scheduler 4110 may use various scheduling and / or work allocation algorithms to allocate work to the clusters 4114A-4114N of the processing array 4112, which may vary depending on the workload generated by each type of program or computation. In at least one embodiment, scheduling may be handled dynamically by the scheduler 4110, or may be partially assisted by compiler logic during the compilation of the program logic configured to be executed by the processing array 4112. In at least one embodiment, different clusters 4114A-4114N of the processing array 4112 may be assigned to process different types of programs or to perform different types of computations.

[0395] In at least one embodiment, the processing array 4112 may be configured to perform various types of parallel processing operations. In at least one embodiment, the processing array 4112 is configured to perform general-purpose parallel computing operations. For example, in at least one embodiment, the processing array 4112 may include logic for performing processing tasks, including filtering video and / or audio data, performing modeling operations, including physical operations, and performing data transformations.

[0396] In at least one embodiment, the processing array 4112 is configured to perform parallel graphics processing operations. In at least one embodiment, the processing array 4112 may include additional logic to support the execution of such graphics processing operations, including but not limited to texture sampling logic for performing texture operations, as well as tessellation logic and other vertex processing logic. In at least one embodiment, the processing array 4112 may be configured to execute shader programs related to graphics processing, such as, but not limited to, vertex shaders, tessellation shaders, geometry shaders, and pixel shaders. In at least one embodiment, the parallel processing unit 4102 may transfer data from system memory via I / O unit 4104 for processing. In at least one embodiment, during processing, the transferred data may be stored in on-chip memory (e.g., parallel processor memory 4122) and then written back to system memory.

[0397] In at least one embodiment, when the parallel processing unit 4102 is used to perform graph processing, the scheduler 4110 may be configured to divide the processing workload into tasks of approximately equal size to better distribute graphics processing operations among the multiple clusters 4114A-4114N of the processing array 4112. In at least one embodiment, portions of the processing array 4112 may be configured to perform different types of processing. For example, in at least one embodiment, a first portion may be configured to perform vertex shading and topology generation, a second portion may be configured to perform tessellation and geometry shading, and a third portion may be configured to perform pixel shading or other screen-space operations to generate a rendered image for display. In at least one embodiment, intermediate data generated by one or more of the clusters 4114A-4114N may be stored in a buffer to allow intermediate data to be transferred between the clusters 4114A-4114N for further processing.

[0398] In at least one embodiment, the processing array 4112 may receive processing tasks to be executed via a scheduler 4110, which receives commands defining the processing tasks from a front end 4108. In at least one embodiment, the processing task may include an index of data to be processed, such as surface (patch) data, raw data, vertex data, and / or pixel data, as well as state parameters and commands defining how the data is processed (e.g., what program to execute). In at least one embodiment, the scheduler 4110 may be configured to acquire an index corresponding to a task, or may receive an index from the front end 4108. In at least one embodiment, the front end 4108 may be configured to ensure that the processing array 4112 is configured to be active before initiating a workload specified by an incoming command buffer (e.g., a batch buffer, push buffer, etc.).

[0399] In at least one embodiment, each of one or more instances of the parallel processing unit 4102 may be coupled to the parallel processor memory 4122. In at least one embodiment, the parallel processor memory 4122 may be accessed via a memory crossbar switch 4116, which may receive memory requests from the processing array 4112 and the I / O unit 4104. In at least one embodiment, the memory crossbar switch 4116 may be accessed via a memory interface 4118. In at least one embodiment, the memory interface 4118 may include a plurality of partition units (e.g., partition units 4120A, 4120B to 4120N), each of which may be coupled to a portion (e.g., a memory cell) of the parallel processor memory 4122. In at least one embodiment, the plurality of partition units 4120A-4120N are configured to be equal to the number of memory units, such that the first partition unit 4120A has a corresponding first memory unit 4124A, the second partition unit 4120B has a corresponding memory unit 4124B, and the Nth partition unit 4120N has a corresponding Nth memory unit 4124N. In at least one embodiment, the number of partition units 4120A-4120N may not be equal to the number of memory devices.

[0400] In at least one embodiment, memory cells 4124A-4124N may include various types of memory devices, including dynamic random access memory (DRAM) or graphics random access memory, such as synchronous graphics random access memory (SGRAM), including graphics double data rate (GDDR) memory. In at least one embodiment, memory cells 4124A-4124N may also include 3D stacked memory, including but not limited to high bandwidth memory (HBM). In at least one embodiment, rendering targets such as frame buffers or texture maps may be stored across memory cells 4124A-4124N, allowing partitioning cells 4120A-4120N to write portions of each rendering target in parallel, to efficiently utilize the available bandwidth of the parallel processor memory 4122. In at least one embodiment, local instances of the parallel processor memory 4122 may be excluded to facilitate a unified memory design that combines system memory with local cache memory.

[0401] In at least one embodiment, any of the clusters 4114A-4114N of the processing array 4112 can process data to be written to any memory cell 4124A-4124N within the parallel processor memory 4122. In at least one embodiment, the memory crossbar switch 4116 can be configured to transfer the output of each cluster 4114A-4114N to any partition cell 4120A-4120N or another cluster 4114A-4114N, and the clusters 4114A-4114N can perform further processing operations on the output. In at least one embodiment, each cluster 4114A-4114N can communicate with the memory interface 4118 via the memory crossbar switch 4116 to read from or write to various external storage devices. In at least one embodiment, the memory crossbar switch 4116 has a connection to a memory interface 4118 for communication with I / O unit 4104, and a connection to a local instance of parallel processor memory 4122, thereby enabling processing units within different processing clusters 4114A-4114N to communicate with system memory or other memory not local to parallel processing unit 4102. In at least one embodiment, the memory crossbar switch 4116 may use virtual channels to separate traffic flows between clusters 4114A-4114N and partition units 4120A-4120N.

[0402] In at least one embodiment, multiple instances of the parallel processing unit 4102 may be provided on a single insert card, or multiple insert cards may be interconnected. In at least one embodiment, different instances of the parallel processing unit 4102 may be configured to interoperate, even if the different instances have different numbers of processing cores, different numbers of local parallel processor memories, and / or other configuration differences. For example, in at least one embodiment, some instances of the parallel processing unit 4102 may include higher-precision floating-point units relative to other instances. In at least one embodiment, a system combining one or more instances of the parallel processing unit 4102 or the parallel processor 4100 can be implemented in various configurations and form factors, including but not limited to desktop, laptop, or handheld personal computers, servers, workstations, game consoles, and / or embedded systems.

[0403] Figure 41BA processing cluster 4194 according to at least one embodiment is illustrated. In at least one embodiment, the processing cluster 4194 is included within a parallel processing unit. In at least one embodiment, the processing cluster 4194 is an example of one of the processing clusters 4114A-4114N of FIG. 41. In at least one embodiment, the processing cluster 4194 may be configured to execute a number of threads in parallel, wherein the term "thread" refers to an instance of a specific program executing on a particular set of input data. In at least one embodiment, a Single Instruction Multiple Data (SIMD) instruction issuing technique is used to support the parallel execution of a large number of threads without providing multiple independent instruction units. In at least one embodiment, a Single Instruction Multiple Threading (SIMT) technique is used to support the parallel execution of a large number of generally synchronous threads, which uses a common instruction unit configured to issue instructions to a set of processing engines within each processing cluster 4194.

[0404] In at least one embodiment, the operation of the processing cluster 4194 can be controlled by a pipeline manager 4132 that assigns processing tasks to the SIMT parallel processors. In at least one embodiment, the pipeline manager 4132 receives instructions from the scheduler 4110 of FIG. 41 and manages the execution of these instructions via the graphics multiprocessor 4134 and / or texture unit 4136. In at least one embodiment, the graphics multiprocessor 4134 is an exemplary instance of a SIMT parallel processor. However, in at least one embodiment, the processing cluster 4194 may include various types of SIMT parallel processors with different architectures. In at least one embodiment, the processing cluster 4194 may include one or more instances of the graphics multiprocessor 4134. In at least one embodiment, the graphics multiprocessor 4134 can process data, and the data crossover switch 4140 can be used to distribute the processed data to one of a number of possible destinations (including other shader units). In at least one embodiment, the pipeline manager 4132 can facilitate the distribution of processed data by specifying the destination of the processed data to be distributed via the data crossover switch 4140.

[0405] In at least one embodiment, each graphics multiprocessor 4134 within the processing cluster 4194 may include the same set of functional execution logic (e.g., arithmetic logic units, load-memory units (LSUs), etc.). In at least one embodiment, the functional execution logic may be configured in a pipelined manner, wherein new instructions may be issued before previous instructions complete. In at least one embodiment, the functional execution logic supports a variety of operations, including integer and floating-point arithmetic, comparison operations, Boolean operations, shift operations, and computation of various algebraic functions. In at least one embodiment, the same functional unit hardware may be used to perform different operations, and any combination of functional units may exist.

[0406] In at least one embodiment, instructions transmitted to the processing cluster 4194 constitute threads. In at least one embodiment, a group of threads executed across a set of parallel processing engines is a thread group. In at least one embodiment, the thread group executes programs on different input data. In at least one embodiment, each thread within the thread group may be assigned to a different processing engine within the graphics multiprocessor 4134. In at least one embodiment, the thread group may include fewer threads than the number of processing engines within the graphics multiprocessor 4134. In at least one embodiment, when the number of threads included in the thread group is less than the number of processing engines, one or more processing engines may be idle during a loop that is processing the thread group. In at least one embodiment, the thread group may also include more threads than the number of processing engines within the graphics multiprocessor 4134. In at least one embodiment, when the thread group includes more threads than the number of processing engines within the graphics multiprocessor 4134, processing can be performed in consecutive clock cycles. In at least one embodiment, multiple thread groups can be executed simultaneously on the graphics multiprocessor 4134.

[0407] In at least one embodiment, the graphics multiprocessor 4134 includes an internal cache memory for performing load and store operations. In at least one embodiment, the graphics multiprocessor 4134 may forgo the internal cache and use a cache memory within the processing cluster 4194 (e.g., L1 cache 4148). In at least one embodiment, each graphics multiprocessor 4134 may also access partition units (e.g., Figure 41A The L2 cache is located within partition units 4120A-4120N, which are shared among all processing clusters 4194 and can be used to transfer data between threads. In at least one embodiment, the graphics multiprocessor 4134 can also access off-chip global memory, which may include one or more of local parallel processor memory and / or system memory. In at least one embodiment, any memory outside of the parallel processing unit 4102 can be used as global memory. In at least one embodiment, the processing cluster 4194 includes multiple instances of the graphics multiprocessor 4134, which can share common instructions and data that can be stored in the L1 cache 4148.

[0408] In at least one embodiment, each processing cluster 4194 may include an MMU 4145 configured to map virtual addresses to physical addresses. In at least one embodiment, one or more instances of the MMU 4145 may reside within the memory interface 4118 of FIG. 41. In at least one embodiment, the MMU 4145 includes a set of page table entries (PTEs) for mapping virtual addresses to physical addresses of tiles (more information about tiles is discussed) and optionally to cache line indices. In at least one embodiment, the MMU 4145 may include an address translation back buffer (TLB) or a cache that may reside within the graphics multiprocessor 4134, the L1 cache 4148, or the processing cluster 4194. In at least one embodiment, physical addresses are processed to allocate surface data access locality for efficient request interleaving between partition units. In at least one embodiment, cache line indices may be used to determine whether a request for a cache line is a hit or a miss.

[0409] In at least one embodiment, the processing cluster 4194 may be configured such that each graphics multiprocessor 4134 is coupled to a texture unit 4136 to perform texture mapping operations, such as determining texture sample locations, reading texture data, and filtering texture data. In at least one embodiment, texture data is read as needed from an internal texture L1 cache (not shown) or from an L1 cache within the graphics multiprocessor 4134, and texture data is also retrieved from an L2 cache, local parallel processor memory, or system memory. In at least one embodiment, each graphics multiprocessor 4134 outputs a processed task to a data crossbar switch 4140 to provide the processed task to another processing cluster 4194 for further processing or to store the processed task in an L2 cache, local parallel processor memory, or system memory via a memory crossbar switch 4116. In at least one embodiment, the pre-ROP unit 4142 is configured to receive data from the graphics multiprocessor 4134 and direct the data to a ROP unit that may be located together with partitioning units described herein (e.g., partitioning units 4120A-4120N of FIG. 41). In at least one embodiment, the PreROP unit 4142 may perform optimizations for color blending, organize pixel color data, and perform address translation.

[0410] Figure 41C A graphics multiprocessor 4196 according to at least one embodiment is illustrated. In at least one embodiment, the graphics multiprocessor 4196 is... Figure 41BThe graphics multiprocessor 4134 is included. In at least one embodiment, the graphics multiprocessor 4196 is coupled to the pipeline manager 4132 of the processing cluster 4194. In at least one embodiment, the graphics multiprocessor 4196 has an execution pipeline including, but not limited to, an instruction cache 4152, an instruction unit 4154, an address mapping unit 4156, a register file 4158, one or more GPGPU cores 4162, and one or more LSUs 4166. The GPGPU cores 4162 and LSUs 4166 are coupled to cache memory 4172 and shared memory 4170 via memory and cache interconnect 4168.

[0411] In at least one embodiment, instruction cache 4152 receives a stream of instructions to be executed from pipeline manager 4132. In at least one embodiment, instructions are cached in instruction cache 4152 and dispatched to instruction unit 4154 for execution. In one embodiment, instruction unit 4154 may dispatch instructions as thread groups (e.g., thread bundles), assigning each thread of the thread group to a different execution unit within GPGPU core 4162. In at least one embodiment, instructions can access any local, shared, or global address space by specifying an address within a unified address space. In at least one embodiment, address mapping unit 4156 may be used to translate addresses in the unified address space into different memory addresses that can be accessed by LSU 4166.

[0412] In at least one embodiment, register file 4158 provides a set of registers for the functional units of graphics multiprocessor 4196. In at least one embodiment, register file 4158 provides temporary storage for operands of data paths connected to functional units of graphics multiprocessor 4196 (e.g., GPGPU core 4162, LSU 4166). In at least one embodiment, register file 4158 is partitioned among each functional unit, such that a dedicated portion of register file 4158 is allocated to each functional unit. In at least one embodiment, register file 4158 is partitioned among different thread groups being executed by graphics multiprocessor 4196.

[0413] In at least one embodiment, each of the GPGPU cores 4162 may include an FPU and / or an ALU for executing instructions of the graph multiprocessor 4196. The GPGPU cores 4162 may be architecturally similar or may differ in architecture. In at least one embodiment, a first portion of the GPGPU core 4162 includes a single-precision FPU and an integer ALU, while a second portion of the GPGPU core includes a double-precision FPU. In at least one embodiment, the FPU may implement the IEEE 754-4108 standard for floating-point algorithms or enable variable-precision floating-point algorithms. In at least one embodiment, the graphics multiprocessor 4196 may additionally include one or more fixed-function or special-function units to perform specific functions, such as copying rectangles or pixel blending operations. In at least one embodiment, one or more of the GPGPU cores 4162 may also include fixed-function or special-function logic.

[0414] In at least one embodiment, the GPGPU core 4162 includes SIMD logic capable of executing a single instruction on multiple sets of data. In at least one embodiment, the GPGPU core 4162 can physically execute SIMD4, SIMD8, and SIMD16 instructions, and logically execute SIMD1, SIMD2, and SIMD32 instructions. In at least one embodiment, the SIMD instructions for the GPGPU core can be generated by a shader compiler at compile time, or automatically generated when executing a program written and compiled for a Single Program Multiple Data (SPMD) or SIMT architecture. In at least one embodiment, multiple threads of a program configured for a SIMT execution model can be executed using a single SIMD instruction. For example, in at least one embodiment, eight SIMD threads performing the same or similar operations can be executed in parallel using a single SIMD8 logic unit.

[0415] In at least one embodiment, the memory and cache interconnect 4168 is an interconnect network connecting each functional unit of the graphics multiprocessor 4196 to the register file 4158 and the shared memory 4170. In at least one embodiment, the memory and cache interconnect 4168 is a cross-switch interconnect that allows the LSU 4166 to perform load and store operations between the shared memory 4170 and the register file 4158. In at least one embodiment, the register file 4158 can operate at the same frequency as the GPGPU core 4162, resulting in very low latency for data transfer between the GPGPU core 4162 and the register file 4158. In at least one embodiment, the shared memory 4170 can be used to enable communication between threads executing on functional units within the graphics multiprocessor 4196. In at least one embodiment, the cache memory 4172 can be used, for example, as a data cache to cache texture data communicated between functional units and texture units 4136. In at least one embodiment, the shared memory 4170 can also be used as a program-managed cache. In at least one embodiment, in addition to the automatically cached data stored in cache memory 4172, the thread executing on GPGPU core 4162 can also programmatically store data in shared memory.

[0416] In at least one embodiment, a parallel processor or GPGPU, as described herein, is communicatively coupled to the host / processor core to accelerate graphics operations, machine learning operations, pattern analysis operations, and various general-purpose GPU (GPGPU) functions. In at least one embodiment, the GPU may be communicatively coupled to the host processor / core via a bus or other interconnect (e.g., high-speed interconnects such as PCIe or NVLink). In at least one embodiment, the GPU may be integrated with the core on the same package or chip and communicatively coupled to the core via an internal processor bus / interconnect (i.e., within the package or chip). In at least one embodiment, regardless of how the GPU is connected, the processor core may assign work to the GPU in the form of a sequence of commands / instructions contained in the WD. In at least one embodiment, the GPU then uses dedicated circuitry / logic to efficiently process these commands / instructions.

[0417] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0418] General computing

[0419] The following figures illustrate, but are not limited to, exemplary software configurations used in general computing to implement at least one embodiment.

[0420] Figure 42 A software stack of a programming platform according to at least one embodiment is illustrated. In at least one embodiment, the programming platform is a platform for accelerating computational tasks by utilizing hardware on a computing system. In at least one embodiment, software developers can access the programming platform through libraries, compiler instructions, and / or extensions to programming languages. In at least one embodiment, the programming platform may be, but is not limited to, CUDA, Radeon Open Computing Platform (“ROCm”), OpenCL (OpenCL developed by Khronosgroup). TM ), SYCL or Intel One API.

[0421] In at least one embodiment, the software stack 4200 of the programming platform provides an execution environment for the application 4201. In at least one embodiment, the application 4201 may include any computer software capable of being launched on the software stack 4200. In at least one embodiment, the application 4201 may include, but is not limited to, artificial intelligence (“AI”) / machine learning (“ML”) applications, high-performance computing (“HPC”) applications, virtual desktop infrastructure (“VDI”) or data center workloads.

[0422] In at least one embodiment, application 4201 and software stack 4200 run on hardware 4207. In at least one embodiment, hardware 4207 may include one or more GPUs, CPUs, FPGAs, AI engines, and / or other types of computing devices supporting a programming platform. In at least one embodiment, such as using CUDA, software stack 4200 may be vendor-specific and compatible only with devices from a specific vendor. In at least one embodiment, such as using OpenCL, software stack 4200 may be used with devices from different vendors. In at least one embodiment, hardware 4207 includes a host connected to one or more devices that can be accessed via application programming interface (API) calls to perform computational tasks. In at least one embodiment, compared to the host within hardware 4207, which may include, but is not limited to, a CPU (but may also include computing devices) and its memory, devices within hardware 4207 may include, but are not limited to, GPUs, FPGAs, AI engines, or other computing devices (but may also include CPUs) and their memory.

[0423] In at least one embodiment, the software stack 4200 of the programming platform includes, but is not limited to, multiple libraries 4203, a runtime 4205, and a device kernel driver 4206. In at least one embodiment, each library 4203 may include data and programming code that can be used by a computer program and utilized during software development. In at least one embodiment, library 4203 may include, but is not limited to, pre-written code and subroutines, classes, values, type specifications, configuration data, documentation, help data, and / or message templates. In at least one embodiment, library 4203 includes functions optimized for execution on one or more types of devices. In at least one embodiment, library 4203 may include, but is not limited to, functions for performing mathematical, deep learning, and / or other types of operations on the device. In at least one embodiment, library 4303 is associated with a corresponding API 4302, which may include one or more APIs that expose functions implemented in library 4303.

[0424] In at least one embodiment, the application 4201 is written as source code, which is compiled into executable code, as follows: Figure 47 This will be discussed in more detail. In at least one embodiment, the executable code of application 4201 may run at least partially on an execution environment provided by software stack 4200. In at least one embodiment, during the execution of application 4201, code that needs to run on the device (compared to the host) may be obtained. In this case, in at least one embodiment, runtime 4205 may be invoked to load and start the necessary code on the device. In at least one embodiment, runtime 4205 may include any technically feasible runtime system capable of supporting the execution of application 4201.

[0425] In at least one embodiment, runtime 4205 is implemented as one or more runtime libraries associated with a corresponding API (shown as API 4204). In at least one embodiment, one or more such runtime libraries may include, but are not limited to, functions for memory management, execution control, device management, error handling, and / or synchronization, etc. In at least one embodiment, memory management functions may include, but are not limited to, functions for allocating, dealing with, and copying device memory, as well as functions for transferring data between host memory and device memory. In at least one embodiment, execution control functions may include, but are not limited to, functions for launching functions on the device (sometimes referred to as "kernels" when the function is a global function that can be called from the host), and functions for setting attribute values ​​in buffers maintained by the runtime libraries for a given function to be executed on the device.

[0426] In at least one embodiment, the runtime library and the corresponding API 4204 can be implemented in any technically feasible manner. In at least one embodiment, one (or any number of) APIs may expose a low-level set of functions for fine-grained control of the device, while another (or any number of) APIs may expose such a higher-level set of functions. In at least one embodiment, a high-level runtime API can be built on top of the low-level APIs. In at least one embodiment, one or more runtime APIs may be language-specific APIs layered on top of language-independent runtime APIs.

[0427] In at least one embodiment, device kernel driver 4206 is configured to facilitate communication with the underlying device. In at least one embodiment, device kernel driver 4206 may provide APIs such as API 4204 and / or low-level functions upon which other software depends. In at least one embodiment, device kernel driver 4206 may be configured to compile intermediate representation (“IR”) code into binary code at runtime. In at least one embodiment, for CUDA, device kernel driver 4206 may compile non-hardware-specific parallel thread execution (“PTX”) IR code into binary code (cached compiled binary code) for a specific target device (sometimes referred to as “final” code) at runtime. In at least one embodiment, doing so allows the final code to run on the target device, which may not exist when the source code was initially compiled into PTX code. Alternatively, in at least one embodiment, the device source code may be compiled into binary code offline, without requiring device kernel driver 4206 to compile the IR code at runtime.

[0428] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0429] Figure 43 The illustration shows an embodiment according to at least one of the embodiments. Figure 42The software stack 4200 is a CUDA implementation. In at least one embodiment, the CUDA software stack 4300 on which an application 4301 can be launched includes a CUDA library 4303, a CUDA runtime 4305, a CUDA driver 4307, and a device kernel driver 4308. In at least one embodiment, the CUDA software stack 4300 executes on hardware 4309, which may include a CUDA-enabled GPU developed by NVIDIA Corporation of Santa Clara, California.

[0430] In at least one embodiment, application 4301, CUDA runtime 4305, and device kernel driver 4308 can respectively perform functions similar to those of application 4201, runtime 4205, and device kernel driver 4206, in combination with the above. Figure 42 The CUDA driver 4307 is described in at least one embodiment. In at least one embodiment, the CUDA driver API 4307 includes a library (libcuda.so) that implements the CUDA driver API 4306. In at least one embodiment, similar to the CUDA runtime API 4304 implemented by the CUDA runtime library (cudart), the CUDA driver API 4306 may expose, but is not limited to, functions for memory management, execution control, device management, error handling, synchronization, and / or graphics interoperability. In at least one embodiment, the CUDA driver API 4306 differs from the CUDA runtime API 4304 in that the CUDA runtime API 4304 simplifies device code management by providing implicit initialization, context (similar to processes) management, and module (similar to dynamically loaded libraries) management. In contrast to the high-level CUDA runtime API 4304, in at least one embodiment, the CUDA driver API 4306 is a low-level API that provides finer-grained control over the device, particularly regarding context and module loading. In at least one embodiment, the CUDA driver API 4306 may expose functions for context management that are not exposed by the CUDA runtime API 4304. In at least one embodiment, the CUDA driver API 4306 is also language-independent and supports, in addition to the CUDA runtime API 4304, OpenCL, for example. Furthermore, in at least one embodiment, development libraries, including the CUDA runtime 4305, can be considered separate from the driver components, including the user-mode CUDA driver 4307 and the kernel-mode device driver 4308 (sometimes also referred to as the "display" driver).

[0431] In at least one embodiment, CUDA library 4303 may include, but is not limited to, mathematical libraries, deep learning libraries, parallel algorithm libraries, and / or signal / image / video processing libraries, which parallel computing applications (e.g., application 4301) may utilize. In at least one embodiment, CUDA library 4303 may include mathematical libraries, such as the cuBLAS library, which is an implementation of basic linear algebra subroutines (“BLAS”) for performing linear algebra operations; the cuFFT library for computing the Fast Fourier Transform (“FFT”); and the cuRAND library for generating random numbers, etc. In at least one embodiment, CUDA library 4303 may include deep learning libraries, such as the cuDNN library for primitives of deep neural networks and the TensorRT platform for high-performance deep learning inference, etc.

[0432] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0433] Figure 44 The illustration shows an embodiment according to at least one of the embodiments. Figure 42 The software stack 4200 is a ROCm implementation. In at least one embodiment, the ROCm software stack 4400 on which the application 4401 can be launched includes a language runtime 4403, a system runtime 4405, a thunk 4407, a ROCm kernel driver 4408, and a device kernel driver 4409. In at least one embodiment, the ROCm software stack 4400 executes on hardware 4409, which may include a ROCm-enabled GPU developed by AMD Inc. of Santa Clara, California.

[0434] In at least one embodiment, application 4401 can perform actions in conjunction with the above. Figure 42 The discussed application 4201 has similar functionality. Additionally, in at least one embodiment, the language runtime 4403 and system runtime 4405 can perform functions combined with the above. Figure 42The runtime 4205 discussed has similar functionality. In at least one embodiment, the language runtime 4403 and the system runtime 4405 differ in that the system runtime 4405 is a language-independent runtime that implements the ROCr system runtime API 4404 and utilizes the Heterogeneous System Architecture (“HAS”) runtime API. In at least one embodiment, the HAS runtime API is a thin-user mode API that exposes interfaces for accessing and interacting with AMD GPUs, including functions for memory management, kernel execution control dispatched via architecture, error handling, system and agent information, and runtime initialization and shutdown, etc. In at least one embodiment, compared to the system runtime 4405, the language runtime 4403 is an implementation of a language-specific runtime API 4402 layered on top of the ROCr system runtime API 4404. In at least one embodiment, the language runtime API may include, but is not limited to, the Portable Heterogeneous Computing Interface (“HIP”) language runtime API, the Heterogeneous Computing Compiler (“HCC”) language runtime API, or the OpenCL API, etc. In particular, the HIP language is an extension of the C++ programming language, a functionally similar version with CUDA mechanisms, and in at least one embodiment, the HIP language runtime API includes elements combined with the above. Figure 43 The discussion focuses on functions similar to CUDA runtime API 4304, such as those used for memory management, execution control, device management, error handling, and synchronization.

[0435] In at least one embodiment, the thunk (ROCt) 4407 is an interface that can be used to interact with the underlying ROCm driver 4408. In at least one embodiment, the ROCm driver 4408 is a ROCk driver, which is a combination of an AMD GPU driver and a HAS kernel driver (amdkfd). In at least one embodiment, the AMD GPU driver is a device kernel driver for GPUs developed by AMD, which performs the above-described combination. Figure 42 The device kernel driver 4206 discussed has similar functionality. In at least one embodiment, the HAS kernel driver is a driver that allows different types of processors to share system resources more efficiently via hardware features.

[0436] In at least one embodiment, various libraries (not shown) may be included in the ROCm software stack 4400 above the language runtime 4403, and provide integration with the above. Figure 43 The discussed CUDA library 4303 has similar functionality. In at least one embodiment, various libraries may include, but are not limited to, mathematical, deep learning, and / or other libraries, such as the hipBLAS library which implements functions similar to CUDA cuBLAS, the rocFFT library which is similar to CUDA cuFFT for computing FFT, etc.

[0437] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0438] Figure 45 The illustration shows an embodiment according to at least one of the embodiments. Figure 42 The software stack 4200 is an OpenCL implementation. In at least one embodiment, the OpenCL software stack 4500 on which the application 4501 can be launched includes an OpenCL framework 4505, an OpenCL runtime 4506, and a driver 4507. In at least one embodiment, the OpenCL software stack 4500 executes on non-vendor-specific hardware 4309. In at least one embodiment, because devices developed by different vendors support OpenCL, specific OpenCL drivers may be required for interoperability with hardware from such vendors.

[0439] In at least one embodiment, the application 4501, the OpenCL runtime 4506, the device kernel driver 4507, and the hardware 4508 can respectively execute the combination described above. Figure 42 The application 4201, runtime 4205, device kernel driver 4206, and hardware 4207 discussed have similar functionality. In at least one embodiment, application 4501 also includes an OpenCL kernel 4502 with code that will execute on the device.

[0440] In at least one embodiment, OpenCL defines a "platform" that allows a host to control devices connected to that host. In at least one embodiment, the OpenCL framework provides a platform layer API and a runtime API, shown as Platform API 4503 and Runtime API 4505. In at least one embodiment, Runtime API 4505 uses a context to manage the execution of the kernel on the device. In at least one embodiment, each identified device can be associated with a respective context, which Runtime API 4505 can use to manage the device's command queue, program objects and kernel objects, shared memory objects, etc. In at least one embodiment, Platform API 4503 discloses functions that allow the device context to select and initialize devices, submit work to devices via command queues, and enable data transfers to and from devices, etc. Additionally, in at least one embodiment, the OpenCL framework provides various built-in functions (not shown), including mathematical functions, relational functions, and image processing functions, etc.

[0441] In at least one embodiment, compiler 4504 is also included in the OpenCL framework 4505. In at least one embodiment, the source code can be compiled offline before executing the application or online during application execution. Unlike CUDA and ROCm, the OpenCL application in at least one embodiment can be compiled online by compiler 4504, which is included to represent any number of compilers that can be used to compile source code and / or IR code (e.g., Standard Portable Intermediate Representation (“SPIR-V”) code) into binary code. Alternatively, in at least one embodiment, the OpenCL application can be compiled offline before executing such an application.

[0442] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0443] Figure 46Software supported by a programming platform according to at least one embodiment is illustrated. In at least one embodiment, the programming platform 4604 is configured to support various programming models 4603, middleware and / or libraries 4602, and frameworks 4601 that an application 4600 may depend on. In at least one embodiment, the application 4600 may be an AI / ML application implemented using, for example, a deep learning framework (MXNet, PyTorch, or TensorFlow), which may depend on libraries such as cuDNN, the NVIDIA Collective Communications Library (“NCCL”), and / or the NVIDIA Developer Data Loading Library (“DALI”) CUDA library to provide accelerated computation on the underlying hardware.

[0444] In at least one embodiment, the programming platform 4604 can be a combination of the above-described components. Figure 43 , Figure 44 and Figure 45 One of the described CUDA, ROCm, or OpenCL platforms. In at least one embodiment, the programming platform 4604 supports multiple programming models 4603, which are abstractions of the underlying computing system that allow for the expression of algorithms and data structures. In at least one embodiment, the programming model 4603 may expose features of the underlying hardware to improve performance. In at least one embodiment, the programming model 4603 may include, but is not limited to, CUDA, HIP, OpenCL, C++ Accelerated Massive Parallelism (“C++AMP”), Open Multiprocessing (“OpenMP”), Open Accelerator (“OpenACC”), and / or Vulcan Compute.

[0445] In at least one embodiment, the library and / or middleware 4602 provides an abstract implementation of the programming model 4604. In at least one embodiment, such a library includes data and programming code that can be used by a computer program and utilized during software development. In at least one embodiment, in addition to those available from the programming platform 4604, such middleware also includes software that provides services to applications. In at least one embodiment, the library and / or middleware 4602 may include, but is not limited to, cuBLAS, cuFFT, cuRAND, and other CUDA libraries, or rocBLAS, rocFFT, rocRAND, and other ROCm libraries. Additionally, in at least one embodiment, the library and / or middleware 4602 may include NCCL and ROCm communication collection library (“RCCL”) libraries, which provide communication routines for GPUs, the MIOpen library for deep learning acceleration, and / or intrinsic libraries for linear algebra, matrix and vector operations, geometric transformations, numerical solvers, and related algorithms.

[0446] In at least one embodiment, the application framework 4601 depends on libraries and / or middleware 4602. In at least one embodiment, each application framework 4601 is a software framework for implementing a standard structure of application software. In at least one embodiment, AI / ML applications can be implemented using frameworks such as Caffe, Caffe2, TensorFlow, Keras, PyTorch, or MxNet deep learning frameworks.

[0447] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0448] Figure 47 Compilation code according to at least one embodiment is shown to be used in Figures 42-45 The application is executed on one of the programming platforms. In at least one embodiment, compiler 4701 receives source code 4700, which includes both host code and device code. In at least one embodiment, compiler 4701 is configured to convert source code 4700 into host executable code 4702 for execution on a host and device executable code 4703 for execution on a device. In at least one embodiment, source code 4700 may be compiled offline before executing the application or compiled online during application execution.

[0449] In at least one embodiment, source code 4700 may include code in any programming language supported by compiler 4701, such as C++, C, Fortran, etc. In at least one embodiment, source code 4700 may be included in a single-source file, which has a mixture of host code and device code, and indicates the location of the device code therein. In at least one embodiment, the single-source file may be a .cu file including CUDA code or a .hip.cpp file including HIP code. Alternatively, in at least one embodiment, source code 4700 may include multiple source code files instead of a single-source file, in which the host code and device code are separate.

[0450] In at least one embodiment, compiler 4701 is configured to compile source code 4700 into host executable code 4702 for execution on a host and device executable code 4703 for execution on a device. In at least one embodiment, compiler 4701 performs operations including resolving source code 4700 into an abstract system tree (AST), performing optimizations, and generating executable code. In at least one embodiment where source code 4700 comprises a single source file, compiler 4701 may separate device code and host code within such a single source file, compile the device code and host code into device executable code 4703 and host executable code 4702 respectively, and link device executable code 4703 and host executable code 4702 together in a single file, as described below. Figure 36 To be discussed in more detail.

[0451] In at least one embodiment, the host executable code 4702 and the device executable code 4703 can be in any suitable format, such as binary code and / or IR code. In the case of CUDA, in at least one embodiment, the host executable code 4702 may include native object code, while the device executable code 4703 may include code in a PTX intermediate representation. In at least one embodiment, in the case of ROCm, both the host executable code 4702 and the device executable code 4703 can include target binary code.

[0452] In at least one embodiment, referring to the figures, one or more circuits, processors, computing systems, or other devices or techniques are adapted to identify the cause of performance degradation by comparing performance metrics associated with user interactions with a first set of network-based services with performance metrics associated with a second set of user interactions with network-based services. In at least one embodiment, this is done according to the provisions of this document regarding... Figure 1-10 The embodiments described in the figures are implemented to achieve the desired results.

[0453] At least one embodiment of this disclosure may be described in accordance with the following terms:

[0454] 1. A processor, comprising:

[0455] One or more circuits are configured to compare one or more performance metrics of the network-based service in response to interaction with a first group of users of the network-based service and in response to interaction with a second group of users of the network-based service.

[0456] 2. The processor as described in Clause 1, wherein the one or more circuits are configured to determine that the performance of the network-based service has degraded by at least the following means:

[0457] The resampled time series is generated by at least randomly reassigning points from the time series of one or more performance metrics of the network-based service to buckets of the resampled time series; and

[0458] Transition points in the resampled time series are identified, at least in part, based on statistical comparisons of segments of the resampled time series.

[0459] 3. The processor as described in Clause 1 or 2, wherein the one or more circuits are configured to compare the rate of change of the one or more performance metrics of the network-based service in response to the first set of user interactions with the rate of change of the one or more performance metrics of the network-based service in response to the second set of user interactions.

[0460] 4. The processor as described in any one of Clauses 1-3, wherein the one or more circuits are configured to compare the proportion of the first group of user interactions with the proportion of the second group of user interactions.

[0461] 5. The processor as described in any one of Clauses 1-4, wherein the first set of user interactions is associated with a first attribute in the attribute category, and the second set of user interactions is associated with a second attribute in the attribute category.

[0462] 6. The processor as described in any one of Clauses 1-5, wherein the one or more circuits are configured to determine, at least in part, based on a metric obtained by comparing one or more performance metrics of the first set of user interactions with one or more performance metrics of the second set of user interactions, that an attribute associated with the first set of user interactions may be the cause of performanc...

Claims

1. A processor, comprising: One or more circuits are configured to compare one or more performance metrics of the network-based service in response to interaction with a first group of users of the network-based service and in response to interaction with a second group of users of the network-based service. The one or more circuits are also configured to determine that the performance of the network-based service has degraded by at least the following means: The resampled time series is generated by at least randomly reassigning points of the time series of one or more performance metrics of the network-based service to buckets of the resampled time series. as well as Transition points in the resampled time series are identified, at least in part, based on statistical comparisons of segments of the resampled time series, wherein the transition points reflect changes in the operational characteristics of the network-based service.

2. The processor of claim 1, wherein the one or more circuits are configured to compare the rate of change of the one or more performance metrics of the network-based service in response to the first set of user interactions with the rate of change of the one or more performance metrics of the network-based service in response to the second set of user interactions.

3. The processor of claim 1, wherein the one or more circuits are configured to compare the proportion of the first group of user interactions with the proportion of the second group of user interactions.

4. The processor of claim 1, wherein the first set of user interactions is associated with a first attribute in the attribute category, and the second set of user interactions is associated with a second attribute in the attribute category.

5. The processor of claim 1, wherein the one or more circuits are configured to determine, at least in part, based on metrics obtained by comparing one or more performance metrics of the first set of user interactions with one or more performance metrics of the second set of user interactions, that an attribute associated with the first set of user interactions may be the cause of performance degradation of the network-based service.

6. The processor of claim 1, wherein the one or more circuits are configured to at least partially base their comparisons on recursively comparing user interaction groups, wherein each recursive level is at least partially based on an attribute category that differs from the attribute category in earlier recursive levels.

7. The processor of claim 1, wherein user interaction includes utilization of the network-based service by a client device associated with the user.

8. A system comprising: One or more computing devices, including one or more processors, are configured to compare one or more performance metrics of the network-based service in response to interaction with a first group of users of the network-based service and in response to interaction with a second group of users of the network-based service. The one or more processors are also configured to determine that the performance of the network-based service has degraded by at least the following means: The resampled time series is generated by at least randomly reassigning points of the time series of one or more performance metrics of the network-based service to buckets of the resampled time series. as well as Transition points in the resampled time series are identified, at least in part, based on statistical comparisons of segments of the resampled time series, wherein the transition points reflect changes in the operational characteristics of the network-based service.

9. The system of claim 8, wherein the one or more processors are configured to compare the rate of change of the one or more performance metrics of the network-based service in response to the first group of user interactions with the rate of change of the one or more performance metrics of the network-based service in response to the second group of user interactions.

10. The system of claim 8, wherein the comparison of the one or more performance metrics of the network-based service in response to the first group of user interactions and the one or more performance metrics of the network-based service in response to the second group of user interactions comprises: A comparison of the interaction ratio with the first group of users and the interaction ratio with the second group of users.

11. The system of claim 8, wherein the first set of user interactions is generated by segmenting user interactions based on attributes associated with attribute categories.

12. The system of claim 8, wherein the one or more processors are configured to determine, at least in part, based on a metric obtained by comparing the rate of change of the proportion of the first group of user interactions and the second group of user interactions with a performance metric, that an attribute associated with the first group of user interactions may be the cause of the performance degradation of the network-based service.

13. The system of claim 8, wherein the one or more processors are configured to recursively compare user interaction groups, wherein the groups compared at each recursive level are generated at least in part based on the attribute categories selected for that recursive level.

14. A machine-readable medium having a set of instructions stored thereon, which, when executed by one or more processors, causes said one or more processors to at least: Compare one or more performance metrics of the web-based service in response to a first group of users interacting with the web-based service and one or more performance metrics of the web-based service in response to a second group of users interacting with the web-based service. The machine-readable medium also stores a set of instructions that, if executed by one or more processors, cause at least: Randomly reassign points of the time series of one or more performance metrics of the network-based service to buckets of the resampled time series; as well as Transition points in the resampled time series are identified, at least in part, based on statistical comparisons of segments of the resampled time series, wherein the transition points reflect changes in the operational characteristics of the network-based service.

15. The machine-readable medium of claim 14, having stored thereon a set of instructions that, if executed by one or more processors, cause the one or more processors to at least: compare the rate of change of the one or more performance metrics of the network-based service in response to the first set of user interactions with the rate of change of the one or more performance metrics of the network-based service in response to the second set of user interactions.

16. The machine-readable medium of claim 14, having stored thereon a set of instructions that, if executed by one or more processors, cause the one or more processors to at least compare the proportion of the first set of user interactions with the proportion of the second set of user interactions.

17. The machine-readable medium of claim 14, wherein the first set of user interactions is associated with a first attribute of the attribute category, and the second set of user interactions is associated with a second attribute of the attribute category.

18. The machine-readable medium of claim 14, having stored thereon a set of instructions, which, if executed by one or more processors, cause the one or more processors to determine, at least in part, based on a metric obtained by comparing one or more performance metrics of the first set of user interactions with one or more performance metrics of the second set of user interactions, that an attribute associated with the first set of user interactions is a potential cause of performance degradation of the network-based service.

19. The machine-readable medium of claim 14, having stored thereon a set of instructions that, if executed by one or more processors, cause the one or more processors to at least: at least partially base their actions on a recursive comparison of user interaction groups, wherein each recursive level is at least partially based on an attribute category that differs from the attribute categories in earlier recursive levels.

20. A system comprising: One or more computing devices are used to generate output for a computerized gaming service, wherein the one or more computing devices compare one or more performance metrics of the service in response to a first set of interactions with the service and one or more performance metrics of the service in response to a second set of interactions with the service. The one or more computing devices are also used for at least: Performance degradation is identified by randomly reassigning points from the time series of at least one or more performance metrics to buckets of the resampled time series; and Transition points in the resampled time series are identified, at least in part, based on statistical comparisons of segments of the resampled time series, wherein the transition points reflect changes in the operational characteristics of the service.

21. The system of claim 20, wherein the comparison is based at least in part on the rate of change of one or more performance metrics of the service in response to the first set of interactions.

22. The system of claim 20, wherein the one or more computing devices are configured to at least compare the proportion of the first group of interactions with the proportion of the second group of interactions.

23. The system of claim 20, wherein the first set of interactions is generated at least in part based on attributes common to all interactions in the first set of interactions.

24. The system of claim 20, wherein the one or more computing devices are configured to at least identify one or more attributes that may be the cause of performance degradation by calculating values ​​indicating information gain based at least in part on statistics associated with groupings of user interactions and at least in part on the analysis.

Images