HIERARCHICAL REPRESENTATION OF DATA VALUES IN WORK MEMORY

Abstract

In some examples, a system receives data values ​​relating to a computing environment and creates an in-memory representation of the data values ​​in memory. This representation comprises a hierarchical array of bins, where each bin contains a statistical metric based on an aggregate of a subset of the data values. Bins at different hierarchical levels within the array represent different resolutions at which the data values ​​are aggregated. The system receives a query requesting a computation, which includes a query filter. Based on the query filter, the system identifies a hierarchical level from the array of bins and performs the computation using bins at that identified level.

Description

Cross-reference to related applications

[0001] This application claims, pursuant to 35 USC § 119(e), the benefits of the preliminary US patent application No. 63 / 679,694 entitled “In-Memory Hierarchical Representation of Data Values”, filed on August 6, 2024 (Attorney Docket No. P174910PRV), which is hereby incorporated by reference in its entirety. background

[0002] A computer environment comprises various types of electronic devices. Examples of electronic devices include user devices, server computers, network devices, storage systems, and other types of electronic equipment. Brief description of the drawings

[0003] Some embodiments of the present disclosure are described with reference to the following figures. Fig.Figure 1 is a block diagram of an arrangement comprising a computer environment, an analysis system containing an in-memory data store, and a client device in accordance with some examples. Fig. Figure 2 is a block diagram of a hierarchy of bins that store summarized representations of data values, according to some examples. Fig. Figure 3 is a flowchart of a procedure according to some examples. Fig. Figure 4 is a block diagram of a system according to some examples. Fig. Figure 5 is a block diagram of a storage medium that stores machine-readable instructions according to some examples.

[0004] In the drawings, identical reference numbers denote similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated for better illustration of the example shown. Furthermore, the drawings contain examples and / or embodiments that correspond to the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description

[0005] In-memory analytics can be performed on telemetry data collected in a computing environment. In-memory analytics involves calculations performed on telemetry data stored in memory, rather than in slower secondary storage such as disk-based storage. In-memory computing allows for faster data access and manipulation compared to disk-based storage. However, a challenge for in-memory analytics systems is scaling the storage capacity to accommodate the large telemetry datasets that can be collected in a computing environment over extended periods. Furthermore, an in-memory analytics system can receive a relatively large number of queries from multiple requesters, which may trigger different analytical calculations.Given a large volume of telemetry data and multiple queries, the in-memory analysis system may experience significant latency when performing the requested calculations.

[0006] In accordance with some implementations of the present disclosure, a system creates an in-memory representation of data values ​​relating to a computing environment in memory. This in-memory representation contains a hierarchical array of bins, each with a statistical metric based on an aggregate of a respective subset of the data values. The bins at different hierarchical levels of the array represent different resolutions at which the data values ​​are aggregated. In response to a query requesting a computation, the system identifies a hierarchical level from the array of bins based on a query filter in the query. The system then performs the requested computation using bins at the identified hierarchical level.

[0007] Techniques or mechanisms according to some examples in this disclosure improve computer functionality and related technology by enhancing storage efficiency in analysis systems designed to process large amounts of data within a computing environment. The computing environment may include a relatively large system, such as an exascale system or another type of supercomputer, containing a relatively large amount of electronic equipment. Storage efficiency is achieved by maintaining a constant storage requirement for each bin. This constant storage requirement is achieved by storing a set of statistical metrics (or, more simply, "statistics") for each bin representing the data values ​​received by the analysis system. The number of statistics in the set of statistics for a bin does not change even if more data values ​​are received for that bin.

[0008] The analysis of large datasets continuously received by an analysis system during the operation of a computer environment can be performed promptly and accurately. Accuracy is achieved by aggregating data values ​​into appropriate bins using statistics. The analysis system can ingest data for the computer environment in real time, allowing queries requiring calculations on the ingested data to be processed promptly. By managing data values ​​in various data aggregation resolutions, corresponding to different hierarchical levels of bins in an in-memory representation, the data analysis system can select bins at a specific hierarchical level to which a requested calculation should be applied.Different queries can request calculations for different groups of data values, and the data analysis system can process such queries using bins at corresponding different hierarchical levels of the in-memory representation.

[0009] Fig. Figure 1 is a block diagram of an example setup with a computer environment 102, an analysis system 104, and a client device 106. Further examples may include multiple computer environments, multiple analysis systems, or multiple client devices.

[0010] Examples of a computing environment 102 may include a high-performance computing (HPC) system such as a supercomputer, an artificial intelligence (AI) system, or any other computing environment with a large number of electronic devices connected by a high-speed connection 110 (or multiple connections). An “interconnect” may refer to a network or any other type of communication medium. Examples of electronic devices 108 include user devices, server computers, network devices, storage systems, control systems, or other types of electronic equipment.

[0011] The computer environment 102 also includes sensors 112, which may be hardware sensors or software sensors (e.g., monitoring agents running on processing resources). Some of the sensors 112 may be part of the electronic devices 108, while other sensors 112 are located outside of the electronic devices 108. The sensors 112 can collect telemetry data, which includes metrics relating to any of the following characteristics of the computer environment 102: performance metrics, health metrics, usage metrics, or other types of metrics.

[0012] The analysis system 104 receives telemetry data from the sensors 112 of the computer environment 102 and applies analysis to this telemetry data. Examples of analysis include monitoring the computer environment 102 to derive properties related to the computer environment 102, troubleshooting problems that have occurred in the computer environment 102, performing adaptive resource management of the computer environment 102, or for other purposes. Examples of properties that can be derived include one or a combination of the following: network performance, workload performance, communication patterns, load conditions, fault conditions, or other properties. Troubleshooting can include identifying the root cause of a problem and identifying corrective actions in response to the problem.Adaptive resource management can include the dynamic allocation of resources from Computing Environment 102 to workloads running within Computing Environment 102. Examples of workloads include artificial intelligence (AI) workloads, image processing workloads, chatbot workloads, or other types of workloads.

[0013] In some examples, telemetry data can be analyzed using machine learning models. A machine learning model can be used for pattern and anomaly detection and predictive analytics. The telemetry data can be aggregated and provided as input for the machine learning model. For example, the 104 analytics system can perform analyses related to artificial intelligence for information technology operations (AIOps), which applies AI to IT operations. AIOps analytics can automate IT operations, intelligently detect patterns, adapt to changing workloads, provide insights into IT operations, and solve IT problems.

[0014] Due to the large amounts of telemetry data that can be collected from the computer environment 102 over extended periods, the storage space required for this data can be substantial. Implementing a storage system with sufficient capacity to handle large volumes of telemetry data can be costly. Furthermore, storing telemetry data on relatively slow disk-based systems can lead to increased latency when accessing and analyzing the data. Additionally, processing large volumes of telemetry data can increase the consumption of processing resources (and thus higher resource costs) and take longer.

[0015] The analysis system 104 includes data storage for storing data, including telemetry data received from the computer environment 102. The data storage comprises an in-memory data storage 116 and a secondary data storage 118. The in-memory data storage 116 is implemented with one or more storage devices, such as dynamic random-access memory (DRAM), static random-access memory (SRAM), flash memory, or other types of storage devices.

[0016] In some examples, the in-memory data store 116 is included as part of a compute node 120-1. The compute node 120-1 further includes a data store controller 114, which manages the storage (during data acquisition) and retrieval of telemetry data in the in-memory data store 116. The data store controller 114 can be implemented with machine-readable instructions that are executed in the analysis system 104.

[0017] The secondary data store 118 can be implemented with one or more persistent storage devices, such as disk-based storage devices or other types of storage devices. An in-memory data store 116 has a lower input / output access latency than a persistent storage device of the secondary data store 118. Consequently, analyses performed by the analysis system 104 using data in the in-memory data store 116 are associated with lower latency than analyses performed using data in the slower secondary data store 118.

[0018] To enable the in-memory data storage 116 to scale with the constantly growing telemetry data, the telemetry data can be aggregated (e.g., by the data storage controller 114) and stored in aggregated form in the in-memory data storage 116. The aggregation of telemetry data is continuously updated as further telemetry data is received from the computer environment 102. Although the storage capacity of the in-memory data storage 116 may be limited, the dynamic aggregation of telemetry data upon receipt allows the constantly growing amount of telemetry data to be represented by the in-memory data storage 116 without increasing the storage space used for the telemetry data.

[0019] The secondary data store 118 can store a copy of the aggregated telemetry data stored in the in-memory data store 116. Alternatively or additionally, the secondary data store 118 can store the raw telemetry data received from the computer environment 102. In other examples, the secondary data store 118 can be omitted.

[0020] When the in-memory data store 116 fills up, the data store controller 114 can apply a swapping scheme to remove in-memory telemetry data from the in-memory data store 116 to make room for other telemetry data. In this way, the in-memory data store 116 can continuously receive telemetry data despite having limited storage capacity.

[0021] In some examples, the in-memory data store 116 is a distributed in-memory data store, where the data is distributed across the storage devices of the distributed in-memory data store across multiple compute nodes 120-1 to 120-N (N ≥ 2). A "compute node" can refer to a physical computer. In other examples, the in-memory data store 116 is implemented in only one compute node, e.g., compute node 120-1. In examples implementing the distributed in-memory data store, multiple instances of the data store controller 114 are present in the respective compute nodes 120-1 to 120-N to manage access to the respective in-memory data stores.

[0022] In some examples, the analysis system 104 includes an extraction-transformation-load (ETL) module 122 (which can be implemented with machine-readable instructions) to retrieve telemetry data from the computing environment 102. The ETL module 122 can apply transformations to the telemetry data, for example, to convert raw telemetry data into a target format. The transformed data is then loaded by the ETL module 122 into the in-memory data store 116.

[0023] Queries can be received by the analysis system 104 from one or more client devices, including client device 106. A query can request a specific analysis operation to be performed by the analysis system 104. Client device 106 includes a user interface (UI) 130, such as a graphical user interface (GUI), which is displayed on a screen of client device 106. A user of client device 106 can submit queries to the analysis system 104 via the UI 130. The UI 130 also enables interactive analytical queries, where the user can submit queries based on results presented to the user in the UI 130 to obtain further results from the analysis system 104.

[0024] The analysis system 104 includes a query processing engine 160, which generates result data based on a query received from a client device, such as client device 106. The result data generated by the query processing engine 160 can be displayed on the user interface 130 of client device 106.

[0025] As used here, an “engine” can refer to one or more hardware processing circuits, which may include any or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry. Alternatively, an “engine” can refer to a combination of one or more hardware processing circuits and machine-readable instructions (software and / or firmware) that can be executed on the one or more hardware processing circuits.

[0026] The client device 106 accesses the analysis system 104 via an interface 132. In some examples, the interface 132 includes an application programming interface (API), such as a Representational State Transfer (REST) ​​API (also known as a RESTful API). The REST API supports the transmission of web requests, which in some examples include queries to the analysis system 104. In other examples, other types of interfaces (such as a Structured Query Language (SQL) interface, a Hypertext Transfer Protocol (HTTP) interface, or another type of interface) can be used to enable client devices to interact with the analysis system 104.

[0027] Queries transmitted from client device 106 can search for result data aggregated across various domains, such as arbitrary temporal and telemetry boundaries. A "temporal boundary" can refer to a specific time range. A query searching for result data across a temporal boundary will result in analyses being performed on telemetry data within that specific time range.

[0028] A "telemetry bound" can refer to a specific segment (also called a "domain") of Computer Environment 102, for example, a segment consisting of one or more ports identified by one or more port numbers, a segment consisting of a single switch on Link Line 110, or a segment consisting of a group of switches on Link Line 110. A "switch" refers to a network device that forwards data based on network addresses (and possibly port numbers) in data packets. Although specific examples of telemetry bounds are given above, telemetry bounds in other examples may refer to other segments of Computer Environment 102.

[0029] Results data generated by the analysis system 104 according to the areas defined by queries can be visualized on the user interface 130 of the client device 106. The results data can display information on resource utilization for workloads, identify jobs associated with the workloads, or provide other information about the workloads. The user interface 130 can include an interactive interface that allows the user of the client device 106 to delve deeper into the results data or query other insights based on the results data.

[0030] In some examples, the 104 analysis system supports a live operating mode where the analysis is applied to the telemetry data of the most recent time interval, e.g., the most recent 10-minute interval or another time interval. In other examples, the analysis can be based on any subset of past telemetry data (historical operating mode).

[0031] In some examples, the in-memory data store 116 is a Ray-based in-memory data store. Ray is a unified, open-source framework for scaling AI and Python applications. In Ray, tasks and actors create data objects, identified by object references, and perform computations on them. The data objects are stored in Ray's distributed shared-memory object store, with one object store per compute node in a cluster of compute nodes.

[0032] In other examples, the in-memory data store 116 can be implemented with other technologies (e.g. Apache Spark) that are capable of storing aggregated telemetry data. HIERARCHICAL IN-MEMORY DATA STORAGE

[0033] In some examples from the present revelation, such as in Fig. As shown in Figure 2, the data model used by the in-memory data store 116 is a hierarchical data model that arranges data objects with aggregated telemetry data in a hierarchy of bins 200. The hierarchy of bins 200 comprises several hierarchical levels, where a hierarchical level can contain one or more bins.

[0034] A "bin" can refer to a unit of information containing a summarized representation of a corresponding portion of the telemetry data ("telemetry data portion"). The information unit also includes bin metadata associated with the summarized representation of the telemetry data portion. Bins at different hierarchical levels of the Bin 200 hierarchy represent different resolutions at which telemetry data values ​​are aggregated. A bin serves as the unit for the data aggregation and analysis performed by the Analysis System 104.

[0035] In the example of Fig.The hierarchical levels comprise a port level 202, a switch level 204, a group level 206, a day level 212, a month level 214, and a year level 216. Hierarchy levels 202, 204, and 206 divide the data into telemetry domains (ports, individual switches, and groups of switches). Hierarchy levels 212, 214, and 216 divide the data into temporal boundaries (days, months, and years). The bins of the bin hierarchy 200 are logically organized as a multirelational property graph, with each hierarchical level grouping bins of a specific telemetry or temporal resolution.

[0036] In other examples, a hierarchy of bins containing aggregated representations of telemetry data may have a different organization or other hierarchical levels.

[0037] In the example of Fig.In two examples, the hierarchical levels are arranged in descending order of resolution (a hierarchical level, e.g., port level 202, representing higher-resolution data, is lower in the hierarchy than a hierarchical level, e.g., group level 206, representing lower-resolution data). In other examples, the hierarchical levels are arranged in ascending order of resolution. Higher-resolution data refers to higher-resolution data, while lower-resolution data refers to lower-resolution data.

[0038] Port level 202 comprises port bins 220-1 through 220-5, with each port bin 220 storing a aggregated representation of telemetry data for its corresponding port. Port bins 220-1 through 220-5 store aggregated representations of telemetry data for the respective ports of a first switch, represented by switch bin 222-1. Switch bin 222-1 is part of port level 204, and it contains an aggregated representation of telemetry data for the first switch. Switch level 204 also includes switch bins 222-2 through 222-4, which contain aggregated representations of telemetry data for other switches. Although not shown, other port bins at port level 202 contain aggregated telemetry data for ports on the other switches.

[0039] Group level 206 contains a group bin 224, which provides a summarized representation of the telemetry data for a group of switches (including the switches represented by switch bins 222-1 through 222-4). Although in Fig. While only one group bin is shown in example 2, other examples may show multiple group bins representing other groups of switches.

[0040] Similarly, the daily level 212 comprises various daily bins, including a daily bin 226, which contains a summary of the telemetry data for a specific day. The monthly level 214 comprises various monthly bins, including a monthly bin 228, which contains a summary of the telemetry data for a specific month. The yearly level 216 comprises various yearly bins, including a yearly bin 230, which contains a summary of the telemetry data for a specific year.

[0041] This mapping between the granularity of the telemetry data, represented by the bins of the bin hierarchy 200, and the respective hierarchy levels enables the query processing engine 160 to quickly identify a set of bins to be used to fulfill a query based on a filter in the query. For example, if the filter in the query seeks a result based on telemetry data for a specific switch or a specific time of day, the query processing system 160 can traverse the hierarchy of bins 200 to find out which bins contain the telemetry data that satisfies the query filter.

[0042] The hierarchy of Bins 200 is built using bin metadata. Examples of bin metadata include: a time range that defines a time interval (e.g., day, month, year), a group identifier (ID) that identifies a group of switches, a switch ID that identifies a switch, and a port number that identifies a port.

[0043] In some examples, a representation of the Bin hierarchy 200 may comprise a set of key-value data structures. Each key-value data structure forms a Bin of the Bin hierarchy 200.

[0044] The value of the key-value data structure is a summarized representation of a telemetry data segment, and the key of the key-value data structure contains bin metadata with segment information that specifies which segment (time range or computer environment segment 102) the value corresponds to. The bin metadata of a given bin also specifies a neighborhood of the given bin. The neighborhood of the given bin includes one or more parent bins and one or more child bins located above and below the given bin in the hierarchy of locations 200.

[0045] For example, the partial information in the bin metadata of port bin 220-3 includes a tuple {G4, S2, P99}, where G4 is a group ID identifying a group of switches, S2 is a switch ID, and P99 is a port number. Using the above tuple, the data storage controller 114 can readily determine that the parent storage location of port bin 220-3 is the switch storage location (e.g., 222-2) with the following storage location metadata: {G4, S2}. In another example, a tuple for the port location might also include time information, e.g., {2023-12-23, G4, S2, P99}, where 2023-12-23 specifies a particular day. More generally, the tuple that forms the partial information of a bin's metadata can contain {time range, group ID, switch ID, port range}, where the port range can identify one or more ports. Some elements of the tuple may be omitted in some bins.For example, the tuple for a switch bin excludes the port range, while the tuple for a group bin excludes both the switch ID and the port range.

[0046] To identify a subordinate bin, the analysis system 104 can perform a prefix match between the metadata of the subordinate bin and the metadata of the parent bin. For example, if a turnout has the turnout position metadata {G4, S2}, then the positions with the following subordinate bin metadata are subordinate positions of the turnout: {G4, S2, P70}, {G4, S2, P71}, and {G4, S2, P72}.

[0047] By using the set of key-value data structures to represent the hierarchy of bins, explicit links between the bins in the key-value data structure set do not need to be maintained, which can reduce the overall size of the in-memory data store. Eliminating the representation of links in the hierarchy of bins can save a significant amount of memory when the bins are used to represent a large computing environment with many components. In addition to saving memory, using a set of key-value data structures allows the data storage controller to avoid traversing a tree of bins, which can be a costly operation in terms of processing resources.

[0048] In other examples, the hierarchy of storage locations 200 can be represented using a tree structure containing nodes that represent the storage locations, as well as connections between the nodes that define the relationships between the bins.

[0049] In some examples, the data store controller 114 enables the conditional persistence of lower-resolution (higher granularity) bins in the in-memory data store 116 by easily deriving hierarchical relationships between bins using a set of key-value data structures. The lowest-level bins (leaf-level bins) (representing ports in the example bin hierarchy 200) are persisted when the data store controller 114 generates appropriate aggregated telemetry data for the lowest-level bins. "Persisting" a bin refers to storing the bin in the in-memory data store 116. During the acquisition of telemetry data by the analysis system 104, none of the higher-level (non-leaf) bins need to be stored in the in-memory data store 116.Maintaining higher-resolution bins allows for the dynamic calculation of lower-resolution bins during query evaluations, as lower-level (higher-resolution) bins can be quickly merged to create higher-level (lower-resolution) bins. DATA SUMMARY IN BINS

[0050] Incoming telemetry data is aggregated by the data storage controller 114 to produce aggregated telemetry data, which forms summarized representations of telemetry data sections. For example, based on metadata of the incoming telemetry data, the data storage controller 114 identifies a telemetry data section TD_S2 (including telemetry data points) for the switch S2 and aggregates the telemetry data points of telemetry data section TD_S2 to produce aggregated telemetry data (including a single set of values, such as a key-value data structure), which is stored as a summarized representation of telemetry data section TD_S2 in switch bin 222-2 for the switch S2.

[0051] Storing aggregated telemetry data in the bins reduces redundancy and storage requirements because the data size for each subregion remains constant regardless of the volume of incoming telemetry data. This approach is useful for handling large incoming telemetry data streams from the computing environment 102, as these large streams can quickly overwhelm storage systems and slow down analysis if aggregation is not applied according to some examples in this disclosure. Pre-aggregating telemetry data enables more cost-effective data storage and faster, real-time query processing.

[0052] In some examples, Welford's online algorithm is used for the aggregation of telemetry data points, e.g. according to the pseudocode in Table 1 below.

[0053] In Table 1, m1 represents the mean of the data points, m2 the variance of the data points, m3 the skewness of the data points (a measure of the asymmetry of a distribution of data points), m4 the kurtosis of the data points (a measure of the taillessness of a distribution of data points, where taillessness refers to how frequently data outliers occur), and n the number of data points received.

[0054] The variables m1, m2, m3, m4, and n are statistics provided by Welford's online algorithm to represent the incoming data. The set of values ​​contained in a bin of the hierarchy of bins 200 comprises a set of the aforementioned statistics. Although this example illustrates the use of specific statistics, other examples may use a different number of statistics or different types of statistics.

[0055] In the "Initialize" section of Table 1's pseudocode, m1, m2, m3, m4, and n are each initialized to zero. For each newly received data point x, Table 1's pseudocode aggregates the data point x in the "Update" section of Table 1's pseudocode. The updated set of statistics m1, m2, m3, m4, and n replaces the previous set of statistics m1, m2, m3, m4, and n stored in the respective bin.

[0056] It is important to note that updating the statistics avoids recalculating them from scratch (i.e., based on the underlying data points), resulting in significantly faster updates. Consequently, updates to the bins in the Bins 200 hierarchy can be performed online (in real time) as soon as new data points are received. Furthermore, the number of statistics stored in the bins (m1, m2, m3, m4, and n) remains the same even after receiving a new data point x, resulting in a constant memory requirement in the in-memory data store 116. A "constant" memory requirement means that the same amount of memory is used to store a set of values. Therefore, even if the Analysis System 104 receives a large number of new telemetry data points, the memory requirement for each bin in the Bins 200 hierarchy remains constant.Each bin represents the aggregated telemetry data for each configurable telemetry boundary or time limit of the incoming telemetry data.

[0057] Due to the efficient use of storage capacity of the in-memory data store 116, the in-memory data store 116 can store representations of telemetry data over longer periods (e.g., over several days instead of several minutes). As a result, analyses can be performed over longer periods, which can be useful for understanding how the computer environment 102 behaves over such extended periods.

[0058] In some cases, the analysis system 104 may determine, upon receiving telemetry data, that new bins need to be added to the bin hierarchy 200, for example, for new switches or in response to a change in the computer environment 102. If the in-memory data store 116 is full or nearly full, some bins may be removed.

[0059] In one example, the telemetry data from computer environment 102 contains the following parameters: rxBW (receive bandwidth), txBW (transmit bandwidth), rxCongestion (receive congestion), and txCongestion (transmit congestion). Other examples may include different or additional parameters in the telemetry data. For each parameter, a set of statistics is provided by Welford's online algorithm and stored in a bin. If there are four parameters, four sets of statistics are contained in one bin.

[0060] In some examples, the storage space required by the bins remains constant even when more telemetry data points are received, but the statistics stored in each bin (e.g., those provided by the Welford online algorithm) offer an accurate representation of the telemetry data range represented by the statistics. Therefore, all query results generated using these statistics provide accurate insights.

[0061] In examples where the in-memory data store 116 is distributed across multiple compute nodes 120-1 to 120-N, the in-memory data store 116 is divided across multiple in-memory data store partitions, with each compute node containing a corresponding in-memory data store partition. In some examples, the partitioning of the in-memory data store 116 can be based on groups of components, such as groups of switches. For example, a first in-memory data store partition is for a first group of switches (or a first collection of groups of switches), and a second in-memory data store partition is for a second group of switches (or a second collection of groups of switches).

[0062] In the event of a cold start of the analysis system 104 (if the analysis system 104 starts in a state where the hierarchy of bins 200 is empty), the analysis system 104 can first construct the hierarchy of bins 200 in the in-memory data store 116 using previous data, e.g., from the secondary data store 118. The hierarchy of bins 200 can then be updated when new telemetry data is received, and the clearing process can be applied to remove data from the hierarchy of bins 200 (clearing is explained further below).

[0063] In other examples, other types of statistical computational algorithms can be used to generate statistical metrics to represent telemetry data, such as a sliding window algorithm that calculates mean and standard deviation, a Flajolet-Martin sketch algorithm that estimates the cardinality (number of distinct elements) in the data, or any other algorithm that calculates statistical metrics. STORAGE-EFFICIENT ONLINE AGENTS

[0064] Analysis system 104 can decide whether or not to store higher-level bins in in-memory data storage 116 while it is acquiring telemetry data. This decision can be based on the storage capacity of in-memory data storage 116. Skipping the storage of some higher-level bins can reduce memory consumption during telemetry data acquisition. The higher-level bins (lower-resolution bins) can be calculated later, for example, during query evaluation, from lower-level bins (higher-resolution bins) by merging them.

[0065] In examples where the Welford Online algorithm is used to perform aggregates, the Welford Online algorithm is also used for bin merging. Merging a first and a second bin simply involves updating the statistics of either the first or second bin with the statistics of the other bin using the Welford Online algorithm.

[0066] Bin merging can also occur in a distributed environment where bins in in-memory data storage partitions of different compute nodes have overlapping boundaries. In such cases, the statistics of bins in the different compute nodes are merged during query evaluation. DATA CLEANSING

[0067] As mentioned above, the data storage controller 114 can apply a swapping scheme when the in-memory data store 116 fills up (overflows) to remove in-memory telemetry data from the in-memory data store 116 to make room for other telemetry data. Swapping bins from the in-memory data store 116 can be performed in parallel with other operations of the analysis system 104, including reading telemetry data from the computer environment 102 and performing query evaluation. In some examples, the data storage controller 114 can perform bin clearing intermittently at configurable intervals.

[0068] In some examples, in response to a memory overflow, the analysis system 104 prioritizes retaining the most recent bins to enable real-time analysis of the latest telemetry data. For instance, the data storage controller 114 may set a threshold below the total number of bins that can be stored in the in-memory data store 116. In this example, when the total number of memory locations in the in-memory data store 116 reaches the set threshold, the data storage controller 114 may begin removing memory locations to make room for new ones as more telemetry data is received from the computing environment 102.

[0069] In another example, the data store controller 114 can specify a certain time threshold at which any bin associated with a timestamp (part of the bin metadata) that precedes the current timestamp (representing the current time) by the specified time threshold is a candidate for swapping out. In both cases, as part of the swapping scheme, the in-memory data store 116 removes older bins to make room for newer bins containing the latest information, resulting in a sliding time window of in-memory bins in some examples. This targeted deletion approach allows the analysis system 104 to obtain a fresh and up-to-date in-memory representation of the telemetry data.

[0070] In some examples, if a storage location selected for removal is a parent storage location with one or more child storage locations, the data store controller 114 can remove the parent storage location's child storage location(s). For example, the parent storage location selected for removal might have the following storage location metadata: {G4, S2} (e.g., the parent storage location for the switch is S2, and the switch is part of the group G4). The data store controller 114 can easily identify its child storage locations as those that share a common prefix (e.g., G4, S2 in the example above). Thus, the child bins with the following storage location metadata would be candidates for removal: {G4, S2, P70}, {G4, S2, P71}, and {G4, S2, P72}.In other words, all child bins with bin metadata that match a prefix of the parent bin's bin metadata for the S2 switch in the example above can be selected for offloading. DATA COLLECTION AND MAINTENANCE

[0071] When telemetry data is read from the analysis system 104, the bins of the in-memory data store 116 are updated by the data store controller 114. As mentioned above, in some cases the leaf-level bins are updated in response to newly read data, but the non-leaf bins may be updated gradually at a later time, for example, during query evaluation. In such cases, non-leaf bins associated with leaf bins that have been updated are considered stale.

[0072] In some examples, bin status information (e.g., in the form of a hierarchical bitmap) can be managed to identify which non-leaf bins are outdated. The bin status information can include, for example, expiration indicators (e.g., bits). An expiration indicator set to a first value (e.g., "1") indicates that a corresponding bin is outdated, while a second value (e.g., "0") indicates that the corresponding bin is not outdated. Bin locations marked as outdated by the bin status information are updated during query evaluation.

[0073] In other examples where the entire hierarchy of storage locations is updated during data acquisition, the information about the state of the storage locations can be omitted by the analysis system 104.

[0074] In examples where the analysis system 104 comprises several compute nodes 120-1 to 120-N, the data storage controller 114 can process the recorded data independently in each respective compute node by updating the corresponding hierarchy of bins in the respective compute node. QUERY EVALUATION

[0075] An example query that can be processed by the query processing engine 160 of the analysis system 104 is listed in Table 2 below. TABLE 2 1 SELECT AVG(rxCongestion) 2 FROM Table_Telemetry 3 WHERE GROUP = G3 AND 4 SWITCH IN {Switch_List} and 5 PORT IN {Port_Range} and 6 TIME IN {Query_Time} 7 GROUP BY GROUP / SWITCH / CONNECTION, DAY / MONTH / YEAR

[0076] The preceding query is an example SQL query. Line 1 of the query specifies that one output to be generated is the calculation of the average of the parameter rxCongestion. Averaging a parameter is an example of an aggregation performed on the parameter. Line 2 of the query specifies that the query evaluation is based on a table named Table_Telemetry, which is the name of the table containing the hierarchy of bins 200 ( Fig. 2) contains.

[0077] Lines 3-6 of the query form the predicate (filter) of the query. The predicate (WHERE clause) specifies that the calculation is based on telemetry data for group G3, for the switches of group G3 specified in Switch_List, for the port(s) of the specified switch(es) specified in Port_Range, and for a time range specified in Query_Time.

[0078] The GROUP BY clause in line 7 of the query can specify how a result generated by the aggregation in the SELECT clause (e.g., AVG(rxCongestion)) should be grouped. The GROUP BY clause can, for example, specify grouping based on one of the following: a group of switches, a single switch, or a single port. If the grouping is based on a single switch, for instance, AVG(rxCongestion) produces result data grouped by switches in Switch_List. If Switch_List identifies switches S3 and S4, for example, AVG(rxCongestion) produces first result data containing the average rxCongestion for switch S3, and second result data containing the average rxCongestion for switch S4. The first and second result data are made available to the user interface 130 on client device 106.

[0079] As another example, the GROUP BY clause can specify a grouping based on time, such as day, month or year.

[0080] The GROUP BY clause selects which hierarchical level of the bin hierarchy 200 is used to perform the aggregation specified by the SELECT clause. If the GROUP BY clause specifies grouping based on GROUP, the query processing engine 160 retrieves statistics from the bins at the group level 206 when performing the aggregation. Another example: If the GROUP BY clause specifies grouping based on PORT, then the query processing engine 160 retrieves statistics from bin(s) at the port level 202 when performing the aggregation. Another example: If the GROUP BY clause specifies grouping based on MONTH, then the query processing system 160 retrieves statistics from bin(s) at the month level 214 when performing the aggregation.

[0081] To calculate the average of a parameter, the statistical mean (e.g., m1 in Table 1) in the respective bins can be used. In other examples, different queries may require other calculations, which can be performed using other statistics in the bins, including m2, m3, m4, or other statistics.

[0082] In examples where the in-memory data storage 116 is distributed across multiple compute nodes 120-1 to 120-N, the result data generated by the query processing machines 160 in the multiple compute nodes 120-1 to 120-N are combined (e.g. by one of the compute nodes) and the combined result data are presented to the user interface 130 of the client device 106.

[0083] By using efficient representations of large amounts of current telemetry data in the in-memory data store 116, the analysis system 104 is able to perform faster evaluations because the latency associated with accessing the in-memory data store 116 is lower, and the amount of data (in the form of statistics) that needs to be retrieved from the bins of the in-memory data store 116 is reduced. A user can specify a wide range of temporal or telemetric resolutions in the queries submitted to the analysis system 104.

[0084] The Analysis System 104 supports the acquisition of high-throughput, real-time telemetry data, which can be generated at high speed from large computing environments, such as high-performance computing (HPC) environments. By storing aggregated representations of telemetry data, large volumes of telemetry data can be acquired without significantly increasing the storage requirements of the In-Memory Data Store 116. In fact, the storage requirement of any given area of ​​the In-Memory Data Store 116 remains constant, as the statistics for that area are updated in response to newly received telemetry data. The efficient storage of aggregated telemetry data improves the scalability of the Analysis System 104 to support growing data volumes in large computing environments.

[0085] In some examples, the statistics generated by the Welford online algorithm in the bins provide accurate aggregated representations of the telemetry data, so that the result data generated from the statistics are also accurate. MORE EXAMPLES

[0086] Fig. Figure 3 is a flowchart of a process 300 according to some examples in the present disclosure. The process 300 can be carried out by a system such as the analysis system 104 of Fig. 1. be carried out. Although Fig. 3 shows a sequence of tasks; in other examples, the tasks may be performed in a different order, some of the tasks may be omitted, and other tasks may be added.

[0087] Process 300 involves receiving (at 302) data values ​​relating to a computer environment (e.g., 102 in). Fig.1) In some examples, the data values ​​include telemetry data points collected by sensors in the computer environment. In other examples, the data values ​​may include other data values ​​generated by or for the computer environment.

[0088] Process 300 involves creating (at 304) an in-memory representation of the data values ​​in memory. The in-memory representation comprises a hierarchical arrangement of bins, as shown in Fig.2. Hierarchy of Bins 200. A "hierarchical arrangement" of bins refers to bins at different levels, where a bin at one level is related to a bin at another level. A bin contains a statistical metric based on an aggregate of a subset of the data values. The bins at different hierarchical levels of the hierarchical arrangement of bins represent different resolutions at which the data values ​​are aggregated. A relationship between the bins can be a parent-bin or a child-bin relationship. A child bin of a given bin represents data with a higher resolution than that of the given bin. A parent bin of a given bin represents data with a lower resolution than that of the given bin.

[0089] The “subset” of data values ​​that are aggregated to create the statistical metric for a bin refers to data values ​​of a segment (or range) of the computer environment or to data values ​​of a time range.

[0090] Process 300 involves receiving (at 306) a query requesting a computation, where the query contains a query filter. A "query filter" refers to one or more conditions specified in the query regarding the data that are relevant for performing a computation specified by the query.

[0091] Process 300 involves identifying (in process 308) a hierarchical level from among the various hierarchical levels of the hierarchical arrangement of bins based on the query filter. "Identifying" the hierarchical level includes determining which hierarchical level corresponds to a grouping (e.g., in a GROUP BY clause) specified by the query.

[0092] Process 300 involves performing (at 308) the computation using bins at the identified hierarchical level. The "computation" can refer to any operation requested by the query. The computation uses statistical measures in the bins at the identified hierarchical level to generate result data for the query.

[0093] In some examples, the system assigns a subset of data values ​​to a first bin in the hierarchical arrangement of bins based on metadata associated with the data values. This metadata can specify time information for the data values ​​and segments of the computing environment to which the data values ​​are assigned. The system then calculates a first statistical metric based on data values ​​from this subset and adds this metric to the first bin.

[0094] In some examples, the system receives a new data value for the first bin, and the system updates the first statistical metric using a statistical computation algorithm (e.g., Welford's Online Algorithm) that updates the current value of the first statistical metric based on the combination of the current value of the first statistical metric with an update value derived from the new data value. For example, in Table 1, to update m1 (an example of a statistical metric), the system computes m1 ← m1 + δ / n, which updates the current value of m1 by an update value δ / n. Another example: To update m3 (another example of a statistical metric), the system computes m3 ← m3 + δ2 * (x- m1)- 3 * m2 * δ / n, whereby the current value of m3 is updated with a value δ2 * (x-m1)- 3 * m2 * δ / n is updated.

[0095] In some examples, the system calculates a second statistical metric based on the data values ​​of the subset of data values ​​and adds the second statistical metric to the first bin. The first bin contains a set of statistical metrics, including the first statistical metric and the second statistical metric.

[0096] In some examples, the different hierarchical levels of the hierarchical arrangement of bins comprise hierarchical levels that correspond to different segments of the computer environment.

[0097] In some examples, the different hierarchical levels of the hierarchical arrangement of bins include hierarchical levels that correspond to different time ranges.

[0098] In some examples, the first bin contains bin metadata that specifies the hierarchical level to which the first bin belongs and one or more neighboring bin locations of the first bin. The neighboring locations can include a parent bin of the first location, a child bin of the first location, or both parent and child bins of the first location.

[0099] In some examples, the system identifies a neighboring bin of the first bin based on the bin metadata.

[0100] In some examples, the neighboring bin is located at a hierarchical level higher than the first bin. As part of the computation in response to the query, a statistical metric is calculated for the neighboring bin.

[0101] In some examples, the statistical metric for the neighbor bin is not calculated during the recording of the data values ​​into the system.

[0102] In some examples, the system comprises a plurality of compute nodes, such as compute nodes 120-1 to 120-N of Fig. 1. The in-memory representation of the data values ​​is a first in-memory representation of a first subset of the data values. The memory containing the first in-memory representation of the data values ​​is located in a first compute node of the plurality of compute nodes. The system creates a second in-memory representation of a second subset of the data values, including a hierarchical arrangement of bins, in a second memory of a second compute node of the plurality of compute nodes. The computation performed in response to the query further uses bins at a hierarchical level of the second in-memory representation.

[0103] In some examples, the system manages a data structure (e.g., a bitmap) with indicators of which bins of the in-memory representation are outdated. A given memory location is outdated if a statistical metric in a sub-bin of the given memory location has been updated and a statistical metric in the given memory location itself has not yet been updated.

[0104] In some examples, the system removes a selected memory location from storage based on a timestamp of that location. Based on the removal of the selected bin, the system also removes child bins of that selected bin.

[0105] Fig. Figure 4 is a block diagram of a System 400 according to some examples in the present disclosure. One example of System 400 is Analysis System 104 of Fig.1. The System 400 can be implemented with one or more computers.

[0106] System 400 includes a memory 402 for storing an in-memory representation 404 of data values. The in-memory representation 404 comprises a hierarchical arrangement of bins, such as the hierarchy of the bins 200 in Fig. 2.

[0107] The System 400 further includes a Hardware Processor 406 (or multiple hardware processors). A hardware processor can include a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry.

[0108] The System 400 also includes a storage medium 408 on which machine-readable instructions are stored that can be executed on the hardware processor 406 to perform various tasks. Machine-readable instructions executable on a hardware processor can refer to instructions that can be executed on a single hardware processor or on multiple hardware processors.

[0109] The machine-readable instructions in storage medium 408 contain instructions for receiving data values ​​410 to receive data values ​​from a computer environment. The "receiving" of data values ​​can refer to receiving raw data values ​​from the computer environment or to receiving processed data values ​​(e.g., as by the ETL module 122 of Fig. 1 carried out).

[0110] The machine-readable instructions in storage medium 408 contain instructions for updating the statistical metrics 412 in response to received data values. These instructions update the statistical metrics in a collection of bins within the hierarchical arrangement of bins. A "collection" of bins can comprise a single bin or multiple bins. Updating a statistical metric can involve calculating a new value for the statistical metric, which combines a current value with an updated value based on a newly received data value.

[0111] The machine-readable instructions in storage medium 408 contain instructions for receiving queries 414 to receive a query that requests a computation, where the query includes a query filter. The query can be, for example, from a user on a client device (e.g., 106 in Fig. 1) be asked.

[0112] The machine-readable instructions in storage medium 408 contain instructions for identifying hierarchical level 416, in order to identify a hierarchical level of the various hierarchical levels of the hierarchical arrangement of bins based on the query filter.

[0113] The machine-readable instructions in the storage medium 408 contain instructions for the computing power 418 to perform the computation using bins at the identified hierarchical level.

[0114] In some examples, the collection of bins where statistical metrics are updated includes leaf bins in the hierarchical arrangement of bins. A "leaf" bin refers to a bin that represents the highest resolution of the data values. The machine-readable instructions can defer the update of a statistical metric from a parent bin of the leaf bins.

[0115] In some examples, the machine-readable instructions can update the statistical metric of the parent bin during query evaluation.

[0116] In some examples, updating the statistical metrics involves updating a first statistical metric in a first bin from the collection of bins using a statistical calculation algorithm. This algorithm updates the current value of the first statistical metric based on a combination of that value and an update value derived from a received data value. The storage space required for the first bin remains constant after the update.

[0117] Fig.Figure 5 is a block diagram of a non-transitory machine-readable or computer-readable storage medium 500 that stores machine-readable instructions which, when executed, cause a system to perform various tasks. The system could, for example, be the analysis system 104 of Fig. 1. Act.

[0118] The machine-readable instructions contain instructions for receiving incoming data values ​​(502) to receive incoming data values ​​relating to a computer environment. The incoming data values ​​may include telemetry data values ​​or other types of data values.

[0119] The machine-readable instructions include 504 instructions for updating the in-memory representation. These instructions update a representation of the data values ​​in memory based on incoming data values. This representation contains a hierarchical array of bins. Each bin contains a statistical metric based on an aggregate of a subset of data values. Bins at different hierarchical levels represent different resolutions of data aggregation. "Updating" a statistical metric can refer to calculating a new value for that metric.

[0120] The machine-readable instructions contain instructions for receiving queries (506) to receive a query requesting a computation. The query includes a query filter.

[0121] The machine-readable instructions contain instructions for identifying hierarchy level 508, in order to identify one hierarchy level among the various hierarchy levels of the hierarchical arrangement of the subjects based on the query filter.

[0122] The machine-readable instructions contain instructions for computational power 510 to perform the computation using bins at the identified hierarchical level.

[0123] A storage medium (e.g., 408 in Fig. 4 or Fig. 500 in Fig.5) may include any or a combination of the following: a semiconductor storage device such as DRAM or SRAM, an erasable and programmable read-only memory (EPROM), an electrically erasable and programmable read-only memory (EEPROM), and flash memory; a magnetic disk such as a hard disk, a floppy disk, and a removable disk; another magnetic medium, including tape; an optical medium such as a compact disc (CD) or a digital video disc (DVD); or any other type of storage device. It should be noted that the instructions described above may be provided on a single computer- or machine-readable storage medium or, alternatively, on multiple computer- or machine-readable storage media distributed throughout a large system, possibly with multiple nodes.Such a computer-readable or machine-readable storage medium or media are considered part of an article (or manufactured item). An article or manufactured item may refer to any single manufactured component or to multiple components. The storage medium or media may be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded for execution over a network.

[0124] In this revelation, the use of the terms "a," "an," or "the" also includes the plural forms, unless the context clearly indicates otherwise. Similarly, the terms "comprises," "including," "encompasses," "have," or "have," when used in this revelation, specify the presence of the elements indicated but do not exclude the presence or addition of other elements.

[0125] The foregoing description includes numerous details to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and deviations from the details described above. It is intended that the accompanying claims cover such modifications and variations. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] US 63 / 679,694

[0001]

Claims

[1] A procedure, comprising: Receiving data values ​​relating to a computer environment on a system that includes a hardware processor; The system generates an in-memory representation of the data values ​​in a memory that comprises a hierarchical arrangement of bins, wherein one bin of the bins comprises a statistical metric based on an aggregate of a subset of the data values, and the bins at different hierarchical levels of the hierarchical arrangement of bins represent different resolutions in which the data values ​​are aggregated; Receiving a query requesting a calculation by the system, where the query contains a query filter; Identifying a hierarchical level from the various hierarchical levels of the hierarchical arrangement of bins by the system based on the query filter; and The system performs the calculation using bins at the identified hierarchical level. [2] The method of claim 1, comprising: Assigning a subset of the data values ​​to a first bin of the hierarchical arrangement of bins based on metadata associated with the data values; and Calculating an initial statistical metric based on data values ​​from the subset of data values; and Adding the first statistical metric to the first bin. [3] The method of claim 2, comprising: Receiving a new data value for the first bin and Updating the first statistical metric using a statistical calculation algorithm that updates a current value of the first statistical metric based on combining the current value of the first statistical metric with an update value derived from the new data value. [4] The method of claim 2, comprising: Calculating a second statistical metric based on the data values ​​of the subset of data values ​​and Adding the second statistical metric to the first bin, where the first bin contains a set of statistical metrics including the first statistical metric and the second statistical metric. [5] The method of claim 1, wherein the different hierarchical levels of the hierarchical arrangement of bins comprise hierarchical levels corresponding to the different segments of the computer environment [6] Method according to claim 1, wherein the different hierarchical levels of the hierarchical arrangement of bins comprise hierarchical levels corresponding to different time domains. [7] Method according to claim 1, wherein a first bin of the bins comprises metadata specifying a hierarchical level to which the first bin belongs and specifying one or more post-bins of the first bin. [8] The method of claim 7, comprising: The system uses the bin metadata to identify a neighboring bin of the first bin. [9] Method according to claim 8, wherein the neighboring bin is located at the identified hierarchy level which is higher than a hierarchy level of the first place, wherein the method comprises: as part of the calculation in response to the query, calculating a statistical metric for the neighboring bin. [10] Method according to claim 9, wherein the statistical metric for the neighbor bin is not calculated during the reading of the data values ​​into the system. [11] Method according to claim 1, wherein the system comprises a plurality of compute nodes, the memory is located in a first compute node of the plurality of compute nodes, and the in-memory representation of the data values ​​is a first in-memory representation of a first subset of the data values, and wherein the method further comprises: The system generates a second in-memory representation of a second subset of the data values, comprising a hierarchical arrangement of bins, in another memory of a second compute node of the plurality of compute nodes. the computation performed in response to the query continues to use bins at a hierarchical level of the second in-memory representation. [12] Method according to claim 1, comprising: The system traverses a data structure containing indicators of which bins of the in-memory representation are outdated, where a given bin is outdated if a statistical metric in a child bin of the given bin has been updated and a statistical metric in the given bin has not yet been updated. [13] The method of claim 1, comprising: The system clears a selected bin from memory based on a timestamp of the bin; and Based on the clearing of the selected bin, the system clears sub-bins of the selected bin. [14] Method according to claim 1, wherein the query filter specifies a time range or a domain in the computer environment. [15] A system, comprehensive: a memory for storing an in-memory representation of data values ​​comprising a hierarchical arrangement of bins, wherein one bin of the bins comprises a statistical metric based on an aggregate of a subset of the data values, and the bins at different hierarchical levels of the hierarchical arrangement of bins represent different resolutions in which the data values ​​are aggregated; a processor and A non-transitory storage medium that stores instructions executable on the processor for: Receiving data values ​​from a computer environment; In response to the received data values, statistical metrics are updated in a collection of bins in the hierarchical arrangement of bins; Receiving a query that requests a calculation, where the query contains a query filter; Identifying a hierarchical level from the various hierarchical levels of the hierarchical arrangement of bins based on the query filter and Performing the calculation using bins at the identified hierarchical level. [16] System according to claim 15, wherein the collection of bins in which the statistical metrics are updated comprises leaf bins of the hierarchical arrangement of bins, and wherein the instructions are executable on the processor to: Deferring the update of a statistical metric of a parent bin of the leaf bins. [17] System according to claim 16, wherein the instructions are executable on the processor to: Updating the statistical metric of the parent bin during query evaluation. [18] System according to claim 15, wherein the updating of the statistical metrics comprises: Updating a first statistical metric in a first bin from the collection of bins using a statistical calculation algorithm that updates a current value of the first statistical metric based on the combination of the current value of the first statistical metric with an update value derived from a received data value, while keeping the storage space requirement of the first bin constant after the update. [19] A non-transitory, machine-readable storage medium containing instructions which, when executed, cause a system to: Receiving incoming data values ​​at the system that relate to a computer environment; Based on the incoming data values, updating an in-memory representation of the incoming data values, which comprises a hierarchical arrangement of bins, where one bin of the bins comprises a statistical metric based on an aggregate of a subset of data values, and the bins at different hierarchical levels of the hierarchical arrangement of bins represent different resolutions of data aggregation; Receiving a query at the system that requests a calculation, where the query contains a query filter; Identifying a hierarchical level from the various hierarchical levels of the hierarchical arrangement of bins based on the query filter and Performing the calculation using bins at the identified hierarchical level. [20] Non-transitory machine-readable storage medium according to claim 19, wherein the statistical metric is updated using Welford's online algorithm.

Images