Dynamic monitoring to collect data to determine an operating characteristic of the computing environment

Abstract

A non-transitory machine-readable storage medium (300) containing instructions that, when executed, cause a controller (102) to: receive metadata from an agent (104) connected to a sensor (112) at the controller (102), the metadata comprising an operating characteristic of the sensor (112); determine, based on the metadata received from the agent (104), a monitoring configuration related to the collection of sensor data by the sensor (112) connected to the agent (104), such that the monitoring configuration is received by the agent (104) during registration; store in a repository (130) mapping information correlating various indexing information with various metadata comprising operating characteristics of sensors (112); receive first sensor data from the agent (104) collected according to the monitoring configuration at the controller (102), wherein the first sensor data is included in a first message that includes first indexing information and excludes metadata that includes the operating characteristics of the sensor (112); use the first indexing information in the first message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information; update the monitoring configuration; receive second sensor data from the agent (104) collected according to the updated monitoring configuration to the controller (102), wherein the second sensor data is included in a second message that includes the first indexing information and excludes the metadata that includes the operating characteristics of the sensor (112); and use the first indexing information in the second message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information.

Description

[0001] This invention was made with government support under Contract No. DE-AC52-07NA27344, awarded by the National Nuclear Security Administration of the Department of Energy (DOE / NNSA). The government has certain rights in the invention. background

[0002] System management can be performed to control various electronic devices in response to events (e.g., security events such as a malware attack, a malfunction, or a failure, etc.) or to achieve specific operational goals. System monitoring can be performed to collect data to determine an operational characteristic of the computing environment.

[0003] US 2017 / 0 329 808 A1 discloses a system for real-time data acquisition in a network with multiple sensors. A central processing device first receives a message from each sensor, after which the central processing device sends instructions to receive data to the sensors, so that each sensor transmits its measurement data in a format configured in a data format received by the central processing device.

[0004] It is an object of the present invention to provide dynamic monitoring of a system, wherein new sensors or agents can be added or existing ones removed particularly simply and efficiently, and wherein the collected data can be quickly adapted to changing user requirements and specifications. This object is achieved by a storage medium according to claim 1, a system according to claim 13, and a method according to claim 16. Brief description of the drawings

[0005] Some embodiments of the present disclosure are described with reference to the following figures. Fig. is a block diagram of an arrangement with a monitoring system manager, according to some examples. Fig. is a block diagram of a monitoring system according to some examples. Fig. is a block diagram of a storage medium that stores machine-readable instructions according to some examples. Fig. is a block diagram of a system according to some examples. Fig. is a flowchart of a method according to some examples.

[0006] In the drawings, identical reference numbers indicate similar, but not necessarily identical, elements. The illustrations are not necessarily to scale, and the size of some parts may be exaggerated to clarify the example shown. Furthermore, the drawings contain examples and / or embodiments consistent with the description; however, the description is not limited to the examples and / or embodiments shown in the drawings. Detailed description

[0007] In this disclosure, the use of the term "a," "an," or "the" includes the plural forms unless the context clearly indicates otherwise. Likewise, the term "comprises," "including," "comprises," "includes," "has," or "have," when used in this disclosure, specifies the presence of the specified elements but does not preclude the presence or addition of other elements.

[0008] A monitoring system for a computing environment may include agents that collect monitoring data. An "agent" may refer to an entity that collects monitoring data of a computing environment. An agent may comprise one or more devices. A device may refer to a (logical or physical) unit for organizing sensors. A sensor refers to an entity that collects measurement data (also referred to as "sensor data") related to the computing environment. The sensor may be a hardware sensor or a sensor implemented with machine-readable instructions. Examples of measurement data that may be collected by sensors include any one or a combination of the following data: resource usage (e.g., usage of a processing resource, a communication resource, a storage resource, etc.), event information (e.g.,Information about anomalies such as data errors, program errors, hardware failures, information indicating an attack by an unauthorized entity such as malware or a human, a temperature exceeding a threshold, resource performance below a threshold, etc.), environmental data (e.g., data exceeding a threshold, data falling below a threshold, etc.), environmental data (e.g., temperature, pressure, information about system infrastructure such as cooling systems, etc.), etc.

[0009] In some examples, a monitoring system may use a static configuration, where the data collection characteristics are statically determined and / or the agents used to collect sensor data are static. For example, the number of agents may be fixed, or a monitoring system may require that agents be provided by specific entities (e.g., companies, individuals, organizations, etc.). The monitoring system may not accept monitoring data from agents not provided by the specified entities.

[0010] In addition, different types of agents may use different sensor data formats, which may raise the question of compatibility with the monitoring system.

[0011] In accordance with some implementations of the present disclosure, a dynamic monitoring system dynamically configures agents including corresponding sensors to monitor a computing environment, which may include electronic devices. The computing environment may include a network to which the electronic devices are connected. Examples of networks include local area networks, wide area networks, storage area networks, public networks, etc. Examples of electronic devices include computers (e.g., desktop computers, notebook computers, server computers, tablet computers, etc.), storage systems, communication nodes, vehicles, vehicle controllers, appliances, etc. In some examples, a computing environment may also include supporting infrastructure such as a cooling system, a power system, etc.

[0012] A computing environment may include a computing cloud environment, a data center, or any other type of environment containing electronic devices.

[0013] The dynamic monitoring system receives sensor data and metadata separate from the sensor data. The term "sensor data" refers to all data collected as part of monitoring a data processing environment. Sensor data can include raw measurement data from a sensor or transformed / processed data based on the raw measurement data. Sensor data can also contain information about detected events.

[0014] Separating sensor data and metadata allows metadata to be transmitted once or infrequently, so that sensor data transmitted more frequently does not need to include metadata. Messages containing sensor data without metadata can be smaller in size than messages containing both sensor data and metadata. Furthermore, in some examples, the sensor data may conform to a specific format, corresponding to a syntax that governs the formatting of sensor data transmitted by various agents interacting with the dynamic monitoring system.

[0015] Fig. is a block diagram of an example arrangement including a monitoring system manager 102 capable of interacting with agents 104 that collect sensor data related to various aspects of a computing environment 106. It should be noted that agents 104 may be deployed as part of computing environment 106, e.g., as machine-readable instructions and / or hardware components that are part of or coupled to electronic devices 105 and / or other devices of computing environment 106. In other examples, agents 104 may be separate from computing environment 106 and either connected to computing environment 106 via a network (e.g., 108).

[0016] The manager of the monitoring system 102 includes a number of computers that can communicate with the agents 104 via the network 108. As used herein, a "number" of elements may refer to one element or multiple elements.

[0017] In the examples according to Fig. Each agent 104 comprises a number of devices 110. A device 110 is a logical or physical unit that organizes a number of sensors 112. A device can be, for example, a thermostat that has a number of sensors. Another example is a network card with multiple ports, where each port can be connected to a corresponding sensor. Another example: A device can be a group of sensors.

[0018] In the examples according to Fig. Although the devices 110 and sensors 112 are shown as part of the respective agent 104, it is noted that a device and / or a sensor may be located outside the agent 104 but is communicatively connected to the agent 104.

[0019] In other examples, the concept of a device 110 for organizing a number of sensors 112 is not used. In such examples, a sensor 112 may either be part of the agent 104 or communicatively coupled to the agent 104.

[0020] The manager of the monitoring system 102 includes a dynamic configuration module 114. As used herein, an "engine" may refer to a hardware processing circuit, which may include any one or a combination of a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuit. Alternatively, an "engine" may refer to a combination of a hardware processing circuit and machine-readable instructions (software and / or firmware) executable on the hardware processing circuit.

[0021] The dynamic configuration module 114 is capable of dynamically configuring a set of properties associated with the monitoring performed by a set of agents 104, for example, to meet changing specifications (e.g., from customers or the monitoring framework itself). Such dynamically configured properties are part of a monitoring configuration.

[0022] Examples of properties that may be dynamically configured by the dynamic configuration engine 114 include any or some of the following collections: a frequency of collecting sensor data, an amount of sensor data to collect, specific sensors to use (which may include enabling and / or disabling sensors and / or removing or adding new sensors), and so on.

[0023] In some examples, changing the frequency or amount of sensor data collection allows the dynamic configuration engine 114 to adjust the input rate of incoming sensor data, for example, to meet customer-specified requirements. Changing the characteristics of the monitoring performed by the agents 104 may also change the load on resources associated with sensor data collection. For example, an increased input rate of incoming sensor data may result in increased use of storage and communication resources. Processing a larger amount of incoming sensor data may also result in increased consumption of processing resources. Processing sensor data may include converting the sensor data into another form, such as a human-readable form, a queryable database format, etc.

[0024] The dynamic configuration performed by the dynamic configuration engine 114 may be performed for each specific monitoring job to be performed by the monitoring system manager 102. A monitoring job may refer to an operation that involves collecting sensor data: for a specific number of electronic devices 105 and / or other devices of the computing environment 106, at specific times, for a specific number of metrics (e.g., a resource utilization metric, etc.), in response to specific events, etc.

[0025] A monitoring job may be requested via a job request 116 received from a consumer 118 by a job scheduler 120. The job scheduler 120 may be implemented using a variety of computing devices. The job scheduler 120 may be part of the monitoring system manager 102 and / or external to the monitoring system manager 102.

[0026] A "consumer" may refer to an electronic device, a program (including machine-readable instructions), or a user. The job request 116 is sent from the consumer 118 to the job scheduler 120 over a network (e.g., 108) or received at a user interface of the job scheduler 120.

[0027] A job request can specify information about a monitoring job, such as the number of metrics to monitor, the number of electronic devices and / or other assets to monitor, a schedule of monitoring times, events that trigger the collection of sensor data, and so on.

[0028] The job scheduler 120 is to schedule a monitoring job for execution. In some cases, the job scheduler 120 may receive multiple job requests from one or more consumers 118. The job requests received by the job scheduler 120 may be stored in a job request queue (e.g., in memory) of the job scheduler 120. The job scheduler 120 may then execute monitoring jobs for the respective jobs in a particular order, e.g., based on the time a job is received, the priority of the job, and / or another factor or combination of factors.

[0029] For a monitoring job scheduled by a job scheduler, the dynamic configuration engine 114 of the monitoring system manager 102 can dynamically configure a number of properties associated with the monitoring performed by the agents 104. The dynamic configuration can be performed before the start of a particular monitoring job and / or during the given monitoring job.

[0030] In some examples, various events 122 (e.g., a time event, an anomaly event, or another event) may cause the dynamic configuration engine 114 to change the monitoring configuration of a monitoring job. For example, the dynamic configuration engine 114 may change the monitoring configuration of the monitoring job at different times. Another example is that the dynamic configuration module 114 may change the monitoring configuration of the monitoring job for different levels of anomalies (e.g., data errors, program errors, hardware failures, an attack, etc.).

[0031] In further examples, the monitoring system manager 102 may trigger a monitoring job that is not requested in a job request from a consumer 118. For example, the monitoring system manager 102 may trigger a monitoring job in response to certain events 122, such as an anomaly. For such a monitoring job triggered by an event 122, the dynamic configuration engine 114 may dynamically configure a set of properties associated with the monitoring performed by the agents 104 in response to the event 122.

[0032] Dynamically changing a monitoring configuration for a first monitoring task can result in a change in the utilization of resources used for monitoring. For example, increasing the frequency of data collection by the sensors can result in increased data storage, increased data communication, and increased processing load for the first monitoring task. If multiple monitoring tasks compete for the same resources, the increased resource allocation for the first monitoring task can be offset by a reduced resource allocation for a second monitoring task by changing the monitoring configuration for the second monitoring task.

[0033] In addition to the ability to dynamically configure monitoring jobs, a monitoring framework provided by the monitoring system manager 102 also supports the separation of metadata and sensor data, allowing agents 104 to send sensor data to the monitoring system manager 102 without including the metadata in the messages containing the sensor data. A "message" can refer to any unit of data, such as a packet, a frame, etc., used to transmit sensor data. A fairly large amount of metadata can be associated with the sensor data, so including metadata in messages containing sensor data can increase the overall size of each message, which in turn consumes network bandwidth.

[0034] The metadata for a data collection entity (e.g., an agent 104, a device 110, and / or a sensor 112) may be transmitted from the data collection entity to the manager of the monitoring system 102 only once or less frequently than sensor data. For example, the metadata may be transmitted to the manager of the monitoring system 102 as part of a registration process (see below).

[0035] The monitoring system manager 102 may store the metadata 128 received from the monitoring system manager 102 in a repository 130. The repository 130 may be implemented with a variety of storage devices, such as any one or a combination of a disk-based storage device, a solid-state storage device, etc. The repository 130 may be part of the monitoring system manager 102 or external to it.

[0036] In some examples, the mapping information 132 may correlate various data collection entity identifiers (IDs) 134 with corresponding metadata 128. In some examples, the mapping information 132 may be in the form of a lookup table containing entries that each correlate a corresponding ID 134 with a corresponding piece of metadata 128. An identifier may refer to a number or a string of characters (e.g., numbers, letters, etc.) for uniquely identifying a corresponding data collection entity, such as an agent 104, a device 110, and / or a sensor 112. An example of an ID is a universally unique identifier (UUID).

[0037] In some examples, a message containing sensor data transmitted from an agent 104 to the monitoring system manager 102 may also include a data collection entity ID that may be used by the monitoring system manager 102 to retrieve corresponding metadata 128 based on the mapping information 132 correlating the data collection entity ID in the message with the corresponding metadata 128.

[0038] Furthermore, messages containing sensor data have a common format according to a schema (or multiple schemas) defined by schema information 124. Such messages with the common format may be referred to as "standardized messages" that the monitoring system manager 102 may expect from the agents 104. The schema information 124 may be stored in a repository 126. The repository 126 may be implemented with a variety of storage devices. The repository 126 is connected to the network 108 so that entities (including the agents 104 and the monitoring system manager 102) on the network 108 can access the schema information 124.

[0039] In other examples, the schema information 124 may be stored in a memory of the monitoring system manager 102, e.g., in the repository 130.

[0040] In some examples, the schema information 124 may be part of a schema registry according to Apache Avro™. Avro™ provides a data serialization system for communicating data in a specific format. In other examples, other types of schemas may be used for messages containing sensor data.

[0041] In some examples, the schema information 124 can be extended or modified to support changing formats for messages containing sensor data and to meet new specifications without losing backward or forward compatibility. The ability to extend or modify the schema information 124 enables a dynamic data model for the lifetime of the monitoring framework, allowing for changing customer specifications for data monitoring to be accommodated.

[0042] Furthermore, any agent 104 from any source can be added to the monitoring framework provided by the monitoring system manager 102. As a result, the monitoring framework would not have to limit data collection to agents from specific sources; rather, any agent from any source can be added to the monitoring framework, as long as the agent supports message formatting according to the schema information 124.

[0043] In some examples, agents from different sources may be transparent to the monitoring system manager 102 through the use of standardized messages (in other words, the monitoring system manager 102 does not need to know specific specifications of an agent 104 and can still receive messages containing sensor data from the agent 104). In this way, the monitoring system manager 102 can work with newly added agents 104 without having to change the monitoring framework (e.g., changing the monitoring framework to support new formats of newly added agents).

[0044] Fig. is a block diagram of an example monitoring framework 200 according to some embodiments of the present disclosure. The monitoring framework 200 supports control data provided via a control channel 202, sensor data 204 transmitted by an agent 104 via a message bus 206, and metadata 208 transmitted by the monitoring system manager 102 via the message bus 206.

[0045] As used herein, a "message bus" refers to a communication connection with properties governed by a protocol, which may be a standardized protocol, an open-source protocol, or a proprietary protocol. For example, the message bus 206 may be a software bus provided by Apache Kafka™, which provides an open-source platform. Using such a message bus 206, receivers (e.g., consumers 118 or the manager of the monitoring system 102) can subscribe to a data feed (containing sensor data) provided by producers (e.g., agents 104).

[0046] Another message-based system that may be used is RabbitMQ™, which provides message-oriented middleware to enable the communication of messages (including sensor data) between the agents 104 and other entities, such as the consumers 118 and the monitoring system manager 102.

[0047] In other examples, other types of communication connections may also be used. An interface (e.g., Kafka™'s subscription-based interface or RabbitMQ™'s message-oriented broker) may be connected to a communication connection to allow a receiver (e.g., consumer 118 or monitoring system manager 102) to receive sensor data from an agent 104.

[0048] The "control channel" 202 may refer to any interface through which control messages can be sent, e.g., from agent 104 to monitoring system manager 102 or vice versa. The control channel 202 enables dynamic configuration of system monitoring by agents 104 and helps reduce the amount of monitoring data transmitted.

[0049] For example, agent 104 may register (210) with the manager of monitoring system 102 via control channel 202 before sending sensor data. A registration procedure enables agent 104 to identify itself to the monitoring system and allows agent 104 to register its device(s) 110 and sensor(s) 112. The registration may provide the manager of the monitoring system 102 with metadata about the agent 104 (e.g., type of agent, identifier, description, etc.), its connected device(s) 110 (e.g., location, identifier, type, etc.), and device sensor(s) 112 (e.g., identifier, minimum data collection frequency, maximum data collection frequency, default data collection frequency, valid value range of a metric collected by a sensor 112, unit of measurement, type of metric measured, etc.).

[0050] Although examples of metadata have been provided above, it should be noted that in other examples, additional or alternative metadata may be provided by the agents 104 during registration.

[0051] The metadata can be used by the monitoring system manager 102 to dynamically control the connected agents 104 (e.g., changing the collection frequency for an agent 104, a particular device 110, or a particular sensor 112, disabling or enabling a sensor 112, performing automatic checks on the collected sensor data, etc.).

[0052] As part of the registration, the sensors 112 may be linked to corresponding metadata, e.g., based on the mapping information 132 updated by the monitoring system manager 102 from Fig. , which can, for example, correlate sensor IDs with metadata. With such a mapping, the sensors 112 can send messages with sensor data, but without the metadata (thus reducing message size), which was already transmitted to the manager of the monitoring system 102 during agent registration.

[0053] For example, a message containing sensor data from a sensor 112 transmitted by an agent 104 may have the following triplet form: {timestamp, sensor ID, and value}, where "value" represents a sensor measurement from sensor 112. When the monitoring system manager 102 receives the message, the sensor ID is used to retrieve the corresponding metadata, which can then be used by the monitoring system manager 102 to control monitoring operations or other tasks.

[0054] In other examples, a message containing sensor data may take a different form.

[0055] As in Fig. As further illustrated, the agent 104 may perform a deregistration (212) with the monitoring system manager 102 via the control channel 202. The deregistration may prevent the sensor(s) 112, the device(s) 110, or the entire agent 104 from being active in the monitoring frame, ie, the deregistered sensor(s) 112, the device(s) 110, or the agent 104 would no longer transmit sensor data.

[0056] The monitoring system manager 102 may also perform other configuration operations with respect to the agent 104 or its connected devices 110 or sensors 112. For example, the monitoring system manager 102 may enable or disable (214) one or more sensors 112, for example, by sending one or more enable or disable messages. Activating a sensor 112 causes the sensor to be ready to collect data, while deactivating a sensor 112 causes the sensor to no longer be able to collect data.

[0057] As another example, the manager of monitoring system 102 can start or stop data collection (216) by a sensor 112, for example, by sending a start or stop message. A start message can cause an active sensor 112 to start data collection. A stop message can cause an active sensor 112 to stop data collection.

[0058] As another example, the manager of the monitoring system 102 may change the data collection behavior (218) of the sensors, e.g., by changing the frequency with which one or more sensors 112 collect data, by changing the time interval at which one or more sensors 112 collect data, by changing the metric(s) collected, and so on.

[0059] As another example, the monitoring system manager 102 may update a monitoring configuration (218) of the agent 104 or its connected devices 110 or sensors 112. This may be done by sending a configuration update message to the agent 104. Note that enabling or disabling sensors (214), starting or stopping sensor collections (216), and changing sensor behavior (218) are examples of a configuration update. Rather than listing configuration operations 214, 216, and 218 separately, these configuration operations may be included as part of configuration updates 218.

[0060] In some examples, it is also possible for agent 104 to initiate a configuration operation (one of 214, 216, 218, and 220) with monitoring system manager 102. For example, agent 104 may send its monitoring configuration to monitoring system manager 102 so that monitoring system manager 102 can set the monitoring configuration for agent 104. As another example, agent 104 may update the monitoring configuration and send the updated monitoring configuration (e.g., a sensor has been enabled or disabled, a sensor has been started or stopped, etc.) to monitoring system manager 102.

[0061] Agent 104 may store its copy of the configuration data (which may also be stored by monitoring system manager 102). The configuration data may contain information defining a monitoring configuration. Should agent 104 lose its configuration data for any reason (e.g., due to a failure, malfunction, or data error), agent 104 may request the configuration data (220) from monitoring system manager 102. Similarly, monitoring system manager 102 may request configuration data from agent 104.

[0062] Metadata 208 sent by the monitoring system manager 102 over the message bus 206 may be stored in a persistent storage 224 (e.g., in the repository 130 of Fig. ). Sensor data 204 sent by agent 104 over message bus 206 may also be stored by monitoring system manager 102 in persistent memory 224 for later retrieval, e.g., by a consumer 118. Furthermore, the manager of monitoring system 102 may forward the sensor data to another entity, e.g., a consumer 118.

[0063] An example scenario is described below. A consumer 118 may submit a job request specifying a frequency for monitoring the computing environment 106 ( Fig. ). The job scheduler 120 processes the job request and schedules the requested job for execution on specific resources (e.g., one or more compute nodes).

[0064] The job scheduler 120 may notify the monitoring system manager 102 that a monitoring job is requested, for example, where data collection (e.g., from a list of compute nodes) should occur at a specific frequency. The job notification from the job scheduler 120 to the monitoring system manager 102 may also specify, for example, the selected sensors 112 to be used, a category of sensors 112 (e.g., temperature sensor, power consumption sensor, processor usage tracking sensor, etc.), specified specific sensors, a category (e.g., performance and energy values), metrics to be collected, etc.

[0065] The monitoring system manager 102 can then contact the agents 104 (already running) on ​​the computer node(s) in the list to start data monitoring. Furthermore, the monitoring system manager 102 can start the agents 104 on the computer node(s). The monitoring system manager 102 can provide the contacted and / or started agents 104 with configuration data for the monitoring job. The configuration data can identify the sensor(s) 112 to be used for data collection, the data collection frequency, and so on.

[0066] In some examples, a consumer 118 submitting a job request may specify the creation of job-specific topics in the monitoring framework. A job-specific topic may relate to a category of data to be collected, e.g., a topic related to measurements taken by sensors, a topic related to events, etc. Each job-specific topic may be associated with a corresponding queue for storing the respective data.

[0067] After the configuration of the agents and the corresponding devices and / or sensors is successfully completed, the monitoring system manager 102 can inform the job scheduler 120.

[0068] After successful dynamic setup of the monitoring infrastructure, the monitoring job is started. If job-specific topics are used, agents 104 can write data for the topics to the respective queue. The consumer 118 that submitted the job can access the collected sensor data, possibly in corresponding queues.

[0069] In another example scenario, a dynamic change in data collection by the monitoring framework may be based on the detection of a specific event, such as an anomaly. For example, if a program's performance (expressed as the number of input / output transactions per time interval) falls below a certain threshold, information about the anomaly is transmitted to the manager of the monitoring system 102.

[0070] The monitoring system manager 102 retrieves information about the resources on which the program is running and retrieves a monitoring configuration associated with the anomaly. The monitoring configuration may, for example, specify an increased collection frequency for metrics related to the program's I / O transactions.

[0071] The monitoring system manager 102 can send the monitoring configuration to the appropriate agent(s) 104 to collect data about the anomaly. The agent(s) 104 collect(s) the data, which is then made available to the monitoring system manager 102 and / or a consumer 118. The collected data can be subjected to root cause analysis, for example, to determine the cause of the anomaly.

[0072] In some examples, the dynamic monitoring framework discussed here can provide some of the following examples. Standardizing framework messages through the use of schemas allows consumers to conveniently connect to the monitoring framework through an interface. The dynamic monitoring framework is capable of adapting to customer requirements without requiring changes to the underlying framework.

[0073] Furthermore, the resources allocated to monitoring by the monitoring framework can be adjusted as the monitoring load changes. For example, the monitoring framework can initially be allocated a relatively small amount of resources, and as the monitoring load increases, additional resources can be allocated.

[0074] In addition, agents from different sources can be dynamically added to or removed from the monitoring framework, provided the agents support messages according to the specified schema(s).

[0075] Dynamic configuration of the monitoring framework can also provide flexibility in the metrics to be monitored to support the addition of new services that may be based on specific metrics.

[0076] Separating sensor data and metadata allows the monitoring framework to more efficiently communicate large amounts of sensor data collected by agents. In some examples, a publish-subscribe communication model (where a consumer subscribes to the sensor data published by an agent) also enables a more efficient technique for delivering data to consumers. Extensible or modifiable schemas allow message formats to change over time to support new devices and / or sensors, while maintaining backward or forward compatibility.

[0077] Fig. is a block diagram of a non-transitory machine-readable or computer-readable storage medium 300 on which machine-readable instructions are stored that, when executed, cause a controller to perform various tasks. The controller may, for example, include the monitoring system manager 102 of Fig. include.

[0078] The machine-readable instructions include dynamic configuration instructions 302 to dynamically configure a property associated with the monitoring performed by an agent (e.g., 104 in Fig. ). Dynamically configuring a property can refer to dynamically configuring one or more properties. Dynamic configuration can be done for a monitoring job requested by a consumer or for a monitoring job initiated in response to a specific event.

[0079] The machine-readable instructions include metadata storage instructions 304 for storing metadata related to the agent in a repository (e.g., 130). In some examples, the metadata includes information about a property of a sensor that is part of or coupled to the agent. The metadata may also include information related to a device and the agent. The sensor property refers to an operational property (e.g., the frequency of data collection by the sensor, etc.) of the sensor or a data property of the data collected by the sensor (e.g., the valid range of values).

[0080] The machine-readable instructions include sensor data receive instructions 306 for receiving first sensor data from the agent, excluding the metadata. In some examples, the first sensor data conforms to a format defined by a syntax that governs the formatting of sensor data from agents interacting with the controller. The metadata may be received from the agent as part of a registration the agent performs with the controller.

[0081] The machine-readable instructions include metadata retrieval instructions 308 for using indexing information (e.g., an identifier of a sensor or other data collection entity) in the first sensor data to retrieve the metadata from the repository. The retrieved metadata can be used to determine the monitoring configuration and other details associated with the sensor data, such as identifiers and locations of the corresponding agent 104 and device 110.

[0082] Fig. 4 is a block diagram of a system 400 according to some examples. System 400 may be, for example, the manager of monitoring system 102. System 400 includes a hardware processor 402 (or multiple hardware processors). A hardware processor may be a microprocessor, a core of a multi-core microprocessor, a microcontroller, a programmable integrated circuit, a programmable gate array, or other hardware processing circuitry.

[0083] System 400 further includes a storage medium 404 storing machine-readable instructions executable on hardware processor 402 to perform various tasks. Machine-readable instructions executable on a hardware processor may refer to instructions executable on a single hardware processor or to instructions executable on multiple hardware processors.

[0084] The machine-readable instructions include instructions for receiving metadata 406 to receive metadata from an agent relating to the agent as part of the registration performed by the agent. The metadata received by the system 400 may be stored in a repository (e.g., 130 in Fig. ) containing mapping information (e.g. 132) correlating sensor identifiers (or identifiers of other data acquisition units) with corresponding metadata.

[0085] The machine-readable instructions include instructions for determining the monitoring configuration 408 to determine a monitoring configuration of the agent based on the metadata. The monitoring configuration specifies properties associated with the collection of sensor data by agents.

[0086] The machine-readable instructions contain instructions for receiving sensor data from the agent, excluding the metadata. The sensor data is collected according to the monitoring configuration. The sensor data is included in a message that excludes the metadata. This allows the size of the message to be reduced.

[0087] Fig. is a flowchart of a method 500 according to some examples.

[0088] The process 500 includes a controller (e.g., the monitoring system manager 102 of Fig. ), which (at 502) receives metadata from an agent. The metadata can be received, for example, when the agent registers with the controller.

[0089] The controller determines (at 504) a monitoring configuration related to the collection of sensor data by a sensor connected to the agent. The monitoring configuration may be dynamically determined by the controller based on the metadata, or the monitoring configuration may be determined based on configuration data sent from the agent to the controller.

[0090] The controller receives (at 506) initial sensor data from the agent collected according to the monitoring configuration. The initial sensor data is included in a first message that excludes the metadata.

[0091] The control device updates the monitoring configuration (at 508). The monitoring configuration update can be initiated by the control device or requested by the agent.

[0092] The controller receives (at 510) second sensor data from the agent collected according to the updated monitoring configuration. The second sensor data is included in a second message that excludes the metadata.

[0093] A storage medium (e.g., 300 in Fig. in Fig.) may include any one or a combination of the following: a semiconductor memory device such as dynamic or static random access memory (DRAM or SRAM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory, or other type of non-volatile memory device; a magnetic disk such as a hard disk, a floppy disk, and a removable disk; other magnetic medium including tape; an optical medium such as a compact disk (CD) or digital video disk (DVD); or other type of storage device. It should be noted that the instructions described above may be provided on a single computer- or machine-readable storage medium, or alternatively, on multiple computer- or machine-readable storage media distributed throughout a large system, possibly with multiple nodes.Such computer-readable or machine-readable storage medium or media are considered part of an article (or article of manufacture). An article or article of manufacture may refer to each manufactured component or multiple components. The storage medium or media may be located either in the machine on which the machine-readable instructions are executed or at a remote location from which machine-readable instructions can be downloaded over a network for execution.

[0094] In the foregoing description, numerous details are set forth to provide an understanding of the subject matter disclosed herein. However, implementations may be practiced without some of these details. Other implementations may include modifications and variations from the details described above. The appended claims are intended to cover such modifications and variations.

Claims

[1] Non-transitory machine-readable storage medium (300) containing instructions that, when executed, cause a controller (102) to: receive metadata from an agent (104) connected to a sensor (112) at the controller (102), the metadata comprising an operating characteristic of the sensor (112); determine, based on the metadata received from the agent (104), a monitoring configuration related to the collection of sensor data by the sensor (112) connected to the agent (104), such that the monitoring configuration is received by the agent (104) during registration; store in a repository (130) mapping information correlating various indexing information with various metadata comprising operating characteristics of sensors (112); receive first sensor data from the agent (104) collected according to the monitoring configuration at the controller (102), wherein the first sensor data is included in a first message that includes first indexing information and excludes metadata that includes the operating characteristics of the sensor (112); use the first indexing information in the first message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information; update the monitoring configuration; receive second sensor data from the agent (104) collected according to the updated monitoring configuration to the controller (102), wherein the second sensor data is included in a second message that includes the first indexing information and excludes the metadata that includes the operating characteristics of the sensor (112); and use the first indexing information in the second message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information. [2] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): receive the first sensor data conforming to a format defined by a syntax that governs the formatting of sensor data from agents (104) interacting with the controller (102). [3] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): receive the metadata comprising the operating characteristics of the sensor (112) from the agent as part of a registration performed by the agent (104) with the controller (102). [4] The non-transitory machine-readable storage medium (300) of claim 1, wherein the operating characteristic of the sensor (112) contained in the metadata comprises a frequency of data acquisition by the sensor (112). [5] The non-transitory machine-readable storage medium (300) of claim 1, wherein the operating characteristic of the sensor (112) contained in the metadata comprises a valid range of values ​​of the data acquired by the sensor (112). [6] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): dynamically configure a property associated with the monitoring performed by the agent (104) by sending configuration data to the agent (104) to operate according to a monitoring configuration determined by the controller (102). [7] The non-transitory machine-readable storage medium (300) of claim 6, wherein the instructions, when executed, cause the controller (102) to dynamically configure the property associated with the monitoring performed by the agent (104) in response to a job request specifying a monitoring operation. [8] The non-transitory machine-readable storage medium (300) of claim 1, wherein the first indexing information comprises an identifier of the sensor (112) and the different indexing information in the association information comprises identifiers of different sensors (112). [9] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): update the monitoring configuration of the agent (104) by sending a configuration update to the agent (104). [10] The non-transitory machine-readable storage medium (300) of claim 1, wherein the first sensor data is received via a communication link connected to an interface to enable access to the first sensor data during transmission from the agent (104) to the controller (102). [11] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): perform a deregistration with respect to the agent (104) to remove the agent (104) or the sensor (112) connected to the agent (104). [12] The non-transitory machine-readable storage medium (300) of claim 1, wherein the instructions, when executed, cause the controller (102): dynamically control the agent (104) based on the metadata. [13] System (400) comprising: a processor (402); and a non-transferable storage medium (404) storing instructions executable on the processor (402) to: receive from an agent (104) at the system (400) metadata about the agent (104) as part of the registration performed by the agent (104), the metadata comprising an operational characteristic associated with the agent (104); store in a repository (130) mapping information correlating various indexing information with various metadata comprising operating characteristics of sensors (112); based on the metadata received from the agent (104), determining a monitoring configuration related to the collection of sensor data by the sensor (112) associated with the agent (104), such that the monitoring configuration is received by the agent (104) during registration; receiving from the agent (104) at the system (400) first sensor data collected according to the monitoring configuration, wherein the first sensor data is included in a first message comprising first indexing information and excluding metadata comprising the operating characteristics of the sensor (112); Using the first indexing information in the first message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information; Updating the monitoring configuration; Receiving second sensor data from the agent (104) collected according to the updated monitoring configuration at the system (400), wherein the second sensor data is included in a second message that includes the first indexing information and excludes the metadata that includes the operating characteristics of the sensor (112); and Using the first indexing information in the second message to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information. [14] The system (400) of claim 13, wherein the sensor data corresponds to a syntax represented by syntax information stored in the repository (130). [15] The system (400) of claim 13, wherein the operating characteristic of the sensor (112) contained in the metadata comprises a frequency of data acquisition by the sensor (112) or a valid range of values ​​of the data acquired by the sensor (112). [16] Procedure comprising: Receiving (502), by a controller (102), metadata from an agent (104) connected to a sensor (112), the metadata comprising an operating characteristic of the sensor (112); based on the metadata received from the agent (104), determining (504) by the controller (102) a monitoring configuration related to the acquisition of sensor data by the sensor (112) connected to the agent (104), such that the monitoring configuration is received by the agent (104) during registration; Storing mapping information in a repository (130) correlating various indexing information with various metadata comprising operating characteristics of sensors (112); Receiving (506) by the controller (102) first sensor data from the agent (104) collected according to the monitoring configuration, wherein the first sensor data is included in a first message that includes first indexing information and excludes the metadata that includes the operating characteristics of the sensor (112); Using the first indexing information in the first message, the controller (102) retrieves the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information; Updating (508) the monitoring configuration by the controller (102); Receiving (510) second sensor data from the agent (104) collected according to the updated monitoring configuration by the controller (102), wherein the second sensor data is included in a second message that includes the first indexing information and that excludes the metadata that includes the operating characteristics of the sensor (112), and Using the first indexing information in the second message, the controller (102) to retrieve the metadata comprising the operating characteristics of the sensor (112) from the repository (130) using the mapping information. [17] The method of claim 16, wherein the operating characteristic of the sensor (112) contained in the metadata comprises a frequency of data acquisition by the sensor (112).

Images