Data-driven Thing Graph for Data Center Monitoring

CN114365505BActive Publication Date: 2024-11-15ALIBABA GROUP HOLDING LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201980099754.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2019-11-07
Publication Date
2024-11-15
Estimated Expiration
2039-11-07

AI Technical Summary

Technical Problem

[0005]当前的系统无法从此类传感器生成的大量原始数据中充分显现有用的信息

Benefits of technology

[0006] The disclosed embodiments address these and other problems in IDC. The disclosed embodiments enable safer and more efficient operation of such IDCs by employing a graph of things (GoT) that can model sensors and events.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114365505B_ABST
    Figure CN114365505B_ABST
Patent Text Reader

Abstract

A technique for building and using a sensor transaction graph to associate sensors based on past events is disclosed. In one embodiment, the disclosed method includes retrieving raw sensor data collected by a plurality of sensors; identifying a plurality of events based on the raw sensor data; building a sensor graph based on the plurality of events, the sensor graph having nodes representing the plurality of sensors and a set of edges connecting the nodes, each edge in the set of edges being associated with a weight calculated based on a number of related events detected in the plurality of events; and querying the sensor graph in response to a new event associated with a sensor in the plurality of sensors, the querying including identifying at least one related sensor connected to the sensor in the sensor graph.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Copyright Notice

[0002] This application contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent disclosure as it appears in the Patent and Trademark Office files or records, but otherwise reserves all copyright rights whatsoever. Background Art

[0003] The disclosed embodiments relate to management of sensors, and more particularly, to techniques for managing sensor data of an Internet data center (IDC).

[0004] Modern IDCs typically employ one or more monitoring systems to monitor outputs from a variety of sensors. The data generated by these sensors includes a large amount of raw time series data. In current systems, human technicians or automated systems need to understand the information behind this data, especially when there are real-time conditions that require intervention. For example, an increase in temperature sensor readings may be caused by a fan failure in a computer room air handler (CRAH) or the sudden fall of a heavy object placed on a server rack.

[0005] Current systems are unable to fully extract useful information from the large amounts of raw data generated by such sensors. In fact, many of these systems rely on human interpretation of events and trial-and-error techniques to resolve anomalous sensor readings. Summary of the invention

[0006] The disclosed embodiments address these and other problems in IDCs.The disclosed embodiments enable safer and more efficient operation of such IDCs by employing a graph of things (GoT) that can model sensors and events.

[0007] In one embodiment, a method is disclosed, including retrieving raw sensor data collected by a plurality of sensors; identifying a plurality of events based on the raw sensor data; constructing a sensor graph based on the plurality of events, the sensor graph having nodes representing the plurality of sensors and a set of edges connecting the nodes, each edge in the set of edges being associated with a weight calculated based on a number of related events detected in the plurality of events; and querying the sensor graph in response to a new event associated with a sensor in the plurality of sensors, the query including identifying at least one related sensor connected to a sensor in the sensor graph.

[0008] In another embodiment, a non-transitory computer-readable storage medium for tangibly storing computer program instructions executable by a computer processor, the computer program instructions defining the following steps: retrieving raw sensor data collected by a plurality of sensors; identifying a plurality of events based on the raw sensor data; constructing a sensor graph based on the plurality of events, the sensor graph having nodes representing the plurality of sensors and a set of edges connecting the nodes, each edge in the set of edges being associated with a weight calculated based on a number of related events detected in the plurality of events; and querying the sensor graph in response to a new event associated with a sensor in the plurality of sensors, the querying comprising identifying at least one related sensor connected to a sensor in the sensor graph.

[0009] In another embodiment, an apparatus is disclosed, comprising: a processor; and a storage medium for tangibly storing program logic thereon for execution by the processor, the stored program logic causing the processor to perform the following operations: retrieving raw sensor data collected by a plurality of sensors, identifying a plurality of event data based on the raw sensors, building a sensor graph based on the plurality of events, the sensor graph having nodes representing the plurality of sensors and a set of edges connecting the nodes, each edge in the set of edges being associated with a weight calculated based on a number of related events detected in the plurality of events, and querying the sensor graph in response to a new event associated with a sensor in the plurality of sensors, the query comprising identifying at least one related sensor connected to a sensor in the sensor graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The foregoing and other objects, features, and advantages of the present disclosure will be apparent from the following description of embodiments illustrated in the accompanying drawings, in which reference characters refer to the same parts throughout the different views. The drawings are not necessarily drawn to scale, emphasis instead being placed upon illustrating the principles of the present disclosure.

[0011] Figure 1 is a system block diagram for generating and analyzing GoT according to some embodiments of the present disclosure.

[0012] Figure 2 is a flowchart illustrating a method for generating a GoT according to some embodiments of the present disclosure.

[0013] Figures 3A to 3F An event graph and a method of constructing the same according to some embodiments of the present disclosure are shown.

[0014] Figure 4 A sensor transaction diagram according to some embodiments of the present disclosure is shown.

[0015] Figure 5A is a flow chart illustrating a method for determining whether a peak or drop event occurs according to some embodiments of the present disclosure.

[0016] Figure 5B is a flow chart illustrating a method for determining whether a mean shift event occurs according to some embodiments of the present disclosure.

[0017] Figure 5C is a flow chart illustrating a method for determining whether a positive or negative trend change event has occurred according to some embodiments of the present disclosure.

[0018] Figure 5D is a flow chart illustrating a method for determining whether a variance change event has occurred according to some embodiments of the present disclosure.

[0019] Figure 6 is a flow chart illustrating a method for analyzing GoT according to some embodiments of the present disclosure.

[0020] Figure 7 FIG. 1 is a diagram showing device hardware for generating or analyzing GoT according to some embodiments of the present disclosure. DETAILED DESCRIPTION

[0021] Figure 1 is a system block diagram for generating and analyzing GoT according to some embodiments of the present disclosure.

[0022] In the illustrated embodiment, the system (100) includes a plurality of sensors (102a, 102b, 102c, 102n; collectively 102). Examples of sensors include devices such as temperature sensors, voltage sensors, current sensors, humidity sensors, airflow sensors, humidity or water sensors, smoke sensors, door sensors, video surveillance equipment, power consumption / switch sensors, and other sensors that measure real-world properties of a data center or other environment in which the system (100) is employed. Alternatively, in conjunction with the foregoing, the sensors may include software-based sensing components such as server load monitoring sensors, log file monitoring sensors, intrusion detection systems, load peak detection software, and various other software for monitoring services provided by, for example, a data center.

[0023] The sensors (102) generate time series data. That is, each sensor generates a data value at a given time and generates such data over a period of time. Examples of time series data include continuous temperature monitors (i.e., periodic temperatures reported by a temperature sensor) and the number of inbound connections to a server at any given time (e.g., as reported by the server). In general, any data that can be expressed as a function of time can be considered to be time series data generated by the sensors (102), and the disclosed embodiments are not limited to a particular sensor or type or format of time series data.

[0024] In the disclosed embodiments, the sensors (102) are primarily described in the context of IDCs, but the sensors (102) may be installed in any type of environment. Furthermore, there is no limitation on the number, type, or location of the sensors (102).

[0025] As shown, each of the sensors (102) is connected to a network (118). In one embodiment, each sensor (102) is physically coupled to the network (118) and the network (118) may include a controller area network (CAN) bus, Ethernet, or other type of data communication medium. In other embodiments, the sensor (102) includes a wireless transceiver and may communicate via a wireless medium instead of a physical bus. Examples of such communication networks include Wireless Fidelity (Wi-Fi), cellular, or satellite networks. In some embodiments, the sensor (102) may be directly connected to other devices in the system (e.g., monitoring system 116 or preprocessor 104) and may not communicate via the network (118). In some embodiments, the system (100) may employ multiple networks (and directly connected devices). For example, a server employing load monitoring may communicate via Ethernet, while a temperature sensor may communicate via a Wi-Fi network. The disclosed embodiments are not limited to any one type (or combination) of network technologies used.

[0026] In the illustrated embodiment, the data generated by the sensor (102) is referred to as raw data. The sensor transmits this data to a preprocessor (104) that forms the initial stage of the event extraction phase. This stage is designed to filter the raw data into actionable events and is described in more detail for various flow charts. As shown, the event extraction phase includes a preprocessor (104), a time series data analysis processor (106), and an event detection processor (108). In some embodiments, each processor (104, 106, 108) includes a dedicated hardware processing element. In other embodiments, the event extraction phase can be implemented on one or more hardware devices, and the processor (104, 106, 108) can be implemented as software running on these devices.

[0027] In the illustrated embodiment, the preprocessor (104) receives raw data from the sensor (102) via the network (118). As previously described, this raw data includes data values ​​and time values. In the illustrated embodiment, the preprocessor (104) cleans and smoothes the received data. The details of this operation are described in Figure 2 , particularly in the description of step 204.

[0028] After cleaning and smoothing the sensor data, the preprocessor (104) transmits the data to the time series data analysis processor (106). The time series data analysis processor (106) decomposes the cleaned and smoothed time series data into trend, seasonality, and residual components. The details of this operation are described in Figure 2 , particularly in the description of step 206.

[0029] The time series data analysis processor (106) then transmits the decomposed components to the event detection processor (108). The event detection processor (108) processes the individual components of the time series data points and identifies the actionable events represented by the components. The output of the event detection processor (108) includes a vector that includes a value for each event type, the time of the sensor data, and the identity of the sensor. The event detection processor (108) transmits this data to the event memory (110). Details of the operation of the event detection processor (108) are described in Figure 2 , particularly in step 208, and Figure 5A , 5B and in the description of 5C.

[0030] In the illustrated embodiment, the event store (110) includes a storage device (physical or logical) that stores events detected during the event extraction phase. In one embodiment, the event store (110) includes a relational database management system (RDBMS) or other type of database. In some embodiments, the event store (110) may include a key-value data store or other less intensive data store (e.g., an object store). In some embodiments, the event store (110) is configured to store events in chronological order based on the time identified by the event detection processor (108). In this manner, the event store (110) operates similar to a queue.

[0031] In the illustrated embodiment, a separate drawing phase is shown. During the drawing phase, a graph is constructed based on events stored in the event store (110) and updated periodically. In the illustrated embodiment, the graph builder (112) accesses the event store (110) to retrieve events. In some embodiments, the graph builder (112) actively queries the event store (110). In other embodiments, the graph builder (112) subscribes to the event store (110) and periodically receives events as they are added to the event store (110).

[0032] The graph builder (112) also accesses a graph memory (114). The graph memory (114) may include a graph database or may include an RDBMS. Typically, the graph memory (114) stores a set of nodes, a set of edges, and weights of the edges. In the illustrated embodiment, the nodes include a sensor identifier, a time point, and an event type. The graph builder (112) updates the data stored in the graph memory (114) based on the events received in the event memory (110). In this manner, the graph builder (112) is responsible for updating the event graph captured by the sensor (102) and processed during the event extraction phase. Details of the operation of the graph builder (112) and the graph memory (114) are described in detail in Figure 2 , particularly in step 212 and Figures 3A to 3F And in the description of 4.

[0033] As shown, the system (100) includes a monitoring system (116). In the illustrated embodiment, the monitoring system (116) receives processed event data from the event detection processor (108). Alternatively, in combination with the foregoing, the monitoring system (116) may also receive raw data from the sensor (102). The monitoring system (116) is also communicatively coupled to the graph memory (114). In the illustrated embodiment, the monitoring system (116) may include a hardware device (or multiple devices) that is responsible for analyzing the sensor data and providing actionable intelligence to the operator of the device. In other embodiments, the monitoring system (116) may also automatically take action in response to the event (e.g., alerting the fire department when an increase in temperature is detected).

[0034] To provide such insight or take such action, the monitoring system (116) queries the graph memory (114) to identify relevant sensors in response to the event and perform root cause analysis. The details of the operation of the monitoring system (116) are described in Figure 6 provided in the description.

[0035] Figure 2 is a flowchart illustrating a method of generating a GoT according to some embodiments of the present disclosure.

[0036] In step 202, the method (200) receives sensor data. As described above, the method (200) may receive raw data generated by one or more sensors via a network or other communication medium.

[0037] In step 204, the method (200) cleans and smoothes the data.

[0038] In some embodiments, the method (200) performs various cleaning operations on the received data, such as filtering anomalies, removing abnormal sensors, smoothing short-term fluctuations in sensor data, and other operations. In some embodiments, cleaning can also include interpolating missing values ​​and / or normalizing values. In some embodiments, the method (200) can use moving average smoothing to smooth and clean the data. Examples of such moving average methods include weighted moving average (WMA), exponentially weighted moving average (EWMA), autoregressive integrated moving average (ARIMA) smoothing, and other techniques. Other techniques used in addition to (or in place of) these techniques and the disclosed embodiments are not intended to limit the specific or single technique used to clean and smooth the data from the sensor. In fact, multiple techniques can be used simultaneously, and the technique for each sensor may be unique based on the type of time series data generated and the observation of fluctuations in the data generated by the sensor.

[0039] In step 206, the method (200) decomposes the cleaned and smoothed data into trend, seasonality, and residual data.

[0040] In the illustrated embodiment, the time series data received in step 202 may be affected by three components: trend, seasonality, and residual. In short, the seasonal component of a time series refers to a pattern that repeats at a fixed time interval. The trend component refers to the overall change in the data over a longer period of time. Finally, the residual refers to the residual data remaining in the time series after removing the trend and seasonal data.

[0041] In one embodiment, the time series can be represented by a function Y(t), where t represents the elapsed time. In this embodiment, Y(t) can be represented as a series of vectors Y(t) = [y(1), y(2), ... y(n)]. In one embodiment, the values ​​of Y(t) can therefore be decomposed into an additive set of trend (T), seasonal (S), and residual (R) functions: Y(t) = T(t) + S(t) + R(t), where t again represents the elapsed time. An additive or multiplicative model can be selected based on the underlying sensor.

[0042] In one embodiment, the method (200) receives raw time series data and first detects a trend component. In one embodiment, the method (200) may use a centered moving average algorithm to calculate the trend T(t) of the time series Y(t), but other algorithms (e.g., Fourier transform) may also be used. The method (200) may then remove the trend T(t) from the time series Y(t). The method (200) may then analyze the underlying detrended time series (Y(t)-T(t)) to determine the period of the resulting data. After determining the period, the method (200) extracts the seasonal component S(t), which leaves a residual or noise component R(t).

[0043] In step 208, the method (200) extracts events from the decomposed data.

[0044] In one embodiment, the method (200) stores a list of predefined events. Generally speaking, an event includes one or more conditions that utilize basic components (trend, seasonality, residual) as input. That is, if one or more of the decomposed components meets the conditions, the event is assumed to have occurred. The disclosed embodiment describes four examples of events: peak / drop, mean shift, trend change, and variance shift. However, the disclosed embodiment is not intended to be limited to only these events, and other events can be constructed based on the basic data or requirements of the system implementing the method (200).

[0045] A spike / dip is a rapid increase or decrease in the value of the residual component of a time series. As mentioned above, a trend is a gradual change in a time series, while seasonality is a cyclical (and regular) fluctuation. Therefore, a spike or dip in a time series can usually be attributed to the residual component of the time series. Figure 5A Detection of peaks / drops is more fully described in the description of .

[0046] A mean shift event refers to an overall change in the moving average of the time series data. In this embodiment, a sliding window is used to calculate the average value of the time series at a given time t, and this value is used to determine whether a mean shift has occurred. Figure 5B Detection of mean shift is more fully described in the description of .

[0047] A trend change event is when the trend component of a time series reverses, either from positive (upward) to negative (downward) or from negative (downward) to positive (upward). Figure 5C Detection of trend changes is more fully described in the description of .

[0048] A variance shift event occurs when the variance of the underlying time series data changes. Figure 5D Detection of variance shift is more fully described in the description of .

[0049] In general, other features can be extracted from the time series, including but not limited to fluctuations in mean, variance, skewness, kurtosis, median, mode, quantiles, etc. Information measures such as entropy or autoregressive coefficients can also be included. The disclosed embodiments are not intended to be limited to Figures 5A to 5D Specific examples are provided in .

[0050] In the illustrated embodiment, the method (200) generates an n-item tuple representing the detected event. Thus, using the three examples above, the method (200) will generate a three-item tuple representing whether the event is present in the time series. Thus, after executing step 208, the method (200) represents the event in the time series as y j (t i )=[e1(t1),e2(t1),e3(t1),e4(t1)] j , where e1, e2, e3, and e4 (respectively) represent the presence of a peak / drop, mean shift, trend change, or variance change event, Indicates the processing result (such as Figures 5A to 5D As described in), j represents the sensor, t i represents a given time, and y j represents the time series of sensor j.

[0051] In the above manner, the entire time series can therefore be defined as Y j (t) = [y j (t i ),j=1...k,i=1...m]=[[e1(t i ),e2(t i ),e3(t i )] j ,j=1…k,i=1…m], where k represents k sensors, t i …t m represents the time from t1 to t m Observation.

[0052] In step 210, the method (200) stores the extracted events.

[0053] As described above, the output of step 208 includes a data packet including a numerical value indicating the presence / absence of each type of event, a time component, and a sensor identifier (referred to as an event data structure). These components are stored in a database. In some embodiments, the event data structure may be stored in an RDBMS or other type of storage medium for future access. As described above, in some embodiments, the event data structure may be stored in a queue type data structure. In other embodiments, the event data structure may be stored in a database that supports a publish / subscribe (pub / sub) architecture pattern (e.g., REDIS), so that the event stream may be subscribed to by a downstream processing device.

[0054] In step 212, the method (200) builds or updates a graph based on the extracted events.

[0055] In the illustrated embodiment, the method (200) may first construct an event graph representing events stored in a database. In some embodiments, the graph represents each event type, sensor, and time as a node in the graph. In the graph, a given sensor node is connected to an event node or a time node, while event nodes and time nodes are only connected to sensor nodes (i.e., not to other time nodes or event nodes). In some embodiments, the method (200) stores the graph in a graph database. In other embodiments, the graph may be converted to a relational form and stored in an RDBMS. An example of constructing a graph from a set of events is shown in Figures 3A to 3F , which is incorporated herein by reference. In some embodiments, the event graph may include a separately stored graph. In other embodiments, the event graph may include a temporary graph (e.g., stored only in memory). In this embodiment, the event graph is used to guide the following and Figure 4 A diagram of the (thing) sensor described in the description.

[0056] From the event graph, the method (200) further synthesizes a sensor graph (also referred to as a sensor graph of things or simply GoT). In this graph, nodes contain only sensors, and the edges between nodes are weighted based on the connectivity of the event graph. In the illustrated embodiment, the sensor graph is constructed by analyzing the event graph to identify related sensors. In one embodiment, sensors are related if they experience an event at the same point in time. Thus, as an example, if two sensors experience a peak or a dip at the same time, the two sensors are related. In one embodiment, the weight of an edge of the sensor graph represents the number of time points at which the two sensors experienced the event at the same time.

[0057] Figures 3A to 3F An event graph is shown, and a method of constructing the same according to some embodiments of the present disclosure.

[0058] In the illustrated embodiment, Figures 3A to 3FThe graph in can be generated using a set of events generated using historical sensor data. Alternatively, in combination with the foregoing, the graph can be generated using real-time events extracted from a real-time stream of raw sensor data.

[0059] exist Figure 3A In the example, the first event is analyzed, including a peak or drop of sensor 1 occurring at time 1, and the event type (peak and drop), sensor ID (1) and time point (1) are used as nodes in the graph (300a) and the sensor node is connected to the event type node and the time point node through undirected edges. Figures 3A to 3F As shown, all edges are undirected, and this will not be repeated for the sake of brevity.

[0060] exist Figure 3B , also at time 1, a second event is processed, where sensor 3 detects a peak or a drop. In response, a node for sensor 3 is added to the graph (300b). The time 1 node is reused, and the sensor 3 node is connected to the time 1 node via an edge. In addition, the sensor 3 node is connected to the existing peak and drop nodes via an edge. In some embodiments, new, repeated peak and drop nodes may be added; however, as shown, event nodes (such as time nodes) may be reused.

[0061] exist Figure 3C In , a third event is received at time 2. This event comprises a mean shift type event detected by sensor 2. In response, a new component is added to the graph (300c), comprising a new sensor 2 node connected to the new time 2 node and the new mean shift node. This Figure 3C , graph (300c) consists of two separate components.

[0062] exist Figure 3D , a fourth event is received that occurs at time 2. Since the time 2 node has already been created, it is reused. However, a sensor 4 node is added, as well as a new trend change node corresponding to the detected trend change event. As will be discussed in more detail, Figure 3D The graph depicted in (300d) begins to show the correlation between sensors.

[0063] exist Figure 3E In the example, the fifth event is received at time 3. Figure 3DAs shown, a new Time 3 node is added to the graph (300e). In addition, the Sensor 1 node is connected to the new Time 3 node. However, since the Sensor 1 node is already connected to the Peak and Drop nodes, this connection is reused. As described above, in some embodiments, a new duplicate Peak and Drop node can be added and connected to the Sensor 1 node. Alternatively, in some embodiments, the weight of the edge between the Sensor 1 node and the Peak and Drop nodes can be increased by one.

[0064] exist Figure 3F , the sixth (and final) event is received at time 3. Similar to the fifth event, Figure 3F The events in represent a peak and a drop detected at sensor 3. In this case, all nodes exist (sensor 3, time 3, peak and drop), so no new nodes are created in the graph (300f). However, as mentioned above, event type nodes may be repeated again. Therefore, in Figure 3F New edges are added between the Sensor3 node and the Time3 node, and the edges between the Sensor3 node and the Peak and Drop nodes are reused (or in some embodiments, weighted).

[0065] As the following figure will describe, you can use Figure 3F Generate a sensor map from the graph in .

[0066] Figure 4 A sensor transaction graph according to some embodiments of the present disclosure is shown. In the illustrated embodiment, the sensor graph (400) is based on Figure 3F The event graph (300f) shown in FIG. Figure 4 As shown, the difference between the sensor graph (400) and the event graph (300f) is that all nodes of the sensor graph (400) include sensor nodes. In addition, the edges between sensor nodes are undirected and only include numerical weights. Therefore, when generating the sensor graph (400), data on event type and time are deleted.

[0067] In one embodiment, each sensor node in the event graph (300f) is included in the sensor graph (400). In addition, each node may be connected to each other node with a zero weight. Alternatively, each node in the sensor graph (400) may initially be unconnected and connected only to other nodes as described herein.

[0068] In the illustrated embodiment, for a given sensor, a map generator (e.g. Figure 1(described in) analyzes the event graph (300f) to determine the number of paths between a given sensor and another sensor that is at a distance of 2 and passes through a time node. To accomplish this, the graph builder may select the sensor nodes and identify the connected time nodes. The graph builder may then identify, for each time node, any sensor nodes that are connected to the time node. The resulting sensor nodes are then counted and the count is used as a weight between the given sensor and the other sensor. Thus, in graph (300f), sensor 1 is connected to sensor 3 via two time nodes (and vice versa), so an edge between sensor 1 and sensor 3 is added to the sensor graph (400) and the weight is set to 2. Additionally, sensors 2 and 4 are connected to each other via one time node (time 2). Therefore, an edge between sensors 2 and 4 is added to the sensor graph (400) and the weight is set to 1. As shown in the figure, all other edges are set to zero weight or omitted entirely.

[0069] Figure 5A is a flow chart illustrating a method for determining whether a peak or drop event occurs according to some embodiments of the present disclosure.

[0070] In step 502a, the method (500a) receives a residual component R(t) comprising the remaining portion of a given time series calculated as described above.

[0071] In step 504, the method (500a) compares the residual component with the peak threshold (Th spike ) and falling threshold (Th dip ) comparison. In one embodiment, these thresholds include static values ​​above and below the center point. For example, Th spike can be set to a positive value (e.g., +5), while Th dip Can be set to negative values ​​(e.g., -5). The specific values ​​of these thresholds are not intended to be limiting. In one embodiment, Th spike Must be greater than zero and Th dip Must be less than zero.

[0072] In step 506a, the method (500a) determines whether the value of R(t) exceeds a preconfigured peak threshold Th spike and marking the time (t) as a peak. In one embodiment, marking the time as a peak includes setting the event output (e.g., e1) value to one (1). This value is then passed to and combined with Figures 5B to 5D Combined with other output generated in Figure 2 This is more fully described in the description of step 208 of .

[0073] In step 510a, the method (500a) determines whether the value of R(t) is below a preconfigured falling threshold Th dip, and marks the time (t) as falling. In one embodiment, marking the time as falling includes setting the event output (e.g., e1) value to negative one (-1). This value is then passed to and combined with Figures 5B to 5D Combined with other output generated in Figure 2 This is more fully described in the description of step 208 of .

[0074] In step 508a, if the value of R(t) is not greater than Th spike and not less than Th dip , then the method (500a) bypasses the peak / fall event detection and sets the event output (e.g., e1) value to zero (0). Therefore, in the illustrated embodiment, the method (500a) generates an output value for the event denoted as e1 using the following rule:

[0075]

[0076] In some embodiments, the method (500a) may use output values ​​other than -1, 0, and 1. For example, the method (500a) may set the output value to represent the distance of the value of R(t) from a corresponding threshold.

[0077] In one embodiment, two thresholds (Th) are determined using an empirical formula of (t) spike And Th dip ), so that the event occurs only with a small probability (can be adjusted and can be set to 1%) occurs. This is equivalent to respecting the prior knowledge that interesting events are rare. In most cases, the time series is "normal" and does not contain much information. In some embodiments, the two thresholds (Th spike And Th dip ) has a threshold value (TH) greater than (>) 0 abs ), which can represent the prior knowledge of human experts. spike The settings are as follows:

[0078] TH spike =max(TH spike ,TH abs ),TH dip =(TH dip ,-TH abs )

[0079] Figure 5B is a flow chart illustrating a method for determining whether a mean shift event occurs according to some embodiments of the present disclosure.

[0080] In step 502b, the method (500b) defines a sliding window for the time series data set. In some embodiments, the sliding window includes a fixed duration during which the remaining steps analyze the time series data. The specific duration of this window is not limited and can be set according to the basic data or needs of the system.

[0081] In one embodiment, method (500b) calculates the mean of the trend component T(t) from two sliding windows and obtains the difference between them. In this embodiment, the left window consists of less recent data:

[0082] [T(tL-R+1),T(tL-R+2),…,T(tR)],

[0083] And the right window contains the latest data:

[0084] [T(t-R+1),T(t-R+2),…,T(t)],

[0085] Where L and R are the sizes of the left and right time windows respectively.

[0086] In this embodiment, μ L (t), μ R The value of (t) is calculated by calculating the mean of T(t) in the left and right time windows, and the difference Δμ(t) = μ K (t)-μ L (t) is used as the average sensor time series over the window (denoted as a(t)).

[0087] In step 504b, the method (500b) averages the values ​​of the time series data points that appear within the window (ie, the values ​​of Y(t)). In some embodiments, the average includes the average value of Y(t) during the above window.

[0088] In step 506b, the method (500b) adds the calculated average value of the given window (a(t)) to the mean increase threshold (Th incerase ) and the mean reduction threshold (Th decrease ) comparison. In one embodiment, these thresholds include static values ​​above and below the center point. For example, Th increase can be set to a positive value (e.g., +5), while Th decrease Can be set to negative values ​​(e.g., –5). The specific values ​​of these thresholds are not intended to be limiting. However, Th increase The value should be higher than Th decrease value to avoid Th decrease >a(t)>Th increase situation.

[0089] In step 508b, the method (500b) determines whether the value of a(t) exceeds a preconfigured mean increase threshold Th increase and marking the time (t) as a mean increase. In one embodiment, marking the time as a mean increase includes setting the event output (e.g., e2) value to one (1). This value is then passed to and combined with Figure 5A , 5C and other outputs generated in 5D, such as Figure 2 This is more fully described in the description of step 208 of .

[0090] In step 512b, the method (500b) determines whether the value of a(t) is below a preconfigured mean reduction threshold Th decrease , and marks the time (t) as a mean decrease in the sliding window. In one embodiment, marking the time as a mean decrease includes setting the event output (e.g., e2) value to negative one (-1). This value is then passed to and combined with Figure 5A , 5C and other outputs generated in 5D, such as Figure 2 This is more fully described in the description of step 208 of .

[0091] In step 510b, if the value of a(t) is not greater than Th increase and not less than Th decrease , then the method (500b) bypasses the average increase / decrease event detection and sets the event output (e.g., e2) value to zero (0). Therefore, in the illustrated embodiment, the method (500b) generates the output value of the event using the following rules, as shown in e2:

[0092]

[0093] In some embodiments, the method (500b) may use output values ​​other than -1, 0, and 1. For example, the method (500b) may set the output value to represent the distance of the value of a(t) from a corresponding threshold.

[0094] Figure 5C is a flow chart illustrating a method for determining whether a positive or negative trend change event has occurred according to some embodiments of the present disclosure.

[0095] In step 502c, the method (500c) receives a trend component T(t) comprising the trend portion of a given time series calculated as described above.

[0096] In step 504c, the method (500c) calculates the time difference between the current trend value (T(t)) and the previous trend value (T(t-1)), and then calculates the exponential moving average of ΔT(t)=T(t)-T(t-1), which is represented as D(t).

[0097] In step 506c, the method (500c) compares the trend component with the positive trend threshold (Th positive ) and the negative trend threshold (Th negative ) comparison. In one embodiment, these thresholds include static values ​​above and below the center point. For example, Th positive can be set to a positive value (e.g., +5), while Th negative Can be set to negative values ​​(e.g., –5). The specific values ​​of these thresholds are not intended to be limiting. However, Th positive The value should be higher than Th negative value to avoid Th negative >D(t)>Th positive situation.

[0098] In step 508c, the method (500c) determines whether the value of D(t) exceeds a preconfigured positive trend threshold Th positive and marking the time (t) as a positive value. In one embodiment, marking the time as positive includes setting the event output (e.g., e3) value to one (1). This value is then passed to and combined with Figure 5A , 5B and other outputs generated in 5D, such as Figure 2 This is more fully described in the description of step 208 of .

[0099] In step 512c, the method (500c) determines whether the value of D(t) is below a preconfigured negative trend threshold Th negative , and marks the time (t) as a negative value. In one embodiment, marking the time as a negative trend event includes setting the event output (e.g., e3) value to negative one (-1). This value is then passed to and combined with Figure 5A , 5B and other outputs generated in 5D, such as Figure 2 This is more fully described in the description of step 208 of .

[0100] In step 510c, if the value of D(t) is not greater than Th positive and not less than Th negative , then the method (500c) bypasses the positive / negative trend event detection and sets the event output (e.g., e3) value to zero (0). Therefore, in the illustrated embodiment, the method (500c) generates the output value of the event using the following rules, as shown in e3:

[0101]

[0102] In some embodiments, the method (500c) may use output values ​​other than -1, 0, and 1. For example, the method (500c) may set the output value to represent the distance of the value of D(t) from a corresponding threshold.

[0103] Figure 5D is a flow chart illustrating a method for determining whether a variance change event has occurred according to some embodiments of the present disclosure.

[0104] In step 502d, the method (500d) receives the calculated standard deviation σ of the left and right sliding window residual components R(t) of the time series data, respectively. L (t) and σ R (t). The left and right sliding windows can be calculated as described above and will not be described again here.

[0105] In step 504d, the method (500c) calculates the standard deviation difference between the right and left standard deviations over time, denoted as Δσ(t).

[0106] In step 506d, the method (500d) compares the standard deviation difference with the positive variance threshold (Th positive ) and the negative variance threshold (Th negative ) comparison. In one embodiment, these thresholds include static values ​​above and below the center point. For example, Th positive can be set to a positive value (e.g., +5), while Th negative Can be set to negative values ​​(e.g., –5). The specific values ​​of these thresholds are not intended to be limiting. However, Th positive The value should be higher than Th negative value to avoid the occurrence of Th negative >Δσ(t)>Th positive situation.

[0107] In step 508d, the method (500d) determines whether the value of Δσ(t) exceeds a preconfigured positive variance threshold Th positive , and marks the time (t) as positive. In one embodiment, marking the time as positive includes setting the event output (e.g., e4) value to one (1). This value is then passed to and combined with Figures 5A to 5C Combined with other output generated in Figure 2 This is more fully described in the description of step 208 of .

[0108] In step 512d, the method (500d) determines whether the value of Δσ(t) is below a preconfigured negative variance threshold Th negative , and marks the time (t) as a negative number. In one embodiment, marking the time as a negative variance event includes setting the event output (e.g., e4) value to negative one (-1). This value is then passed to and combined with Figures 5A to 5C Combined with other outputs generated in Figures 5A to 5C As shown in Figure 2 This is more fully described in the description of step 208 of .

[0109] In step 510d, if the value of Δσ(t) is not greater than Th positive and not less than Th negative , then the method (500d) bypasses the positive / negative variance event detection and sets the event output (e.g., e4) value to zero (0). Therefore, in the illustrated embodiment, the method (500d) generates the output value of the event using the following rules, as shown in e4:

[0110]

[0111] In some embodiments, the method (500d) may use output values ​​other than -1, 0, and 1. For example, the method (500d) may set the output value to represent the distance of the value of Δσ(t) from a corresponding threshold.

[0112] Figure 6 is a flow chart illustrating a method for analyzing GoT according to some embodiments of the present disclosure.

[0113] In step 602, the method (600) monitors sensor data. In some embodiments, the method (600) monitors raw sensor data output accessed via a communication bus, such as Figure 1 as described in the description.

[0114] In step 604, the method (600) determines whether an event has occurred.

[0115] In one embodiment, the method (600) is performed by utilizing Figure 2 , 5A , 5B and 5C to determine whether an event has occurred. In some embodiments, these events can be added to the event storage (such as Figure 2 ) and processed by the method (600). In the illustrated embodiment, the event detected in step 604 includes a sensor identifier (i) that indicates the sensor associated with the event. In some embodiments, steps 602 and 604 may be optional. In this embodiment, the method (600) directly Figure 1 The event extraction phase depicted in receives pre-processed events.

[0116] If the method (600) does not detect an event (e.g., no peak / drop, mean shift, trend change, or variance shift is detected), the method (600) continues to monitor the system for events. Alternatively, if the method (600) detects an event, the method (600) proceeds to step 608.

[0117] In step 606, the method (600) retrieves the sensor identifier (i) from the event detected in step 606. In some embodiments, the sensor identifier may include a globally unique identifier, an Internet Protocol (IP) address, or other unique identification information.

[0118] In the illustrated embodiment, the method (600) uses the sensor identifier to query a graph storage storing a sensor graph. That is, the method (600) uses the sensor identifier as a query key to retrieve data about the identified sensor from the graph storage. In one embodiment, the method (600) requests a first level connection list of nodes associated with the sensor identifier from the sensor graph.

[0119] In step 608, the method (600) determines whether the above query returns an empty set; that is, if the node associated with the sensor identifier does not have any first-level connections. Figure 4 As shown, a first level connection means that another node in the graph connected to the node associated with the sensor identifier has a distance of 1.

[0120] In step 610, if the sensor does not have any first level connections, the method (600) marks the event for further analysis. In this scenario, the method (600) cannot identify any sensor associated with the sensor identifier extracted in step 606. Therefore, when an event occurs (as identified in step 604), it cannot be associated using the sensor map and requires further intervention by, for example, an operator of the monitoring system.

[0121] In step 612, if the method (600) determines that one or more first-level connections exist, the method (600) retrieves the first-level connections from the sensor graph. In some embodiments, steps 610 and 612 may be combined. In the illustrated embodiment, the first-level connections include a set of nodes associated with other sensors. Each of these nodes may include one or more events (and times of these events) previously recorded for the corresponding sensor. In some embodiments, further analysis or details may be included for each event.

[0122] In step 614, the method (600) analyzes the identified sensor nodes to determine whether the same type of event is logged for any sensor in the first level connection list. In some embodiments, the method (600) limits the analysis in step 614 to concurrent events. In other embodiments, the method (600) may limit the analysis to events that occurred within a predefined time from the current event detected in step 604. Alternatively, in conjunction with the foregoing, the method (600) may analyze the shape of the trend of the time series data for each sensor to determine whether the trend is similar to the trend of the sensor identified by the sensor identifier.

[0123] In step 616, the method (600) sets a set of related sensors equal to those sensors having the same or similar events or trends identified in step 614. In the illustrated embodiment, the method (600) stores identifiers associated with the related sensors and then tags the related sensors for further analysis (previously described in conjunction with step 610).

[0124] In steps 618, 620, and 622, the method (600) continues to analyze the second level connections of the sensor identified in step 606. As used herein, the second level connection refers to the nodes in the sensor graph that have a distance of 2 from the sensor identified in step 606. Steps 618, 620, and 622 are similar to steps 608, 612, and 614, and the details of these steps are not repeated here.

[0125] In general, steps 608, 612, and 614 (as well as 618, 620, and 622) may be performed for any level of connectivity of the sensors identified in step 606. That is, in addition to analyzing first and second level connectivity (as shown), the method (600) may also analyze nodes that are three or more levels apart. In some embodiments, the number of levels to be analyzed may be preconfigured by the method (600). In other embodiments, the method (600) may continue searching each level until at least one relevant sensor node is found.

[0126] In the illustrated embodiment, the method (600) stops after identifying at least one relevant sensor in a given level. In other embodiments, even when relevant sensors are found at a given level, the method (600) may continue analyzing other levels. In this and other embodiments, the method (600) may rank the relevant sensors based on their distance from the sensor identified in step 606. For example, a relevant first level sensor (step 614) may be ranked higher than a relevant second level sensor (step 622).

[0127] Figure 7 FIG. 1 is a diagram showing device hardware for generating or analyzing GoT according to some embodiments of the present disclosure.

[0128] The device (700) may include Figure 1 The components shown in FIG. 7 may be more or less than those shown in FIG. However, the components shown are sufficient to disclose exemplary embodiments for implementing the present disclosure. The device (700) may represent, for example, the components discussed above with reference to FIG. Figure 1 Related equipment.

[0129] like Figure 7 As shown, the device (700) includes a processing unit (CPU) (702) that communicates with a mass storage (730) via a bus (724). The device (700) also includes one or more network interfaces (750), an audio interface (752), a display (754), a keyboard (756), an illuminator (758), an input / output interface (760), and a camera or other optical, thermal or electromagnetic sensor (762). As will be appreciated by those skilled in the art, the device (700) may include one camera / sensor (762) or multiple cameras / sensors (762).

[0130] The device (700) may optionally communicate with a base station (not shown), or directly with another computing device. The network interface (750) includes circuitry for coupling the device (700) to one or more networks, and is configured for use with one or more communication protocols and techniques. The network interface (750) is sometimes referred to as a transceiver, a transceiver device, or a network interface card (NIC).

[0131] The audio interface (752) is arranged to generate and receive audio signals, such as sounds of a human voice. For example, the audio interface (752) can be coupled to a speaker and a microphone (not shown) to enable remote communication with others and to generate audio confirmations for certain operations. The display (754) can be a liquid crystal display (LCD), a gas plasma, a light emitting diode (LED), or any other type of display used with a computing device. The display (754) can also include a touch-sensitive screen arranged to receive input from an object, such as a stylus or finger from a human hand.

[0132] The keyboard (756) may include any input device arranged to receive input from a user. For example, the keyboard (756) may include a button numeric pad or a keyboard. The keyboard (756) may also include command buttons associated with selecting and sending images. The illuminator (758) may provide a status indication and provide light. The illuminator (758) may remain activated for a specific period of time or in response to an event. For example, when the illuminator (758) is active, it may backlight the buttons on the keyboard (756) and remain illuminated when the device is powered on. In addition, when a specific action is performed (such as dialing another device), the illuminator (758) may backlight these buttons in various modes. The illuminator (758) may also cause a light source located in a transparent or translucent housing of the device to emit light in response to an action.

[0133] The device (700) also includes an input / output interface (760) for communicating with external devices. The input / output interface (760) can use one or more communication technologies, such as USB, infrared, Bluetooth TM wait.

[0134] The mass storage (730) includes RAM (732), ROM (724), and other storage devices. The mass storage (730) illustrates another example of a computer storage medium for storing information, such as computer readable instructions, data structures, program modules, or other data. The mass storage (730) stores the basic input / output system ("BIOS") (740) for controlling the low-level operation of the device (700). The mass storage may also store an operating system for controlling the operation of the device (700). It is appreciated that this component may include a general-purpose operating system, such as UNIX or LINUX. TM version, or a dedicated client communication operating system, such as Windows Client TM , or Symbian operating system. The operating system may include, or interface with, a Java virtual machine module that is capable of controlling hardware components and operating system operations through Java applications.

[0135] The RAM (732) stores and executes one or more application programs (734). In some embodiments, these application programs include software configured to perform one or more operations associated with the aforementioned figures. In some embodiments, the device (700) further includes a persistent storage (e.g., a hard disk, a solid state drive, etc.) for storing the application programs (34) prior to execution in the RAM (732).

[0136] For purposes of this disclosure, a module is a software, hardware or firmware (or combination thereof) system, process or function or component thereof that performs or facilitates the processes, features and functions described herein (with or without human-computer interaction or enhancement). A module may include submodules. The software components of a module may be stored on a computer-readable medium for execution by a processor. A module may be integrated into one or more servers, or loaded and executed by one or more servers. One or more modules may be combined into an engine or application.

[0137] Those skilled in the art will recognize that the methods and systems of the present disclosure can be implemented in a variety of ways and are therefore not limited by the foregoing exemplary embodiments and examples. In other words, functional elements performed by various combinations of single or multiple components, hardware and software or firmware, and individual functions can be distributed between software applications at the client level or server level or both. In this regard, any number of features of the different embodiments described herein can be combined into a single or multiple embodiments, and alternative embodiments with fewer or more than all of the features described herein are possible.

[0138] Functionality may also be distributed in whole or in part among multiple components, in ways now known or to become known. Thus, countless software / hardware / firmware combinations are possible in implementing the functions, features, interfaces, and preferences described herein. Furthermore, the scope of the present disclosure encompasses conventionally known ways of performing the described features and functions and interfaces, as well as those changes and modifications that may be made to the hardware or software or firmware components described herein, which changes and modifications may be understood by those skilled in the art.

[0139] In addition, the embodiments of the methods presented and described as flow charts in the present disclosure are provided by way of example to provide a more complete understanding of the technology. The disclosed methods are not limited to the operations and logical flows presented herein. Alternative embodiments are contemplated in which the order of the various operations is changed and sub-operations described as part of a larger operation are performed independently.

[0140] Although various embodiments are described for the purposes of this disclosure, such embodiments should not be considered to limit the teachings of this disclosure to these embodiments. Various changes and modifications may be made to the above elements and operations to obtain results within the scope of the systems and processes described in this disclosure.

Claims

1. A method for generating a thing graph, characterized in that include: Retrieve raw sensor data collected by multiple sensors; Identifying a plurality of events based on the raw sensor data includes: decomposing the raw sensor data into trend, seasonality, and residual components; identifying the plurality of events based on one or more of the trend, seasonality, and residual components; and A sensor graph is constructed based on the multiple events, the sensor graph having nodes representing the multiple sensors and a group of edges connecting the nodes, each edge in the group of edges being associated with a weight calculated based on the number of related events between two nodes connected by the edge.

2. The method according to claim 1, characterized in that Further including: The sensor graph is queried in response to a new event associated with a sensor of the plurality of sensors, the querying comprising identifying at least one related sensor connected to the sensor in the sensor graph.

3. The method according to claim 1, characterized in that Each event of the plurality of events comprises an event selected from the group consisting of a peak or drop event, a mean shift event, a variance shift, or a trend change event.

4. The method according to claim 3, characterized in that The identifying of the plurality of events includes identifying a peak or a falling event by comparing the residual component to a peak threshold and a falling threshold.

5. The method according to claim 3, characterized in that: The identifying of the plurality of events includes identifying a mean shift event by: averaging the raw sensor data over a sliding time window; and The average is compared to mean increase and mean decrease thresholds.

6. The method according to claim 3, characterized in that The identifying a plurality of events includes identifying a trend change event by comparing the trend component to a positive trend or negative trend threshold.

7. The method according to claim 1, characterized in that The constructing of the sensor graph comprises: constructing an event graph based on the plurality of events, the event graph storing event types, times, and sensor identifiers as nodes; and The sensor graph is constructed based on the event graph.

8. The method according to claim 2, characterized in that: The querying the sensor map comprises: detecting a second event for a second sensor; identifying, in the sensor map, first-order connected sensors for the second sensor, the first-order connected sensors comprising sensors having a distance of 1 from the second sensor in the sensor map; and The first-stage connection sensor is used as the correlation sensor.

9. A non-volatile computer-readable storage medium for tangibly storing computer program instructions executable by a computer processor, characterized in that: The computer program instructions define the following steps: Retrieve raw sensor data collected by multiple sensors; Identifying a plurality of events based on the raw sensor data includes: decomposing the raw sensor data into trend, seasonality, and residual components; identifying the plurality of events based on one or more of the trend, seasonality, and residual components; and A sensor graph is constructed based on the multiple events. The sensor graph has nodes representing the multiple sensors and a group of edges connecting the nodes. Each edge in the group of edges is associated with a weight calculated based on the number of related events between two nodes connected by the edge.

10. The non-volatile computer-readable storage medium according to claim 9, wherein: The instructions further define the step of querying the sensor map in response to a new event associated with a sensor of the plurality of sensors, the querying comprising identifying at least one related sensor connected to the sensor in the sensor map.

11. The non-volatile computer-readable storage medium according to claim 10, wherein: Each event of the plurality of events comprises an event selected from the group consisting of a peak or drop event, a mean shift event, a variance shift, or a trend change event.

12. The non-volatile computer-readable storage medium according to claim 10, wherein: The identifying of the plurality of events includes identifying a peak or a falling event by comparing the residual component to a peak threshold and a falling threshold.

13. The non-volatile computer-readable storage medium according to claim 10, wherein: The identifying of the plurality of events includes identifying a mean shift event by: averaging the raw sensor data over a sliding time window; and The average is compared to mean increase and mean decrease thresholds.

14. The non-volatile computer-readable storage medium according to claim 10, wherein: The identifying a plurality of events includes identifying a trend change event by comparing the trend component to a positive trend or negative trend threshold.

15. The non-volatile computer-readable storage medium according to claim 9, wherein: The constructing of the sensor graph comprises: constructing an event graph based on the plurality of events, the event graph storing event types, times, and sensor identifiers as nodes; and The sensor graph is constructed based on the event graph.

16. The non-volatile computer-readable storage medium according to claim 10, wherein: The query sensor graph comprises: detecting a second event for a second sensor; identifying, for the second sensor, a first level of connected sensors in the sensor map, the first level of connected sensors comprising sensors having a distance of 1 from the second sensor in the sensor map; and The first-stage connection sensor is used as the correlation sensor.

17. A device for generating a thing graph, characterized in that include: processor; as well as A storage medium for tangibly storing program logic thereon for execution by a processor, the stored program logic causing the processor to perform the following operations: Retrieve raw sensor data collected by multiple sensors, Identifying a plurality of events based on the raw sensor data includes: decomposing the raw sensor data into trend, seasonality, and residual components; identifying the plurality of events based on one or more of the trend, seasonality, and residual components; and A sensor graph is constructed based on the multiple events, the sensor graph having nodes representing the multiple sensors and a group of edges connecting the nodes, each edge in the group of edges being associated with a weight calculated based on the number of related events between two nodes connected by the edge.

18. The device according to claim 17, characterized in that The operations further include querying the sensor graph in response to a new event associated with a sensor of the plurality of sensors, the querying including identifying at least one related sensor connected to the sensor in the sensor graph.

19. The device according to claim 17, characterized in that The constructing of the sensor graph comprises: constructing an event graph based on the plurality of events, the event graph storing event types, times, and sensor identifiers as nodes; and The sensor graph is constructed based on the event graph.

20. The device according to claim 18, characterized in that Querying the sensor graph includes: detecting a second event for a second sensor; identifying, for the second sensor, a first level of connected sensors in the sensor map, the first level of connected sensors comprising sensors having a distance of 1 from the second sensor in the sensor map; and Use the first level connection sensor as the relevant sensor.