Sensor log data analysis method, device, equipment and medium

Abstract

The present application discloses a sensor log data analysis method, apparatus, device and medium, which relate to the field of data analysis, including: obtaining original log data of each sensor on a server within a preset time period, merging the original log data for the same sensor so that each sensor corresponds to one merged log data, thereby reducing the number of logs required for analysis; then determining the log data that meets the alarm conditions corresponding to the corresponding sensor from the merged log data of each sensor to obtain the merged log data of the first sensor, querying each associated sensor directly associated with any sensor in the first sensor from a sensor relationship graph, and when there is a target sensor whose merged log data meets the alarm conditions among the associated sensors, generating a fusion alarm for any sensor and the target sensor, and by fusing the alarms of the associated sensors, reducing the number of generated alarms and reducing the operation and maintenance costs of the server.

Description

Technical Field

[0001] The present application relates to the field of data analysis, and in particular to a sensor log data analysis method, apparatus, device, and medium. Background Art

[0002] As data centers expand and server clusters become more complex, collecting and analyzing server sensor logs has become a critical technology for ensuring server stability. Servers are typically deployed with multiple sensors that generate massive amounts of log data in real time. This log data is then used to monitor server health and predict potential failures, generating server alerts to alert operations personnel of potential server failures.

[0003] Server alarms are generated by capturing sensor logs on the server and analyzing them to generate alarms. In order to detect server failures as early as possible, the frequency of capturing sensor logs can be increased accordingly. However, the faster the capture frequency, the more alarms will be generated, which will greatly increase the server operation and maintenance costs. Summary of the Invention

[0004] The present application provides a sensor log data analysis method, apparatus, device and medium, which can perform alarm fusion on related sensors to significantly reduce the number of generated alarms, thereby at least solving the problem of high server operation and maintenance costs in related technologies.

[0005] This application provides a sensor log data analysis method, including:

[0006] Acquire the first raw log data of each sensor on the server within a preset time period, and merge the first raw log data of the same sensor to obtain merged log data of each sensor;

[0007] Building a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data that meets the alarm condition corresponding to the corresponding sensor in the merged log data of each sensor;

[0008] A current sensor relationship graph is queried to determine whether associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on correlations between sensors; and any sensor is any sensor in the first sensor;

[0009] If all associated sensors are found and the target sensor exists among the associated sensors, a fusion alarm for any sensor and the target sensor is generated based on the combined log data of any sensor and the target sensor; the combined log data of the target sensor is in the data list.

[0010] The present application also provides a sensor log data analysis device, comprising:

[0011] The log data merging module is used to obtain the first original log data of each sensor on the server within a preset time period, and merge the first original log data of the same sensor to obtain the merged log data of each sensor;

[0012] A data list building module, configured to build a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data that meets the alarm condition corresponding to the corresponding sensor in the merged log data of each sensor;

[0013] a relationship graph query module, configured to query the current sensor relationship graph to determine whether associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on the correlation between the sensors; and any sensor is any sensor in the first sensor;

[0014] The alarm fusion module is used to generate a fusion alarm for any sensor and the target sensor based on the merged log data of any sensor and the target sensor if all associated sensors are queried and the target sensor exists among the associated sensors; the merged log data of the target sensor is located in the data list.

[0015] The present application also provides an electronic device, comprising:

[0016] memory for storing computer programs;

[0017] A processor is configured to implement the steps of any of the above-mentioned sensor log data analysis methods when executing a computer program.

[0018] The present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned sensor log data analysis methods are implemented.

[0019] The present application also provides a computer program product, including a computer program, which implements the steps of any of the above-mentioned sensor log data analysis methods when executed by a processor.

[0020] Through this application, after obtaining the original log data of each sensor on the server within a preset time period, the original log data for the same sensor can be merged, so that each sensor corresponds to a merged log data. Compared with analyzing all the original log data of each sensor, this application can significantly reduce the number of logs required for analysis by analyzing the merged log data of each sensor, thereby reducing the consumption of computing resources and reducing the operation and maintenance costs of the server. Afterwards, this application determines the log data that meets the alarm conditions corresponding to the corresponding sensor from the merged log data of each sensor to obtain the merged log data of the first sensor, that is, the merged log data of the first sensor meets the alarm conditions corresponding to the first sensor. At this time, it is necessary to generate an alarm for the first sensor. This application can avoid generating alarms for sensors whose merged log data does not meet the alarm conditions by screening whether the merged log data meets the alarm conditions, thereby reducing the generation of unnecessary alarms. Furthermore, the present application queries the associated sensors that are directly associated with any sensor in the first sensor from the sensor relationship diagram, and when there is a target sensor in the associated sensors whose merged log data meets the alarm conditions, generates a fused alarm for any sensor and the target sensor associated with it based on the merged log data of any sensor and the target sensor associated with it. Compared with generating an alarm for each sensor in the first sensor, the present application can greatly reduce the number of generated alarms by fusing alarms for associated sensors, which not only reduces the operation and maintenance costs of the server, but also facilitates the operation and maintenance personnel to quickly identify the root cause sensor of the alarm event based on the fused alarms of the associated sensors. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0022] Figure 1 A flow chart of a sensor log data analysis method provided in an embodiment of the present application;

[0023] Figure 2 A specific sensor relationship diagram provided in an embodiment of the present application;

[0024] Figure 3 Another specific sensor relationship diagram provided in an embodiment of the present application;

[0025] Figure 4 A flow chart for constructing a sensor relationship diagram provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of an alarm probability provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of the structure of a sensor log data analysis device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0028] The following will be combined with the accompanying drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0029] It should be noted that, in the description of this application, the terms "comprises," "includes," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. The terms "first," "second," etc., in this application are used to distinguish similar objects, and are not used to describe a particular order or sequence.

[0030] In order to enable those skilled in the art to better understand the present application, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0031] Server alarms are generated by capturing and analyzing sensor logs on the server. To detect server failures as early as possible, the frequency of sensor log capture can be increased. However, a faster capture frequency also generates more alarms, significantly increasing server operation and maintenance costs. To address this, this application provides a sensor log data analysis method that significantly reduces the number of generated alarms by fusing alarms from related sensors, thereby at least addressing the high server operation and maintenance costs associated with related technologies.

[0032] The embodiment of the present application provides a sensor log data analysis method, combined with Figure 1 The execution process of the sensor log data analysis method shown is used to describe the method of the present application in detail.

[0033] Step S11: Acquire the first original log data of each sensor on the server within a preset time period, and merge the first original log data of the same sensor to obtain merged log data of each sensor.

[0034] The sensor log data analysis method proposed in the embodiment of the present application can be applied to the server itself or to a third-party monitoring platform that establishes a communication connection with the server. Among them, the third-party monitoring platform that establishes a communication connection with the server can still analyze the sensor log data and issue an alarm when the server fails and cannot operate normally.

[0035] It should be noted that the sensors on the server include, but are not limited to, temperature sensors, voltage sensors, fan speed sensors, and humidity sensors. The temperature sensor collects server temperature data, the voltage sensor collects server voltage data, the fan speed sensor collects server fan speed data, and the humidity sensor collects server humidity data. Furthermore, after collecting server data, each server sensor generates corresponding sensor log data.

[0036] Specifically, this application obtains the first raw log data for each sensor on the server within a preset time period from a time series database. The preset time period can be the historical time period closest to the current time, such as 10 minutes or 20 minutes, and the time series database can be InfluxDB. The first raw log data for the same sensor is then merged to obtain merged log data for each sensor. That is, each sensor corresponds to a merged log data, thereby reducing the number of log entries required for analysis, thereby reducing computing resource consumption and server operation and maintenance costs.

[0037] According to one specific embodiment, the present application periodically obtains first raw log data from each sensor on a server based on a preset time period, and stores the first raw log data in a time series database; then periodically obtains first raw log data from each sensor within a preset time period from the time series database, and merges the first raw log data for the same sensor to obtain merged log data for each sensor. The preset time period is no longer than the preset time period, for example, the preset time period can be 5 minutes.

[0038] Since there are multiple sensors deployed on the server, in order to improve the efficiency of obtaining sensor log data, this application can configure multiple acquisition modules so that one acquisition module corresponds to one sensor or one acquisition module corresponds to multiple sensors, thereby obtaining the first original log data from each sensor on the server in parallel through multiple acquisition modules.

[0039] The collection module can obtain log data from sensors through protocols such as HTTP (Hypertext Transfer Protocol), SNMP (Simple Network Management Protocol), and IPMI (Intelligent Platform Management Interface).

[0040] Before storing the first original log data of each sensor into the time series database, the present application may further perform format conversion on the first original log data through a data conversion module so that the converted first original log data has a unified data format.

[0041] Furthermore, before storing the first original log data of each sensor into the time series database, the present application can also determine whether the first original log data of each sensor meets the alarm conditions corresponding to the corresponding sensor, and configure the corresponding flag bit for the first original log data of each sensor based on the judgment result, and then store the first original log data of each sensor into the time series database.

[0042] Among them, if the first original log data 1 of sensor 1 meets the alarm condition corresponding to sensor 1, the first preset flag bit is configured for the first original log data 1 of sensor 1; if the first original log data 1 of sensor 1 does not meet the alarm condition corresponding to sensor 1, the second preset flag bit is configured for the first original log data 1 of sensor 1.

[0043] It should be noted that the flag bit is used to indicate whether the log data of the sensor meets the alarm condition corresponding to the sensor; and the first preset flag bit and the second preset flag bit use different characters, such as 1 and 0, A and B, + and -, etc., which are not specifically limited here.

[0044] It should also be noted that different sensors may correspond to different alarm conditions, and these conditions include, but are not limited to, threshold alarm conditions and frequency band alarm conditions. Specifically, a threshold alarm condition may be: if the sensor's log data is greater than and / or less than a threshold, an alarm is generated for the sensor; a frequency band alarm condition may be: if the frequency band corresponding to the sensor's log data is outside the sensor's normal frequency band, an alarm is generated for the sensor. Frequency bands include low frequency, medium frequency, and high frequency.

[0045] For example, the alarm condition corresponding to the temperature sensor may specifically be: if the log data of the temperature sensor is greater than a preset temperature threshold, it is determined that an alarm needs to be generated for the temperature sensor, wherein the preset temperature threshold may be set to 60 degrees.

[0046] The alarm conditions corresponding to each sensor on the server can be configured through the alarm condition configuration module, including adding alarm conditions, modifying alarm conditions, and deleting alarm conditions. If the user does not configure alarm conditions through the alarm condition configuration module, the default alarm conditions will be used.

[0047] Step S12: constructing a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data in the merged log data of each sensor that meets the alarm condition corresponding to the corresponding sensor.

[0048] After obtaining the merged log data of each sensor, it is necessary to determine the log data that meets the alarm conditions corresponding to the corresponding sensor from the merged log data of each sensor to obtain the merged log data of the first sensor. That is, the merged log data of the first sensor meets the alarm conditions corresponding to the first sensor. At this time, the first sensor is the sensor that needs to generate an alarm.

[0049] The process of determining the merged log data of the first sensor may specifically include: determining the preset alarm condition corresponding to each sensor based on the sensor type of each sensor; determining the first target log data that meets the preset alarm condition from the first original log data of each sensor; determining a second ratio between the number of first target log data and the number of first original log data of each sensor; if the second ratio is less than or equal to the second preset ratio, determining that the merged log data of each sensor does not meet the preset alarm condition; if the second ratio is greater than the second preset ratio, determining that the merged log data of each sensor meets the preset alarm condition, and determining the merged log data of each sensor as the merged log data of the first sensor.

[0050] Exemplarily, for each sensor in each sensor, taking the first original log data of each sensor as 5 and the second preset proportion as 60% as an example, if the first target log data that meet the preset alarm condition determined from the first original log data of each sensor is 2, then the second proportion is 40%. Since the second proportion is less than the second preset proportion, it is determined that the merged log data of each sensor does not meet the preset alarm condition; if the first target log data that meet the preset alarm condition determined from the first original log data of each sensor is 4, then the second proportion is 80%. Since the second proportion is greater than the second preset proportion, it is determined that the merged log data of each sensor meets the preset alarm condition.

[0051] The process of determining the merged log data of the first sensor may further include: for each sensor among the sensors, determining the first target log data whose flag bit is the first preset flag bit from the first original log data of each sensor; determining a second ratio between the number of first target log data items and the number of first original log data items of each sensor; if the second ratio is less than or equal to the second preset ratio, determining that the merged log data of each sensor does not meet the preset alarm condition; if the second ratio is greater than the second preset ratio, determining that the merged log data of each sensor meets the preset alarm condition, and determining the merged log data of each sensor as the merged log data of the first sensor.

[0052] Furthermore, after the merged log data of the first sensor is determined, a data list may be constructed based on the merged log data of the first sensor. That is, the merged log data stored in the data list is the merged log data that meets the alarm condition.

[0053] Step S13: query the current sensor relationship graph to determine whether associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on the correlation between the sensors; and any sensor is any sensor in the first sensor.

[0054] In an embodiment of the present application, for any sensor in the first sensor, the current sensor relationship graph is queried to determine whether the associated sensors directly associated with any sensor are queried from the current sensor relationship graph, that is, whether there are associated sensors directly associated with any sensor in the current sensor relationship graph.

[0055] The current sensor relationship graph is constructed based on the correlations between sensors. For example, sensors with correlations greater than a preset correlation are connected to form the current sensor relationship graph. The preset correlation can be a default correlation, such as 85, or a user-defined correlation, such as 90. Furthermore, the sensor relationship graph can be stored in a relational database, such as MySQL.

[0056] like Figure 2The sensor relationship diagram shows that there are direct, indirect, and unrelated relationships between the sensors in the diagram. A direct relationship refers to a direct connection between two sensors via a line, an indirect relationship refers to an indirect connection between two sensors via other sensors, and an unrelated relationship refers to no connection between two sensors. For example, sensors A and B, A and C, A and D, and B and E all have direct relationships, with correlations of 85, 91, 95, and 88, respectively. Sensor A and E have an indirect relationship, while sensors A and F, B and F, C and F, D and F, and E and F all have unrelated relationships.

[0057] In addition, this application records other sensors in addition to the first sensor as the second sensor. Before querying the current sensor relationship diagram, it is necessary to determine whether there is an alarm for the second sensor in the current list of unresolved alarms; if there is an alarm for the second sensor in the current list of unresolved alarms, the alarm for the second sensor is removed from the current list of unresolved alarms, and the alarm for the second sensor is added to the list of historical resolved alarms.

[0058] Since the merged log data of the first sensor is the log data in the merged log data of each sensor that meets the alarm condition corresponding to the corresponding sensor, and the second sensor is the sensors other than the first sensor, the merged log data of the second sensor is the log data in the merged log data of each sensor that does not meet the alarm condition corresponding to the corresponding sensor, that is, the merged log data of the second sensor does not meet the alarm condition corresponding to the second sensor. At this time, the second sensor is a sensor that does not need to generate an alarm.

[0059] Based on this, if there is an alarm for the second sensor in the current list of unresolved alarms, since the merged log data of the second sensor no longer meets the alarm conditions corresponding to the second sensor, that is, the alarm for the second sensor has been resolved or cleared at this time, the alarm for the second sensor can be removed from the current list of unresolved alarms, and the alarm for the second sensor can be added to the historical resolved alarm list.

[0060] Step S14: If all associated sensors are found and the target sensor exists among the associated sensors, a fusion alarm is generated for any sensor and the target sensor based on the combined log data of the sensor and the target sensor; the combined log data of the target sensor is in the data list.

[0061] In an embodiment of the present application, if the associated sensors directly associated with any sensor are queried from the current sensor relationship diagram, it is further determined whether the target sensor exists in the associated sensors; wherein the merged log data of the target sensor is located in the data list, or the merged log data of the target sensor meets the alarm condition corresponding to the target sensor.

[0062] If a target sensor exists among the associated sensors, it indicates that an alarm needs to be generated for the target sensor. At this time, a fused alarm can be generated for any sensor and its associated target sensors based on the merged log data of any sensor and the merged log data of the target sensor directly associated with any sensor. In this way, this application can significantly reduce the number of generated alarms by fusing alarms for associated sensors, which not only reduces the operation and maintenance costs of the server, but also makes it easier for operation and maintenance personnel to quickly identify the root cause sensor of the alarm event based on the fused alarms of the associated sensors.

[0063] Generally, there is a chain effect between associated sensors, that is, if the log data of a sensor in the associated sensors meets the alarm condition, the log data of other sensors in the associated sensors will most likely also meet the alarm condition.

[0064] For example, the temperature sensor and fan speed sensor are related sensors. If the temperature sensor's log data meets the alarm condition (the temperature is higher than the preset temperature), the fan speed will be increased accordingly to reduce the temperature. This may eventually cause the fan speed sensor's log data to also meet the alarm condition (the fan speed is higher than the preset speed).

[0065] Based on this, if the log data of only a few of the associated sensors meet the alarm conditions, it indicates that the alarm event at this time may be just sporadic and may recover on its own without the intervention of operation and maintenance personnel; if the log data of most of the associated sensors meet the alarm conditions, it indicates that the alarm event at this time is real and is likely to require intervention from operation and maintenance personnel to resolve.

[0066] Therefore, after generating a fusion alarm for any sensor and target sensor, this application needs to further determine the first ratio between the number of target sensors and the number of associated sensors; if the first ratio is greater than the first preset ratio, the current list of unresolved alarms is updated based on the fusion alarm; if the first ratio is not greater than the first preset ratio, the list of occasional alarms is updated based on the fusion alarm.

[0067] It is understandable that, since there is a target sensor among the associated sensors whose merged log data meets the alarm conditions, in order to avoid generating an alarm for each sensor, the present application performs alarm fusion on any sensor and the target sensor to generate a fused alarm for any sensor and the target sensor. The fused alarm for any sensor and the target sensor also needs to refer to the associated sensors. If the first ratio between the number of target sensors and the number of associated sensors is large, it indicates that there are many associated sensors that need to generate alarms. At this time, it can be determined that the fused alarm for any sensor and the target sensor is the current alarm to be resolved, and the current alarm list to be resolved can be updated based on the fused alarm for any sensor and the target sensor; if the first ratio between the number of target sensors and the number of associated sensors is small, it indicates that there are few associated sensors that need to generate alarms. At this time, it can be determined that the fused alarm for any sensor and the target sensor is an accidental alarm, and the accidental alarm list can be updated based on the fused alarm for any sensor and the target sensor. In this way, when there are fewer associated sensors in the merged log data that meet the alarm conditions, this application will indicate that the current fusion alarm is an occasional alarm, and update the occasional alarm list based on the fusion alarm to avoid adding unnecessary alarms to the current list of alarms to be resolved, thereby reducing the server operation and maintenance costs.

[0068] If the target sensor does not exist in the associated sensors and the number of associated sensors is one, it means that there are fewer sensors directly associated with any sensor. At this time, the alarm for any sensor can be generated without referring to the associated sensors, and the alarm for any sensor can be directly determined as the current alarm to be resolved. That is, this application can directly generate an alarm for any sensor based on the merged log data of any sensor, and update the current alarm list to be resolved based on the alarm for any sensor.

[0069] If the target sensor does not exist in the associated sensors and the number of associated sensors is more than one, it means that there are more sensors directly associated with any sensor. At this time, the alarm for any sensor needs to refer to the associated sensors, and the merged log data of the associated sensors does not meet the alarm conditions. In other words, there is no need to generate an alarm for the associated sensors, only for any sensor. Therefore, it can be determined that the alarm for any sensor is an accidental alarm, that is, the present application can directly generate an alarm for any sensor based on the merged log data of any sensor, and update the occasional alarm list based on the alarm for any sensor.

[0070] If no associated sensors directly associated with any sensor are found from the current sensor relationship diagram, it means that any sensor is not directly associated with other sensors. At this time, the alarm for any sensor can be directly determined as the current alarm to be resolved, that is, this application can directly generate an alarm for any sensor based on the merged log data of any sensor, and update the current alarm list to be resolved based on the alarm for any sensor.

[0071] by Figure 3 The sensor relationship diagram shown below is used as an example to illustrate the three possible associations: one-to-many, one-to-one, and no association. Furthermore, the correlations between sensors A and B, A and C, and A and D are 85, 91, and 95, respectively. The correlation between sensors E and F is 88, and the correlation between sensors G and H is 92.

[0072] For a one-to-many association, if the merged log data of sensor A meets the alarm condition corresponding to sensor A, the associated sensors B, C, and D directly associated with sensor A are queried from the current sensor relationship graph. In a first embodiment, if the associated sensors include target sensor B whose merged log data meets the alarm condition, the first percentage is determined to be 33.3%. Since the first percentage is less than the first preset percentage of 60%, after generating a fused alarm for sensors A and B, the occasional alarm list is updated based on the fused alarm. In a second embodiment, if the associated sensors include target sensors B and C whose merged log data meets the alarm condition, the first percentage is determined to be 66.7%. Since the first percentage is greater than the first preset percentage of 60%, after generating a fused alarm for sensors A, B, and C, the current pending alarm list is updated based on the fused alarm. In a third embodiment, if the associated sensors do not include target sensor B whose merged log data meets the alarm condition, after generating an alarm for sensor A, the occasional alarm list is updated based on the alarm for sensor A.

[0073] For a one-to-one association, if the merged log data of sensor E meets the alarm condition corresponding to sensor E, the associated sensor F directly associated with sensor E is queried from the current sensor relationship graph. In a first specific embodiment, if the associated sensors do not contain a target sensor whose merged log data meets the alarm condition, then after an alarm is generated for sensor E, the current list of pending alarms is updated based on the alarm for sensor E. In a second specific embodiment, if the associated sensors contain a target sensor F whose merged log data meets the alarm condition, the first percentage is determined to be 100%. Since the first percentage is greater than the first preset percentage of 60%, after generating a fused alarm for sensors E and F, the current list of pending alarms is updated based on the fused alarm.

[0074] For no association, if the merged log data of sensor I meets the alarm condition corresponding to sensor I, since there is no associated sensor directly associated with sensor I in the current sensor relationship diagram, after the alarm for sensor I is generated, the current list of unresolved alarms is updated based on the alarm for sensor I.

[0075] When updating the current pending alarm list or the sporadic alarm list based on a target alarm (which can be a fused alarm for multiple sensors or an alarm for a single sensor), the system checks whether any sub-alarms exist in the alarm list. If so, the sub-alarms in the alarm list are updated based on the target alarm. If not, the target alarm is added to the alarm list. The sensor corresponding to the target alarm includes the sensors corresponding to the sub-alarms. For example, if the target alarm is a fused alarm for sensors A, B, and C, and an alarm for sensor A exists in the alarm list, the alarm for sensor A is identified as a sub-alarm, and the sub-alarms in the alarm list are updated based on the target alarm, replacing the sub-alarm with the target alarm. This method of updating the alarm list avoids duplicate alarms from being added to the alarm list, effectively utilizing the alarm list and reducing the storage capacity required for the alarm list, thereby lowering server operation and maintenance costs.

[0076] Furthermore, in the embodiment of the present application, generated alarms can be sent to the user terminal via a preset communication method for the user to view; the preset communication method includes but is not limited to email, SMS, etc. Furthermore, in the embodiment of the present application, alarm sending records can be saved, where each alarm sending record can include information such as the communication method used for the current sending, the sending content, the sending time, and whether the sending was successful.

[0077] Furthermore, the application subject of the present application also has a display module, which can display the log data of the sensor, the sensor relationship diagram, and the alarm list (currently unresolved alarm list, historical resolved alarm list, and occasional alarm list).

[0078] Sensor log data can be displayed not only directly but also in the form of data charts. Specifically, a corresponding chart display method, such as a line chart, area chart, bar chart, or pie chart, is selected based on the characteristics of the sensor log data. The sensor log data is then processed using the chart display method and the resulting data chart is displayed.

[0079] Through this application, after obtaining the original log data of each sensor on the server within a preset time period, the original log data for the same sensor can be merged, so that each sensor corresponds to a merged log data. Compared with analyzing all the original log data of each sensor, this application can significantly reduce the number of logs required for analysis by analyzing the merged log data of each sensor, thereby reducing the consumption of computing resources and reducing the operation and maintenance costs of the server. Afterwards, this application determines the log data that meets the alarm conditions corresponding to the corresponding sensor from the merged log data of each sensor to obtain the merged log data of the first sensor, that is, the merged log data of the first sensor meets the alarm conditions corresponding to the first sensor. At this time, it is necessary to generate an alarm for the first sensor. This application can avoid generating alarms for sensors whose merged log data does not meet the alarm conditions by screening whether the merged log data meets the alarm conditions, thereby reducing the generation of unnecessary alarms. Furthermore, the present application queries the associated sensors that are directly associated with any sensor in the first sensor from the sensor relationship diagram, and when there is a target sensor in the associated sensors whose merged log data meets the alarm conditions, generates a fused alarm for any sensor and the target sensor associated with it based on the merged log data of any sensor and the target sensor associated with it. Compared with generating an alarm for each sensor in the first sensor, the present application can greatly reduce the number of generated alarms by fusing alarms for associated sensors, which not only reduces the operation and maintenance costs of the server, but also facilitates the operation and maintenance personnel to quickly identify the root cause sensor of the alarm event based on the fused alarms of the associated sensors.

[0080] The embodiment of the present application provides a process for constructing a current sensor relationship graph, combining Figure 4 The construction process of the current sensor relationship graph shown is a detailed description of the construction process of the sensor relationship graph of this application.

[0081] Step S21: Acquire the second original log data of each sensor in each historical time period.

[0082] In the embodiment of the present application, the second raw log data of each sensor in each historical time period can be periodically obtained from the time series database. For example, the second raw log data of each sensor in each historical time period can be obtained from the time series database at 24:00 every day. In this case, each historical time period can be selected from T1 (9:00-9:20), T2 (10:00-10:20), T3, T4, etc.

[0083] Step S22: Determine the log data that meets the alarm condition corresponding to the corresponding sensor from the second original log data of each sensor in any historical time period, so as to obtain the second target log data of each sensor in any historical time period.

[0084] For any historical time period in each historical time period and each sensor in each sensor, the log data that meets the alarm condition corresponding to each sensor is determined from the second original log data of each sensor in any historical time period to obtain the second target log data of each sensor in any historical time period. By analogy, the second target log data of each sensor in any historical time period can be obtained.

[0085] Step S23: Determine the alarm probability of each sensor in any historical time period based on the number of second target log data of each sensor in any historical time period.

[0086] After determining the second target log data of each sensor in any historical time period, for each of the sensors, the ratio between the number of second target log data and the number of second original log data of each sensor in any historical time period is determined to obtain the alarm probability of each sensor in any historical time period. By analogy, the alarm probability of each sensor in any historical time period can be obtained.

[0087] For example, if the number of second target log data items for each sensor in any historical time period is 3, and the number of second original log data items for each sensor in any historical time period is 5, then the ratio between the number of second target log data items and the number of second original log data items for each sensor in any historical time period is 0.6.

[0088] Step S24: Determine the correlation between any two sensors based on the alarm probability of each sensor in each historical time period.

[0089] The embodiment of the present application can obtain the alarm probability of each sensor in each historical time period based on steps S21 to S23, such as Figure 5 As shown in the figure, the alarm probabilities of sensors A, B, C, D, and E in historical time periods such as T1, T2, T3, and T4 are displayed.

[0090] Furthermore, the present application utilizes the Pearson correlation coefficient (Pearson correlation coefficient) and can determine the correlation between any two sensors based on the alarm probabilities of any two sensors in various historical time periods. The Pearson correlation coefficient is used to measure the correlation between two variables.

[0091] Specifically, first, based on the alarm probability of one of the two sensors in each historical time period, an average first alarm probability is determined. Based on the alarm probability of the other of the two sensors in each historical time period, an average second alarm probability is determined. Then, using the first and second average alarm probabilities, and based on the alarm probabilities of the two sensors in each historical time period, the correlation between the two sensors is determined. The specific calculation method involved is as follows:

[0092] ;

[0093] r represents the correlation, represents the average value of the first alarm probability, and n represents the total number of each historical time period. represents the alarm probability of one of the sensors in the i-th historical time period, represents the alarm probability of another sensor in the i-th historical time period, Indicates the average second alarm probability.

[0094] It should be noted that in order to make the correlation a hundredths value, the present application can multiply r by 100 so that the correlation value range is between 0 and 100. Moreover, the closer the correlation is to 100, the stronger the correlation between the two sensors.

[0095] Step S25: Connect sensors with correlation greater than a preset correlation among the sensors to construct an initial sensor relationship graph, and display the initial sensor relationship graph on a preset interactive interface.

[0096] After determining the correlation between any two sensors among the sensors, sensors with correlations greater than a preset correlation among the sensors are connected to construct an initial sensor relationship graph, and the initial sensor relationship graph is displayed on a preset interactive interface.

[0097] Furthermore, after determining the correlation between any two sensors among the sensors, if a correlation adjustment instruction is obtained, the present application adjusts the correlation between the corresponding two sensors based on the correlation adjustment instruction, and after completing the correlation adjustment, connects the sensors among the sensors whose correlation is greater than the preset correlation to construct an initial sensor relationship diagram.

[0098] Step S26: If a sensor connection adjustment instruction is obtained through the preset interactive interface, the connection relationship between the corresponding sensors in the initial sensor relationship diagram is adjusted based on the sensor connection adjustment instruction to obtain the current sensor relationship diagram; wherein the sensor connection adjustment instruction includes a sensor connection addition instruction and / or a sensor connection deletion instruction.

[0099] In the embodiment of the present application, if a sensor connection adjustment instruction is obtained through a preset interactive interface, the connection relationship between corresponding sensors in the initial sensor relationship diagram is adjusted based on the sensor connection adjustment instruction to obtain a current sensor relationship diagram, and the current sensor relationship diagram is saved in the relationship database. If a sensor connection adjustment instruction is not obtained, the initial sensor relationship diagram is determined as the current sensor relationship diagram, and the current sensor relationship diagram is saved in the relationship database.

[0100] Taking the sensor connection adjustment instruction as a sensor connection adding instruction as an example, based on the sensor connection adding instruction, the connection relationship between the corresponding sensors is added to the initial sensor relationship graph to obtain the current sensor relationship graph.

[0101] Taking the sensor connection adjustment instruction as a sensor connection deletion instruction as an example, based on the sensor connection deletion instruction, the connection relationship between corresponding sensors in the initial sensor relationship graph is deleted to obtain the current sensor relationship graph.

[0102] It can be seen that the present application constructs a sensor relationship graph, and after determining the first sensor whose merged log data meets the alarm condition, queries the associated sensors directly associated with any sensor in the first sensor from the sensor relationship graph, and when there is a target sensor whose merged log data meets the alarm condition in the associated sensors, based on the merged log data of any sensor and the target sensor associated with it, generates a fused alarm for any sensor and the target sensor associated with it. Compared with generating an alarm for each sensor in the first sensor, the present application can greatly reduce the number of generated alarms by fusing alarms for associated sensors, which not only reduces the operation and maintenance costs of the server, but also facilitates the operation and maintenance personnel to quickly identify the root cause sensor of the alarm event based on the fused alarms of associated sensors.

[0103] The embodiment of the present application also provides a sensor log data analysis device, combined with Figure 6 The structural diagram of the sensor log data analysis device shown is used to describe the device of the present application in detail.

[0104] The log data merging module 11 is configured to obtain first original log data of each sensor on the server within a preset time period, and merge the first original log data of the same sensor to obtain merged log data of each sensor.

[0105] The data list construction module 12 is configured to construct a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data in the merged log data of each sensor that meets the alarm condition corresponding to the corresponding sensor.

[0106] The relationship graph query module 13 is used to query the current sensor relationship graph to determine whether the associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on the correlation between the sensors; and any sensor is any sensor in the first sensor.

[0107] The alarm fusion module 14 is configured to generate a fusion alarm for any sensor and the target sensor based on the merged log data of any sensor and the target sensor if all associated sensors are found and the target sensor exists among the associated sensors; the merged log data of the target sensor is in the data list.

[0108] For the description of the features in the embodiment corresponding to the sensor log data analysis device, please refer to the relevant description of the embodiment corresponding to the sensor log data analysis method, which will not be repeated here.

[0109] Through this application, after obtaining the original log data of each sensor on the server within a preset time period, the original log data for the same sensor can be merged, so that each sensor corresponds to a merged log data. Compared with analyzing all the original log data of each sensor, this application can significantly reduce the number of logs required for analysis by analyzing the merged log data of each sensor, thereby reducing the consumption of computing resources and reducing the operation and maintenance costs of the server. Afterwards, this application determines the log data that meets the alarm conditions corresponding to the corresponding sensor from the merged log data of each sensor to obtain the merged log data of the first sensor, that is, the merged log data of the first sensor meets the alarm conditions corresponding to the first sensor. At this time, it is necessary to generate an alarm for the first sensor. This application can avoid generating alarms for sensors whose merged log data does not meet the alarm conditions by screening whether the merged log data meets the alarm conditions, thereby reducing the generation of unnecessary alarms. Furthermore, the present application queries the associated sensors that are directly associated with any sensor in the first sensor from the sensor relationship diagram, and when there is a target sensor in the associated sensors whose merged log data meets the alarm conditions, generates a fused alarm for any sensor and the target sensor associated with it based on the merged log data of any sensor and the target sensor associated with it. Compared with generating an alarm for each sensor in the first sensor, the present application can greatly reduce the number of generated alarms by fusing alarms for associated sensors, which not only reduces the operation and maintenance costs of the server, but also facilitates the operation and maintenance personnel to quickly identify the root cause sensor of the alarm event based on the fused alarms of the associated sensors.

[0110] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus the necessary general hardware platform, and of course it can also be implemented by hardware, but in many cases the former is a better implementation method.

[0111] An embodiment of the present application further provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to run the computer program to execute the steps in any of the above-mentioned sensor log data analysis method embodiments.

[0112] An embodiment of the present application further provides a computer-readable storage medium, in which a computer program is stored. The computer program is configured to execute the steps of any of the above-mentioned sensor log data analysis method embodiments when running.

[0113] In an exemplary embodiment, the computer-readable storage medium may include, but is not limited to, various media that can store computer programs, such as a USB flash drive, a read-only memory (ROM), a random access memory (RAM), a mobile hard disk, a magnetic disk, or an optical disk.

[0114] An embodiment of the present application further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the steps of any of the above-mentioned sensor log data analysis method embodiments are implemented.

[0115] An embodiment of the present application further provides another computer program product, including a non-volatile computer-readable storage medium, wherein the non-volatile computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of any of the above-mentioned sensor log data analysis method embodiments are implemented.

[0116] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the above description has generally described the components and steps of each example according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0117] The above is a detailed introduction to a sensor log data analysis method provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core ideas. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of the present application.

Claims

1. A sensor log data analysis method, characterized in that: include: Acquire the first raw log data of each sensor on the server within a preset time period, and merge the first raw log data of the same sensor to obtain merged log data of each sensor; Building a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data that meets the alarm condition corresponding to the corresponding sensor in the merged log data of each sensor; querying a current sensor relationship graph to determine whether associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on correlations between sensors; the any sensor is any one of the first sensors; If the associated sensors are found and the target sensor exists among the associated sensors, generating a fusion alarm for the any sensor and the target sensor based on the merged log data of the any sensor and the target sensor; The merged log data of the target sensor is located in the data list; The process of determining the merged log data of the first sensor includes: determining, based on a sensor type of each of the sensors, a preset alarm condition corresponding to each of the sensors; Determining first target log data that meets the preset alarm condition from the first original log data of each sensor; Determining a second ratio between the number of pieces of the first target log data and the number of pieces of the first original log data of each sensor; If the second proportion is greater than a second preset proportion, determining that the combined log data of each sensor meets the preset alarm condition, and determining the combined log data of each sensor as the combined log data of the first sensor; The process of constructing the current sensor relationship graph includes: Obtain the second original log data of each sensor in each historical time period; Determining, from the second original log data of each sensor within any of the historical time periods, log data that meets the alarm condition corresponding to the corresponding sensor, to obtain second target log data of each sensor within any of the historical time periods; Determining an alarm probability of each sensor in any of the historical time periods based on the number of second target log data of each sensor in any of the historical time periods; Determining the correlation between any two sensors based on the alarm probabilities of the sensors in each of the historical time periods; Connecting sensors with correlation greater than a preset correlation among the sensors to construct an initial sensor relationship graph, and displaying the initial sensor relationship graph on a preset interactive interface; If a sensor connection adjustment instruction is obtained through the preset interactive interface, adjusting the connection relationship between corresponding sensors in the initial sensor relationship diagram based on the sensor connection adjustment instruction to obtain the current sensor relationship diagram; The sensor connection adjustment instruction includes a sensor connection adding instruction and / or a sensor connection deleting instruction.

2. The sensor log data analysis method according to claim 1, characterized in that: Before querying the current sensor relationship graph, the method further includes: Determine whether there is an alarm for a second sensor in the current list of pending alarms; the second sensor is a sensor other than the first sensor among the sensors; If there is an alarm for the second sensor in the current unresolved alarm list, the alarm for the second sensor is removed from the current unresolved alarm list, and the alarm for the second sensor is added to a historical resolved alarm list.

3. The sensor log data analysis method according to claim 1, characterized in that: Also includes: If the associated sensors are not found, an alarm for any sensor is generated based on the merged log data of any sensor, and a current list of unresolved alarms is updated based on the alarm for any sensor.

4. The sensor log data analysis method according to claim 1, characterized in that: Also includes: If the target sensor does not exist in the associated sensors and the number of the associated sensors is one, generating an alarm for the any sensor based on the merged log data of the any sensor, and updating a current unresolved alarm list based on the alarm for the any sensor; If the target sensor does not exist in the associated sensors and the number of the associated sensors is more than one, an alarm for the any sensor is generated based on the merged log data of the any sensor, and an occasional alarm list is updated based on the alarm for the any sensor.

5. The sensor log data analysis method according to claim 1, characterized in that: After generating a fusion alarm for the any one sensor and the target sensor based on the combined log data of the any one sensor and the target sensor, the method further includes: determining a first ratio between the number of the target sensors and the number of the associated sensors; If the first proportion is greater than a first preset proportion, updating the current list of unresolved alarms based on the fused alarm; If the first proportion is not greater than a first preset proportion, the occasional alarm list is updated based on the fused alarm.

6. A sensor log data analysis device, characterized in that: include: The log data merging module is used to obtain the first original log data of each sensor on the server within a preset time period, and merge the first original log data of the same sensor to obtain the merged log data of each sensor; A data list building module, configured to build a data list based on the merged log data of the first sensor; the merged log data of the first sensor is the log data that meets the alarm condition corresponding to the corresponding sensor in the merged log data of each sensor; a relationship graph query module, configured to query a current sensor relationship graph to determine whether associated sensors directly associated with any sensor are found; the current sensor relationship graph is a relationship graph constructed based on correlations between sensors; and the any sensor is any one of the first sensors; an alarm fusion module, configured to generate a fusion alarm for any one of the sensors and the target sensor based on the merged log data of the any one of the sensors and the target sensor if the associated sensors are found and the target sensor exists among the associated sensors; The merged log data of the target sensor is located in the data list; The process of determining the merged log data of the first sensor includes: determining, based on a sensor type of each of the sensors, a preset alarm condition corresponding to each of the sensors; Determining first target log data that meets the preset alarm condition from the first original log data of each sensor; Determining a second ratio between the number of pieces of the first target log data and the number of pieces of the first original log data of each sensor; If the second proportion is greater than a second preset proportion, determining that the combined log data of each sensor meets the preset alarm condition, and determining the combined log data of each sensor as the combined log data of the first sensor; The process of constructing the current sensor relationship graph includes: Obtain the second original log data of each sensor in each historical time period; Determining, from the second original log data of each sensor within any of the historical time periods, log data that meets the alarm condition corresponding to the corresponding sensor, to obtain second target log data of each sensor within any of the historical time periods; Determining an alarm probability of each sensor in any of the historical time periods based on the number of second target log data of each sensor in any of the historical time periods; Determining the correlation between any two sensors based on the alarm probabilities of the sensors in each of the historical time periods; Connecting sensors with correlation greater than a preset correlation among the sensors to construct an initial sensor relationship graph, and displaying the initial sensor relationship graph on a preset interactive interface; If a sensor connection adjustment instruction is obtained through the preset interactive interface, adjusting the connection relationship between corresponding sensors in the initial sensor relationship diagram based on the sensor connection adjustment instruction to obtain the current sensor relationship diagram; The sensor connection adjustment instruction includes a sensor connection adding instruction and / or a sensor connection deleting instruction.

7. An electronic device, characterized in that: include: memory for storing computer programs; A processor, configured to implement the steps of the sensor log data analysis method according to any one of claims 1 to 5 when executing the computer program.

8. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the sensor log data analysis method according to any one of claims 1 to 5 are implemented.

Images