Intelligent monitoring method and system for multi-source data analysis

Abstract

The application discloses an intelligent monitoring method and system for multi-source data analysis, and belongs to the field of intelligent monitoring, which comprises the following steps: collecting data from multiple data sources according to a first monitoring object to obtain a multi-source data set; collecting historical data from the multiple data sources and performing analysis to obtain multiple abnormal analysis result sets; setting multi-source monitoring coding rules based on the multiple abnormal analysis result sets; performing multi-source data coding on the multi-source data set through the multi-source monitoring coding rules to obtain a data coding result; determining the number of abnormal data sources and obtaining abnormal values of the abnormal data sources; and performing data abnormality early warning according to the number of abnormal data sources and the abnormal values. The application solves the technical problems of imperfect monitoring data collection, untimely and inaccurate data abnormality early warning in the prior art, achieves the technical effects of collecting and integrating multiple data sources, and improves the accuracy of monitoring and the processing efficiency of abnormality early warning.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of intelligent monitoring, in particular to an intelligent monitoring method and system for multi-source data analysis. BACKGROUND

[0002] Real-time monitoring technology has been widely used in various industries, which can help enterprises and organizations to master the key business process, find potential problems and deal with them in time. However, the traditional monitoring system usually relies on a single data source, which is difficult to collect information from different data sources and comprehensively judge the overall state of the monitoring object. This easily causes one-sidedness of monitoring, misses key indicators, and leads to problems such as untimely and inaccurate abnormal warning of monitoring data. SUMMARY

[0003] The present application provides an intelligent monitoring method and system for multi-source data analysis, aiming to solve the technical problems of imperfect monitoring data collection, untimely and inaccurate abnormal warning of data in the prior art.

[0004] In view of the above problems, the present application provides an intelligent monitoring method and system for multi-source data analysis.

[0005] The first aspect of the present application provides an intelligent monitoring method for multi-source data analysis, which comprises: collecting data from multiple data sources according to a first monitoring object to obtain a multi-source data set; collecting historical data from the multiple data sources, performing data anomaly analysis according to the historical data collection results to obtain a plurality of abnormal analysis result sets; setting a multi-source monitoring coding rule based on the plurality of abnormal analysis result sets; performing multi-source data coding on the multi-source data set through the multi-source monitoring coding rule to obtain a data coding result; determining the number of abnormal data sources according to the data coding result and obtaining the abnormal value of the abnormal data sources; and performing data anomaly warning according to the number of abnormal data sources and the abnormal value.

[0006] Another aspect of the present application provides an intelligent monitoring system for multi-source data analysis, which comprises: a multi-source data set module for collecting data from multiple data sources according to a first monitoring object to obtain a multi-source data set; an abnormal analysis result module for collecting historical data from the multiple data sources, performing data anomaly analysis according to the historical data collection results to obtain a plurality of abnormal analysis result sets; a coding rule setting module for setting a multi-source monitoring coding rule based on the plurality of abnormal analysis result sets; a data coding result module for performing multi-source data coding on the multi-source data set through the multi-source monitoring coding rule to obtain a data coding result; a data source abnormal value module for determining the number of abnormal data sources according to the data coding result and obtaining the abnormal value of the abnormal data sources; and a data anomaly warning module for performing data anomaly warning according to the number of abnormal data sources and the abnormal value.

[0007] One or more technical solutions provided in the present application have at least the following technical effects or advantages:

[0008] 1. The technical solution of collecting data from multiple data sources by monitoring objects, obtaining a multi-source data set, obtaining historical data collection results according to the multi-source data set for data anomaly analysis, obtaining a plurality of anomaly analysis result sets, formulating corresponding coding rules for anomaly analysis results for early warning processing, solves the technical problem of imperfect monitoring data collection in the prior art monitoring process, and achieves the technical effect of collecting and integrating multiple data sources.

[0009] 2. The technical solution of setting multi-source monitoring coding rules based on a plurality of anomaly analysis result sets, performing multi-source data coding through the multi-source monitoring coding rules, obtaining data coding results and determining the number of abnormal data sources, obtaining the abnormal value of the abnormal data source, and performing data anomaly early warning through the number of abnormal data sources and the abnormal value, solves the technical problems of the prior art that the anomaly early warning of the monitoring data is not timely and the accuracy is not high, and achieves the technical effects of improving the accuracy of monitoring and the processing efficiency of anomaly early warning.

[0010] The above description is only a summary of the technical solutions of the present application. In order to more clearly understand the technical means of the present application, the present application can be implemented according to the content of the specification, and in order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the following specific embodiments of the present application are described. BRIEF DESCRIPTION OF DRAWINGS

[0011] Figure 1 A possible flowchart of an intelligent monitoring method for multi-source data analysis is provided for the embodiments of the present application;

[0012] Figure 2 A possible flowchart of setting multi-source monitoring coding rules in an intelligent monitoring method for multi-source data analysis is provided for the embodiments of the present application;

[0013] Figure 3 A possible flowchart of performing data anomaly early warning in an intelligent monitoring method for multi-source data analysis is provided for the embodiments of the present application;

[0014] Figure 4 A possible structure diagram of an intelligent monitoring system for multi-source data analysis is provided for the embodiments of the present application.

[0015] Explanation of reference signs: multi-source data set module 11, anomaly analysis result module 12, coding rule setting module 13, data coding result module 14, data source abnormal value module 15, data anomaly early warning module 16. DETAILED DESCRIPTION

[0016] The overall concept of the technical solution provided in this application is as follows:

[0017] This application provides an intelligent monitoring method and system for multi-source data analysis. It collects data from multiple data sources based on the monitored object to obtain a multi-source dataset. Then, it collects historical data from these multiple data sources and performs anomaly analysis on the historical data to obtain multiple sets of anomaly analysis results. Next, it sets multi-source monitoring coding rules based on these sets and encodes the multi-source data using these rules to obtain the encoded data. Finally, it provides data anomaly warnings based on the number of abnormal data sources and outliers, comprehensively monitoring multi-source data, promptly detecting data anomalies, and providing accurate early warning information.

[0018] After introducing the basic principles of this application, various non-limiting embodiments of this application will be described in detail below with reference to the accompanying drawings.

[0019] Example 1

[0020] like Figure 1 As shown in the figure, this application provides an intelligent monitoring method for multi-source data analysis, the method including:

[0021] Step S100: Collect data from multiple data sources based on the first monitored object to obtain a multi-source dataset;

[0022] Specifically, the primary monitoring object refers to the specific object to be monitored, which can be network equipment, industrial equipment, business systems, etc. Multiple data sources refer to various different data sources related to the primary monitoring object, including equipment logs, operation and maintenance systems, sensors, databases, etc., used to reflect different status data of the monitored object. Multi-source datasets are obtained through methods such as database queries, log parsing, and API calls. A multi-source dataset is a dataset composed of collected data, containing data from different data sources related to the same monitored object. This dataset includes both structured and unstructured data, all related to the status monitoring of the monitored object.

[0023] By collecting data from multiple data sources for the first monitored object, the resulting multi-source dataset provides basic data support for subsequent anomaly detection, encoding, and decoding, thereby determining the overall status of the monitored object and achieving accurate monitoring and early warning.

[0024] Step S200: Collect historical data based on the multiple data sources, perform data anomaly analysis based on the historical data collection results, and obtain multiple anomaly analysis result sets;

[0025] Specifically, the historical data collection refers to collecting data related to the monitoring object in a historical time period according to multiple data sources. According to the multiple data sources, the historical data collection needs to set the historical monitoring content to be collected, pre-set the specific data content and format collected by each data source, then determine the historical data time range, determine the historical data time range according to the business characteristics, key monitoring indicators and the like of the monitoring object, for example, the network equipment collects historical data for more than 6 months, the key business system for 1-2 years and the like. For each data source, the data is repeatedly collected in the determined historical time range with a predetermined collection period. Then, the collected historical data is cleaned to eliminate invalid and abnormal data, and the data collected by different data sources is integrated and converted into the same data format and structure.

[0026] According to the characteristics of the monitoring object, the abnormal detection data is selected, for example, statistical algorithm, machine learning algorithm and the like, the historical data is used to train the abnormal detection algorithm, the normal data model of each data source is established, and the model reflects the data distribution characteristics of the monitoring object of the data source in the normal state. For each data source, an abnormal judgment rule is formulated, such as threshold limit, lower limit of confidence and the like, to judge whether there is an abnormality in the collected multi-source data set. The normal data model is used to detect each data source, the abnormal analysis result of each data source is obtained, and then a plurality of abnormal analysis result sets are obtained.

[0027] The multiple data sources are analyzed for abnormality, and a plurality of abnormal analysis result sets are formed, which provide a basis for subsequent judgment of the state of the monitoring object and setting of the multi-source monitoring rule.

[0028] Step S300: setting a multi-source monitoring coding rule based on the plurality of abnormal analysis result sets;

[0029] Specifically, the multi-source monitoring coding rule refers to the data monitoring rule set for multiple related data sources, which is used to code the collected data to judge the state of the monitoring object. First, the correlation between the abnormal analysis results of different data sources is analyzed to determine which results have an important influence on the state of the monitoring object, and these data sources should be used as key monitoring elements, and the results of other data sources should be used as auxiliary judgment elements. Then, according to the determined key monitoring elements, specific monitoring indicators and parameters corresponding to the key monitoring elements are selected as monitoring elements, such as device temperature, network flow, system response time and the like, which are used to construct the coding rule. Then, according to the monitoring elements, the value range of the elements under different states of the monitoring object is set, for example, the temperature is 80-90 degrees in the normal state, 90-95 degrees in the warning state, and more than 95 degrees in the alarm state.

[0030] The collected data and monitoring elements are clustered and divided according to historical data by using a clustering algorithm, data blocks representing different states of the monitored object are obtained, and the centers of the data blocks are used as encoding cluster centers. According to the state range and the cluster center, an encoding rule is created, which defines which encoding corresponds to the monitoring element in different value ranges, for example, temperature 80-90 degrees corresponds to encoding 0, indicating a normal state. According to the monitoring effect, the monitoring elements, state ranges, encoding rules, etc. are continuously optimized and adjusted to improve the accuracy of encoding and decoding, and precise monitoring is realized.

[0031] Based on the abnormality analysis, a multi-source monitoring encoding rule is created, which can be used as a reference for subsequent encoding of collected data, and can effectively determine the state of the monitored object and realize precise monitoring.

[0032] Step S400: According to the multi-source data set, the multi-source data is encoded by the multi-source monitoring encoding rule to obtain a data encoding result;

[0033] Specifically, after generating the monitoring encoding rule of each data source according to the historical data and the abnormality analysis result, setting the encoding rule for the data of the multi-source data set, reading the multi-source monitoring encoding rule corresponding to each data source, traversing each data in the multi-source data set, determining which value range the data value is in the encoding rule, and encoding the corresponding data source data into the corresponding value, the encoding values obtained from different data sources are summarized to form the final data encoding result.

[0034] Based on the accurate encoding rule prepared in advance, automatic data encoding and state determination are realized. Compared with directly using raw data for judgment, the encoding method can filter out the influence of some noise data and improve the accuracy of state determination of the monitored object.

[0035] Step S500: According to the data encoding result, determine the number of abnormal data sources, and obtain the abnormal value of the abnormal data source;

[0036] Specifically, the data encoding result and the original multi-source data set are obtained, wherein the data encoding result contains the data encoding values of different data sources, representing the data state of each data source; the original data set contains the specific data information detected by different data sources. Traverse the encoding values obtained from each data source in the data encoding result, if the encoding value is "abnormal", record the data source number, and count the number of records to obtain the number of abnormal data sources.

[0037] According to the recorded abnormal data source number, the data information detected by the corresponding data source in the original data set is extracted to form the abnormal value of each abnormal data source. Then, the number of abnormal data sources and the abnormal value of each abnormal data source are recorded, and the number of abnormal data sources reflects the distribution degree of abnormal phenomenon, and the abnormal value of each abnormal data source represents the specific abnormal degree. The recorded number of abnormal data sources and the corresponding abnormal value of each abnormal data source are output.

[0038] By determining the number of abnormal data sources according to the data encoding result, and obtaining the abnormal value of the abnormal data source, the normal data source is filtered out, the abnormal source is focused and the specific abnormal information is obtained, which provides an important basis for early warning classification and response decision.

[0039] Step S600: data anomaly early warning is performed according to the number of abnormal data sources and the abnormal value.

[0040] Specifically, the number of abnormal data sources and the abnormal value of each source are input, it is judged whether the abnormal values are the same, if yes, the early warning level is the value, if not, the standard deviation of the abnormal value is calculated, it is judged whether the preset threshold is met, if yes, the early warning level is the mean value, otherwise, the median of the abnormal value is calculated, it is judged whether the number of abnormal values in the abnormal data source is greater than the median of the abnormal value, if more than half of the total number of abnormal sources, the early warning level is the maximum value, otherwise, the median. The early warning level is output, and early warning information is generated and sent.

[0041] By analyzing the distribution of abnormal data sources and the dispersion degree of abnormal values, an accurate early warning level is set. The technical effects of collecting and integrating multiple data sources, and improving the accuracy of monitoring and the processing efficiency of abnormal early warning are achieved.

[0042] Further, the embodiments of the present application also include:

[0043] Step S310: a first abnormal analysis result set corresponding to a first data source is obtained according to the plurality of abnormal analysis result sets;

[0044] Step S320: a first abnormal analysis result set is analyzed according to the first abnormal analysis result set to analyze the abnormal influence degree set of the abnormal data corresponding to the first data source on the first monitoring object;

[0045] Step S330: in this way, a plurality of abnormal influence degree sets of the abnormal data corresponding to the plurality of data sources on the first monitoring object are obtained;

[0046] Step S340: same level abnormal degree data is obtained by performing same level abnormal degree statistical analysis according to the plurality of abnormal influence degree sets.

[0047] Step S350: the multi-source monitoring coding rule is set according to the same level abnormal degree data.

[0048] Specifically, the first data source is one of a plurality of data sources collected for the monitoring object, and an abnormal analysis result set corresponding to the data source is obtained as the first abnormal analysis result set. A corresponding relationship between an abnormal type of different data sources and a state of the monitoring object is analyzed, and an abnormal influence degree judgment rule is formulated, which can be set as general influence, moderate influence, important influence, and great influence, for example, network source flow abnormality has general influence on the state of the monitoring object, and database response time abnormality has important influence on the state of the monitoring object. Using these rules, the source abnormal analysis result is judged one by one, and a plurality of abnormal influence degree sets are obtained.

[0049] The same level abnormality degree statistics refers to statistics of the abnormal influence degree set of different data sources, to obtain the statistical number of abnormal data of each data source under a certain abnormality level, and to form the same level abnormality degree data. The abnormal influence degree set of different data sources is traversed and counted, and the result is stored using a structure such as Map<String, Integer>, wherein the key is the abnormal level, and the value is the abnormal number of the corresponding data source. This storage result is the data of the same level abnormality degree. According to the statistical data of the influence degree of the data source abnormal data on the monitoring object, the corresponding rules under different influence degrees are automatically learned through a machine learning model, for example, when there are 3 “general influences” or 2 “moderate abnormalities” in the same level abnormality degree data, the model outputs code 1 to represent a first level abnormality.

[0050] By analyzing the abnormal influence degree, the same level abnormality degree data is obtained, and the multi-source monitoring coding rule is set to avoid false positives caused by abnormality of a single data source or missed reports caused by unclear states of several data sources, so that the state judgment of the monitoring object has higher accuracy, and provides a basis for realizing reliable monitoring and timely warning.

[0051] Further, as shown in Figure 2 embodiments of the present application further include:

[0052] Step S351: setting a normal coding value of a normal value, wherein the normal coding values corresponding to the plurality of data sources are the same, and a normal coding rule is obtained;

[0053] Step S352: setting the same abnormal coding value for the same level abnormality data corresponding to the plurality of data sources according to the same level abnormality data, and obtaining an abnormal offset coding rule;

[0054] Step S353: generating the multi-source monitoring coding rule according to the normal coding rule and the abnormal offset coding rule.

[0055] Specifically, the normal encoding value refers to encoding the data of different data sources in a normal state into the same value, such as encoding 0 representing normal. The normal encoding rule can specify the encoding values corresponding to the normal data of different data sources, such as source A normal data encoding 0 and source B normal data encoding 0. The abnormal encoding value refers to encoding the data of the same abnormal level of different data sources into the same value, such as encoding 1 representing a first-level abnormality. The abnormal encoding rule can specify the encoding values corresponding to the data of the same abnormal level of different data sources, such as source A and source B first-level abnormal data encoding 1. According to the normal encoding rule and the abnormal encoding rule, the historical data and the abnormal analysis results of different data sources are traversed, the uniform encoding value of the normal data is set to 0, the uniform encoding value of the data of the same abnormal level is set, such as the first-level abnormality being 1, and a complete encoding rule from normal to severe is set for each data source.

[0056] By setting a uniform encoding rule for multiple data sources, the same encoding corresponds to the same state regardless of the type and characteristics of the data sources, so that the overall state of the monitored object can be simply and directly judged from the encoding value, such as all sources encoding 0 being normal and some sources encoding 1 being a first-level abnormality. Through the simple and direct mapping relationship, the processing efficiency of the monitoring on the abnormality is further improved.

[0057] Further, the embodiments of the present application also include:

[0058] Step S410: obtaining the normal data range of multiple data sources based on the normal encoding rule;

[0059] Step S420: filtering out data meeting the normal data range from the multi-source data set for normal value encoding;

[0060] Step S430: obtaining the abnormal data range of multiple abnormal encoding values corresponding to the multiple data sources respectively based on the abnormal bias encoding rule

[0061] Step S440: filtering out data meeting the abnormal data range from the multi-source data set for abnormal value encoding;

[0062] Step S450: composing the data encoding result with the normal value encoding result and the abnormal value encoding result.

[0063] Specifically, the normal encoding rule corresponding to the normal data of different data sources is set, such as source A normal encoding 0 and source B normal encoding 0. The value range of the normal data of the data source determined according to the normal encoding rule is determined, such as source A temperature 80-90 degrees and source B flow 500-800 MB. The abnormal data encoding rule corresponding to the abnormal data of different data sources is set, such as source A and source B first-level abnormal data encoding 1. The value range of the abnormal data of the data source determined according to the abnormal encoding rule is determined, such as source A temperature > 95 degrees and source B flow > 1000 MB.

[0064] The normal encoding rule and the abnormal encoding rule are analyzed to determine the normal data range and the abnormal data range of each data source, and the data collected by the multi-source data set is read to determine whether each data is within the normal / abnormal range of a data source, and if so, the corresponding encoding is performed to obtain the final encoding result.

[0065] The collected data is encoded, classified and judged based on the pre-set encoding rule, the data within the normal data range is encoded as normal, and the data within the abnormal data range is encoded as abnormal, and the final data encoding result reflects the state of the monitored object, so that the encoding result is more intuitive, which helps to improve the efficiency of the monitoring system in handling the abnormality.

[0066] Further, as shown in Figure 3 The embodiment of the application further comprises:

[0067] Step S610: determining whether the abnormal values are the same according to the number of abnormal data sources and the abnormal values, and if the same, taking the same abnormal value as the warning level;

[0068] Step S620: if at least one of the abnormal values is different from the other abnormal values, performing standard deviation calculation on the abnormal values;

[0069] Step S630: if the standard deviation calculation result meets the pre-set threshold, taking the mean of the abnormal values as the warning level;

[0070] Step S640: generating and sending warning information according to the warning level.

[0071] Specifically, the number of abnormal data belonging to different sources and the specific abnormal values detected by each data source are obtained, and they are recorded and stored in the database. Then, it is checked whether the recorded specific abnormal values are completely the same, if the same, the warning level is directly set to the abnormal value. If not, the measurement index of the dispersion degree between the recorded abnormal values is calculated, that is, the standard deviation calculation of the abnormal values is performed. If the measurement index is less than the pre-set threshold, the operation continues. If greater than or equal to the threshold, the warning level is set to the average of the abnormal values, and the operation ends.

[0072] For example, assume that there are four data sources A, B, C and D, and the abnormal temperature values detected by them are 92 degrees, 105 degrees, 97 degrees and 103 degrees. First, record that the number of data sources is four, the specific abnormal temperature values are 92 degrees, 105 degrees, 97 degrees and 103 degrees, and the standard deviation threshold is 5 degrees. Then, check whether the four temperature values are the same, and the detection result is that the abnormal values are different. Then, calculate the standard deviation of the four temperature values. The calculation result is 6.04 degrees, which is greater than the preset threshold. Then, set the alarm level to the average of the temperature values, which is 99.25 degrees, generate alarm information containing the alarm level 99.25 degrees, and send it to the monitoring personnel.

[0073] By generating corresponding early warning information according to different situations of abnormal values, the technical effect of integrating multiple data sources and improving the accuracy of monitoring is achieved.

[0074] Further, the embodiments of the present application also include:

[0075] Step S650: If the standard deviation calculation result does not meet the preset threshold, the median of the abnormal values is extracted according to the abnormal values.

[0076] Step S660: Obtain the number of data sources whose abnormal values are greater than the median of the abnormal values.

[0077] Step S670: Determine whether the number of data sources is greater than or equal to half of the total number of data sources. If yes, the maximum value in the abnormal values is extracted as the early warning level. If no, the median of the abnormal values is set as the early warning level.

[0078] Specifically, when the standard deviation calculation result of the abnormal values of different data sources does not meet the preset threshold, the intermediate value is extracted from the recorded abnormal values. Check the number of abnormal data sources greater than the intermediate value. If it is greater than or equal to half of the total number of data sources, set the alarm level to the maximum value in the abnormal values; if it is less than half, set the alarm level to the intermediate value.

[0079] For example, assuming there are 3 data sources A, B and C, the detected abnormal temperature values are 97 degrees, 103 degrees and 99 degrees, and the preset threshold is 5 degrees. First, record the number of data sources as 3, and the specific abnormal temperature values are 97 degrees, 103 degrees and 99 degrees. Then, check whether the three temperature values are the same, and the abnormal value is different. Then, calculate the standard deviation of the three temperature values, and the calculation result is 3 degrees, which is less than the preset threshold, so the middle value 99 degrees is extracted from the abnormal temperature values 97 degrees, 103 degrees and 99 degrees. The number of abnormal data sources greater than 99 degrees is 2, which is more than half of the total number 3, so the alarm level is set to the maximum value 103 degrees. Finally, alarm information containing the alarm level 103 degrees is generated, which is sent to the monitoring personnel through different communication methods, greatly improving the timeliness and accuracy of the state judgment of the monitored object.

[0080] In summary, the intelligent monitoring method for multi-source data analysis provided by the embodiments has the following technical effects:

[0081] According to the first monitoring object, data is collected from multiple data sources to obtain a multi-source data set, realizing multi-dimensional detection of the monitoring object and obtaining comprehensive data. According to the historical data collection of multiple data sources, data anomaly analysis is performed according to the historical data collection results to obtain a plurality of abnormal analysis result sets, realizing learning and understanding of the data characteristics and abnormal patterns of each data source and obtaining abnormal analysis results of multiple abnormal data sources, providing a basis for subsequent formulation of multi-source monitoring coding rules. Based on the plurality of abnormal analysis result sets, a multi-source monitoring coding rule is set to determine the abnormal judgment standard and formulate the coding rule, providing a basis for real-time data anomaly judgment. According to the multi-source data set, the multi-source data is coded through the multi-source monitoring coding rule to obtain a data coding result, realizing abnormal judgment and state classification of real-time collected data and providing a basis for subsequent accurate positioning of abnormal sources. According to the data coding result, the number of abnormal data sources is determined, and the abnormal value of the abnormal data source is obtained, accurately identifying the abnormal source and obtaining the corresponding abnormal value, providing data support for abnormal early warning. According to the number of abnormal data sources and the abnormal value, data anomaly early warning is performed, and abnormal early warning grading and judgment are performed according to the abnormal source distribution and the severity of the abnormal value, which plays a key role in the safe operation of the monitoring object. Through multi-source data collection, analysis learning, rule formulation and judgment application, automatic multi-source monitoring and abnormal early warning are realized, achieving the technical effects of collecting and integrating multiple data sources, and further improving the accuracy of monitoring and the processing efficiency of abnormal early warning.

[0082] Embodiment two

[0083] Based on the same inventive concept as the intelligent monitoring method for multi-source data analysis in the foregoing embodiments, as Figure 4As shown, the embodiment of the present application provides a multi-source data analysis intelligent monitoring system, which comprises:

[0084] A multi-source data set module 11 is configured to collect data from a plurality of data sources according to a first monitoring object, and obtain a multi-source data set;

[0085] An abnormality analysis result module 12 is configured to collect historical data from the plurality of data sources, perform data abnormality analysis according to the historical data collection result, and obtain a plurality of abnormality analysis result sets;

[0086] An encoding rule setting module 13 is configured to set a multi-source monitoring encoding rule based on the plurality of abnormality analysis result sets;

[0087] A data encoding result module 14 is configured to perform multi-source data encoding on the multi-source data set through the multi-source monitoring encoding rule, and obtain a data encoding result;

[0088] A data source abnormal value module 15 is configured to determine an abnormal data source quantity according to the data encoding result, and obtain an abnormal value of the abnormal data source;

[0089] A data abnormality early warning module 16 is configured to perform data abnormality early warning according to the abnormal data source quantity and the abnormal value.

[0090] Further, the embodiment of the present application further comprises:

[0091] An abnormality result set module is configured to obtain a first abnormality analysis result set corresponding to a first data source according to the plurality of abnormality analysis result sets;

[0092] An abnormality influence degree module is configured to analyze a plurality of abnormality influence degree sets of abnormal data of the first data source on the first monitoring object according to the first abnormality analysis result set;

[0093] A plurality of abnormality obtaining modules are configured to obtain a plurality of abnormality influence degree sets of abnormal data of the plurality of data sources on the first monitoring object;

[0094] A same-grade abnormality module is configured to perform same-grade abnormality degree statistical analysis according to the plurality of abnormality influence degree sets, and obtain same-grade abnormality degree data;

[0095] A monitoring encoding rule module is configured to set the multi-source monitoring encoding rule according to the same-grade abnormality degree data.

[0096] Further, the embodiment of the present application further comprises:

[0097] a normal encoding value module, configured to set a normal encoding value of a normal value, wherein the normal encoding value corresponding to the plurality of data sources is the same, and a normal encoding rule is obtained;

[0098] an abnormal bias code rule module, configured to set the same abnormal encoding value corresponding to the plurality of data sources according to the same grade abnormal data, and an abnormal bias code rule is obtained;

[0099] a multi-source monitoring encoding module, configured to generate the multi-source monitoring encoding rule according to the normal encoding rule and the abnormal bias code rule.

[0100] Further, the embodiment of the present application further comprises:

[0101] a normal data range module, configured to obtain a normal data range of the plurality of data sources based on the normal encoding rule;

[0102] a normal value encoding module, configured to filter data meeting the normal data range from the multi-source data set, and perform normal value encoding;

[0103] an abnormal data range module, configured to obtain an abnormal data range of a plurality of abnormal encoding values corresponding to the plurality of data sources respectively based on the abnormal bias code rule;

[0104] an abnormal value encoding module, configured to filter data meeting the abnormal data range from the multi-source data set, and perform abnormal value encoding;

[0105] a data encoding result module, configured to generate the data encoding result by using the normal value encoding result and the abnormal value encoding result.

[0106] Further, the embodiment of the present application further comprises:

[0107] a warning level module, configured to determine whether the abnormal values are the same according to the number of abnormal data sources and the abnormal values, and if the abnormal values are the same, use the same abnormal values as a warning level;

[0108] a standard deviation calculation module, configured to perform standard deviation calculation on the abnormal values if at least one of the abnormal values is different from other abnormal values;

[0109] a warning level setting module, configured to use a mean value of the abnormal values as the warning level if a standard deviation calculation result meets a preset threshold;

[0110] a warning information module, configured to generate and send warning information according to the warning level.

[0111] Further, the embodiment of the present application further comprises:

[0112] The abnormal value median module is configured to extract an abnormal value median from the abnormal values if the standard deviation calculation result does not satisfy the preset threshold value.

[0113] The data source number module is configured to obtain a data source number of abnormal values greater than the abnormal value median.

[0114] The data source number judgment module is configured to judge whether the data source number is greater than or equal to half of the total data source number. If yes, the maximum value in the abnormal values is extracted as the early warning level. If no, the abnormal value median is taken as the early warning level.

[0115] Any step of the above method can be stored in a computer memory without limitation as computer instructions or programs, and can be called and recognized by a computer processor without limitation to implement any method in the embodiments of the present application, and no redundant limitation is made herein.

[0116] Further, the above first or second may not only represent an order relationship, but also may represent a specific concept, and / or refer to the selection of a plurality of elements individually or collectively. Obviously, those skilled in the art can make various modifications and variations to the present application without departing from the scope of the present application. Thus, if these modifications and variations of the present application belong to the scope of the present application and its equivalent technologies, the present application intends to include these modifications and variations.

Claims

1. An intelligent monitoring method for multi-source data analysis, characterized in that, The method includes: Data is collected from multiple data sources based on the first monitored object to obtain a multi-source dataset; Historical data is collected from the multiple data sources, and data anomaly analysis is performed based on the historical data collection results to obtain multiple sets of anomaly analysis results. Multi-source monitoring coding rules are set based on the aforementioned sets of multiple anomaly analysis results; Based on the multi-source dataset, multi-source data is encoded using the multi-source monitoring coding rules to obtain the data encoding results; The number of abnormal data sources is determined based on the data encoding results, and the abnormal values ​​of the abnormal data sources are obtained. Data anomaly warnings are issued based on the number of abnormal data sources and the abnormal values. The setting of multi-source monitoring coding rules includes: Based on the multiple anomaly analysis result sets, obtain the first anomaly analysis result set corresponding to the first data source; Based on the first set of anomaly analysis results, analyze the set of anomalies that affect the first monitored object by the abnormal data corresponding to the first data source. Similarly, obtain a set of multiple abnormal impact levels of the abnormal data corresponding to the multiple data sources on the first monitored object; Based on the multiple sets of abnormal impact levels, perform statistical analysis of the same level of abnormality to obtain data on the same level of abnormality. The multi-source monitoring coding rules are set based on the anomaly level data of the same level.

2. The method as described in claim 1, characterized in that, The setting of the multi-source monitoring coding rules based on the anomaly level data of the same level includes: Set the normal encoding value for the normal value, wherein the normal encoding values ​​corresponding to the multiple data sources are the same, and obtain the normal encoding rule; Based on the abnormal data of the same level, the abnormal data of the same level corresponding to the multiple data sources are set with the same abnormal code value to obtain the abnormal bias code rule; The multi-source monitoring coding rule is generated based on the normal coding rule and the abnormal bias coding rule.

3. The method as described in claim 2, characterized in that, The step of encoding multi-source data according to the multi-source dataset and the multi-source monitoring coding rules to obtain the data encoding result includes: Based on the aforementioned normal encoding rules, the normal data range of multiple data sources is obtained; Data that conforms to the normal data range is selected from the multi-source dataset and then encoded as normal values. Based on the aforementioned abnormal bias code rule, obtain the abnormal data range of multiple abnormal code values ​​corresponding to the multiple data sources respectively; Data that falls within the specified abnormal data range is selected from the multi-source dataset and then encoded as outliers. The data encoding result is composed of the normal value encoding result and the abnormal value encoding result.

4. The method as described in claim 1, characterized in that, The method of issuing a data anomaly warning based on the number of abnormal data sources and the abnormal values ​​includes: Based on the number of abnormal data sources and the abnormal values, determine whether the abnormal values ​​are the same. If they are the same, use the same abnormal value as the warning level. If at least one of the outliers is different from the other outliers, the standard deviation of the outlier is calculated. If the standard deviation calculation result meets the preset threshold, then the mean of the outliers is used as the warning level; A warning message is generated and sent based on the warning level.

5. The method as described in claim 4, characterized in that, The method further includes: If the standard deviation calculation result does not meet the preset threshold, then the median of the outliers is extracted based on the outliers; Obtain the number of data sources whose outliers are greater than the median of the outliers; Determine whether the number of data sources is greater than or equal to half of the total number of data sources. If so, use the maximum value among the outliers as the warning level; otherwise, use the median of the outliers as the warning level.

6. An intelligent monitoring system for multi-source data analysis, characterized in that, The system includes: A multi-source dataset module is used to collect data from multiple data sources based on a first monitored object to obtain a multi-source dataset. An anomaly analysis result module is used to collect historical data based on the multiple data sources, perform data anomaly analysis based on the historical data collection results, and obtain multiple anomaly analysis result sets. The coding rule setting module sets multi-source monitoring coding rules based on the multiple anomaly analysis result sets. The data encoding result module is used to encode multi-source data according to the multi-source dataset and the multi-source monitoring encoding rules, and obtain data encoding results. A data source anomaly module is used to determine the number of abnormal data sources based on the data encoding result and to obtain the anomaly values ​​of the abnormal data sources. A data anomaly warning module is used to issue a data anomaly warning based on the number of abnormal data sources and the abnormal value. An anomaly result set module is used to obtain the first anomaly analysis result set corresponding to the first data source based on the multiple anomaly analysis result sets; The abnormal impact degree module is used to analyze the abnormal impact degree set of the abnormal data corresponding to the first data source on the first monitored object based on the first abnormal analysis result set; Multiple anomaly acquisition modules are used to acquire a set of multiple anomaly impact degrees of the multiple data sources corresponding to the multiple anomaly data on the first monitored object; The same-level anomaly module is used to perform statistical analysis of the same-level anomaly degree based on the multiple sets of anomaly impact degrees, and to obtain the same-level anomaly degree data. The monitoring coding rule module is used to set the multi-source monitoring coding rules based on the anomaly data of the same level.

Images