Abnormal positioning method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention provides an anomaly positioning method and device, electronic equipment and a storage medium. The method comprises the following steps: acquiring first time sequence data under a preset dimension in a normal time period of a system and second time sequence data under the preset dimension in an abnormal time period of the system; on the basis of the information entropy corresponding to the first time series data and the information entropy corresponding to the second time series data, determining an abnormal dimension with an influence coefficient on system abnormity greater than a threshold value in the preset dimensions; determining an abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time sequence data and the second time sequence data under the abnormal dimension; and outputting the abnormal dimension and the abnormal dimension value. According to the embodiment of the invention, the difficulty of the abnormal dimension in multiple dimensions can be reduced, and the accuracy of abnormal dimension positioning is improved.

Description

Technical Field

[0001] The present application relates to the field of abnormality location technology, and in particular to an abnormality location method, device, electronic device and storage medium. Background Art

[0002] Multidimensional Metric Anomaly Localization is an important data analysis task, which is usually used to monitor the normal operation of indicators in complex systems (such as cloud services, network systems and industrial equipment) to quickly discover and solve performance degradation or failure problems. In the operation and maintenance of data centers, cloud computing platforms and large-scale enterprise IT (Internet Technology) infrastructure, multidimensional indicators are used to locate anomalies of system performance indicators (such as CPU (Central Processing Processor) utilization, memory usage, network traffic, etc.).

[0003] At present, the problem of multi-dimensional indicator anomaly location is mainly solved by real-time monitoring system plus rich visualization tools. However, as the number of dimensions continues to increase, the complexity of data grows exponentially. Manual location of faults through visualization reports requires expensive time. By the time the problem is actually located, the golden time for troubleshooting has passed. In addition, multiple indicators in the system may be interrelated, and individual analysis may not accurately reflect the real status of the system. The need to pay attention to multiple indicators at the same time increases the difficulty of dimension location, and the accuracy of abnormal dimension location is poor. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide an abnormality location method, device, electronic device and storage medium to achieve the following specific technical solutions:

[0005] In a first aspect of the implementation of the present application, a method for locating an abnormality is first provided, comprising:

[0006] Acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system;

[0007] Based on the information entropy corresponding to the first time series data and the second time series data respectively, determining an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension;

[0008] Determine an abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension;

[0009] The abnormal dimension and the abnormal dimension value are output.

[0010] In a second aspect of the present application, there is provided an abnormality locating device, comprising:

[0011] A time series data acquisition module, used to acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system;

[0012] An abnormal dimension determination module, used to determine an abnormal dimension whose influence coefficient on system abnormality is greater than a threshold in the preset dimensions based on the information entropy corresponding to the first time series data and the second time series data respectively;

[0013] An abnormal dimension value determination module, used to determine the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension;

[0014] The abnormal dimension value output module is used to output the abnormal dimension and the abnormal dimension value.

[0015] In another aspect of the present application, there is also provided an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0016] Memory, used to store computer programs;

[0017] The processor is used to implement any of the above-mentioned method steps when executing the program stored in the memory.

[0018] In another aspect of the implementation of the present application, a computer-readable storage medium is provided, wherein instructions are stored in the computer-readable storage medium, and when the computer-readable storage medium is run on a computer, the computer executes any of the above-mentioned abnormality locating methods.

[0019] In another aspect of the implementation of the present application, a computer program product comprising instructions is also provided, which, when executed on a computer, enables the computer to execute any of the above-mentioned abnormality locating methods.

[0020] The solution provided by the embodiment of the present application is to obtain the first time series data under the preset dimension within the normal period of the system, and the second time series data under the preset dimension within the abnormal period of the system. Based on the information entropy corresponding to the first time series data and the second time series data, respectively, the abnormal dimension in the preset dimension whose influence coefficient on the system abnormality is greater than the threshold is determined. Based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension, the abnormal dimension value under the abnormal dimension is determined, and the abnormal dimension and the abnormal dimension value are output. The embodiment of the present application first calculates the information entropy of the preset dimension in the abnormal period and the normal period to determine the importance of different dimensions in contributing to the system abnormality. Then, the abnormal dimension value under the important abnormal dimension is located by using the similarity, so that the dimension and dimension value important to the abnormality can be provided quickly and effectively, thereby improving the efficiency and accuracy of dimensional positioning of system abnormalities. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0022] Figure 1 A flowchart of a method for locating an abnormality provided in an embodiment of the present application;

[0023] Figure 2 A flowchart of a time series data screening method provided in an embodiment of the present application;

[0024] Figure 3 A flowchart of the steps of an abnormal dimension screening method provided in an embodiment of the present application;

[0025] Figure 4 A flowchart of a method for obtaining an abnormal dimension value provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of an abnormal dimension positioning process provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of an abnormal dimension positioning system provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of the structure of an abnormality locating device provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0030] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0031] Figure 1 A flowchart of a method for locating an abnormality provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the abnormality positioning method may include: step 101, step 102, step 103 and step 104.

[0032] Step 101: Acquire first time series data under a preset dimension within a normal system period, and second time series data under the preset dimension within an abnormal system period.

[0033] The embodiments of the present application can be applied to the scenario of combining the information entropy and similarity of multiple dimensions in the abnormal period and the normal period to quickly locate the abnormal dimension and the abnormal dimension value under the abnormal dimension.

[0034] In the real-time monitoring system, once an anomaly is detected, the multi-dimensional indicator anomaly location process will be started in time to quickly locate the cause of the problem and reduce the impact of the failure on the system. Of course, in the real-time monitoring scenario, there may be a delay in anomaly monitoring. When the system is monitored to have an anomaly, the system anomaly has occurred for a period of time. At this time, the start time of the system anomaly can be determined based on the recorded log information, and the time series data from the start time of the system anomaly to the current time and the time series data of the system during the normal period can be obtained.

[0035] Of course, in some cases, the multi-dimensional indicator anomaly location may not be performed in real time, but may be performed regularly according to a certain time period (such as daily, weekly or monthly, etc.), or according to actual needs. This method is suitable for scenarios where system performance is periodically evaluated or specific problems are analyzed in depth.

[0036] The system is the system that needs to be anomaly located. In this example, the system can be: IT (Internet Technology) system (such as cloud service, data center, network architecture, etc.), financial system (such as trading system, payment system, risk management system, etc.), Internet of Things system (such as a system composed of various sensors, devices and networks, etc.), multimedia system (such as video playback platform, etc.), etc.

[0037] The preset dimensions refer to the dimensions that may affect the anomaly and need to be monitored in the system. In this example, corresponding preset dimensions can be pre-configured for different types of systems.

[0038] For example, when the system is a multimedia system, the preset dimensions may include: regional dimension and film source dimension, etc. Among them, the regional dimension refers to the dimension that divides system data or events according to geographic location, which allows analysts to observe the operating status and abnormal distribution of the system from a geographic space perspective. The time series data under the regional dimension may include: the number of multimedia files played in different regions, normal / abnormal information on multimedia file playback, system performance indicator data, etc. The film source dimension refers to the dimension that is divided according to the source or type of video content. It helps analysts understand the impact of different film sources on system performance, user behavior, etc. The time series data under the film source dimension may include: playback data of specific movies, TV series or variety shows (such as the number of plays of popular content, bandwidth consumption, cache hit rate, etc.).

[0039] When the system is an IT system, the preset dimensions may include: service component dimension (such as database, web server, load balancer, etc.) and geographic location dimension (such as data center, geographic location of server, etc.). The time series data under the service component dimension may include: response time, error rate, resource utilization (CPU, memory, disk), etc. of each component. The time series data under the geographic location dimension may include: access delay data and connection failure data of users in a specific region.

[0040] When the system is a financial system, the preset dimensions may include: transaction type dimension (such as stock trading, bond trading, foreign exchange trading, etc.), transaction channel dimension (such as online banking, mobile banking, etc.), system component dimension (such as trading engine, risk control system, clearing system, etc.), etc. The time series data under the transaction type dimension may include: transaction success data, transaction failure data, transaction delay data, etc. The time series data under the transaction channel dimension may include: transaction volume of each channel, number of failures, etc. The time series data under the system component dimension may include: component response time, number of errors, resource consumption data, etc.

[0041] When the system is an IoT system, the preset dimensions may include: geographic location dimension (such as the geographic location or area where the device is deployed), network environment dimension (such as the wireless network or wired network status to which the device is connected), device type dimension (such as smart light bulbs, smart door locks, smart sensors, etc.), etc. The time series data under the geographic location dimension may include: connection stability data and signal strength data of devices in a specific area. The time series data under the network environment dimension may include: network bandwidth usage, packet loss data, delay time, etc. The time series data under the device type dimension may include: online rate, failure rate, energy consumption data, etc. of each type of device.

[0042] Of course, in actual applications, different preset dimensions can be set according to different system types, such as time dimension (time granularity such as year, month, day, hour, etc., used to analyze the performance changes and anomalies of the system in different time periods), user dimension (user ID, user type (such as new users, old users, active users, inactive users, etc.), user behavior (such as clicks, browsing, purchases, etc.), etc., used to analyze the impact of user behavior on system anomalies), service dimension (service type (such as API interface, database service, cache service, etc.), service response time, service success rate, etc., used to monitor and analyze the stability and performance of system services), business dimension (dimensions defined according to specific business scenarios, such as product category, order status, payment channel, etc., used to analyze anomalies in business processes), etc.

[0043] In a specific implementation, the first time series data of the system under a preset dimension during a normal period of the system and the second time series data of the system under a preset dimension during an abnormal period of the system may be acquired in advance.

[0044] It can be understood that the first time series data in the normal system period can be a time series data obtained in advance when the system is normal, and the second time series data in the abnormal system period can be a time series data of the system in the abnormal system period obtained when the system is abnormal.

[0045] Among them, time series data refers to a series of data points arranged in chronological order, and each data point usually contains two main elements: a timestamp and the data at the timestamp. In this example, it can include: system performance indicator data, log information, user activity data, system status data, etc. Corresponding time series data can be established for different types of data. For example, taking performance indicator data and log information as an example, the performance indicator data at each time point within the normal period of the system can be obtained to form the time series data of the performance indicator data. And the log information at each time point within the normal period of the system is obtained to form the time series data of the log information, etc.

[0046] In practical applications, when an abnormality occurs in the system, time series data under preset dimensions can be obtained, and the time series data can include time series data in a normal period and time series data in an abnormal period. Then, the obtained time series data can be preprocessed and anomaly detected to determine the abnormal period, thereby extracting the corresponding first time series data and second time series data. In practical applications, for the first time series data, tools such as monitoring systems and logging systems can be used to collect time series data according to preset dimensions during the normal period of the system. For the second time series data, third-party services (such as data collection tools, performance analysis tools, etc.) can be used to capture time series data when the system is abnormal.

[0047] The implementation process of obtaining the first time series data and the second time series data will be combined in the following embodiment. Figure 2 A detailed description is given below and this embodiment will not be described in detail here.

[0048] After obtaining the first time series data under the preset dimension in the normal period of the system and the second time series data under the preset dimension in the abnormal period of the system, step 102 is performed.

[0049] Step 102: Based on the information entropy corresponding to the first time series data and the second time series data respectively, determine the abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension.

[0050] In this example, information entropy can be used to measure the influence of preset dimensions on system anomalies. In practical applications, information entropy is very sensitive to changes in data distribution. When the operating state of the system changes (such as from a normal state to an abnormal state), the distribution of time series data will also change, and this change will be reflected in the value of information entropy. Therefore, information entropy can objectively and sensitively reflect the changes in system states and the level of uncertainty.

[0051] The threshold refers to a pre-set impact coefficient threshold used to measure the impact on system abnormalities.

[0052] After obtaining the first time series data under the preset dimension in the normal period of the system and the second time series data under the preset dimension in the abnormal period of the system, the information entropy of the first time series data and the second time series data can be calculated respectively, and then, the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold is screened out from the preset dimension by combining the two information entropies. In a specific implementation, two information entropies can be calculated based on the first time series data and the second time series data, and then, based on the relationship between the difference between the two information entropies and the threshold, the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold is screened out from the preset dimension. In this example, when the system changes from a normal state to an abnormal state, its internal structure and behavior pattern will change, and this change will be reflected in the distribution of the time series data. Therefore, the difference in information entropy between the first time series data (normal period) and the second time series data (abnormal period) is large, which means that the system state has changed significantly, indicating that the system has changed significantly in the preset dimension, and the preset dimension can be determined as an abnormal dimension.

[0053] The implementation process of filtering abnormal dimensions will be combined in the following embodiments. Figure 3 A detailed description is given below and this embodiment will not be described in detail here.

[0054] After determining the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold in the preset dimensions based on the information entropy corresponding to the first time series data and the second time series data, step 103 is executed.

[0055] Step 103: Determine the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension.

[0056] Similarity is a quantitative method to evaluate the similarity between two data vectors. In time series analysis, similarity is usually used to measure the closeness of two time series data. Commonly used similarity measurement methods include Euclidean distance, Manhattan distance, cosine similarity, dynamic time warping (DTW), etc.

[0057] After determining the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold value in the preset dimension based on the information entropy corresponding to the first time series data and the second time series data respectively, the abnormal dimension value under the abnormal dimension can be determined based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension. For example, the abnormal dimensions determined in the above steps are: regional dimension and film source dimension. For the regional dimension, the time series data of each region under the regional dimension (i.e., the dimension value in the following steps in this example) can be similarly calculated to determine the region that affects the system abnormality (i.e., the abnormal dimension value). For the film source dimension, the time series data of each film and television drama under the film source dimension can be similarly calculated to determine the film and television drama that affects the system abnormality (i.e., the abnormal dimension value), etc.

[0058] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0059] In a specific implementation, the similarity corresponding to each dimension value can be calculated according to the data vectors of the first time series data and the second time series data under each dimension under the abnormal dimension, and the abnormal dimension value can be screened out from each dimension value according to the similarity. Specifically, the dimension value with lower similarity can be screened out from each dimension value as the abnormal dimension value. In the time series data with multiple abnormal dimensions, the data changes under each abnormal dimension reflect different aspects of the system or process. When the similarity of two time series data in a certain abnormal dimension is low, it means that the data under the abnormal dimension has undergone significant changes or anomalies. By setting a similarity threshold and screening out dimension values ​​below the threshold, these abnormal dimension values ​​can be effectively identified, so as to further analyze and handle abnormal situations.

[0060] The screening process for abnormal dimension values ​​will be combined in the following embodiments. Figure 4A detailed description is given below and this embodiment will not be described in detail here.

[0061] Step 104: Output the abnormal dimension and the abnormal dimension value.

[0062] After determining the abnormal dimension and the abnormal dimension value under the abnormal dimension, the abnormal dimension and the abnormal dimension value may be output. In this example, the output method may include at least one of the following:

[0063] 1. Report output method. Generate a detailed text report that lists important abnormal dimensions and their corresponding dimension values, including the specific manifestations of the abnormality, the scope of impact, possible causes, etc. Alternatively, use charts (such as bar charts, line charts, heat maps, etc.) to intuitively display the changing trends of abnormal dimensions and dimension values, which is convenient for quick understanding and analysis.

[0064] 2. Monitoring interface output method. Set up a special interface in the real-time monitoring system to display the status of abnormal dimensions and dimension values ​​in real time, so that operation and maintenance personnel can grasp the system status at any time. Alternatively, when an important abnormal dimension or dimension value is detected, the system automatically triggers an alarm to remind relevant personnel through sound, pop-up windows, etc.

[0065] 3. Data interface: Provide API interface to output the data of abnormal dimensions and dimension values ​​in the form of API interface for other systems or applications to call and realize data sharing and integration.

[0066] 4. Email or SMS notification: When an important anomaly is detected, the system automatically sends an email or SMS to notify relevant personnel to ensure that they can obtain key information in a timely manner.

[0067] The embodiments of the present application play an important role in quickly locating problems, optimizing system performance, assisting decision making, improving user experience, and enhancing system security by outputting abnormal dimensions and abnormal dimension values.

[0068] It can be understood that the four methods of outputting abnormal dimensions and abnormal dimension values ​​listed above are only examples listed for a better understanding of the technical solution of the embodiment of the present application. In actual applications, other methods can also be used to output abnormal dimensions and abnormal dimension values, such as conversation software, etc. This embodiment does not limit this.

[0069] The embodiment of the present application first calculates the information entropy of the preset dimensions during the abnormal period and the normal period to determine the importance of different dimensions in contributing to the system abnormality. Then, the abnormal dimension values ​​under the important abnormal dimensions are located using similarity, so that the dimensions and dimension values ​​important to the abnormality can be provided quickly and effectively, thereby improving the efficiency and accuracy of dimension location of system abnormalities.

[0070] Next, combine Figure 2 The process of acquiring time series data is described in detail.

[0071] Reference Figure 2 , shows a flow chart of the steps of a time series data screening method provided by an embodiment of the present application. Figure 2 As shown, the time series data screening method may include: step 201, step 202, step 203 and step 204.

[0072] Step 201: Acquire initial time series data of the monitored system under the preset dimension.

[0073] In this embodiment, the initial time series data of the monitored system under the preset dimension can be obtained. In a specific implementation, a data monitoring program or monitoring tool can be pre-set in the system to monitor the data generated by the system under the preset dimension in real time. When an abnormality occurs in the system, the data within a preset time period before the time point (such as 24 hours before the abnormality occurs) can be obtained according to the time point when the system abnormality occurs, as the initial time series data.

[0074] In this example, the initial time series data may include, but is not limited to, data such as CPU usage, memory usage information, and network traffic.

[0075] After the initial time series data of the monitored system under the preset dimension is acquired, step 202 is executed.

[0076] Step 202: preprocess the initial time series data to obtain time series data.

[0077] After obtaining the initial time series data of the monitored system under the preset dimension, the initial time series data can be preprocessed to obtain time series data. Specifically, the initial time series data can be first cleaned to remove noise and outliers in the initial time series data, thereby obtaining cleaned time series data. Then, it can be checked whether there is missing data in the cleaned time series data. If there is missing data, the cleaned time series data can be filled with missing data (such as interpolation using an interpolation algorithm to fill the missing data, etc.) to obtain filled time series data. Finally, the filled time series data can be normalized (or standardized) to normalize the filled time series data to the specified data range, thereby obtaining time series data.

[0078] After the initial time series data is preprocessed to obtain the time series data, step 203 is executed.

[0079] Step 203: Perform anomaly detection on the time series data based on an anomaly detection algorithm to obtain an anomaly detection result.

[0080] After preprocessing the initial time series data to obtain the time series data, an anomaly detection algorithm can be used to perform anomaly detection on the time series data to obtain an anomaly detection result. The anomaly detection result can be used to indicate which period of data in the time series data is data when the system is abnormal and which period of data is data when the system is normal.

[0081] In a specific implementation, a statistical method or anomaly detection algorithm (such as a histogram, Isolation Forest algorithm, etc.) can be used to identify abnormal events or time periods.

[0082] After anomaly detection is performed on the time series data based on the anomaly detection algorithm to obtain an anomaly detection result, step 204 is executed.

[0083] Step 204: Based on the abnormality detection result, the first time series data in the normal period of the system and the second time series data in the abnormal period of the system are filtered out from the time series.

[0084] After performing anomaly detection on the time series data based on the anomaly detection algorithm to obtain the anomaly detection result, the first time series data in the normal system period and the second time series data in the abnormal system period can be screened out from the time series based on the anomaly detection result. Specifically, the normal system period and the abnormal system period can be determined according to the anomaly detection result, and the data in a certain time interval in the normal system period (such as any continuous time interval of 2 minutes in the normal system period, etc.) can be screened out from the time series data in the normal system period as the data in the normal period, as the first time series data. At the same time, the data in a certain time interval in the abnormal system period (such as any continuous time interval of 3 minutes in the abnormal system period, etc.) in the time series data can be selected as the second time series data, etc.

[0085] The embodiment of the present application can remove noise and outliers in the time series data and fill in missing data through the above-mentioned processing of time series data, thereby avoiding the problem of inaccurate positioning of dimensional anomalies caused by data noise and outliers, as well as missing data.

[0086] Next, combine Figure 3 The process of determining abnormal dimensions is described in detail.

[0087] Reference Figure 3 , shows a flowchart of the steps of an abnormal dimension screening method provided by an embodiment of the present application. Figure 3 As shown, the abnormal dimension screening method may include: step 301, step 302 and step 303.

[0088] Step 301: Based on the first time series data, calculate and obtain the first information entropy of the preset dimension during the normal period of the system.

[0089] In this embodiment, after the first time series data is acquired, the first information entropy of the preset dimension during the normal period of the system can be calculated based on the first time series data.

[0090] In some examples, the average value of the first time series data may be first calculated, and then the average value divided by the sum of the first time series data may be calculated to obtain the first information entropy.

[0091] In some examples, the median of the first time series data may be obtained first, and then the median divided by the sum of the first time series data may be calculated to obtain the first information entropy.

[0092] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0093] Step 302: Based on the second time series data, calculate and obtain the second information entropy of the preset dimension during the system abnormality period.

[0094] After the second time series data is acquired, the second information entropy of the preset dimension during the system abnormality period may be calculated based on the second time series data.

[0095] The calculation method of the second information entropy is similar to the calculation method of the first information entropy, and will not be described in detail in this embodiment.

[0096] Step 303: Based on the first information entropy and the second information entropy, select abnormal dimensions whose influence coefficient on system abnormality is greater than a threshold from the preset dimensions.

[0097] After the first information entropy and the second information entropy are calculated through the above steps 301 and 302, the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold value can be screened out from the preset dimensions based on the first information entropy and the second information entropy. Specifically, the entropy difference between the first information entropy and the second information entropy can be calculated. And the preset dimensions whose entropy difference value is greater than the entropy difference threshold value are screened out from the preset dimensions, and the preset dimensions whose entropy difference value is greater than the entropy difference threshold value are used as the abnormal dimensions whose influence coefficient on the system abnormality is greater than the threshold value.

[0098] Of course, in a specific implementation, the preset dimensions can also be sorted in descending order according to the entropy difference value, so that the dimension with the largest entropy difference value is placed in front, and then the first N (N is a positive integer greater than or equal to 1) preset dimensions are selected from the sorted preset dimensions as abnormal dimensions, etc.

[0099] The embodiment of the present application calculates the information entropy of preset dimensions during the abnormal period and the normal period to determine the importance of different dimensions in contributing to the system abnormality, thereby improving the accuracy of abnormal dimension positioning.

[0100] Next, combine Figure 4 The process of obtaining abnormal dimension values ​​is described in detail.

[0101] Reference Figure 4 , shows a flowchart of the steps of a method for obtaining an abnormal dimension value provided by an embodiment of the present application. Figure 4 As shown, the method for obtaining abnormal dimension values ​​may include: step 401, step 402 and step 403.

[0102] Step 401: Obtain a first data vector of first time series data of each dimension value under the abnormal dimension, and a second data vector of second time series data of each dimension value under the abnormal dimension.

[0103] In this embodiment, after determining the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold value in the preset dimension, the first data vector of the first time series data of each dimension value under the abnormal dimension and the second data vector of the second time series data of each dimension value under the abnormal dimension can be obtained. For example, the abnormal dimension takes the regional dimension as an example, and the dimension values ​​included in the regional dimension are: City A, City B and City C. The first data vector of the first time series data of City A, City B and City C, and the second data vector of the second time series data can be obtained respectively.

[0104] It can be understood that the above examples are merely examples listed for a better understanding of the technical solutions of the embodiments of the present application, and are not intended to be the sole limitation to the embodiments.

[0105] After obtaining the first data vector of the first time series data of each dimension value under the abnormal dimension and the second data vector of the second time series data of each dimension value under the abnormal dimension, step 402 is performed.

[0106] Step 402: Calculate and obtain the similarity between the first data vector and the second data vector.

[0107] After obtaining the first data vector of the first time series data of each dimension value under the abnormal dimension and the second data vector of the second time series data of each dimension value under the abnormal dimension, the similarity between the first data vector and the second data vector can be calculated.

[0108] In a specific implementation of the present application, the cosine similarity between the first data vector and the second data vector may be calculated and used as the similarity.

[0109] In another specific implementation of the present application, the Euclidean distance between the first data vector and the second data vector may be calculated, and the Euclidean distance may be used as the similarity.

[0110] It can be understood that the above examples are only two similarity calculation methods listed for a better understanding of the technical solution of the embodiment of the present application. In practical applications, other methods can also be used to calculate the similarity between the first data vector and the second data vector, and this embodiment does not limit this.

[0111] After the similarity between the first data vector and the second data vector is calculated, step 403 is performed.

[0112] Step 403: Dimension values ​​whose similarity is less than a similarity threshold are screened out from the dimension values ​​under the abnormal dimension to serve as the abnormal dimension value.

[0113] After calculating the similarity between the first data vector and the second data vector, dimension values ​​whose similarity is less than a similarity threshold value can be screened out from the dimension values ​​under the abnormal dimension to serve as abnormal dimension values.

[0114] Understandably, in time series data with multiple anomaly dimensions, the data changes under each anomaly dimension reflect different aspects of the system or process. When the similarity of two time series data under a certain anomaly dimension is low, it means that the data under the anomaly dimension has undergone significant changes or anomalies. By setting a similarity threshold and filtering out dimension values ​​below the threshold, these anomaly dimension values ​​can be effectively identified, so as to further analyze and handle the anomalies.

[0115] The embodiment of the present application locates abnormal dimension values ​​under important abnormal dimensions by utilizing similarity, thereby being able to quickly and effectively provide dimension values ​​under dimensions that are important to the abnormality.

[0116] The process of locating abnormal dimensions and abnormal dimension values ​​can be combined with Figure 5 This is described in detail as follows.

[0117] Reference Figure 5 , showing a schematic diagram of an abnormal dimension positioning process provided in an embodiment of the present application.

[0118] like Figure 5 As shown, the abnormal dimension positioning process may include:

[0119] 1. Data preprocessing, that is, collecting multi-dimensional time series data from the monitored system, which can include performance indicators, log information, user activities, system status, etc. Then, the time series data can be preprocessed, such as cleaning the data to remove noise and outliers, filling missing values, and standardizing or normalizing the data range to ensure the accuracy of subsequent algorithms.

[0120] 2. Calculate the entropy value of normal time periods in each dimension. Select a time period of data as a representative of normal behavior. During the selected "normal" data period, calculate the information entropy (i.e., the first information entropy in this embodiment) of the dimensional statistical value (such as the mean or median, etc.) of the time series indicator within the time interval set for each dimension (such as province, operator dimension, etc.) as a reference baseline for anomaly detection.

[0121] 3. Calculate the entropy value of abnormal time periods in each dimension. Specifically, statistical methods or anomaly detection algorithms can be used to identify abnormal events or time periods. For the detected abnormal data, the information entropy (i.e., the second information entropy in this embodiment) is calculated for the drilled-down subdivisions (such as provinces, operator dimensions, etc.), and the dimensional statistical values ​​(such as mean or median, etc.) of the time series indicators within the set time interval after the anomaly are specifically selected to calculate the information entropy.

[0122] 4. The importance of each dimension is determined according to the degree of entropy reduction. Specifically, for each dimension, the entropy reduction in normal and abnormal states (i.e., the entropy difference in this example) can be calculated. Then, the abnormal dimensions whose entropy difference is greater than the threshold can be screened out from each dimension. Alternatively, based on the size of the entropy reduction change, the dimensions are sorted, with the dimensions with the largest change in front, and the dimensions ranked in the top N are screened out from the sorted dimensions as abnormal dimensions, etc.

[0123] 5. Determine dimension anomalies by cosine similarity. Specifically, for each important dimension (i.e., the abnormal dimension in this embodiment), obtain the data vectors of the time series data of each dimension value under normal and abnormal conditions. For the obtained data vectors, calculate the similarity (such as cosine similarity, etc.) between the two groups of data vectors for each dimension value.

[0124] 6. Determine the abnormal dimension and abnormal dimension value. Dimension values ​​with low similarity (i.e., dimension values ​​with similarity less than the similarity threshold) are identified as abnormal dimension values. After determining the abnormal dimension and abnormal dimension value, the abnormal dimension and abnormal dimension value can be output. Specifically, the importance can be marked according to the degree of entropy reduction and provided to the user, indicating that these are abnormal dimension values ​​that the user may need to care about, so as to assist the user in quickly locating the direction of investigation.

[0125] In the above implementation process, the algorithm automatically locates the anomaly to solve the problem of slow manual location of dimensional anomalies. At the same time, the importance of the contribution of different dimensions to this anomaly is automatically determined through entropy reduction sorting, which solves the problem of difficulty in quickly determining the direction of investigation when anomalies occur in multiple dimensions.

[0126] In addition, the embodiment of the present application also provides an abnormal dimension positioning system to perform the above abnormal dimension positioning process. Figure 6 As shown, for the data stream obtained by historical data and current monitoring, abnormal time series data (i.e., the second time series data in this example) and normal time series data (i.e., the first time series data in this example) can be detected by the anomaly detection module. Then, the information entropy of each dimension in the normal period and the information entropy in the abnormal period can be calculated by the entropy reduction calculation module, and the difference between the two can be calculated to obtain the information entropy difference, and the dimensions can be sorted according to the information entropy difference. The dimension with the largest change is ranked in front, and it is considered to have a greater contribution to the anomaly. Finally, the dimension value positioning module can be used to calculate the cosine similarity to extract the data vectors in the normal state and the abnormal state for each abnormal dimension located. For the extracted vectors, the cosine similarity is calculated. For dimension values ​​with low similarity, they are identified as abnormal values.

[0127] The anomaly location method provided in the embodiment of the present application obtains the first time series data under the preset dimension within the normal period of the system, and the second time series data under the preset dimension within the abnormal period of the system. Based on the information entropy corresponding to the first time series data and the second time series data, respectively, the abnormal dimension whose influence coefficient on the system anomaly is greater than the threshold value in the preset dimension is determined. Based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension, the abnormal dimension value under the abnormal dimension is determined, and the abnormal dimension and the abnormal dimension value are output. The embodiment of the present application first calculates the information entropy of the preset dimension in the abnormal period and the normal period to determine the importance of different dimensions in contributing to the system anomaly. Then, the abnormal dimension value under the important abnormal dimension is located using the similarity, so that the dimension and dimension value important to the anomaly can be provided quickly and effectively, thereby improving the efficiency and accuracy of dimensional location of system anomalies.

[0128] Reference Figure 7 , shows a schematic diagram of the structure of an abnormality locating device provided in an embodiment of the present application, such as Figure 7 As shown, the abnormality locating device 700 may include the following modules:

[0129] The time series data acquisition module 710 is used to acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system;

[0130] An abnormal dimension determination module 720, configured to determine, based on the information entropy corresponding to the first time series data and the second time series data, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold value in the preset dimensions;

[0131] An abnormal dimension value determination module 730, used to determine the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension;

[0132] The abnormal dimension value output module 740 is used to output the abnormal dimension and the abnormal dimension value.

[0133] Optionally, the time series data acquisition module includes:

[0134] An initial sequence data acquisition unit, used to acquire initial time series data of the monitored system under the preset dimension;

[0135] A time series data acquisition unit, used for preprocessing the initial time series data to obtain time series data;

[0136] An anomaly detection result acquisition unit, used to perform anomaly detection on the time series data based on an anomaly detection algorithm to obtain an anomaly detection result;

[0137] The time series data screening unit is used to screen out the first time series data in the normal period of the system and the second time series data in the abnormal period of the system from the time series based on the abnormality detection result.

[0138] Optionally, the abnormal dimension determination module includes:

[0139] A first information entropy calculation unit, configured to calculate, based on the first time series data, a first information entropy of the preset dimension during a normal period of the system;

[0140] A second information entropy calculation unit, configured to calculate, based on the second time series data, a second information entropy of the preset dimension during the system abnormality period;

[0141] An abnormal dimension screening unit is used to screen out abnormal dimensions whose influence coefficient on system abnormality is greater than a threshold from the preset dimensions based on the first information entropy and the second information entropy.

[0142] Optionally, the abnormal dimension screening unit includes:

[0143] An entropy difference calculation subunit, used to calculate the entropy difference between the first information entropy and the second information entropy;

[0144] The abnormal dimension acquisition subunit is used to screen out the preset dimensions whose entropy difference value is greater than the entropy difference threshold from the preset dimensions, and use the preset dimensions whose entropy difference value is greater than the entropy difference threshold as the abnormal dimensions whose influence coefficient on system abnormality is greater than the threshold.

[0145] Optionally, the abnormal dimension value determination module includes:

[0146] A data vector acquisition unit, used to acquire a first data vector of first time series data of each dimension value under the abnormal dimension, and a second data vector of second time series data of each dimension value under the abnormal dimension;

[0147] A similarity calculation unit, configured to calculate the similarity between the first data vector and the second data vector;

[0148] The abnormal dimension value acquisition unit is used to filter out dimension values ​​whose similarity is less than a similarity threshold from the dimension values ​​under the abnormal dimension, as the abnormal dimension value.

[0149] Optionally, the similarity calculation unit includes:

[0150] A first similarity obtaining subunit is used to calculate the cosine similarity between the first data vector and the second data vector, and use the cosine similarity as the similarity;

[0151] The second similarity obtaining subunit is used to calculate the Euclidean distance between the first data vector and the second data vector, and use the Euclidean distance as the similarity.

[0152] Optionally, the time series data acquisition unit includes:

[0153] Performing data cleaning processing on the initial time series data to obtain cleaned time series data;

[0154] In the case where there are missing data in the cleaned time series data, performing missing data filling processing on the cleaned time series data to obtain filled time series data;

[0155] The filled time series data is normalized to obtain the time series data.

[0156] The abnormality locating device provided in the embodiment of the present application obtains the first time series data under the preset dimension within the normal period of the system, and the second time series data under the preset dimension within the abnormal period of the system. Based on the information entropy corresponding to the first time series data and the second time series data, respectively, the abnormal dimension whose influence coefficient on the system abnormality is greater than the threshold value in the preset dimension is determined. Based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension, the abnormal dimension value under the abnormal dimension is determined, and the abnormal dimension and the abnormal dimension value are output. The embodiment of the present application first calculates the information entropy of the preset dimension in the abnormal period and the normal period to determine the importance of different dimensions in contributing to the system abnormality. Then, the abnormal dimension value under the important abnormal dimension is located by using the similarity, so that the dimension and dimension value important to the abnormality can be provided quickly and effectively, thereby improving the efficiency and accuracy of the dimension positioning of the system abnormality.

[0157] The present application also provides an electronic device, such as Figure 8 As shown, it includes a processor 801, a communication interface 802, a memory 803 and a communication bus 804, wherein the processor 801, the communication interface 802, and the memory 803 communicate with each other through the communication bus 804.

[0158] Memory 803, used for storing computer programs;

[0159] The processor 801 is used to execute the program stored in the memory 803, and implements the following steps:

[0160] Acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system;

[0161] Based on the information entropy corresponding to the first time series data and the second time series data respectively, determining an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension;

[0162] Determine an abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension;

[0163] The abnormal dimension and the abnormal dimension value are output.

[0164] Optionally, the acquiring of first time series data under a preset dimension during a normal period of the system and second time series data under the preset dimension during an abnormal period of the system includes:

[0165] Obtaining initial time series data of the monitored system under the preset dimension;

[0166] Preprocessing the initial time series data to obtain time series data;

[0167] Perform anomaly detection on the time series data based on an anomaly detection algorithm to obtain an anomaly detection result;

[0168] Based on the abnormality detection result, the first time series data in the normal period of the system and the second time series data in the abnormal period of the system are screened out from the time series.

[0169] Optionally, the determining, based on the information entropies respectively corresponding to the first time series data and the second time series data, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension includes:

[0170] Based on the first time series data, a first information entropy of the preset dimension during the normal period of the system is calculated;

[0171] Based on the second time series data, a second information entropy of the preset dimension during the system abnormality period is calculated;

[0172] Based on the first information entropy and the second information entropy, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold is screened out from the preset dimensions.

[0173] Optionally, the filtering out, from the preset dimensions, abnormal dimensions whose influence coefficient on the system abnormality is greater than a threshold based on the first information entropy and the second information entropy, includes:

[0174] Calculating an entropy difference between the first information entropy and the second information entropy;

[0175] The preset dimensions whose entropy difference value is greater than the entropy difference threshold are screened out from the preset dimensions, and the preset dimensions whose entropy difference value is greater than the entropy difference threshold are used as abnormal dimensions whose influence coefficient on system abnormality is greater than the threshold.

[0176] Optionally, determining the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension includes:

[0177] Obtain a first data vector of first time series data of each dimension value under the abnormal dimension, and a second data vector of second time series data of each dimension value under the abnormal dimension;

[0178] Calculating the similarity between the first data vector and the second data vector;

[0179] Dimension values ​​whose similarity is less than a similarity threshold are screened out from the dimensional values ​​under the abnormal dimension to serve as the abnormal dimensional value.

[0180] Optionally, the calculating the similarity between the first data vector and the second data vector includes:

[0181] Calculate and obtain the cosine similarity between the first data vector and the second data vector, and use the cosine similarity as the similarity; or

[0182] The Euclidean distance between the first data vector and the second data vector is calculated and used as the similarity.

[0183] Optionally, the preprocessing the initial time series data to obtain the time series data includes:

[0184] Performing data cleaning processing on the initial time series data to obtain cleaned time series data;

[0185] In the case where there are missing data in the cleaned time series data, performing missing data filling processing on the cleaned time series data to obtain filled time series data;

[0186] The filled time series data is normalized to obtain the time series data.

[0187] The communication bus mentioned in the above terminal can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0188] The communication interface is used for communication between the above terminal and other devices.

[0189] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0190] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0191] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which instructions are stored. When the computer-readable storage medium is run on a computer, the computer executes the abnormality locating method described in any one of the above embodiments.

[0192] In another embodiment provided in the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute any of the abnormality locating methods described in the above embodiments.

[0193] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server, data center, etc. that contains one or more available media integrated. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium (eg, a solid state drive (SSD)).

[0194] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0195] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0196] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A method for locating an abnormality, characterized in that: include: Acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system; Based on the information entropy corresponding to the first time series data and the second time series data respectively, determining an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension; Determine an abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension; The abnormal dimension and the abnormal dimension value are output.

2. The method according to claim 1, characterized in that: The acquiring of first time series data under a preset dimension during a normal period of the system and second time series data under the preset dimension during an abnormal period of the system includes: Obtaining initial time series data of the monitored system under the preset dimension; Preprocessing the initial time series data to obtain time series data; Perform anomaly detection on the time series data based on an anomaly detection algorithm to obtain an anomaly detection result; Based on the abnormality detection result, the first time series data in the normal period of the system and the second time series data in the abnormal period of the system are screened out from the time series.

3. The method according to claim 1, characterized in that The determining, based on the information entropy corresponding to the first time series data and the second time series data, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold in the preset dimension includes: Based on the first time series data, a first information entropy of the preset dimension during the normal period of the system is calculated; Based on the second time series data, a second information entropy of the preset dimension during the system abnormality period is calculated; Based on the first information entropy and the second information entropy, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold is screened out from the preset dimensions.

4. The method according to claim 3, characterized in that The step of screening out, from the preset dimensions, an abnormal dimension whose influence coefficient on the system abnormality is greater than a threshold based on the first information entropy and the second information entropy, includes: Calculating an entropy difference between the first information entropy and the second information entropy; The preset dimensions whose entropy difference value is greater than the entropy difference threshold are screened out from the preset dimensions, and the preset dimensions whose entropy difference value is greater than the entropy difference threshold are used as abnormal dimensions whose influence coefficient on system abnormality is greater than the threshold.

5. The method according to claim 1, characterized in that The determining the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension includes: Obtain a first data vector of first time series data of each dimension value under the abnormal dimension, and a second data vector of second time series data of each dimension value under the abnormal dimension; Calculating the similarity between the first data vector and the second data vector; Dimension values ​​whose similarity is less than a similarity threshold are screened out from the dimensional values ​​under the abnormal dimension to serve as the abnormal dimensional value.

6. The method according to claim 5, characterized in that The calculating and obtaining the similarity between the first data vector and the second data vector includes: Calculate and obtain the cosine similarity between the first data vector and the second data vector, and use the cosine similarity as the similarity; or The Euclidean distance between the first data vector and the second data vector is calculated and used as the similarity.

7. The method according to claim 2, characterized in that The preprocessing of the initial time series data to obtain the time series data includes: Performing data cleaning processing on the initial time series data to obtain cleaned time series data; In the case where there are missing data in the cleaned time series data, performing missing data filling processing on the cleaned time series data to obtain filled time series data; The filled time series data is normalized to obtain the time series data.

8. An abnormality locating device, characterized in that: include: A time series data acquisition module, used to acquire first time series data under a preset dimension during a normal period of the system, and second time series data under the preset dimension during an abnormal period of the system; An abnormal dimension determination module, used to determine an abnormal dimension whose influence coefficient on system abnormality is greater than a threshold in the preset dimensions based on the information entropy corresponding to the first time series data and the second time series data respectively; An abnormal dimension value determination module, used to determine the abnormal dimension value under the abnormal dimension based on the similarity corresponding to the first time series data and the second time series data under the abnormal dimension; The abnormal dimension value output module is used to output the abnormal dimension and the abnormal dimension value.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 7 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

11. A computer program product comprising instructions, characterized in that When the method is executed on a computer, the computer is enabled to execute the method according to any one of claims 1 to 7.

Images