Abnormal index detection method and device for distributed architecture system, and medium
The method enhances distributed architecture system monitoring by using key indicator data and Gaussian fuzzy technology for real-time anomaly detection, addressing delayed fault identification and improving accuracy and stability.
Patent Information
- Application Number
- CN202411229275.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-03
- Publication Date
- 2024-12-06
AI Technical Summary
Existing distributed architecture systems face challenges with delayed fault identification and low fault detection accuracy, leading to long problem discovery cycles and affecting user experience due to their large scale and complex application scenarios, as well as high costs for model training.
A method utilizing key indicator data collection, cluster analysis, and Gaussian fuzzy technology to construct a normal monitoring indicator fuzzy model, enabling real-time anomaly detection and accurate identification of abnormal indicators through data anomaly matching and warning processing.
Enables real-time monitoring and quick, accurate detection of anomalies, improving system stability by reducing noise and outlier impact, and promptly notifying personnel to prevent further deterioration.
Smart Images

Figure CN119088660A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of intelligent operation and maintenance, and in particular to an abnormal indicator detection method, device and medium for a distributed architecture system. Background Art
[0002] With the rapid development of information technology, distributed architecture systems represented by microservice technology are increasingly being used in various fields. As the scale and complexity of systems increase, how to ensure the stable operation of the system has become a problem that many software vendors have to face.
[0003] During the operation of the system, various abnormalities and instabilities such as hardware failures, software defects, network delays, etc. may lead to system performance degradation or even system failure. Therefore, real-time monitoring and abnormality detection of distributed architecture systems are of great significance to ensure stable operation of the system and improve system reliability.
[0004] In operation and maintenance practice, the problem of delayed fault identification is common. With the rapid development of observability technology, the state observation capability of software and hardware has been greatly improved. However, due to various reasons such as the large scale of the system and the high complexity of domain knowledge, it is impossible to judge the system status when a problem occurs, resulting in a long problem discovery cycle and affecting user experience. There are several typical problems that need to be solved urgently: First, the system is large in scale and the application scenarios are complex. The accuracy of abnormal indicators identified by fixed thresholds and other methods is low and cannot meet the requirements of large projects; second, the cost of model training is too high, and it is impossible to train reliable models for systems with different characteristics. Summary of the invention
[0005] The embodiments of the present application provide a method, device and medium for detecting abnormal indicators of a distributed architecture system, which are used to solve the following technical problems: During the operation and maintenance of existing distributed architecture systems, there is a lag in fault identification and low fault detection accuracy, which easily leads to a long problem discovery cycle and affects the user experience.
[0006] The present application embodiment adopts the following technical solutions:
[0007] On the one hand, an embodiment of the present application provides an abnormal indicator detection method for a distributed architecture system, including: collecting key indicator data of an accessed observed system; performing cluster analysis on the key indicator data, and constructing a normal monitoring indicator fuzzy model through Gaussian fuzzy technology; performing data anomaly matching processing on the observed current monitoring indicator data through the normal monitoring indicator fuzzy model to obtain abnormal monitoring indicator data; and performing warning processing on the abnormal monitoring indicator data.
[0008] The embodiment of the present application can understand the operating status of the system in real time and detect abnormal situations in time by collecting key indicator data of the accessed observed system. Then, cluster analysis is performed using the key indicator data to classify similar data points into one category, which helps to discover patterns and laws in the data. And the use of Gaussian fuzzy technology to construct a fuzzy model of normal monitoring indicators can reduce the impact of noise and outliers on the model and improve the accuracy and stability of the model. At the same time, abnormal monitoring indicator data can also be quickly and accurately identified to improve the efficiency and accuracy of anomaly detection. In addition, relevant personnel can be notified in time to take measures to avoid further deterioration of abnormal situations and ensure the stable operation of the system.
[0009] In a feasible implementation manner, collecting key indicator data of the accessed observed system specifically includes: accessing the historical monitoring data in the observed system to the data management module of the distributed architecture system; wherein the historical monitoring data at least includes: Spring Boot Actuator and Prometheus exportor; according to the data characteristics of the observed system, labeling and collecting the indicator data in the historical monitoring data that has a high degree of correlation with the data characteristics, to obtain the key indicator data based on the historical monitoring data; wherein the key indicator data at least includes: CPU occupancy, memory occupancy, disk I / O rate, network traffic, response time, error rate, number of threads / number of processes, queue length, transaction processing volume and cache hit rate.
[0010] In a feasible implementation, cluster analysis is performed on the key indicator data, specifically including: performing data sequence generation processing on the key indicator data to obtain a sample data sequence; calculating the mean and standard deviation of the sample data sequence; identifying and eliminating abnormal data from the sample data sequence through the absolute difference between the mean and the standard deviation in the sample data sequence to obtain normal monitoring indicator data that only retains normal data; performing mean clustering processing on the normal monitoring indicator data under relevant clustering dimensions to obtain multi-category monitoring indicator data; wherein the clustering dimensions include at least: server role, technical field and business cycle; dividing the multi-category monitoring indicator data into normal indicator types to obtain a number of normal category monitoring indicator data; wherein the normal category monitoring indicator data includes: a normal monitoring indicator center and a normal monitoring indicator standard deviation.
[0011] In a feasible implementation, a fuzzy model of normal monitoring indicators is constructed through Gaussian fuzzy technology, specifically including: using the Gaussian fuzzy technology and based on several normal category monitoring indicator data, Gaussian fuzzy models are established respectively; each Gaussian fuzzy model is fuzzy trained with respect to the actual indicator value under the membership function to construct the fuzzy model of normal monitoring indicators.
[0012] In a possible implementation, according to Obtain the membership function of the Gaussian fuzzy model Among them, x t Indicates the actual indicator value of the tth normal category monitoring indicator data, For the Gaussian blur model The center of For the Gaussian blur model Width.
[0013] In a feasible implementation, the observed current monitoring indicator data is processed for data anomaly matching through the normal monitoring indicator fuzzy model to obtain abnormal monitoring indicator data, specifically including: inputting the current monitoring indicator data into the normal monitoring indicator fuzzy model; calculating the membership of the current monitoring indicator data to the normal monitoring indicator fuzzy model according to a preset fuzzy matching threshold; based on the membership of the normal monitoring indicator fuzzy model, calculating the matching degree between the current monitoring indicator data and the normal monitoring indicator fuzzy model to obtain a comprehensive matching degree; if the comprehensive matching degree is zero, then the current monitoring indicator data contains abnormal indicator data, and the abnormal monitoring indicator data is determined; if the comprehensive matching degree is not zero, then the current monitoring indicator data does not contain abnormal indicator data, and is determined to be normal monitoring indicator data.
[0014] In a feasible implementation, the abnormal monitoring indicator data is subjected to warning processing, specifically comprising: generating warning prompt information based on the abnormal monitoring indicator data; wherein the warning prompt information comprises: the time of the abnormal data, the system information where the abnormal data is located, the type of abnormal data, the node information where the abnormal data is located, the offset between the abnormal data and the normal data, and the scenarios to which the abnormal data may be related; performing fault decision analysis on the abnormal monitoring indicator data to determine a disposal action decision based on the observed system; wherein the disposal action decision comprises at least: triggering a system restart and traffic restriction.
[0015] In a feasible implementation, after the observed current monitoring indicator data is processed for data anomaly matching through the normal monitoring indicator fuzzy model to obtain abnormal monitoring indicator data, the method also includes: if the current monitoring indicator data is normal monitoring indicator data, the normal monitoring indicator data is judged as belonging to a category to obtain the belonging category position of the normal monitoring indicator data; based on the belonging category position and according to the normal monitoring indicator center and the normal monitoring indicator standard deviation in the normal category monitoring indicator data, the current monitoring indicator data is updated accordingly; based on the updated normal monitoring indicator center and the normal monitoring indicator standard deviation, the center of the Gaussian fuzzy model and the width of the Gaussian fuzzy model in the normal monitoring indicator fuzzy model are updated to complete the adaptive update of the normal monitoring indicator fuzzy model.
[0016] In a second aspect, an embodiment of the present application also provides an abnormal indicator detection device for a distributed architecture system, the device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor so that the at least one processor can execute a method for detecting abnormal indicators in a distributed architecture system as described in any of the above-mentioned embodiments.
[0017] In a third aspect, an embodiment of the present application further provides a non-volatile computer storage medium, characterized in that the storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by a terminal, the terminal executes an abnormal indicator detection method for a distributed architecture system described in any of the above-mentioned embodiments.
[0018] The present application provides a method, device and medium for detecting abnormal indicators in a distributed architecture system. Compared with the prior art, the embodiments of the present application have the following beneficial technical effects:
[0019] 1. Real-time monitoring: By collecting key indicator data of the accessed observed system, the operating status of the system can be understood in real time and abnormal situations can be discovered in time.
[0020] 2. Cluster analysis: Cluster analysis of key indicator data can group similar data points into one category, which helps to discover patterns and regularities in the data.
[0021] 3. Gaussian blur technology: Using Gaussian blur technology to build a fuzzy model of normal monitoring indicators can reduce the impact of noise and outliers on the model and improve the accuracy and stability of the model.
[0022] 4. Data anomaly matching processing: By matching the observed current monitoring indicator data with the normal monitoring indicator fuzzy model, abnormal monitoring indicator data can be identified quickly and accurately, improving the efficiency and accuracy of anomaly detection.
[0023] 5. Warning processing: Warning processing of abnormal monitoring indicator data can promptly notify relevant personnel to take measures to avoid further deterioration of abnormal conditions and ensure the stable operation of the system. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:
[0025] Figure 1 A flow chart of an abnormal indicator detection method for a distributed architecture system provided in an embodiment of the present application;
[0026] Figure 2 A schematic diagram of the structure of an abnormal indicator detection device for a distributed architecture system provided in an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the technical solutions in this application, the technical solutions in the embodiments of this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are only part of the embodiments of this application, not all of them. Based on the embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this application.
[0028] The present application embodiment provides a method for detecting abnormal indicators in a distributed architecture system, such as Figure 1 As shown, the abnormal indicator detection method of the distributed architecture system specifically includes steps S101-S104:
[0029] S101. Collect key indicator data of the accessed observed system.
[0030] Specifically, first, the historical monitoring data in the observed system needs to be connected to the data management module of the distributed architecture system. The historical monitoring data at least includes: Spring Boot Actuator and Prometheus Explorer.
[0031] Furthermore, according to the data characteristics of the observed system, the indicator data with a high degree of correlation with the data characteristics in the historical monitoring data are annotated and collected to obtain the key indicator data based on the historical monitoring data. Among them, the key indicator data at least includes: CPU occupancy, memory occupancy, disk I / O rate, network traffic, response time, error rate, number of threads / processes, queue length, transaction volume and cache hit rate.
[0032] In one embodiment, the data management module of the distributed architecture system can set and manage access to monitoring data. In combination with the capabilities and characteristics of the observed system, it is possible to configure access to monitoring indicator data from built-in or external observation components, such as Spring Boot Actuator, Prometheus exportor, etc. And according to the characteristics of the observed system, key indicator data with high correlation with system stability can be selected for key collection and analysis. For example, the CPU occupancy rate, memory occupancy rate, disk I / O rate, network traffic, response time, error rate, number of threads / number of processes, queue length, transaction volume, cache hit rate, etc. of each server.
[0033] S102. Perform cluster analysis on key indicator data and construct a fuzzy model of normal monitoring indicators through Gaussian fuzzy technology.
[0034] Specifically, it is also necessary to generate a data sequence for the key indicator data to obtain a sample data sequence. The mean and standard deviation of the sample data sequence are calculated. Through the absolute difference between the mean and the standard deviation in the sample data sequence, the sample data sequence is identified and eliminated for abnormal data to obtain normal monitoring indicator data that only retains normal data.
[0035] In one embodiment, among the various distributed architecture system monitoring indicator data sets constructed, the t-th key indicator data to be processed is a data sequence with N samples: Calculate the mean c of the t-th sample data sequence t and standard deviation sdv t Then determine the state of the sample data sequence: if the i-th data With mean c t If the absolute value of the difference does not exceed 3 times the standard deviation, the data is considered normal, otherwise it is considered abnormal. After the processing is completed, the abnormal data is kicked out and only the normal data is retained. Finally, the remaining M normal data form the normal data sequence of the tth normal monitoring indicator data.
[0036] Furthermore, the normal monitoring indicator data is subjected to mean clustering processing under the relevant clustering dimensions to obtain multi-category monitoring indicator data. The clustering dimensions include at least: server role, technical field and business cycle.
[0037] In one embodiment, according to the data indicator data characteristics and fields, a fast k-means clustering method is used to divide the preprocessed normal monitoring indicator data into several categories to obtain multi-category monitoring indicator data; wherein the clustering dimensions are divided from the server role to which the indicator belongs (database server, web server, middleware server, etc.), technical field (database, application, middleware), business cycle (working hours, non-working hours) and other dimensions.
[0038] Furthermore, the multi-category monitoring indicator data is divided into normal indicator types to obtain a number of normal category monitoring indicator data, wherein the normal category monitoring indicator data includes: a normal monitoring indicator center and a normal monitoring indicator standard deviation.
[0039] In one embodiment, there are K types of normal monitoring indicator data, and the t-th type of multi-category monitoring indicator data is divided into K normal monitoring indicator categories by the k-means clustering algorithm; the centers of the K normal monitoring indicators are The standard deviation of normal monitoring indicators is
[0040] Furthermore, Gaussian fuzzy technology is used to establish Gaussian fuzzy models based on several normal category monitoring indicator data. Each Gaussian fuzzy model is fuzzy trained with the actual indicator value under the membership function to construct a normal monitoring indicator fuzzy model.
[0041] As a feasible implementation method, under the jth normal monitoring indicator category in the normal category monitoring indicator data, and according to Obtain the membership function of the Gaussian fuzzy model Among them, x t Indicates the actual indicator value of the tth normal category monitoring indicator data, Gaussian blur model The center of Gaussian blur model Width.
[0042] S103: Perform data anomaly matching processing on the observed current monitoring indicator data through the normal monitoring indicator fuzzy model to obtain abnormal monitoring indicator data.
[0043] Specifically, the current monitoring index data is first input into the normal monitoring index fuzzy model.
[0044] Furthermore, a preset fuzzy matching threshold is used to calculate the membership of the current monitoring indicator data to the normal monitoring indicator fuzzy model.
[0045] Furthermore, based on the membership degree of the normal monitoring indicator fuzzy model, the matching degree between the current monitoring indicator data and the normal monitoring indicator fuzzy model is calculated to obtain a comprehensive matching degree.
[0046] If the comprehensive matching degree is zero, then there are abnormal indicator data in the current monitoring indicator data, and the abnormal monitoring indicator data is determined.
[0047] If the comprehensive matching degree is not zero, the current monitoring indicator data does not contain abnormal indicator data and is determined to be normal monitoring indicator data.
[0048] In one embodiment, if the newly observed t-th current monitoring indicator data x t If the matching degree with the fuzzy model of each normal monitoring indicator is poor, then the indicator is abnormal monitoring indicator data. The judgment criteria are as follows: set the fuzzy matching threshold ε, ε is 0.01, 0.02 or 0.05, and calculate the newly observed t-th current monitoring indicator data x t The membership degree of K normal monitoring indicator fuzzy models is calculated, and then each normal monitoring indicator fuzzy model is solved, for example: Then calculate the newly observed tth current monitoring indicator data x t The matching degree S with the fuzzy model of normal monitoring indicators of each category j , j=1, 2, ..., K, that is, the comprehensive matching degree S is obtained, and then the abnormal indicator data of the current monitoring indicator data is judged. If S=0, the current monitoring indicator data is detected to be abnormal and the user is warned, otherwise the monitoring indicator data is normal.
[0049] As a feasible implementation method, if the current monitoring indicator data is normal monitoring indicator data, the normal monitoring indicator data is judged to belong to a certain category to obtain the belonging category position of the normal monitoring indicator data. Based on the belonging category position and according to the normal monitoring indicator center and the normal monitoring indicator standard deviation in the normal category monitoring indicator data, the current monitoring indicator data is updated accordingly. Based on the updated normal monitoring indicator center and the normal monitoring indicator standard deviation, the center of the Gaussian fuzzy model and the width of the Gaussian fuzzy model in the normal monitoring indicator fuzzy model are updated to complete the adaptive update of the normal monitoring indicator fuzzy model.
[0050] In one embodiment, the system supports automatic updating of the fuzzy model of normal monitoring indicators. When the comprehensive matching degree S ≥ 1, it is determined to be normal monitoring indicator data, and the normal indicator data x tAdopt an adaptive update algorithm and add it to the category library to monitor the real-time update of the fuzzy model of indicator data, and further improve the accuracy of the fuzzy model. The steps of the adaptive update algorithm include:
[0051] 1) Determine normal monitoring indicator data x t Category of attribution: If Then the data is classified into class j.
[0052] 2) According to the normal monitoring indicator center in the normal category monitoring indicator data and the standard deviation of normal monitoring indicators Update the normal monitoring indicator center of normal indicator category j in the current monitoring indicator data and the standard deviation of normal monitoring indicators
[0053] 3) Monitor indicators based on the new normal and the standard deviation of normal monitoring indicators Update the center of the jth fuzzy model and the fuzzy model width This enables adaptive updating of the fuzzy model of normal monitoring indicators.
[0054] S104. Perform warning processing on abnormal monitoring indicator data.
[0055] Specifically, it is also necessary to generate warning information based on abnormal monitoring indicator data, wherein the warning information includes: the time of abnormal data, the system information where the abnormal data is located, the type of abnormal data, the node information where the abnormal data is located, the offset between the abnormal data and the normal data, and the scenarios that the abnormal data may be related to.
[0056] Furthermore, a fault decision analysis is performed on the abnormal monitoring indicator data to determine a disposal action decision based on the observed system, wherein the disposal action decision at least includes: triggering a system restart and traffic restriction.
[0057] In one embodiment, the abnormal monitoring indicator data found in the inspection is automatically triggered to generate warning prompt information, and the text information of the prompt information includes the time when the abnormal data appears, the system information where the abnormal data is located, the abnormal data type, the node information where the abnormal data is located, the offset between the abnormal data and the normal data, and the scenarios that the abnormal data may be related to. It can also support the addition of quick fault handling actions according to the characteristics of the system, such as triggering a system restart, traffic restriction, etc.
[0058] In addition, the embodiment of the present application also provides an abnormal indicator detection device for a distributed architecture system, such as Figure 2 As shown, the abnormal indicator detection device 200 of the distributed architecture system specifically includes:
[0059] At least one processor 201. And a memory 202 in communication with the at least one processor 201. The memory 202 stores instructions that can be executed by the at least one processor 201, so that the at least one processor 201 can execute:
[0060] Collect key indicator data of the accessed observed system;
[0061] Cluster analysis is performed on key indicator data, and a fuzzy model of normal monitoring indicators is constructed through Gaussian fuzzy technology;
[0062] Through the normal monitoring indicator fuzzy model, the observed current monitoring indicator data is processed for data anomaly matching to obtain abnormal monitoring indicator data;
[0063] Provide warnings for abnormal monitoring indicator data.
[0064] The embodiment of the present application can understand the operating status of the system in real time and detect abnormal situations in time by collecting key indicator data of the accessed observed system. Then, cluster analysis is performed using the key indicator data to classify similar data points into one category, which helps to discover patterns and laws in the data. And the use of Gaussian fuzzy technology to construct a fuzzy model of normal monitoring indicators can reduce the impact of noise and outliers on the model and improve the accuracy and stability of the model. At the same time, abnormal monitoring indicator data can also be quickly and accurately identified to improve the efficiency and accuracy of anomaly detection. In addition, relevant personnel can be notified in time to take measures to avoid further deterioration of abnormal situations and ensure the stable operation of the system.
[0065] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and non-volatile computer storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.
[0066] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.
[0067] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.
[0068] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0069] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.
[0070] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.
[0071] The above describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.
[0072] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the embodiments of the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present application should be included in the scope of the claims of the present application.
Claims
1. A method for detecting abnormal indicators in a distributed architecture system, characterized in that: The method comprises: Collect key indicator data of the accessed observed system; Perform cluster analysis on the key indicator data and construct a fuzzy model of normal monitoring indicators through Gaussian fuzzy technology; Through the normal monitoring indicator fuzzy model, the observed current monitoring indicator data is processed for data anomaly matching to obtain abnormal monitoring indicator data; Perform warning processing on the abnormal monitoring indicator data.
2. The method for detecting abnormal indicators in a distributed architecture system according to claim 1, characterized in that: Collect key indicator data of the accessed observed system, including: Connecting the historical monitoring data in the observed system to the data management module of the distributed architecture system; wherein the historical monitoring data at least includes: Spring Boot Actuator and Prometheus exportor; According to the data characteristics of the observed system, the indicator data with a high degree of correlation with the data characteristics in the historical monitoring data are labeled and collected to obtain the key indicator data based on the historical monitoring data; wherein the key indicator data include at least: CPU occupancy, memory occupancy, disk I / O rate, network traffic, response time, error rate, number of threads / number of processes, queue length, transaction processing volume and cache hit rate.
3. The method for detecting abnormal indicators in a distributed architecture system according to claim 1, characterized in that: Performing cluster analysis on the key indicator data, specifically including: Performing data sequence generation processing on the key indicator data to obtain a sample data sequence; Calculate the mean and standard deviation of the sample data sequence; By using the absolute difference between the mean and the standard deviation in the sample data sequence, the sample data sequence is subjected to identification and elimination processing of abnormal data, so as to obtain normal monitoring indicator data retaining only normal data; Performing mean clustering processing on the normal monitoring indicator data under relevant clustering dimensions to obtain multi-category monitoring indicator data; wherein the clustering dimensions at least include: server role, technical field and business cycle; The multi-category monitoring indicator data are divided into normal indicator types to obtain a number of normal category monitoring indicator data; wherein the normal category monitoring indicator data includes: a normal monitoring indicator center and a normal monitoring indicator standard deviation.
4. The method for detecting abnormal indicators in a distributed architecture system according to claim 3, characterized in that: Through Gaussian fuzzy technology, a fuzzy model of normal monitoring indicators is constructed, including: By using the Gaussian blur technology and based on a number of the normal category monitoring indicator data, Gaussian blur models are established respectively; Each of the Gaussian fuzzy models is subjected to fuzzy training of the actual indicator value under the membership function to construct the normal monitoring indicator fuzzy model.
5. The method for detecting abnormal indicators in a distributed architecture system according to claim 4, characterized in that: according to Obtain the membership function of the Gaussian fuzzy model Among them, x t Indicates the actual indicator value of the tth normal category monitoring indicator data, is the Gaussian blur model The center of For the Gaussian blur model Width.
6. The method for detecting abnormal indicators in a distributed architecture system according to claim 1, characterized in that: Through the normal monitoring indicator fuzzy model, the observed current monitoring indicator data is processed for data anomaly matching to obtain abnormal monitoring indicator data, specifically including: Inputting the current monitoring indicator data into the normal monitoring indicator fuzzy model; According to a preset fuzzy matching threshold, calculating the membership degree of the current monitoring indicator data to the normal monitoring indicator fuzzy model; Based on the membership degree of the normal monitoring indicator fuzzy model, the matching degree between the current monitoring indicator data and the normal monitoring indicator fuzzy model is calculated to obtain a comprehensive matching degree; If the comprehensive matching degree is zero, then the current monitoring indicator data has abnormal indicator data, and the abnormal monitoring indicator data is determined; If the comprehensive matching degree is not zero, the current monitoring indicator data does not contain abnormal indicator data and is determined to be normal monitoring indicator data.
7. The method for detecting abnormal indicators in a distributed architecture system according to claim 1, characterized in that: Performing warning processing on the abnormal monitoring indicator data, specifically including: Generate warning information based on the abnormal monitoring indicator data; wherein the warning information includes: the time of the abnormal data, the system information where the abnormal data is located, the type of abnormal data, the node information where the abnormal data is located, the offset between the abnormal data and the normal data, and the scenarios that the abnormal data may be related to; Performing fault decision analysis on the abnormal monitoring indicator data to determine a handling action decision based on the observed system; wherein the handling action decision at least includes: triggering a system restart and traffic restriction.
8. The method for detecting abnormal indicators in a distributed architecture system according to claim 1, characterized in that: After the observed current monitoring indicator data is subjected to data anomaly matching processing by using the normal monitoring indicator fuzzy model to obtain abnormal monitoring indicator data, the method further includes: If the current monitoring indicator data is normal monitoring indicator data, then performing a classification judgment on the normal monitoring indicator data to obtain a classification position of the normal monitoring indicator data; Based on the belonging category position and according to the normal monitoring indicator center and the normal monitoring indicator standard deviation in the normal category monitoring indicator data, the current monitoring indicator data is updated accordingly; Based on the updated normal monitoring indicator center and the normal monitoring indicator standard deviation, the center of the Gaussian fuzzy model and the width of the Gaussian fuzzy model in the normal monitoring indicator fuzzy model are updated to complete the adaptive update of the normal monitoring indicator fuzzy model.
9. An abnormal indicator detection device for a distributed architecture system, characterized in that: The device comprises: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, so that the at least one processor can execute the abnormal indicator detection method for a distributed architecture system according to any one of claims 1-8.
10. A non-volatile computer storage medium, characterized in that: The storage medium is a non-volatile computer-readable storage medium, and the non-volatile computer-readable storage medium stores at least one program, each of which includes instructions, and when the instructions are executed by the terminal, the terminal executes the abnormal indicator detection method for a distributed architecture system according to any one of claims 1-8.
Citation Information
Cited By
High-reliability multi-mode anomaly detection method and system for micro-service system
CN119739591A
State monitoring method and equipment based on industrial dust removal system and medium
CN121048945A
