High cardinal number management method, device and equipment for intelligent micro-service monitoring system

Abstract

The invention provides an intelligent micro-service monitoring system high cardinal number management method, device and equipment, and the method comprises the steps: carrying out the collection of micro-service monitoring indexes, and enabling the collected micro-service monitoring indexes to comprise universal monitoring indexes needed by all services and business-related customized monitoring indexes; according to the collected micro-service monitoring indexes, carrying out effective convergence on partial monitoring index divergence dimensions; invalid time sequence data in the monitoring indexes are eliminated regularly; performing centralized storage on the monitoring index data; carrying out intelligent cutting on unreasonable monitoring index labels; and carrying out statistics on the stored monitoring index data, and carrying out routing inspection treatment on the high cardinal number indexes regularly. According to the monitoring system full-link high-cardinal-number governance scheme, the possibility of occurrence of the high-cardinal-number problem can be effectively reduced, the fault tolerance and the accuracy of the monitoring system are improved, and the reliability and the performance of the monitoring system are guaranteed.

Description

Technical Field

[0001] This application relates to the field of microservice monitoring, and in particular to a method, apparatus and equipment for managing high cardinality in an intelligent microservice monitoring system. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this application as set forth in the claims. The description herein is not to be construed as prior art simply because it is included in this section.

[0003] Microservices are a software architecture style that builds applications as independently deployed, lightweight service units. Each service implements specific business functions and communicates through an Application Programming Interface (API). In cloud-native microservice architectures, the sheer number of services and their complex inter-service dependencies, coupled with the dynamic scaling of service instances and the complexity of API call chains, significantly increases the demands on monitoring. To provide comprehensive and accurate monitoring information, the number and tag dimensions of monitoring metrics are enormous, resulting in an exponential growth in monitoring metrics. Monitoring metrics used to quantify system operational status include not only general performance metrics such as queries per second (QPS), interface latency, and success rate, but also business-defined metrics. The monitoring system collects and stores these metrics uniformly. To ensure the completeness of monitoring data, metric collection must include multi-dimensional tags such as cluster, instance, and interface. However, various unreasonable metric definitions and tag usage can lead to highly divergent metric dimensions, resulting in a high cardinality problem. This means that a single monitoring metric may experience performance degradation in storage and querying due to an excessively large number of tag combinations (typically exceeding 100,000). High cardinality poses a significant challenge to time-series databases used for monitoring, causing not only slow or failed queries but also write failures leading to lost monitoring data, severely impacting storage stability.

[0004] In existing technologies, a passive "pollute first, clean up later" approach is adopted to address the high initial pollution problem. This passive approach has the following problems: The governance approach is too simplistic: the governance methods for high-base numbers are limited to discontinuing the collection of indicators, resulting in a decrease in monitoring coverage. Poor timeliness: Some invalid time series cannot be detected in time, and continuous collection leads to waste of resources; Insufficient visualization: The system lacks an effective visualization interface for the distribution of indicator bases and the management of high base numbers, making it difficult to fully understand the current status of the monitoring system; Low level of automation: There is a lack of a complete pipeline from baseline detection and root cause analysis to automated governance, and reliance on manual intervention is inefficient; Long governance cycle: Governance relies on manual analysis and investigation, has a long governance cycle and is a short-term governance solution that can only address existing indicators and cannot control new indicators. Poor monitoring stability: The storage layer lacks a pre-filtering mechanism, and high cardinality indicators directly impact the time-series database, easily leading to memory exhaustion and OOM crashes.

[0005] In summary, microservice architectures involve numerous services with highly complex inter-service dependencies. To provide comprehensive and accurate monitoring information, the number of monitoring metrics and tag dimensions are enormous, resulting in high cardinality, which poses a significant challenge to storage stability. Existing monitoring systems have limited solutions for managing high cardinality. Most high cardinality issues are only discovered after storage node failures or query performance degradation, leading to targeted remediation efforts. Furthermore, most solutions rely on manual identification and management of high cardinality metrics by developers or operations personnel, lacking automated processes. This results in high management costs and potential impacts on storage stability, compromising monitoring availability. Therefore, there is an urgent need to propose an intelligent microservice monitoring system with a high cardinality management method, device, and equipment that can address the aforementioned technical challenges. Summary of the Invention

[0006] This application provides a method, apparatus, and equipment for managing high cardinality in an intelligent microservice monitoring system, thereby improving the fault tolerance and accuracy of the monitoring system.

[0007] One aspect of this application provides a method for managing high cardinality in an intelligent microservice monitoring system, the method comprising: Collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. Based on the collected microservice monitoring metrics, the divergent dimensions of some monitoring metrics are effectively converged. Periodically eliminate invalid time-series data from monitoring indicators; Centralized storage of monitoring indicator data; For unreasonable monitoring indicator labels, intelligent cropping is performed; Statistical analysis of stored monitoring data is conducted, and high-base-value indicators are regularly inspected and addressed.

[0008] Furthermore, the effective convergence includes convergence of dynamic interface tags, convergence of static resource files, and convergence of meaningless strings such as random strings. A real-time pattern recognition and regularization processing mechanism is established for automatic convergence and aggregation.

[0009] Furthermore, the update time of the time series data of the monitoring indicators is recorded on the acquisition side, and time series that have not been updated for a certain period of time are automatically eliminated. Specifically, this includes: designing a time series data activity detection algorithm based on time windows to automatically identify and clean up zombie time series that exceed the TTL threshold; and performing periodic aging and elimination on the storage write side for time series without subsequent data.

[0010] Furthermore, the monitoring metrics data are stored using the time-series database VictoriaMetrics, and a storage proxy layer is added in front of VictoriaMetrics to handle dual writes, rate reduction, and high cardinality management, thereby further enhancing the stability of VictoriaMetrics.

[0011] Furthermore, the system performs statistical analysis on the monitoring metrics written to storage and outputs the corresponding results. For unreasonable monitoring metric labels, it performs intelligent trimming to prevent excessive expansion of metric labels. It also introduces a storage agent to provide a global sample statistical view of the monitoring metrics written to storage, and performs further intelligent trimming and management of the monitoring metrics written to storage based on this view.

[0012] Furthermore, based on the statistical analysis of the stored monitoring metrics data, high-baseline metrics are regularly inspected and managed, invalid or unused tags are removed, monitoring metrics that are no longer used are stopped from being collected, and the storage cycle of monitoring metrics that do not have long-term backtracking needs is shortened to avoid the rapid expansion of time series affecting the stability of storage and querying; a multi-dimensional evaluation model is established, including time series growth rate, metric sample size, and metric tag dispersion, to achieve periodic automated inspection and management.

[0013] Another aspect of this application provides a high cardinality management device for an intelligent microservice monitoring system, the device comprising: The collection unit is used to collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. The convergence unit effectively converges some of the divergent dimensions of the collected microservice monitoring metrics. The elimination unit is used to periodically eliminate invalid time-series data from monitoring indicators; The storage unit uses a time-series database to centrally store monitoring indicator data; The trimming unit intelligently trims unreasonable monitoring indicator labels; The governance unit performs statistical analysis on the stored monitoring indicator data and conducts regular inspections and governance of indicators with high baseline values.

[0014] In another aspect, this application provides an electronic device, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which enables the at least one processor to implement the high cardinality governance method of the intelligent microservice monitoring system.

[0015] In another aspect, this application provides a computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor to implement the high cardinality governance method of the intelligent microservice monitoring system.

[0016] In another aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the high cardinality governance method of the intelligent microservice monitoring system.

[0017] The high cardinality governance method, apparatus, and equipment of the intelligent microservice monitoring system provided in this application embodiment introduce an intelligent indicator tag convergence algorithm at the data acquisition end and periodically eliminate invalid data by writing a self-developed software development kit (SDK). Simultaneously, it introduces monitoring indicator cardinality statistics and a visualization interface at the storage end, and automates the governance of some high cardinality indicators through periodic inspections, continuously ensuring the stable availability of the monitoring system. This invention solves the problem that existing monitoring systems have limited solutions for high cardinality issues, and most high cardinality problems are only discovered after storage nodes fail or query performance degrades, requiring targeted governance only then. The high cardinality governance method of the intelligent microservice monitoring system does not rely on manual identification and governance of high cardinality indicators by developers or operations personnel, establishes an automated governance process, has low governance costs, does not affect storage stability, and ensures monitoring availability. Compared with the prior art, this invention has the following technical advantages: Improve operational efficiency: Through intelligent convergence and indicator pruning algorithms, meaningless indicator dimensions can be automatically converged and pruned, reducing the workload of manual processing and improving operational efficiency.

[0018] Provides a visual interface: It provides a global storage statistics view, making the storage status clear at a glance, enabling comprehensive observation of the monitoring system, and facilitating a full understanding of the overall operating status of the monitoring system.

[0019] Accelerate problem localization: When a monitoring system malfunctions, the root cause can be quickly located by examining the monitoring system's observation points, reducing troubleshooting time and improving response speed.

[0020] Low resource overhead: It can promptly detect and discard invalid time series, and stop collecting and writing to monitoring indicators or tags that are no longer used. It also shortens the storage cycle for indicators that do not require long-term storage, further improving resource utilization.

[0021] Improve system stability: By managing high-cardinality indicators, the risk of storage failure can be effectively reduced. At the same time, the visual observation interface can help to discover and resolve potential problems in a timely manner, thereby improving the stability and reliability of the monitoring system.

[0022] Highly adaptable: The system possesses self-optimization capabilities, enabling it to optimize and manage monitoring metrics, continuously improve high-baseline metrics, and enhance the effectiveness and reliability of monitoring metrics storage.

[0023] In summary, this invention proposes a high cardinality governance scheme for the entire monitoring system chain, which can effectively reduce the possibility of high cardinality problems, improve the fault tolerance and accuracy of the monitoring system, and ensure the reliability and performance of the monitoring system. Attached Figure Description

[0024] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 A flowchart illustrating a high cardinality management method for an intelligent microservice monitoring system provided in an embodiment of this application; Figure 2 A flowchart illustrating a high cardinality management method for an intelligent microservice monitoring system provided in another embodiment of this application; Figure 3 A schematic diagram of the structure of a high cardinality governance device for an intelligent microservice monitoring system provided in an embodiment of this application; Figure 4 A schematic diagram of the structure of a high cardinality governance device for an intelligent microservice monitoring system provided in another embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device suitable for implementing the solutions in the embodiments of this application; The same or similar reference numerals in the accompanying drawings represent the same or similar parts. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0027] In a typical configuration of this application, the terminal and the service network devices each include one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0028] Memory may include non-persistent storage in computer-readable storage media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable storage media.

[0029] Computer-readable storage media include permanent and non-permanent, removable and non-removable media, which can store information by any method or technology. Information can be computer program instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, read-only optical disc (CD-ROM), digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0030] This application provides a method for managing high cardinality in an intelligent microservice monitoring system, which includes: Step 1: Collect microservice monitoring metrics. The collected microservice monitoring metrics include general monitoring metrics required by all services and business-related custom monitoring metrics. Step 2: Based on the collected microservice monitoring metrics, effectively converge some of the divergent dimensions of the monitoring metrics; Step 3: Periodically eliminate invalid time-series data from the monitoring indicators; Step 4: Centrally store the monitoring indicator data; Step 5: Perform intelligent cropping for unreasonable monitoring indicator labels; Step 6: Statistically analyze the stored monitoring indicator data and regularly inspect and manage indicators with high baseline values.

[0031] In practical scenarios, the execution entity of this method can be a user device, or a device formed by integrating a user device and a network device through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and wristbands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing. Cloud computing is a type of distributed computing, consisting of a virtual computer composed of a group of loosely coupled computer sets.

[0032] Example 1

[0033] This application proposes a high cardinality management method for intelligent microservice monitoring systems. Figure 1 The present application illustrates a processing flow of a high cardinality governance method for an intelligent microservice monitoring system, which includes at least the following processing steps: Step 101: Collect microservice monitoring metrics. The collected microservice monitoring metrics include general monitoring metrics required by all services and business-related custom monitoring metrics.

[0034] Microservice monitoring primarily falls into two categories: general monitoring metrics required by all services, such as performance metrics like API latency, queries per second (QPS), and success rate, as well as resource metrics like CPU and memory usage; and business-specific custom monitoring metrics, such as order volume and transaction amount. The monitoring system provides comprehensive monitoring metrics for services, ensuring service stability through thorough monitoring.

[0035] Step 102: Intelligent convergence of diverging metrics, that is, based on the collected microservice monitoring metrics, effectively convergence of the diverging dimensions of some monitoring metrics.

[0036] Dimensional divergence refers to the phenomenon where the number of tag combinations in monitoring metrics increases exponentially due to the inclusion of too many dynamic tags (such as random strings and meaningless parameters). To ensure the comprehensiveness and accuracy of monitoring metrics, they carry various dimensional tags. For example, service inbound traffic monitoring typically includes information such as cluster, instance, and interface. In practice, some monitoring metric tags exhibit severe divergence. For instance, if a business uses RESTful (Representational State Transfer) style interfaces, their stateless nature and dynamic URL structure can easily cause dimensional divergence. Alternatively, if a business-defined metric includes temporary debugging tags, it can lead to a high cardinality of related monitoring metrics. This poses a significant stability risk to monitoring storage and prevents the business from obtaining the true interface traffic distribution. Effective convergence is the process of reducing the number of time series data through techniques such as tag rewriting, invalid data eviction, and metric aggregation. Therefore, on the monitoring acquisition side, we designed an effective convergence algorithm to effectively converge overly inflated monitoring metrics, including convergence of dynamic interface tags, static resource files, and meaningless strings such as random strings, ensuring that the dimensions of monitoring metrics remain within a controllable range.

[0037] To address meaningless divergent dimensions, such as dynamic tags generated by RESTful interfaces and random strings, a real-time pattern recognition and regularization mechanism is established for automatic convergence and aggregation. This effectively converges divergent dimensions of some monitoring metrics, reducing high cardinality and ensuring storage stability while improving query performance. Monitoring metrics are automatically aggregated based on business needs, allowing users to directly observe real monitoring information without requiring secondary aggregation during queries, thus enhancing the usability of the monitoring system.

[0038] Step 103: Periodically eliminate invalid time series data. That is, record the update time of time series data on the acquisition side, and automatically eliminate time series that have not been updated for a certain period of time.

[0039] In real-world scenarios, traffic for some services may be time-sensitive. For example, a service interface might receive only one request before no further traffic arrives, or some low-traffic interfaces might only receive new requests after a long interval. In such cases, traditional data collection methods continuously collect time-series data stored in service memory without monitoring the validity of the data or performing aging-out processes. Our self-developed SDK records the update time of the time-series data on the collection side and automatically evicts time-series data that hasn't been updated for a certain period. This not only solves the problem of wasted resources on transmitting and storing invalid time-series data but also effectively reduces the base case of metrics.

[0040] In this embodiment, a time-window-based time-series data activity detection algorithm is designed to automatically identify and clean up zombie time series that exceed the TTL (Time To Live) threshold. For time series with no subsequent data, a self-developed SDK is used to periodically age and eliminate these invalid data on the storage write side, avoiding resource waste in the entire monitoring chain and reducing the pressure on storage acquisition and writing, thereby further reducing monitoring latency.

[0041] Step 104: Use a time-series database to centrally store the monitoring data.

[0042] A time-series database is a storage system specifically designed for timestamped indexed data, employing columnar storage and efficient compression algorithms to process time-series data. In this embodiment, we use the open-source time-series database VictoriaMetrics to store monitoring data. Although VictoriaMetrics boasts excellent performance, its stability remains insufficient when facing high cardinality issues, and cluster avalanche can even occur, rendering monitoring unavailable. To avoid this, we add a storage proxy layer in front of VictoriaMetrics to handle dual writes, rate reduction, and high cardinality management, further enhancing the stability of VictoriaMetrics.

[0043] Step 105: Perform intelligent monitoring and trimming of service metrics. For unreasonable metric labels, perform intelligent trimming to prevent excessive expansion of metric labels.

[0044] We use a self-developed proxy component to statistically analyze write-to-storage monitoring metrics and output corresponding observation panels. For unreasonable metric labels, such as excessively long labels containing random strings, we intelligently prune and remove the random strings to prevent excessive label expansion. By introducing a storage proxy, we provide a global sample statistical view of write-to-storage monitoring metrics and intelligently prune and manage these metrics based on this view. This ensures that label names and values ​​are set reasonably and effectively, avoiding high cardinality issues caused by overly broad value ranges.

[0045] Tags that exceed length limits or contain invalid characters are cleaned and pruned in real time before being written to storage, while providing global storage sample statistics.

[0046] Step 106: Conduct high-baseline inspection and management, which involves statistically analyzing the stored sample indicators and periodically inspecting and managing high-baseline indicators.

[0047] Based on statistical analysis of stored sample metrics, we regularly conduct inspections and remediation of metrics with high base numbers, remove invalid or unused tags, stop collecting data on monitoring metrics that are no longer used, and shorten the storage cycle for monitoring metrics without long-term backtracking needs. These measures aim to prevent the rapid expansion of time series metrics from affecting the stability of storage and queries. We establish a multi-dimensional evaluation model (including time series growth rate, metric sample size, and metric tag dispersion) to achieve periodic automated inspections and remediation.

[0048] Because monitoring data is dynamic, we use a visual UI component to obtain the base statistics of indicators. By regularly inspecting indicators with high base numbers, removing useless indicators, optimizing label design, and shortening storage cycles, we can dynamically ensure the continuous management of monitoring indicators and effectively control the risk of an explosion in the base number of incremental monitoring indicators.

[0049] Example 2

[0050] This application proposes another method for managing high cardinality in intelligent microservice monitoring systems. Figure 2 The present application illustrates a processing flow of a high cardinality governance method for an intelligent microservice monitoring system, which includes at least the following processing steps: Step 201: Collect microservice monitoring metrics. The collected microservice monitoring metrics include general monitoring metrics required by all services and business-related custom monitoring metrics.

[0051] By collecting microservice monitoring metrics, the problem of unified collection of general and custom monitoring metrics under the microservice architecture is solved, providing the system with a complete, rich and accurate monitoring landscape.

[0052] Step 202: Intelligent convergence of diverging metrics, that is, based on the collected microservice monitoring metrics, effectively converge the diverging dimensions of some monitoring metrics.

[0053] By using intelligent algorithms to quickly converge dimensionality-dispersed metrics, the dimensionality of metrics can be effectively reduced, the workload of manual convergence can be reduced, and users can quickly understand the service operation status by monitoring the true label distribution of metrics, thus ensuring the stability and query performance of the storage system.

[0054] Step 203: Periodically eliminate invalid time series data. That is, record the update time of time series data on the acquisition side, and automatically eliminate time series that have not been updated for a certain period of time.

[0055] By periodically eliminating time series data with no subsequent data for a long time based on the updated timestamps of the time series data, the pressure on the write side and the corresponding resource consumption are reduced, the time series in storage are further simplified, and the monitoring query performance is improved.

[0056] Step 204: Perform pre-aggregation to reduce the storage requirements of time series data.

[0057] Pre-aggregation can reduce the storage requirements of time series data, significantly reducing the number of time series data and samples, and effectively alleviating the storage pressure on time series databases.

[0058] Step 205: Use a time-series database to centrally store the monitoring data.

[0059] By building a storage proxy middleware layer, enhanced functions such as dual-write rate reduction, query routing, and high-cardinality circuit breaking are implemented to improve the stress resistance of the VictoriaMetrics cluster.

[0060] Step 206: Perform intelligent monitoring and trimming of service metrics. For unreasonable metric labels, perform intelligent trimming to prevent excessive expansion of metric labels.

[0061] The system uses a storage agent to perform write sample statistics on monitoring metrics and intelligently prunes and manages non-standard monitoring metrics to ensure their effectiveness and reduce the high base problem caused by unreasonable label design.

[0062] Step 207: Conduct high-baseline inspection and management, which involves statistically analyzing the stored sample indicators and periodically inspecting and managing high-baseline indicators.

[0063] Statistical analysis is performed based on storage observation metrics to comprehensively monitor the status, performance, and stability of the system's storage. At the same time, regular inspections are conducted and high-cardinality governance reports are output. Dynamic and long-term continuous high-cardinality governance is carried out to reduce potential failure risks and minimize the possibility of monitoring storage failures and monitoring interruptions.

[0064] Example 3

[0065] like Figure 3 As shown, this application proposes a high cardinality governance device for an intelligent microservice monitoring system, the device comprising: The collection unit 301 is used to collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. Convergence Unit 302 intelligently converges divergence indicators of dimensions, that is, based on the collected microservice monitoring indicators, it effectively converges the divergence dimensions of some monitoring indicators. Elimination unit 303 periodically eliminates invalid time series data. That is, the update time of time series data is recorded on the acquisition side, and time series that have not been updated for a certain period of time are automatically eliminated. Storage unit 304 uses a time-series database to centrally store monitoring data; The trimming unit 305 performs intelligent monitoring and trimming of service indicators. For unreasonable indicator labels, it performs intelligent trimming to prevent the excessive expansion of indicator labels. Governance unit 306 performs high-baseline inspection and governance, which involves statistically analyzing the stored sample indicators and periodically inspecting and governing high-baseline indicators.

[0066] Example 4

[0067] like Figure 4 As shown, this application proposes another intelligent microservice monitoring system high cardinality governance device, which includes: The collection unit 401 is used to collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. Convergence Unit 402 intelligently converges divergence metrics, that is, based on the collected microservice monitoring metrics, it effectively converges the divergence dimensions of some monitoring metrics. Elimination unit 403 periodically eliminates invalid time series data. That is, the update time of time series data is recorded on the acquisition side, and time series that have not been updated for a certain period of time are automatically eliminated. Pre-aggregation unit 404 performs pre-aggregation to reduce the storage amount of time series data; Storage unit 405 uses a time-series database to centrally store monitoring data; The trimming unit 406 performs intelligent monitoring and trimming of service indicators. For unreasonable indicator labels, it performs intelligent trimming to prevent the excessive expansion of indicator labels. Governance unit 407 performs high-baseline inspection and governance, which involves statistically analyzing the stored sample indicators and periodically inspecting and governing high-baseline indicators.

[0068] Example 5

[0069] Based on the same inventive concept, this application also provides an electronic device. The method corresponding to the electronic device can be the high cardinality governance method of the intelligent microservice monitoring system in the foregoing embodiments, and its problem-solving principle is similar to that method. The electronic device provided in this application includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the methods and / or technical solutions of the foregoing embodiments of this application.

[0070] The electronic device can be a user device, or a device formed by integrating user devices and network devices through a network, or it can be an application running on the aforementioned devices. The user device includes, but is not limited to, various terminal devices such as computers, mobile phones, tablets, smartwatches, and smart bands. The network device includes, but is not limited to, network hosts, single network servers, multiple network server sets, or cloud computing-based computer sets, and can be used to implement some processing functions when setting an alarm clock. Here, the cloud consists of a large number of hosts or network servers based on cloud computing.

[0071] Figure 5 The diagram illustrates the structure of an electronic device suitable for implementing the methods and / or technical solutions in the embodiments of this application. The device 500 includes a Central Processing Unit (CPU) 501, which can perform various appropriate actions and processes based on a program stored in a Read Only Memory (ROM) 502 or a program loaded from a storage portion 508 into a Random Access Memory (RAM) 503. The RAM 503 also stores various programs and data required for system operation. The CPU 501, ROM 502, and RAM 503 are interconnected via a bus 504. An Input / Output (I / O) interface 505 is also connected to the bus 504.

[0072] The following components are connected to I / O interface 505: an input section 506 including a keyboard, mouse, touchscreen, microphone, infrared sensor, etc.; an output section 507 including a cathode ray tube (CRT), liquid crystal display (LCD), LED display, OLED display, etc., and speakers, etc.; a storage section 508 including one or more computer-readable storage media such as hard disk, optical disk, magnetic disk, semiconductor memory, etc.; and a communication section 509 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section 509 performs communication processing via a network such as the Internet.

[0073] In particular, the methods and / or embodiments in this application can be implemented as computer software programs. For example, the embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable storage medium, the computer program containing program code for performing the methods shown in the flowchart. When the computer program is executed by the central processing unit (CPU) 501, it performs the functions defined in the methods of this application.

[0074] Example 6

[0075] Another embodiment of this application provides a computer-readable storage medium having computer program instructions stored thereon, which can be executed by a processor to implement the methods and / or technical solutions of any one or more embodiments of this application described above.

[0076] Specifically, this embodiment may employ any combination of one or more computer-readable media. A computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium may be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this document, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0077] The program code contained on a computer-readable medium may be transmitted using any suitable medium, including—but not limited to—wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.

[0078] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0079] The flowcharts or block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-specific system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0080] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0081] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or page components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between devices or units through some interfaces, and may be electrical, mechanical, or other forms.

[0082] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0083] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or in a combination of hardware and software functional units.

[0084] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

[0085] Furthermore, it is clear that the word "comprising" does not exclude other units or steps, and the singular does not exclude the plural. Multiple units or devices recited in a device claim may also be implemented by a single unit or device through software or hardware. The terms "first," "second," etc., are used to indicate names and do not indicate any specific order.

Claims

1. A method for managing high cardinality in an intelligent microservice monitoring system, characterized in that, The method includes: Collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. Based on the collected microservice monitoring metrics, the divergent dimensions of some monitoring metrics are effectively converged. Periodically eliminate invalid time-series data from monitoring indicators; Centralized storage of monitoring indicator data; For unreasonable monitoring indicator labels, intelligent cropping is performed; Statistical analysis of stored monitoring data is conducted, and high-base-value indicators are regularly inspected and addressed.

2. The high cardinality management method for the intelligent microservice monitoring system according to claim 1, characterized in that, in, Effective convergence includes convergence of dynamic interface tags, convergence of static resource files, and convergence of meaningless strings such as random strings. A real-time pattern recognition and regularization processing mechanism is established for automatic convergence and aggregation.

3. The high cardinality management method for the intelligent microservice monitoring system according to claim 1, characterized in that, in, On the acquisition side, the update time of the time series data of the monitoring indicators is recorded. Time series that have not been updated for a certain period of time are automatically eliminated. Specifically, this includes: designing a time series data activity detection algorithm based on time windows to automatically identify and clean up zombie time series that exceed the TTL threshold; and performing periodic aging and elimination on the storage write side for time series without subsequent data.

4. The high cardinality management method for the intelligent microservice monitoring system according to claim 1, characterized in that, in, The VictoriaMetrics time-series database is used to store monitoring metric data, and a storage proxy layer is added in front of VictoriaMetrics to handle dual writes, rate reduction, and high cardinality management, further enhancing the stability of VictoriaMetrics.

5. The high cardinality management method for the intelligent microservice monitoring system according to claim 1, characterized in that, in, The system performs statistical analysis on the monitoring metrics written to storage and outputs the corresponding results. For unreasonable monitoring metric labels, it performs intelligent trimming to prevent the excessive expansion of metric labels. It also introduces a storage agent to provide a global sample statistical view of the monitoring metrics for writes to storage, and based on this, further intelligently prunes and manages the monitoring metrics for writes to storage.

6. The high cardinality management method for the intelligent microservice monitoring system according to claim 1, characterized in that, in, Based on the statistical analysis of the stored monitoring metrics, high-baseline metrics are regularly inspected and managed. Invalid or unused tags are removed, and monitoring metrics that are no longer used are no longer collected. The storage cycle of monitoring metrics that do not have long-term backtracking needs is shortened to avoid the rapid expansion of time series affecting the stability of storage and query. A multi-dimensional evaluation model is established, including time series growth rate, metric sample size, and metric tag dispersion, to achieve periodic automated inspection and management.

7. A high-baseline governance device for an intelligent microservice monitoring system, characterized in that, The device includes: The collection unit is used to collect microservice monitoring metrics, including general monitoring metrics required by all services and business-related custom monitoring metrics. The convergence unit effectively converges some of the divergent dimensions of the collected microservice monitoring metrics. The elimination unit is used to periodically eliminate invalid time-series data from monitoring indicators; The storage unit uses a time-series database to centrally store monitoring indicator data; The trimming unit intelligently trims unreasonable monitoring indicator labels; The governance unit performs statistical analysis on the stored monitoring indicator data and conducts regular inspections and governance of indicators with high baseline values.

8. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 6.

9. A computer-readable storage medium having stored thereon computer program instructions that can be executed by a processor to implement the method as described in any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 6.

Images