Method for realizing fine-grained index information acquisition based on distributed architecture

Abstract

The invention discloses a method for realizing fine-grained index information collection based on a distributed architecture, and the method comprises the steps: data collection and monitoring: combining a technology based on a probe agent with an integrated assembly, and carrying out the comprehensive collection and monitoring of system operation basic information and business interface whole-process response information; and summarizing analysis and performance optimization: summarizing and analyzing various kinds of collected information by utilizing the advantages of an integrated component. And more comprehensive and accurate performance data are provided, so that a more targeted optimization decision can be made.

Description

Technical Field

[0001] The present invention relates to the field of monitoring system information collection, and in particular to a method for collecting fine-grained indicator information based on a distributed architecture. Background Art

[0002] As modern software systems continue to grow in complexity, optimizing system performance has become a critical factor in ensuring user experience and business efficiency. In a distributed architecture, software systems are typically composed of multiple independent yet closely coordinated modules, and the performance of each module directly impacts the performance of the entire system. Therefore, being able to accurately collect performance metrics for each module is crucial for identifying performance bottlenecks and guiding system optimization.

[0003] Traditional performance monitoring tools often only provide overall or coarse-grained performance data, making it difficult to meet the demand for fine-grained metrics. For example, they may only provide the response time of the entire system, but cannot accurately measure the response time and resource usage of specific business logic modules. This limitation limits the depth and accuracy of performance analysis and optimization, making it difficult for developers to quickly identify problems and implement targeted optimizations.

[0004] Solutions of the prior art:

[0005] Integrated component monitoring strategy

[0006] The key to implementing this strategy lies in leveraging the monitoring platform's integrated components. Specifically, when an application requires monitoring, the corresponding monitoring components are integrated into the application. These monitoring components regularly collect key operational information as the application runs.

[0007] The monitoring component periodically pulls information from the application's health check API. This API typically returns the application's current health, including whether it's operating normally and whether any errors have occurred. By regularly pulling this information, you can monitor the application's health in real time and take timely action when problems arise.

[0008] All of this collected data will be used to display and monitor the application interface so that administrators and developers can intuitively understand the running status and health of the application.

[0009] This strategy of relying on integrated components of the monitoring platform fully utilizes the advantages of componentization, making the implementation of monitoring simpler and more efficient.

[0010] Disadvantages of Integrated Component Monitoring Strategies

[0011] While relying on integrated monitoring platform components offers significant advantages in simplifying monitoring implementation, this approach also has some drawbacks. It can't provide sufficiently detailed performance data. Monitoring components typically focus on collecting overall application health and key performance indicators. For deeper performance analysis, such as interface runtime, database query efficiency, or thread blocking details, these monitoring components may not provide sufficient information. This limits developers' ability to conduct precise diagnostics when optimizing system performance.

[0012] This strategy can result in delayed responses to specific issues. Because monitoring components rely primarily on periodic pulls of information from health check interfaces, they may not catch transient performance issues or anomalies in real time. For example, if an interface experiences only a brief increase in latency under high load, this periodic monitoring approach may miss these issues, delaying their discovery and resolution. Summary of the Invention

[0013] In view of the above problems, the present invention is proposed to provide a method for collecting fine-grained indicator information based on a distributed architecture, which overcomes the above problems or at least partially solves the above problems.

[0014] According to one aspect of the present invention, a method for collecting fine-grained indicator information based on a distributed architecture is provided, the information collection method comprising:

[0015] Data collection and monitoring combines probe agent-based technology with integrated components to comprehensively collect and monitor basic system operation information and full-process response information of business interfaces;

[0016] Aggregation analysis and performance optimization: taking advantage of integrated components, we aggregate and analyze various types of collected information.

[0017] Optionally, the comprehensive collection and monitoring of basic system operation information and full-process response information of the business interface specifically includes:

[0018] The probe agent is deployed on key nodes of the system to collect detailed information on the system's internal status, resource consumption, and business logic execution efficiency in real time;

[0019] Use integrated components to collect basic system operation information and business interface response information.

[0020] Optionally, the summary analysis and performance optimization specifically include:

[0021] Leverage existing monitoring platforms and tools to obtain more comprehensive and detailed longitudinal information on system operation;

[0022] Conduct in-depth performance evaluation and optimization decisions based on information, understand the system's operating mechanism, and guide performance tuning and resource allocation;

[0023] Provide new perspectives and tools for system performance monitoring.

[0024] Optionally, the data collection and monitoring specifically includes:

[0025] Probe agent-based technology is combined with integrated components to aggregate key system data in real time within the defined probe code;

[0026] Record class operation parameters and exception information in the integrated component to monitor and analyze the system;

[0027] In the process of processing transactions, the information collected by the probe agent is matched with the information collected by the integration component for each transaction's primary key serial number;

[0028] Ensure a comprehensive understanding of the basic information of system operation and the full-process response of business interfaces;

[0029] The probe agent collects detailed information on the system's internal status, resource consumption, and business logic execution efficiency in real time;

[0030] Use integrated components to collect basic system operation information and business interface response information.

[0031] Optionally, the key data specifically includes: stack memory information and class loading information.

[0032] Optionally, the summary analysis and performance optimization specifically include:

[0033] Use integrated components to aggregate and analyze various types of collected information and integrate data from different sources and formats;

[0034] Conduct in-depth performance evaluation and optimization decisions based on detailed information;

[0035] Through in-depth analysis of system operation data, performance bottlenecks, resource waste and potential risk points can be identified.

[0036] This invention provides a method for collecting fine-grained indicator information based on a distributed architecture. This method includes: data collection and monitoring, combining probe agent-based technology with integrated components to comprehensively collect and monitor basic system operation information and full-process response information from business interfaces; and summary analysis and performance optimization, leveraging the advantages of integrated components to aggregate and analyze various types of collected information. This provides more comprehensive and accurate performance data, facilitating more targeted optimization decisions.

[0037] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0039] Figure 1 A flowchart of a method for collecting fine-grained indicator information based on a distributed architecture provided by an embodiment of the present invention;

[0040] Figure 2 A flowchart of data collection and monitoring provided by an embodiment of the present invention;

[0041] Figure 3 A flowchart of summary analysis and performance optimization provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0043] The terms "comprises" and "comprising" and any variations thereof in the description, embodiments, claims and drawings of the present invention are intended to cover non-exclusive inclusions, for example, including a series of steps or units.

[0044] The technical solution of the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0045] like Figure 1 As shown, a method for collecting fine-grained indicator information based on a distributed architecture includes:

[0046] Data collection and monitoring combines probe agent-based technology with integrated components to comprehensively collect and monitor basic system operation information and full-process response information of business interfaces;

[0047] Aggregation analysis and performance optimization: taking advantage of integrated components, we aggregate and analyze various types of collected information.

[0048] This paper proposes an innovative approach that combines probe agent-based technology with integrated components to comprehensively collect basic system operation information and full-process response information of business interfaces. This not only improves the accuracy of data collection, but also enhances the understanding of system performance.

[0049] The proposed method, through the introduction of probe agent technology, enables monitoring of system operational status at a more granular level. Probe agents, deployed at key system nodes, collect detailed real-time information on system internal status, resource consumption, and business logic execution efficiency. This detailed information is crucial for identifying potential performance bottlenecks, optimizing system configuration, and troubleshooting system issues.

[0050] This invention also leverages the advantages of integrated components, utilizing existing monitoring platforms and tools to aggregate and analyze collected data. This streamlines the monitoring process, reduces the need for additional system resources, and ensures data consistency and accuracy. By summarizing and analyzing basic system operation information and full-process response information from business interfaces, a more comprehensive and detailed longitudinal view of system operation is obtained, which is invaluable for in-depth performance evaluation and optimization decisions.

[0051] The proposed method, based on combining a probe agent with integrated components, provides a new perspective and tool for system performance monitoring. It not only provides more detailed performance data, but also helps better understand the system's operating mechanisms, guiding more effective performance tuning and resource allocation.

[0052] The present invention is divided into two steps as a whole. The first step is data collection and monitoring, and the second step is summary analysis and performance optimization.

[0053] Data Collection and Monitoring: Probe agent-based technology is combined with integrated components to comprehensively collect and monitor basic system operation information and business interface response information throughout the entire process. Probe agents are deployed at key system nodes to collect detailed information such as internal system status, resource consumption, and business logic execution efficiency in real time. Furthermore, integrated components are used to collect basic system operation information and business interface response information, improving data collection accuracy.

[0054] Aggregation, Analysis, and Performance Optimization: Leveraging the advantages of integrated components, we aggregate and analyze various types of collected information. Leveraging existing monitoring platforms and tools, we ensure data consistency and accuracy, providing more comprehensive and detailed longitudinal information on system operations. Based on this information, we conduct in-depth performance evaluations and optimization decisions, gaining a better understanding of system operating mechanisms and guiding more effective performance tuning and resource allocation. This provides new perspectives and tools for system performance monitoring, improving system stability, reliability, and performance.

[0055] like Figure 2 As shown, data collection and monitoring:

[0056] Combining probe agent-based technology with integrated components allows real-time aggregation of key data such as system stack memory information and class loading information within the defined probe code. Furthermore, the integrated components record class execution parameters, exceptions, and other information for more comprehensive system monitoring and analysis.

[0057] During transaction processing, the information collected by the probe agent is matched with the information collected by the integration component for each transaction's unique primary key serial number. This ensures a comprehensive understanding of the basic information of system operation and the full-process response of the business interface.

[0058] The probe agent collects detailed information in real time, including the system's internal status, resource consumption, and business logic execution efficiency. This allows for a better understanding of the system's performance bottlenecks and potential problems, allowing for the implementation of appropriate optimization measures.

[0059] Using integration components to collect basic system operation information and business interface response information can improve the accuracy of data collection. Integration components can integrate this information with data from other system components to form a unified data set, facilitating subsequent analysis and decision-making.

[0060] By combining probe agent technology and integrated components, comprehensive collection and monitoring of basic system operation information and full-process response information of business interfaces can be achieved, thereby improving system stability, performance and maintainability.

[0061] like Figure 3 As shown, summary analysis and performance optimization, including:

[0062] Leverage integrated components to aggregate and analyze the collected information. These components typically feature powerful data processing capabilities and flexible configuration options, effectively integrating data from diverse sources and formats. This approach not only ensures data consistency and accuracy, but also provides a more comprehensive and detailed longitudinal view of system operations.

[0063] Based on this detailed information, we conduct in-depth performance evaluations and optimization decisions. Through in-depth analysis of system operation data, we can identify performance bottlenecks, resource waste, and potential risk points. This analysis not only helps us better understand the system's operating mechanisms but also provides a scientific basis for performance tuning and resource allocation.

[0064] The method presented in this paper provides a new perspective and tool for system performance monitoring. While traditional monitoring systems often focus on real-time data display and alerts, an integrated component-based approach prioritizes deep data mining and long-term analysis. This shift not only enables timely identification and resolution of short-term issues, but also enables macro-level planning and optimization of overall system performance. With the continuous advancement of technology and growing application demand, this comprehensive monitoring approach will undoubtedly become a key trend in future system management and optimization.

[0065] Beneficial effects:

[0066] 1. Comprehensiveness and Accuracy: By deploying probe agents at key nodes, detailed information such as internal system status, resource consumption, and business logic execution efficiency can be collected in real time. This fine-grained monitoring capability provides a deep understanding of system operating mechanisms and promptly identifies and locates potential performance bottlenecks. Compared to traditional monitoring systems, this approach provides more comprehensive and accurate performance data, enabling more targeted optimization decisions.

[0067] 2. Consistency and Accuracy: Combining the advantages of integrated components, we leverage existing monitoring platforms and tools to aggregate and analyze collected data. This not only simplifies the monitoring process and reduces the use of additional system resources, but also ensures data consistency and accuracy. Through in-depth analysis of basic system operation information and full-process response information from business interfaces, we obtain more comprehensive and detailed longitudinal information on system operations, which is invaluable for in-depth performance evaluation and optimization decisions.

[0068] The above specific implementation methods further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for collecting fine-grained indicator information based on a distributed architecture, characterized in that: The information collection method includes: Data collection and monitoring combines probe agent-based technology with integrated components to comprehensively collect and monitor basic system operation information and full-process response information of business interfaces; Aggregation analysis and performance optimization: taking advantage of integrated components, we aggregate and analyze various types of collected information.

2. According to claim 1, a method for collecting fine-grained indicator information based on a distributed architecture is characterized in that: The comprehensive collection and monitoring of basic system operation information and full-process response information of business interfaces specifically includes: The probe agent is deployed on key nodes of the system to collect detailed information on the system's internal status, resource consumption, and business logic execution efficiency in real time; Use integrated components to collect basic system operation information and business interface response information.

3. The method for collecting fine-grained indicator information based on a distributed architecture according to claim 1, characterized in that: The summary analysis and performance optimization specifically include: Leverage existing monitoring platforms and tools to obtain more comprehensive and detailed longitudinal information on system operation; Conduct in-depth performance evaluation and optimization decisions based on information, understand the system's operating mechanism, and guide performance tuning and resource allocation; Provide new perspectives and tools for system performance monitoring.

4. The method for collecting fine-grained indicator information based on a distributed architecture according to claim 1, characterized in that: The data collection and monitoring specifically include: Probe agent-based technology combined with integrated components aggregates key system data in real time within defined probe codes; Record class operation parameters and exception information in the integrated component to monitor and analyze the system; In the process of processing transactions, the information collected by the probe agent is matched with the information collected by the integration component for each transaction's primary key serial number; Ensure a comprehensive understanding of the basic information of system operation and the full-process response of business interfaces; The probe agent collects detailed information on the system's internal status, resource consumption, and business logic execution efficiency in real time; Use integrated components to collect basic system operation information and business interface response information.

5. The method for collecting fine-grained indicator information based on a distributed architecture according to claim 1, characterized in that: The key data specifically includes: stack memory information and class loading information.

6. The method for collecting fine-grained indicator information based on a distributed architecture according to claim 1, characterized in that: The summary analysis and performance optimization specifically include: Use integrated components to aggregate and analyze various types of collected information and integrate data from different sources and formats; Conduct in-depth performance evaluation and optimization decisions based on detailed information; Through in-depth analysis of system operation data, performance bottlenecks, resource waste and potential risk points can be identified.

Images