HTTP (Hyper Text Transport Protocol) request performance diagnosis method and system based on dynamic Agent mechanism and storage medium

Abstract

The invention provides an HTTP request performance diagnosis method and system based on a dynamic Agent mechanism and a storage medium, and relates to the technical field of computer network performance optimizing.The diagnosis method comprises the steps that S1, a dynamic Agent program is accessed to a target server through an Attach mechanism of a JVM tool script; s2, an Agent program acts as a service method of javax.servlet.http.HttpServlet, a specific HTTP request is intercepted, and a background monitoring thread is started to execute the following operations: setting an execution time threshold value, and marking the HTTP request exceeding the time threshold value as an STUCK state; performing thread stack sampling on the STUCK state request at a set frequency; collecting a method execution completion event, and constructing flame map data; s3, generating two result files: head.txt, and recording the total consumed time of the request and a unique identifier UUID; and uurid.txt, storing the flame image data obtained by sampling; s4, calling a flame graph generation tool to generate a flame graph in an SVG format; s5, positioning the performance bottleneck by analyzing the stack width and the sample number of the flame pattern; the width of the flame map is positively correlated with execution time consumption, and the widest stack is optimized preferentially.

Description

Technical Field

[0001] The present invention relates to the technical field of computer network performance optimization, and in particular to a method for diagnosing HTTP request performance. Background Art

[0002] In Java application development, HTTP interface performance issues are a common challenge for online operations and maintenance. To improve the efficiency of online operation and maintenance problem diagnosis, existing tools such as Arthas require manual layer-by-layer tracing of method calls. These tools diagnose HTTP requests by tracing from the control layer inwards. Long method hierarchies require multiple tests, and the number and duration of tests are determined by the method hierarchy and complexity, resulting in low diagnostic efficiency. Summary of the Invention

[0003] The purpose of the present invention is to provide a method, system and storage medium for diagnosing HTTP request performance based on a dynamic Agent mechanism, so as to solve the problem of low efficiency of HTTP request diagnosis.

[0004] To solve the above problems, the present invention first provides an HTTP request performance diagnosis method based on a dynamic Agent mechanism, comprising the following steps: S1. connecting a dynamic Agent program to a target server through the Attach mechanism of a JVM tool script, wherein the Agent program is implemented based on the Java Instrumentation API; S2. the Agent program acts as a proxy for the service method of javax.servlet.http.HttpServlet, intercepts specific HTTP requests, and starts a background monitoring thread to perform the following operations: setting an execution time threshold, marking HTTP requests that exceed the time threshold as a STUCK state; sampling thread stacks for STUCK state requests at a set frequency; collecting method execution completion events, and constructing flame graph data; S3. generating two result files: head.txt, which records the total request time and unique identifier UUID; and uuid.txt, which stores the flame graph data obtained by sampling; S4. calling a flame graph generation tool according to the target server operating system, and converting uuid.txt into a flame graph in SVG format; S5. locating performance bottlenecks by analyzing the stack width and number of samples of the flame graph; the flame graph width is positively correlated with the execution time, and the widest stack is optimized first.

[0005] Furthermore, the S4 includes: the Agent client monitors changes in head.txt, extracts the UUID and matches it with the corresponding uuid.txt; and calls a cross-platform flame graph generation tool to generate an SVG file.

[0006] Furthermore, the flame graph generation tool is Brendan Gregg's FlameGraph open source component, and supports both Windows and Linux operating systems.

[0007] Furthermore, the S5 includes: combining the total time consumption with the number of samples to calculate the theoretical time consumption to verify the CPU load impact, and the calculation formula is theoretical time consumption = sampling interval time × (number of samples + 1).

[0008] Furthermore, the performance bottleneck location rules of S5 include: if there are multiple equal-width stacks in the flame graph, the stack with a higher number of samples is optimized first; when there is an error between the theoretical time consumption and the total time consumption recorded in head.txt and exceeds a threshold, resampling is required.

[0009] The present invention also provides an HTTP request performance diagnosis system based on a dynamic Agent mechanism, comprising: a dynamic implantation module, which implants an Agent program into a target server through a JVM Attach mechanism, wherein the Agent program is implemented based on the Java Instrumentation API; a request monitoring module, which acts as an agent for the request processing method of javax.servlet.http.HttpServlet, marks HTTP requests exceeding a threshold as a STUCK state, and samples thread stacks at a set frequency; a data construction module, which collects sampling data and generates a flame graph data file uuid.txt and a time-consuming record file head.txt; a visual chart generation module, which converts flame graph data into a flame graph in SVG format; a bottleneck location module, which analyzes the stack width and sample number of the flame graph to locate performance bottlenecks; the flame graph width is positively correlated with the execution time, and the widest stack is optimized first.

[0010] Furthermore, the bottleneck location module can calculate the theoretical time consumption by combining the total time consumption and the number of samples to verify the CPU load impact. The calculation formula is: theoretical time consumption = sampling interval time × (number of samples + 1).

[0011] Furthermore, the bottleneck location module is provided with a stack weight evaluation rule: when there are multiple stacks of equal width, the optimization priority is sorted in descending order of the number of samples.

[0012] The present invention also provides an electronic device, characterized in that it includes: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism described in any one of the above technical solutions.

[0013] The present invention also provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed, they are used to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism described in any one of the above technical solutions.

[0014] According to the present invention, a dynamic agent-based HTTP request performance diagnosis method, system, and storage medium are provided. These methods are implemented based on the agent mechanism, do not require business code modification, and can avoid introducing additional risks. Compared with the existing tool Arthas, automatic high-frequency sampling and flame graph visualization greatly reduce manual tracking costs. They support Windows or Linux systems and heavy applications, such as microservice architectures, with strong compatibility, breaking through the environmental limitations of traditional tools. They can combine total time-consuming files with flame graph data to calibrate the impact of CPU load on sampling results with high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0016] Figure 1 This is a flow chart of the HTTP request performance diagnosis method based on the dynamic Agent mechanism provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned objects, features and advantages of the present invention more clearly understood, the following detailed description of the specific embodiments of the present invention is given in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0018] Example 1

[0019] This embodiment provides an HTTP request performance diagnosis method based on a dynamic agent mechanism. It does not require code modification, supports heavy applications and multiple operating systems, and improves the efficiency of locating performance bottlenecks through automated sampling and visual analysis.

[0020] The diagnostic method specifically includes the following steps:

[0021] 1. Dynamic Agent Program Implantation

[0022] Identify the target process. Use the target server management tool, such as the Linux ps command or the Windows Task Manager, to obtain the process ID of the target Java process. For example, in a Linux environment, run the command ps -ef | grepjava to filter out the PID of the HTTP service process.

[0023] Agent loading uses the JVM Attach mechanism to inject the dynamic Agent program into the target process through a tool script.

[0024] Agent initialization is based on the Java Instrumentation API. Agent programs are dynamically loaded at runtime, eliminating the need to restart the JVM. A separate class loader is used to avoid conflicts with application class libraries. Bytecode enhancement technologies, such as the ASM framework, modify the bytecode of javax.servlet.http.HttpServlet to achieve non-intrusive monitoring.

[0025] 2. Request interception and sampling

[0026] Service method proxy, the Agent program intercepts the service method of HttpServlet and times each HTTP request. When the request time exceeds the preset threshold, such as 1000ms, it is marked as a STUCK state request.

[0027] The background monitoring thread is started and the threshold is set: the execution time threshold is configured, which can be adjusted through the Agent parameters. Requests exceeding the threshold enter the sampling process.

[0028] Stack sampling samples the thread stack in the STUCK state at a set frequency, such as a fixed frequency of 200ms, and records the current call chain, such as com.example.service.MethodA to com.example.dao.MethodB.

[0029] Data collection, whenever a method execution is completed, such as a database query or remote call return, the number of executions of the call path is recorded, and flame graph data is constructed in the format of "stack path + execution count".

[0030] 3. Generate result file

[0031] Generate head.txt, recording the unique identifier of the request (UUID, which can be used for subsequent data matching) and the total time consumed, such as UUID: 123456, Total Time: 2500ms.

[0032] Generate uuid.txt to store the sampled flame graph data;

[0033] 4. Generate flame graph

[0034] Agent client monitoring: The client monitors the file change events of head.txt. When a new UUID is detected, it automatically matches the corresponding uuid.txt file. It can be associated by UUID naming or directory.

[0035] Cross-platform conversion, calling BrendanGregg's FlameGraph tool to convert uuid.txt to a flame graph in SVG format

[0036] 5. Performance bottleneck analysis

[0037] To interpret the flame graph, open the SVG flame graph and observe the stack width. The wider the width, the higher the percentage of execution time spent. For example, if the stack width of com.example.dao.OrderDAO.insert is the largest, it indicates that the database insert operation is the performance bottleneck.

[0038] To verify the time consumption, combine the total time consumption of head.txt (e.g., 2500ms) with the number of samples (assuming 10 samples with a 200ms interval), and calculate the theoretical time consumption (200ms × (10 + 1) = 2200ms). Compare this with the actual total time consumption. If the difference exceeds a threshold, such as 10%, it indicates that the CPU load may be affecting the sampling. Adjust the sampling frequency, for example, reduce it to 100ms, and resample until the results converge.

[0039] Optimize priority and prioritize the widest stack paths in the flame graph, such as database operations and complex calculation logic. Improve performance by reducing the number of method calls, optimizing algorithms, or increasing cache.

[0040] 6. Exception handling and parameter adjustment

[0041] Sampling frequency adjustment: Based on server performance, such as CPU load, the sampling frequency can be dynamically adjusted in the range of 100ms-500ms to balance diagnostic accuracy and system overhead.

[0042] Threshold adaptation uses machine learning algorithms, such as dynamic threshold adjustment based on historical request duration, to automatically identify STUCK state requests and reduce manual configuration costs.

[0043] Example 2

[0044] This embodiment provides an HTTP request performance diagnosis system based on a dynamic agent mechanism, which is implemented by the diagnosis method provided in Example 1 and includes:

[0045] Dynamically implant modules, implanting the Agent program into the target server through the JVM Attach mechanism. The Agent program is implemented based on the Java Instrumentation API.

[0046] The request monitoring module, which acts as a proxy for the request processing method of javax.servlet.http.HttpServlet, marks HTTP requests exceeding the threshold as STUCK state and samples the thread stack at a set frequency;

[0047] Data construction module, collects sampling data and generates flame graph data file uuid.txt and time record file head.txt;

[0048] Visual chart generation module, which converts flame graph data into flame graph in SVG format;

[0049] The bottleneck location module analyzes the stack width and sample count of the flame graph to locate performance bottlenecks. The width of the flame graph is positively correlated with the execution time, so the widest stack is optimized first.

[0050] Among them, the bottleneck location module can combine the total time consumption and the number of samples to calculate the theoretical time consumption and verify the impact of CPU load. The calculation formula is theoretical time consumption = sampling interval time × (number of samples + 1); and a stack weight evaluation rule is set: when there are multiple equal-width stacks, the optimization priority is sorted in descending order by the number of samples.

[0051] Microservice link tracing

[0052] In a complex microservices architecture, a request may pass through more than 10 service nodes. By intercepting specific slow requests, you can specifically track the performance bottlenecks of the entire link.

[0053] Example 3

[0054] This embodiment provides an electronic device, including: a processor, and a memory connected to the processor; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism provided in Example 1.

[0055] Example 4

[0056] This embodiment provides a computer-readable storage medium, which stores computer-executable instructions. When the computer-executable instructions are executed, they are used to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism provided in Example 1.

[0057] The present invention is based on the dynamic Agent mechanism, which can non-invasively implant monitoring logic during operation, avoid modifying business code, and ensure the stability of the production environment; it adopts an independent class loader to avoid conflicts with application class libraries, supports heavy applications such as microservices and high-concurrency systems, is suitable for complex scenarios, and improves system compatibility; it can efficiently diagnose and locate bottlenecks, and has an intelligent request screening function, which only samples STUCK state requests that exceed the threshold, reduces the amount of data, focuses on the problem, and greatly shortens the diagnosis time; it generates SVG through FlameGraph, intuitively displays the time consumption ratio, quickly locates performance bottlenecks, and does not require complex debugging, improves analysis efficiency, and non-technical personnel can participate; it supports cross-platform and multi-scenario adaptation, Wind Ows / Linux unifies the tool chain through adaptation scripts, breaks through the environmental limitations of traditional tools, and covers mainstream server systems; data verification and accuracy, theoretical time-consuming calibration and verification of CPU load, automatic resampling when the error exceeds the threshold, ensuring accurate diagnostic results and avoiding errors caused by sampling frequency or load; and setting priority rules to give priority to high-frequency call methods to maximize optimization benefits; can quickly locate deadlocks, reduce resource competition troubleshooting time, improve system availability, and reduce business losses; adopt dynamic agents, intelligent sampling, and cross-platform flame graph combinations to achieve full-process automated diagnosis, which has a qualitative breakthrough in intrusion, efficiency, and compatibility compared to existing technologies, and is highly innovative and practical. The present invention meets the needs of enterprises for zero-intrusion, efficient diagnostic tools, is applicable to scenarios such as JavaWeb, microservices, and big data, has great commercial promotion potential, and provides standardized solutions for enterprise performance optimization.

[0058] Although the present invention is disclosed as above, the present invention is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the scope of protection of the present invention should be based on the scope defined by the claims.

[0059] Finally, it should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0060] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0061] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for diagnosing HTTP request performance based on a dynamic agent mechanism, characterized in that: The following steps are involved: S1 Attach mechanism through the JVM tool script, the dynamic Agent program access to the target server, the Agent program based on the Java Instrumentation API implementation; S2. The Agent program proxies the service method of javax.servlet.http.HttpServlet, intercepts specific HTTP requests, and starts a background monitoring thread to perform the following operations: Set an execution time threshold and mark HTTP requests that exceed the time threshold as STUCK; Request thread stack sampling at a set frequency for STUCK state; Collect method execution completion events and construct flame graph data; S3. Generate two result files: head.txt, records the total request time and unique identifier UUID; and uuid.txt, which stores the sampled flame graph data; S4. Call the flame graph generation tool based on the target server operating system to convert uuid.txt into a flame graph in SVG format. S5. Identify performance bottlenecks by analyzing the stack width and sample count of the flame graph. Since the flame graph width is positively correlated with execution time, prioritize optimizing the widest stack.

2. The HTTP request performance diagnosis method based on the dynamic Agent mechanism according to claim 1 is characterized in that: The S4 includes: The Agent client monitors changes in head.txt, extracts the UUID and matches it with the corresponding uuid.txt; Call the cross-platform flame graph generation tool to generate an SVG file.

3. The HTTP request performance diagnosis method based on the dynamic Agent mechanism according to claim 2 is characterized in that: The flame graph generation tool is Brendan Gregg's FlameGraph open source component, and supports both Windows and Linux operating systems.

4. The HTTP request performance diagnosis method based on the dynamic agent mechanism according to claim 3 is characterized in that: The S5 includes: The total time consumption and the number of samples can be combined to calculate the theoretical time consumption to verify the impact of CPU load. The calculation formula is theoretical time consumption = sampling interval time × (number of samples + 1).

5. The HTTP request performance diagnosis method based on the dynamic Agent mechanism according to claim 4 is characterized in that: The S5 performance bottleneck location rules include: If there are multiple stacks of equal width in the flame graph, the stack with the highest number of samples will be optimized first; When there is a discrepancy between the theoretical time consumption and the total time consumption recorded in head.txt and the discrepancy exceeds the threshold, resampling is required.

6. An HTTP request performance diagnosis system based on a dynamic agent mechanism, characterized in that: include: Dynamic implantation module, which implants the Agent program into the target server through the JVM Attach mechanism. The Agent program is implemented based on the Java Instrumentation API. The request monitoring module, which acts as a proxy for the request processing method of javax.servlet.http.HttpServlet, marks HTTP requests exceeding the threshold as STUCK state and samples the thread stack at a set frequency; Data construction module, collects sampling data and generates flame graph data file uuid.txt and time record file head.txt; Visual chart generation module, which converts flame graph data into flame graph in SVG format; The bottleneck location module analyzes the stack width and sample count of the flame graph to locate performance bottlenecks. The width of the flame graph is positively correlated with the execution time, so the widest stack is optimized first.

7. The HTTP request performance diagnosis system based on the dynamic Agent mechanism according to claim 6 is characterized in that: The bottleneck location module can calculate the theoretical time consumption by combining the total time consumption and the number of samples to verify the CPU load impact. The calculation formula is theoretical time consumption = sampling interval time × (number of samples + 1).

8. The HTTP request performance diagnosis system based on the dynamic Agent mechanism according to claim 7 is characterized in that: The bottleneck location module is provided with a stack weight evaluation rule: when there are multiple stacks of equal width, the optimization priority is sorted in descending order of the number of samples.

9. An electronic device, characterized in that: include: a processor, and a memory connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism as described in any one of claims 1 to 5.

Images