HTTP request performance diagnosis method and system based on dynamic agent mechanism, and storage medium

Abstract

The application provides a HTTP request performance diagnosis method and system based on a dynamic Agent mechanism and a storage medium, relates to the technical field of computer network performance optimization, and the diagnosis method comprises the following steps: S1, a dynamic Agent program is connected to a target server through the Attach mechanism of a JVM tool script; S2, the Agent program proxies the service method of javax.servlet.http.HttpServlet, intercepts specific HTTP requests, and starts a background monitoring thread to perform the following operations: setting an execution time threshold, marking HTTP requests exceeding the time threshold as STUCK states; performing thread stack sampling on the STUCK state requests at a set frequency; collecting method execution completion events and constructing flame graph data; S3, two result files are generated: head.txt, which records the total time consumption of the request and a unique identifier UUID; and uuid.txt, which stores the flame graph data obtained by sampling; S4, a flame graph generation tool is called to generate an SVG format flame graph; and S5, the performance bottleneck is located by analyzing the stack width and the number of samples of the flame graph; the flame graph width is positively correlated with the execution time consumption, and the widest stack is preferentially optimized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer network performance optimization, in particular to an HTTP request performance diagnosis method. BACKGROUND

[0002] In Java application development, HTTP interface performance problems are common challenges in online operation and maintenance. In order to improve the diagnosis efficiency of online operation and maintenance problems, existing tools such as Arthas need to manually track method calls layer by layer, and the way of diagnosing http requests is to track from the control layer to the inner layer, if the method level is long, multiple tests are needed, the test times and time are determined by the method level and complexity, and the diagnosis efficiency is low. SUMMARY

[0003] The purpose of the present application is to provide an HTTP request performance diagnosis method based on a dynamic Agent mechanism, a system and a storage medium, so as to solve the problem of low diagnosis efficiency of http requests.

[0004] In order to solve the above problems, the present application first provides an HTTP request performance diagnosis method based on a dynamic Agent mechanism, comprising the following steps: S1. connecting a dynamic Agent program to a target server through the Attach mechanism of a JVM tool script, the Agent program is realized based on Java Instrumentation API; S2. the Agent program proxies the service method of javax.servlet.http.HttpServlet, intercepts a specific HTTP request, and starts a background monitoring thread to perform the following operations: sets an execution time threshold, and marks an HTTP request exceeding the time threshold as a STUCK state; performs thread stack sampling on the STUCK state request at a set frequency; collects method execution completion events and constructs a flame graph data; S3. generates two result files: head.txt, which records the total time consumption and a unique identifier UUID; and uuid.txt, which stores the flame graph data obtained by sampling; S4. according to the operating system of the target server, calls a flame graph generation tool to convert uuid.txt into a SVG format flame graph; S5. locates the performance bottleneck by analyzing the stack width and sample number of the flame graph; the flame graph width is positively correlated with the execution time consumption, and the widest stack is optimized first.

[0005] Further, the S4 comprises: the Agent client monitors the changes of head.txt, extracts the UUID and matches the corresponding uuid.txt; calls a cross-platform flame graph generation tool to generate an SVG file.

[0006] Further, the flame graph generation tool is Brendan Gregg's FlameGraph open source component, and supports Windows and Linux operating systems at the same time.

[0007] Further, the S5 comprises: the theoretical time consumption can be calculated by combining the total time consumption and the sample number, and the CPU load influence is verified, and the calculation formula is theoretical time consumption = sampling interval time * (sample number + 1).

[0008] Further, the performance bottleneck positioning rule of the S5 comprises: if there are multiple equal-width stacks in the flame graph, the stack with a high sample number is preferentially optimized; and when there is an error between the theoretical time consumption and the total time consumption recorded in the head.txt and the error exceeds a threshold value, resampling is needed.

[0009] The application further provides an HTTP request performance diagnosis system based on a dynamic Agent mechanism, comprising: a dynamic implanting module, which implants an Agent program into a target server through a JVM Attach mechanism, wherein the Agent program is realized based on a JavaInstrumentation API; a request monitoring module, which marks an HTTP request exceeding a threshold value as a STUCK state, and samples a thread stack at a set frequency; a data constructing module, which collects sampling data and generates a flame graph data file uuid.txt and a time consumption record file head.txt; a visual chart generation module, which converts the flame graph data into a flame graph in an SVG format; a bottleneck positioning module, which analyzes the stack width and sample number of the flame graph, and positions a performance bottleneck; the flame graph width is positively correlated with the execution time consumption, and the widest stack is preferentially optimized.

[0010] Further, the bottleneck positioning module can calculate the theoretical time consumption by combining the total time consumption and the sample number, and verify the CPU load influence, and the calculation formula is theoretical time consumption = sampling interval time * (sample number + 1).

[0011] Further, the bottleneck positioning module is provided with a stack weight evaluation rule: when there are multiple equal-width stacks, the optimization priority is sorted in descending order according to the sample number.

[0012] The application further provides an electronic device, characterized by comprising: a processor, and a memory connected with the processor; the memory stores computer execution instructions; and the processor executes the computer execution instructions stored in the memory, so as to realize the HTTP request performance diagnosis method based on the dynamic Agent mechanism.

[0013] The application further provides a computer readable storage medium, characterized in that the computer readable storage medium stores computer execution instructions, and the computer execution instructions are used to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism when executed.

[0014] The HTTP request performance diagnosis method, system and storage medium based on the dynamic Agent mechanism provided by the application are implemented based on the Agent mechanism, do not need to modify business codes, can avoid introducing additional risks, automatically perform high-frequency sampling and visualize a flame chart, greatly reduce manual tracking costs compared with an existing tool Arthas, support Windows or Linux systems and heavy applications such as a microservice architecture, have strong compatibility, break through the environmental limitations of traditional tools, can combine a total time consumption file and flame chart data to calibrate the influence of CPU load on sampling results, and have high accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only illustrate the embodiments of the application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of the provided drawings.

[0016] Figure 1 The application provides an HTTP request performance diagnosis method based on a dynamic Agent mechanism. DETAILED DESCRIPTION

[0017] In order to make the above-mentioned purposes, features and advantages of the application more obvious and easy to understand, the specific embodiments of the application will be described in detail below with reference to the drawings. It should be understood that the specific embodiments described here are only used to explain the application, and are not used to limit the application.

[0018] Embodiment 1

[0019] The embodiment provides an HTTP request performance diagnosis method based on a dynamic Agent mechanism, which does not need code modification, supports heavy applications and multiple operating systems, and improves performance bottleneck positioning efficiency through automatic sampling and visual analysis.

[0020] The diagnosis method specifically includes the following steps:

[0021] 1. Dynamic Agent program implantation

[0022] Target process identification, get the process ID of the target Java process through the target server management tool, such as the Linux ps command or the Windows Task Manager. For example, in a Linux environment, execute ps-ef|grep java to filter out the PID of the HTTP service process.

[0023] Agent loading, use the JVM Attach mechanism to inject a dynamic Agent program into the target process through a tool script.

[0024] Agent initialization, based on the Java Instrumentation API, the Agent program is dynamically loaded at runtime without restarting the JVM. It avoids conflicts with application libraries through a separate class loader and uses bytecode enhancement technology such as the ASM framework to modify the javax.servlet.http.HttpServlet bytecode, achieving zero-intrusive monitoring.

[0025] 2. Request interception and sampling

[0026] Service method proxy, the Agent program intercepts the service method of HttpServlet and times each HTTP request. When the request time exceeds the preset threshold, such as 1000ms, it is marked as a STUCK state request.

[0027] Background monitoring thread startup, threshold setting: configure the execution time threshold, which can be adjusted through Agent parameters. Requests exceeding the threshold enter the sampling process.

[0028] Stack sampling, at a set frequency, such as 200ms fixed frequency, the stack of threads in the STUCK state is sampled, recording the current call chain, such as com.example.service.MethodA to com.example.dao.MethodB.

[0029] Data collection, whenever a method execution is completed, such as a database query or remote call return, the execution frequency of the call path is recorded, and the flame graph data is constructed in the format of "stack path + execution frequency".

[0030] 3. Generate result file

[0031] Generate head.txt, record the unique identifier (UUID, which can be used for subsequent data matching) and total time of the request, such as UUID:123456, Total Time:2500ms.

[0032] Generate uuid.txt, store the flame graph data obtained by sampling;

[0033] 4. Generate flame graph

[0034] Agent client monitoring, monitoring file change events of head.txt, when detecting a new UUID, automatically matching the corresponding uuid.txt file, can be associated by UUID naming or directory.

[0035] Cross-platform conversion, call BrendanGregg's FlameGraph tool to convert uuid.txt to SVG format flame graph

[0036] 5. Performance bottleneck analysis

[0037] Flame graph interpretation, open SVG flame graph, observe stack width, the wider the width, the higher the execution time consumption ratio. For example, if the stack width of com.example.dao.OrderDAO.insert is the largest, it means that the database insertion operation is the performance bottleneck.

[0038] Time-consuming verification, combined with the total time-consuming of head.txt such as 2500ms, and the sample number (assuming sampling 10 times, interval 200ms), calculate the theoretical time-consuming 200ms×(10+1)=2200ms, compared with the actual total time-consuming. If the difference exceeds the threshold value such as 10%, it means that CPU load may affect sampling, and the sampling frequency needs to be adjusted such as reduced to 100ms for resampling until the result converges.

[0039] Optimization priority, prioritize optimizing the widest stack path in the flame graph, such as database operations and complex calculation logic, to improve performance by reducing the number of method calls, optimizing algorithms, or increasing cache.

[0040] 6. Exception handling and parameter adjustment

[0041] Sampling frequency adjustment, according to server performance, such as CPU load, the sampling frequency can be dynamically adjusted in the range of 100ms-500ms to balance the diagnostic accuracy and system overhead.

[0042] Threshold self-adaptation, through machine learning algorithms, such as dynamic threshold adjustment based on historical request time-consuming, automatically identify STUCK state requests, reduce manual configuration cost.

[0043] Embodiment 2

[0044] The embodiment provides an HTTP request performance diagnosis system based on a dynamic Agent mechanism, which is realized by the diagnosis method provided in embodiment 1, comprising:

[0045] Dynamic implantation module, implant Agent program into target server through JVM Attach mechanism, Agent program is based on Java Instrumentation API implementation;

[0046] Request monitoring module, proxy request processing method of javax.servlet.http.HttpServlet, mark HTTP request exceeding threshold as STUCK state, and sample thread stack at set frequency;

[0047] Data construction module, collect sampling data and generate flame graph data file uuid.txt and time consumption record file head.txt;

[0048] Visual chart generation module, convert flame graph data into SVG format flame graph;

[0049] Bottleneck positioning module, analyze stack width and sample number of flame graph, and locate performance bottleneck; flame graph width is positively correlated with execution time consumption, and the widest stack is prioritized for optimization.

[0050] The bottleneck positioning module can calculate theoretical time consumption in combination with total time consumption and sample number, verify CPU load influence, and the calculation formula is theoretical time consumption = sampling interval time x (sample number + 1); and the stack weight evaluation rule is set: when there are multiple equal-width stacks, the optimization priority is sorted in descending order of sample number.

[0051] Microservice link tracking

[0052] In a complex microservice architecture, a request may pass through more than 10 service nodes. After intercepting a specific slow request, the performance bottleneck of the entire link can be tracked.

[0053] Embodiment 3

[0054] The embodiment provides an electronic device, including a processor and a memory connected with the processor; the memory stores computer execution instructions; the processor executes the computer execution instructions stored in the memory to realize the HTTP request performance diagnosis method based on the dynamic Agent mechanism provided in embodiment 1.

[0055] Embodiment 4

[0056] The embodiment provides a computer readable storage medium, and the computer readable storage medium stores computer execution instructions, and the computer execution instructions are used to realize the HTTP request performance diagnosis method based on the dynamic Agent mechanism provided in embodiment 1 when the computer execution instructions are executed.

[0057] The application is based on a dynamic Agent mechanism, and a monitoring logic is implanted in a runtime without invasion, so that business code is avoided from being modified, production environment stability is ensured, an independent class loader is adopted to avoid conflict with application class libraries, heavy applications such as microservices and high-concurrency systems are supported, the application is suitable for complex scenarios, system compatibility is improved, bottlenecks can be diagnosed and positioned efficiently, intelligent request filtering function is provided, only the STUCK state requests exceeding a threshold are sampled, data quantity is reduced, problems are focused, and diagnosis time is greatly shortened, SVG is generated through FlameGraph, time consumption proportion is directly displayed, performance bottlenecks are quickly positioned, complex debugging is avoided, analysis efficiency is improved, and non-technical personnel can participate, cross-platform and multi-scenario adaptation are supported, Windows / Linux unified tool chains are realized through adaptation scripts, environment limitations of traditional tools are broken through, and mainstream server systems are covered, data verification and accuracy are realized, theoretical time consumption calibration verifies CPU load, automatic resampling is realized when error exceeds a threshold, accurate diagnosis results are ensured, and errors caused by sampling frequency or load are avoided, priority rules are set, high-frequency calling methods are processed preferentially, and optimization benefits are maximized, deadlocks and resource competition can be quickly positioned, troubleshooting time is reduced, system usability is improved, and business loss is reduced, dynamic Agent, intelligent sampling and cross-platform flame graph are combined, and full-process automatic diagnosis is realized, compared with the prior art, the application has a qualitative breakthrough in invasiveness, efficiency and compatibility, and has high innovation and practicality.

[0058] Although the application is disclosed as above, the application is not limited thereto. Any person skilled in the art can make various changes and modifications without departing from the spirit and scope of the application, and the protection scope of the application should be defined by the scope defined in the claims.

[0059] Finally, it should also be noted that, in this document, relational terms such as first and second and the like can only be used to distinguish one entity or action from another entity or action, without necessarily requiring or implying that there is any such actual relationship or order between these entities or actions. Moreover, the terms "comprises", "comprising", or any other variations thereof are intended to cover non-exclusive inclusions, so that a process, method, article, or apparatus that comprises a list of elements does not only include those elements, but also includes other elements not explicitly listed or inherent to such process, method, article, or apparatus. Without more limitations, an element defined by the statement "comprises a" does not exclude the presence of additional identical elements in the process, method, article, or apparatus that includes the element.

[0060] The various embodiments described in this specification are intended to be illustrative only and in no way limit the scope of the application. Changes and modifications can be made by those skilled in the art, which employ the principles of the application, without departing from the scope of the application. Accordingly, the application is not limited to the embodiments described herein, but instead has scope to encompass any choice whatsoever that is dependent on, or can be substituted in, the principal, new and inventive features that are described and claimed herein.

[0061] The above description of disclosed embodiments is intended to be illustrative only and not limiting of the application. Numerous modifications to these embodiments can be made by those skilled in the art without departing from the spirit or scope of the application. The scope of the application is not limited to the embodiments described herein, but rather extends to any that are dependent on, or can be substituted in, the principal, new and inventive features that are described and claimed herein.

Claims

1. A method for diagnosing HTTP request performance based on a dynamic agent mechanism, characterized in that, Includes the following steps: S1. The dynamic Agent program is connected to the target server through the Attach mechanism of the JVM tool script. The Agent program is implemented based on the Java Instrumentation API. S2. The Agent program proxies the service methods of javax.servlet.http.HttpServlet, intercepts specific HTTP requests, and starts a background monitoring thread to perform the following operations: Set an execution time threshold to mark HTTP requests that exceed the time threshold as STUCK status; Thread stack sampling is performed on STUCK status requests at a set frequency; Collect the completion event of the method execution and construct the flame graph data; S3. Generate two result files: head.txt records the total request time and the unique identifier UUID; The file uuid.txt stores the flame graph data obtained from sampling. S4. Based on the target server's operating system, call the flame graph generation tool to convert uuid.txt into an SVG format flame graph; S5. By analyzing the stack width and number of samples in the flame graph, performance bottlenecks can be identified; the flame graph width is positively correlated with execution time, so the widest stack should be optimized first.

2. The HTTP request performance diagnosis method based on the dynamic agent mechanism according to claim 1, characterized in that, S4 includes: The Agent client monitors changes to head.txt, extracts the UUID, and matches it with the corresponding uuid.txt. Use a cross-platform flame graph generation tool to generate SVG files.

3. The HTTP request performance diagnosis method based on the dynamic agent mechanism according to claim 2, characterized in that, The flame graph generation tool is Brendan Gregg's FlameGraph open-source component, which supports both Windows and Linux operating systems.

4. The HTTP request performance diagnosis method based on the dynamic agent mechanism according to claim 3, characterized in that, The S5 includes: The theoretical time can be calculated by combining the total time and the number of samples to verify the impact of CPU load. The calculation formula is: theoretical time = sampling interval time × (number of samples + 1).

5. The HTTP request performance diagnosis method based on the dynamic agent mechanism according to claim 4, characterized in that, The performance bottleneck location rules for S5 include: If the flame graph contains multiple stacks of equal width, prioritize optimizing the stack with the higher number of samples. When there is an error between the theoretical time taken and the total time recorded in head.txt, and the error exceeds the threshold, resampling is required.

6. An HTTP request performance diagnostic system based on a dynamic agent mechanism, characterized in that, include: The dynamic implantation module implants the Agent program into the target server through the JVM Attach mechanism. The Agent program is implemented based on the Java Instrumentation API. The request monitoring module proxies the request handling methods of javax.servlet.http.HttpServlet, marks HTTP requests exceeding the threshold as STUCK, and samples the thread stack at a set frequency. The data construction module collects sampled data and generates a flame graph data file uuid.txt and a time consumption record file head.txt; The visualization chart generation module converts flame graph data into flame graphs in SVG format; The bottleneck localization module analyzes the stack width and sample number of the flame graph to locate performance bottlenecks; the flame graph width is positively correlated with the execution time, so the widest stack is optimized first.

7. The HTTP request performance diagnosis system based on the dynamic agent mechanism according to claim 6, characterized in that, The bottleneck localization module can calculate the theoretical time by combining the total time and the number of samples to verify the impact of CPU load. The calculation formula is: theoretical time = sampling interval time × (number of samples + 1).

8. The HTTP request performance diagnosis system based on the dynamic agent mechanism according to claim 7, characterized in that, The bottleneck localization module is equipped with a stack weight evaluation rule: when there are multiple stacks of equal width, the optimization priority is sorted in descending order of the number of samples.

9. An electronic device, characterized in that, include: A processor, and a memory connected to the processor; The memory stores computer-executed instructions; The processor executes the computer execution instructions stored in the memory to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism as described in any one of claims 1-5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed, are used to implement the HTTP request performance diagnosis method based on the dynamic Agent mechanism as described in any one of claims 1-5.

Images