Hotspot symbol analysis method and system supporting multiple programming language environments and application

Abstract

The invention discloses a hotspot symbol analysis method supporting multiple programming language environments, which comprises the following steps: step 1, process initialization: storing all process symbol tables in a target range, judging programming languages of a target process, respectively carrying out initialization operation on different programming languages, binding perf to the target process, and storing the perf in the target range; setting a sampling frequency; step 2, data acquisition: sampling according to the sampling frequency set in the step 1, stopping perf after sampling is completed, reading a call stack and a stack frame symbol of a target process, storing read data, and then restarting perf for next sampling; and step 3, data processing and analysis: releasing binding with perf, performing statistics on call stack data obtained by data acquisition in the step 2, and performing analysis and visualization of hot symbols after data processing. The invention further discloses an analysis system and application for implementing the analysis method.

Description

Technical Field

[0001] The present invention belongs to the field of system optimization, and relates to a hotspot symbol parsing method, system and application that support multiple programming language environments. Background Art

[0002] As the infrastructure of modern society, the performance and efficiency of servers directly affect the operation of the overall system. Usually, the software environment on a server is a multi-programming language environment, that is, multiple processes are running simultaneously in the environment, and these processes are implemented in different programming languages. With the continuous increase of business requirements, the number of processes running on the server is increasing day by day. When the limited CPU resources cannot meet the needs of all processes, the problem of too high CPU usage rate will occur, resulting in longer response times of application processes, increased energy consumption and decreased system performance, thus affecting the quality of application services. Timely addressing and solving the problem of too high CPU usage rate is of great significance for ensuring the stable and efficient operation of application services.

[0003] The problem of too high CPU usage rate can be solved by optimizing the hotspot symbols of processes. Symbols in programming languages refer to functions or methods, usually corresponding to a code block. Hotspot symbols refer to the functions or methods that consume the most CPU clock cycles (cycles) within a certain time range. Cycles are the most basic time unit in a computer. The more cycles a symbol consumes, the longer the time it occupies CPU resources. Optimizing the code of hotspot symbols can effectively reduce the cycles consumed by a process to complete a task, and without affecting its function and performance, reduce the time of occupying CPU resources, and eliminate the problem of too high CPU usage rate.

[0004] When optimizing hotspot symbols, hotspot symbol parsing must be performed, that is, locating hotspot symbols. In a multi-programming language environment, performing hotspot symbol parsing only for a single process has limitations. The manifestations are as follows: First, usually multiple processes are running simultaneously in a server, and each process only occupies a part of the global CPU. Hotspot symbol parsing for a single process only considers the proportion of the cycles consumed by a certain symbol among the cycles consumed by all symbols in this process. However, from the perspective of the overall environment, the cycles consumed by this symbol may account for a relatively small proportion, and the optimization value is relatively low. Second, considering the current maturity of the software industry, some application processes do not have obvious hotspot symbols, and it is impossible to intuitively point out the optimization direction. Finally, there are usually shared symbols among multiple processes in the environment, and the cycles it consumes in a single process are relatively small. Only after summarizing the cycles consumed by this symbol in all processes will there be an obvious proportion. Therefore, hotspot symbol parsing for a single process cannot discover the shared hotspot symbols of different applications, and the provided optimization direction is limited.

[0005] Therefore, to solve the problem of excessive CPU usage in a multi-programming language environment, it is more valuable to parse the hot symbols in the entire environment, that is, the global hot symbols. Usually, multiple processes are running on the system, so the parsing tool needs to have the ability to parse multiple target processes simultaneously. The target processes may be implemented in different programming languages, so the parsing tool also needs to support multiple programming languages. The current tools have the following problems:

[0006] 1. Only for a single programming language. Most tools are only for a single programming language. For example, JProfiler [1] and JFR [2] are common hot symbol parsing tools for Java applications and cannot support parsing in other programming languages; py-spy [3] and cProfile [4] only have the function of parsing hot symbols for Python processes. To use these tools to implement hot symbol parsing in a multi-programming language environment, the usual method is to parse all processes within the target range separately and then summarize and count the hot symbols in the environment. The problem with this method is the lack of a unified way to measure hotspots. Both JProfiler and JFR use cycles as the benchmark for measuring hotspots and count the symbols that take the longest CPU execution time; however, py-spy and cProfile measure by actual time and count the symbols that consume the most actual time. If performance monitoring is performed on an I / O-intensive program, where the symbols consume fewer cycles but more actual time, different parsing tools will produce different results. Therefore, this method cannot count reasonable environmental hot symbols.

[0007] 2. Unable to automatically identify the programming language of the target process. There are a few tools that have the ability to parse more than one programming language. For example, Linux perf_event supports hot parsing of C / C++ and Java applications; VTune supports C / C++, Java, and Python at the same time. They both require the user to specify the programming language of the target process in advance before they can start the hot symbol parsing work. However, in a multi-programming language environment, the programming language used by a process usually cannot be directly viewed, and effective symbol parsing cannot be achieved only through the process ID. This greatly limits their hot symbol parsing ability.

[0008] 3. It is impossible to directly perform hotspot parsing on a running process. The execution state of a process is closely related to factors such as its running duration, input parameters, and system environment, so it is difficult to reproduce. When encountering the problem of too high CPU usage, it is usually necessary to immediately start a hotspot symbol parsing tool to collect hotspot symbols in the current scenario. This requires the hotspot symbol parsing tool to be able to perform hotspot symbol parsing on any running process without modifying the target code and without changing the current execution progress of the target. Existing tools such as cProfile need to modify the process code; Linux perf_event needs to restart the target Java process and modify the Java virtual machine settings before performing hotspot symbol parsing on the target. Summary of the Invention

[0009] To solve the deficiencies of the prior art, the object of the present invention is to provide a method, system and application for parsing hotspot symbols in a multi-programming language environment.

[0010] The technical problem solved by the present invention is how to design and implement a hotspot symbol parsing tool that supports the Linux system and two mainstream CPU architectures, x86_64 and aarch64, can simultaneously parse hotspot symbols of multiple processes within the target range, and supports hotspot parsing of processes such as C / C++, Java, and Python.

[0011] To parse hotspot symbols in a multi-programming language environment, the same hotspot measurement benchmark needs to be adopted for all processes in the environment. Therefore, the present invention solves the problem of using cycles as a unified hotspot symbol measurement benchmark.

[0012] The present invention targets C / C++, Java, and Python processes. Since different programming languages have different running mechanisms and require different symbol parsing methods, the problems of how to perform hotspot symbol analysis on C / C++, Java, and Python processes are respectively solved.

[0013] The present invention also solves the problem of judging the target programming language. Through the target process number, the programming language used by the target can be automatically judged, and an appropriate symbol parsing method can be adopted.

[0014] The hotspot symbol parsing method provided by the present invention, by utilizing technologies such as Linux perf_event, DWARF call stack backtracking, ELF file reading, Java agent, and memory image reading, does not affect the original execution progress of the target process and has extremely low performance impact on it. It can deploy and implement hotspot symbol parsing without the target process being aware of it. Therefore, it is not necessary to modify the source code of the target process or restart the target, so it can be deployed in a production environment to perform real-time hotspot symbol parsing on the online application load.

[0015] The present invention not only analyzes the global environment hot symbols, but also provides the call stack corresponding to the symbols. When a process executes the code block contained in a symbol, it creates a stack frame to save the data in the code block. Since there are call relationships between symbols, the process will create multiple stack frames, and these stack frames form a call stack. The call stack records the code execution path of the process. Providing the call stack corresponding to the hot symbol, that is, providing the runtime background of the hot symbol, can help developers better optimize the hot symbol code.

[0016] In summary, the present invention mainly provides a method, a corresponding system and an application for automatically determining the programming language corresponding to a target process through the target process number, selecting a suitable symbol parsing method to parse symbols according to different programming languages, using cycles as a unified measurement benchmark for hot symbols, and locating hot symbols.

[0017] The present invention provides a method for parsing hot symbols that supports multiple programming language environments, and the method includes the following steps:

[0018] Step 1, process initialization: save all process symbol tables within the target range, determine the programming language of the target process, perform initialization operations for different programming languages respectively, bind perf to the target process, and set the sampling frequency;

[0019] Step 2, data collection: sample according to the sampling frequency set in Step 1, stop perf after sampling is completed, read the call stack and stack frame symbols of the target process, save the read data, and then restart perf for the next sampling;

[0020] Step 3, data processing and analysis: unbind from perf, count the call stack data obtained from data collection in Step 2, and perform analysis and visualization of hot symbols after data processing.

[0021] Step 1 further includes the following steps:

[0022] Step 1.1, save the process symbol table: traverse the memory snapshots of all processes, find all ELF files related to the process, and then parse and save the symbol table information in the ELF files one by one; when the symbol table is not saved in the ELF file, find the backup symbol table in the system according to the build information and debug information of the process;

[0023] Step 1.2. Determine the programming language of the target process: Parse the ELF file corresponding to the target process, traverse the symbol table of the target process, and match and compare the symbols in the symbol table of the target process with the process symbol tables of different programming languages saved in Step 1.1 one by one. If a symbol matching the process symbol table of a different programming language is found in the symbol table of the target process, it is initially determined that the target process belongs to a Java or Python program; if there is no match between the target process and the process symbol tables of different programming languages, it is initially determined that the target process belongs to a C / C++ program;

[0024] For the case where the target process belongs to a Java or Python program, double verification can also be used for further judgment: When it is initially determined that the target process belongs to a Java program, according to the settings in the target Java virtual machine, search whether the socket file created according to the preset rules exists in the file directory address in the system file directory. If it exists, it is further determined that the target process belongs to a Java program; or,

[0025] When it is initially determined that the target process belongs to a Python program, start the executable file of the target process. If a string matching the Python output format can be output, it is further determined that the target process belongs to a Python program; or,

[0026] When the target process belongs to a C / C++ program, parse its ELF file, build information, and debug information to obtain the symbol table data of the target process.

[0027] Step 1.3. Initialization operation: If the programming language of the target process obtained in Step 1.2 is Python, parse the ELF file of the target process, find the address of the global variable in memory, and obtain the data during variable runtime; or,

[0028] If the programming language of the target process obtained in Step 1.2 is Java, let the target process run external code, save the DWARF data in the ELF file, and register a signal handler; or,

[0029] If the programming language of the target process obtained in Step 1.2 is C / C++, the operations in Step 1.2 are sufficient to meet the initialization requirements, and no other operations are required;

[0030] Step 1.4. Bind perf to the target process and set the sampling frequency.

[0031] In Step 1.1, the process symbol table records the mapping relationship between program entities and memory addresses;

[0032] Both the build information and debug information of the process contain the symbol table of the corresponding program. When the symbol table of the ELF file itself is stripped, the build information and debug information files serve as the source of the backup symbol table;

[0033] By reading the / proc / [pid] / maps file, a process memory snapshot is obtained; the memory snapshot contains the files read and loaded into memory by the target process and their corresponding file usage permissions; the file usage permissions consist of three parts: readable, writable, and executable. Traverse the memory snapshot, and for files with executable permissions, determine them as ELF files, and parse them to obtain the symbol table.

[0034] Each program entity (including symbols, global variables, etc.) in the symbol table has a corresponding memory address, indicating the memory storage location of the program entity during program execution. The program entity and the memory address are saved in a one-to-one correspondence. For C / C++ programs, during the call stack reading phase, the memory address of each stack frame is saved; during the stack frame symbol reading phase, the memory address is compared with the previously saved data, and the symbol with the same address is taken as the symbol of the corresponding stack frame.

[0035] and / or,

[0036] In step 1.2, the symbol table of the Java program contains the symbol asyncGetCallTrace, and the symbol table of the Python program contains the global variable PyInterpreterState. Therefore, when traversing the ELF file, these two entities are compared one by one with the symbols or global variables in the symbol table. When the comparison is successful, it can be initially determined that the current process is a Java or Python process;

[0037] and / or,

[0038] In step 1.3, when it is determined in step 1.2 that the current process is a Python program, the global variables such as PyInterpreterState, PyThreadState, PyFrameObject, and PyCodeObject are read respectively. These global variables save the interpreter state data, Python thread state data, Python stack frame state data, Python stack frame code object data, etc. of the current Python process, which can be used for subsequent call stack reading and stack frame symbol reading to analyze the execution status and behavior of the Python program; or,

[0039] When it is determined in step 1.2 that the current process is a Java program, read the DWARF data in the ELF file; DWARF data is a type of data used for debugging, which stores information such as register status, symbol table, and source code addresses; the register status records the values of each register during the execution of the program, which helps analyze the running state of the program; the symbol table contains entity information in the program, such as function names, global variable names, etc.; the source code address provides the address information of the source code file, which helps developers locate the source code position during debugging.

[0040] and / or,

[0041] In step 1.4, set the perf status parameters, which include parameters such as the target process PID (Process Identification Number), sampling frequency, perf wake-up times, perf_event response method, etc. When calling the system call function perf_event_open, pass these parameters in to complete the perf binding work for the target process and achieve the statistics of cycles for the target.

[0042] Target process PID: PID is the unique identification number of each running process. In step 1.4, it is necessary to specify the PID of the target process to be analyzed for performance, which is used to bind with perf and monitor the performance data of this process.

[0043] Sampling frequency: The sampling frequency is the rate at which perf collects performance data. It represents how many times performance data is sampled per second. In step 1.4, by setting an appropriate sampling frequency, balance the performance overhead and data accuracy according to the requirements of performance analysis.

[0044] Perf wake-up times: Perf usually runs in a "sleep-wake" manner, where sleep means it temporarily stops collecting performance data, and then wake means it resumes collecting data. In step 1.4, it is necessary to set the perf wake-up times to determine when to wake up perf to start sampling; by controlling the collection timing of performance data, ensure that sufficient performance data is obtained at appropriate time intervals.

[0045] Perf_event_open system call: Perf_event_open is a Linux system call function used to create and configure performance counters; the function accepts data including the target process PID, sampling frequency, wake-up times, etc., as well as other parameters related to performance analysis. By calling perf_event_open and passing these parameters, perf will be bound to the specified target process and start collecting performance data.

[0046] perf_event response mode: It refers to how perf responds to and records various performance events, such as CPU cycles, cache hits, etc.

[0047] Step 2 further includes the following steps:

[0048] Step 2.1: Sample the data according to the sampling frequency set in Step 1. After sampling is completed, stop perf from counting cycles.

[0049] Step 2.2: Read the call stack: If it is determined that the target process is a C / C++ program, use the ptrace system call to pause the process; then use the method in the libunwind.so library to read the memory addresses of all stack frames in the call stack; or,

[0050] If it is determined that the target process is a Python program, read the Python program stack and the Python interpreter stack composed of C / C++ respectively: Use global variables to read the stack frame objects of the process one by one to complete the reading of the Python program stack; pause the target process through ptrace and read the Python interpreter stack through libunwind.so; after the Python program stack and the Python interpreter stack are spliced, they form a complete Python process call stack; or,

[0051] If it is determined that the target process is a Java program, the Java process call stack is implemented through a signal handler. Use the signal handler to read the information of the Java program stack, which is read through the function in the libjvm.so library. The function returns all stack frame IDs of the call stack of the current process; then read the Java virtual machine stack composed of C / C++ through DWARF data and virtual machine global variables. Finally, splice the Java program stack and the Java virtual machine stack into a complete Java process call stack.

[0052] Step 2.3: Read stack frame symbols: If it is determined that the target process is a C / C++ program, combine the stack frame memory addresses read in Step 2.2 with the symbol table saved in Step 1 to read all symbols corresponding to the stack frames and the source files where the symbols are located; or,

[0053] If it is determined that the target process is a Python program, parse the stack frame objects in Step 2.2. The stack frame objects contain the symbols corresponding to the code blocks of the current stack frame and the source files where they are located, which are obtained through the process_vm_readv system call to complete the symbol reading of the Python stack frames; read the Python interpreter stack frame symbols in the Python program; the acquisition of the Python interpreter stack frame symbols is the same as that of the C / C++ program; or,

[0054] If it is determined that the target process is a Java program, parse the stack frame ID obtained in step 2.2, obtain the stack frame symbols and the source files where the symbols are located in the call stack, and complete the symbol reading of the Java program stack frame; the method for reading the Java virtual machine stack frame symbols is the same as that of C / C++ programs;

[0055] In Python or Java programs, the method for reading stack frame symbols is also to combine the read stack frame memory addresses with the symbol table saved in step one to read all the symbols corresponding to the stack frames and the source files where the symbols are located.

[0056] Step 2.4, save the read data, and the data includes all the symbols in the call stack, the source files where the symbols are located, and metadata such as the process ID and the collection time.

[0057] In step 2.1, perf samples at the set sampling frequency. After perf completes data collection, it will automatically go to sleep until the subsequent call stack reading, stack frame symbol reading, and data saving work are completed, then wake up perf to perform the next sampling and re - count cycles;

[0058] and / or,

[0059] In step 2.2, the call stack of the C / C++ program consists only of C / C++ symbols, the call stack of the Python program consists of the Python program stack and the Python interpreter stack, and the call stack of the Java program consists of the Java program stack and the Java virtual machine stack;

[0060] and / or,

[0061] In step 2.3, since the call stacks of all three programs contain C / C++ symbols, the C / C++ symbol parsing method will be used; for the stack frame symbol reading of Python and Java programs, the Python and Java symbols need to be obtained by "obtaining through the process_vm_readv system call" and "parsing the stack frame ID obtained in step 2.2 to obtain the stack frame symbols and the source files where the symbols are located in the call stack" respectively;

[0062] and / or,

[0063] In step 2.4, the saved metadata serves as the data source for subsequent hotspot statistics.

[0064] Step three further includes the following steps:

[0065] Step 3.1, unbind perf according to the programming language type of the target process;

[0066] Step 3.2: Count all the call stack data in Step 2, summarize the top symbols of the process call stacks for each process, count the occurrence times of the symbols, and determine the hot symbols;

[0067] Step 3.3: Count the call stacks of the hot symbols, find all the call stacks of the same hot symbol, compare the stack frame symbols one by one, merge the call stacks with exactly the same stack frame symbols, accumulate their sampling times, and calculate the call times of the hot symbol in different call stacks;

[0068] Step 3.4: Generate an output visual hot symbol sorting graph, hot symbol call stacks, and flame graph.

[0069] In Step 3.1, according to the different types of programming languages used by the target process, different methods are adopted to unbind perf; for Java programs, run a section of external code to stop the perf operation and unbind it; for Python and C / C++ programs, perf can be directly closed and unbound through system call functions;

[0070] and / or

[0071] In Step 3.2, count the occurrence times of the top symbols of the call stacks, sort the hot symbols according to the number of occurrences, and the more times a symbol appears, the more cycles it consumes;

[0072] and / or

[0073] In Step 3.3, for the top symbols obtained in Step 3.2, count the number of times they appear in different call stacks;

[0074] and / or

[0075] In Step 3.4, sort and display the hot symbols according to the number of cycles consumed from more to less, and at the same time display the call stacks involved in each symbol and the proportion of cycle consumption in different processes; generate a flame graph based on the collected data to visually display the analysis results of the hot symbols.

[0076] The present invention also provides an analysis system for implementing the above hot symbol parsing method, and the analysis system includes: an initialization module, a collection module, and a data processing module;

[0077] The initialization module is used to initialize the process, including saving the target process symbol table, judging the programming language of the target process, binding perf and setting the sampling frequency;

[0078] The collection module samples at the set frequency and automatically adopts different processing methods according to different programming languages to collect and save data;

[0079] The data processing module processes and analyzes the collected symbol data, and visually displays the analysis results.

[0080] The present invention also provides the above-mentioned hot symbol parsing method or parsing system that supports multiple programming language environments, and its applications in hot symbol parsing in multi-programming language environments, hot symbol parsing in server clusters, hot symbol parsing in servers containing containers, performance anomaly detection and diagnosis, etc.

[0081] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0082] The present invention runs in the Linux environment, is compatible with two mainstream CPU architectures, x86_64 and aarch64, and can perform hot parsing on multiple processes on the system simultaneously. These processes can be different applications written in C / C++, Java, and Python languages. The present invention can judge the programming languages used by all processes within the target range, and thus adopt appropriate methods to read the call stack. Then, using perf as the hot spot collection tool and cycles as the measurement benchmark, it collects the hot symbols of all processes within the target range. By summarizing all the collection results, it statistically analyzes the symbols that consume the most cycles within the target range, providing a direction for solving the problem of excessive CPU usage in multi-programming language environments.

[0083] Since it targets all processes in the entire environment and views from the perspective of global CPU consumption, the present invention can statistically analyze global hot symbols. The optimization of global hot symbols is directly reflected in the release of global CPU resources. Since multiple targets are monitored simultaneously and all hot symbols are comprehensively statistically analyzed, hot symbols shared among multi-target processes can be found, and potential optimization directions can be explored.

[0084] At the same time, the present invention does not require any modification to the target program and can directly perform parsing on the running processes. Therefore, it can be deployed in the production environment to parse the hot symbols in the actual environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those skilled in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0086] Figure 1 It is the design architecture diagram in the technical solution of the present invention.

[0087] Figure 2It is a schematic diagram of different language call stacks in the technical solution of the present invention. From left to right, they are the C / C++ process call stack, the Python process call stack, and the Java process call stack respectively.

[0088] Figure 3 It is a schematic diagram of the target / profiler model in the technical solution of the present invention.

[0089] Figure 4 It is a schematic diagram of the combination of target and profiler in the technical solution of the present invention.

[0090] Figure 5 It is the relative relationship between the Java process call stack and the FP and SP registers in the technical solution of the present invention.

[0091] Figure 6 It is a schematic flow chart of merging the hot symbol call stacks in the technical solution of the present invention. In the schematic diagram, taking the hot symbol X as an example, the collection times of X in the call stacks ABC and AB are merged.

[0092] Figure 7 It is a flow chart of Embodiment 1 in the specific implementation manner of the present invention.

[0093] Figure 8 It is a flow chart of Embodiment 2 in the specific implementation manner of the present invention.

[0094] Figure 9 It is a flow chart of Embodiment 3 in the specific implementation manner of the present invention.

[0095] Figure 10 It is a flow chart of Embodiment 4 in the specific implementation manner of the present invention. Specific implementation manner

[0096] Combined with the following specific embodiments and attached drawings, the present invention will be further described in detail. The processes, conditions, experimental methods, etc. for implementing the present invention, except for the specifically mentioned content below, are all common knowledge and well-known common sense in the art, and the present invention has no special restrictions.

[0097] The present invention provides a hot symbol parsing method, system and application that support multiple programming language environments.

[0098] The present invention has the ability to parse multiple processes simultaneously. Whether it is a C / C++ application process, a Java application process or a Python application process, hot symbol parsing can be implemented. The system architecture of the present invention is as Figure 1As shown in the figure, it can be divided into three modules: an initialization module, a collection module, and a data processing module. First is the initialization module, which is mainly responsible for the preparatory work in the early stage, such as judging the target programming language, parsing and reading to save the symbol table, setting perf, etc. Secondly is the collection module, which reads the target call stack at the set frequency, reads the stack frame symbols and saves the data, and automatically adopts different processing methods according to different programming languages. Finally is the data processing module, which is responsible for processing and analyzing the collected symbol data, including hot symbol statistics, call stack summarization, etc., and visualizes the analysis results.

[0099] The present invention uses Linux perf_event (perf) to count the cycles consumed by all target processes. Perf is a performance monitoring tool built into the Linux kernel. Based on the hardware performance detection unit (Performance Monitoring Unit, PMU), it is implemented on two mainstream CPU architectures, x86_64 and aarch64, and includes two basic working modes: sampling and counting. The present invention uses the perf sampling mode, targets the bound process, and counts the duration of the target event. When the duration reaches the threshold, it reads the sampling object of the target. In the present invention, cycles are used as the event, the CPU register is used as the sampling object, and the specified process is used as the target. After specifying the target process, the user can customize the sampling frequency (samplingfrequency), such as 99Hz. Specifically, if the cumulative time for the target process to consume cycles exceeds one over samplingfrequency seconds, perf will perform the sampling operation and read the value of the CPU register. After perf completes the binding, it will create a file descriptor and a corresponding memory buffer. When sampling, the value of the CPU register will be saved to the buffer. Through setting, when a write operation occurs in the buffer, the corresponding file descriptor can be awakened to further notify the collection module to perform corresponding operations.

[0100] 1. Initialization module

[0101] The present invention first runs the initialization module to prepare for hot symbol parsing. For all processes within the target range, the following operations are performed:

[0102] (1) Save the process symbol table. The symbol table is a section of data in an ELF file that contains the correspondence between program entities such as symbols and global variables and memory addresses. By traversing the process memory image, find all ELF files related to the process, parse and save the symbol tables corresponding to different programming languages in each one. In a production environment, for the purpose of reducing file size, the symbol tables in ELF files are stripped. Therefore, if the symbol table cannot be found in the ELF file, the present invention will search for the backup symbol table in the system according to the build information and debug information of the process; the symbol name is the name of a function or method, and the global variable name refers to the name of a certain piece of data, and their corresponding memory addresses are all recorded in the symbol table;

[0103] Global variables are the same as functions. They both have storage locations in memory, are both a section of program entities, and their storage methods in the symbol table are also basically the same as those of functions. Both occupy a section of space in memory. When analyzing the symbol table, the present invention will record the names of each function and global variable, as well as their address ranges in memory. Specific storage examples:

[0104] Symbol: findSymbolByAddress, memory address range: 55cf029be000–55cf029be0aa;

[0105] Global variable: PyInterpreterState, memory address range: 7f33d1c00000–7f33d1c0000f.

[0106] (2) Determine the programming language of the target process. The present invention can parse the hot symbols of C / C++, Java, and Python processes. Depending on the programming language, different preparatory work needs to be done. Java processes and Python processes run through virtual machines and interpreters respectively, and there are certain specific symbols in the symbol tables of their ELF files respectively. Therefore, the symbol table in the target process can be compared with the symbol tables of different programming languages obtained in the previous step one by one. Once a match is found, it can be determined that the programming language used by the current process is Java or Python. If no match is achieved for the current process, it is determined to be a C / C++ program.

[0107] The above operations can preliminarily determine the language used by the target process. In addition, to prevent misjudgment, the present invention provides a second verification means based on the special mechanisms of the Java virtual machine and the Python interpreter. The Java virtual machine will create a socket file for each Java program in a preset specified folder, which will vary according to the settings of the Java virtual machine. By default, it is the / tmp folder. The naming of the socket file is composed of a string and the target process PID, in the format of java-%p, where %p represents the current process PID. For example, if the target process PID is 1234, the file name is java-1234. Search for the existence of this file in the / tmp folder. Using this mechanism, when it is preliminarily determined that the current process belongs to a Java program, search for the existence of the socket file at the specified address. If it exists, it is further determined that the current process belongs to a Java program. As an executable file, the Python interpreter will output a fixed-format string (Python interpreter version number) in the standard output stream when starting. The beginning of this string is composed of "Python" and a three-digit integer separated by dots, such as Python 3.9.4. Therefore, it can be judged whether the string meets the requirements through regular expressions. Using this mechanism, when it is preliminarily determined that the current process belongs to a Python program, start the executable file of this process. If a string matching the above format can be output, it is further determined that the current process belongs to a Python program.

[0108] (3) Analyze the global variables of the Python process. The Python interpreter maintains the runtime state of the Python process in global variables, including: PyInterpreterState, which stores the Python interpreter state data and contains a pointer to PyThreadState; PyThreadState, which stores the Python thread state data and contains a pointer to the top PyFrameObject of the stack; PyFrameObject, which stores the Python stack frame object data and contains pointers to the PyFrameObject of the lower stack frame and the PyCodeObject; PyCodeObject, which stores the Python stack frame code state data and contains the symbol name of the current stack frame. After reading the PyInterpreterState data from the ELF file, the remaining data can be obtained through pointers, and all these data are used for subsequent work of reading the call stack and the stack frame symbols. If it is determined that the current process belongs to a Python program, then parse the ELF file of this process one by one to find the memory address of the global variable. Through the process_vm_readv system call, obtain the runtime data of the variable. The system call process_vm_readv can read the data at the specified memory address of the target process. Therefore, when using it, the PID of the target process, the target memory address, the number of bytes to be read, and the variable used to store this data need to be specified. After the call is successful, the target data is saved to the specified variable.

[0109] (4) Through the system call perf_event_open, implement the perf binding work for the target process. The system call perf_event_open completes the settings of the perf binding object, the sampling frequency, and the buffer address, etc. according to the input parameters. At the same time, it is set that perf automatically enters the sleep state every time a sampling is completed until it is started again. Among them, the perf binding object is the PID of the target process; the sampling frequency is set by the user of the present invention at startup; the buffer address is a memory area allocated through the system call malloc and is used to store the data collected by perf, such as registers. The system call malloc will allocate a memory area for the program, and only the size of the memory area to be allocated needs to be passed to it. After the allocation is successful, the address of the allocated memory area will be returned. After perf_event_open is called, it will bind perf to the target, and then create a file descriptor. This file descriptor has a one-to-one correspondence with the buffer. When perf writes the collected data into the buffer, the file descriptor will be awakened and can be sensed by system calls such as epoll. The present invention will save the file descriptor, the PID of the target process, and the buffer. Through any one of them, the other two can be found.

[0110] So far, the initialization of C / C++ and Python processes has been completed.

[0111] For Java processes, the present invention uses the JavaAgent technology. The JavaAgent technology enables a target Java program to execute a piece of external code during runtime. The external code needs to be encapsulated into a library file, and then the address of this file is written into the socket file of the target Java program. After the writing is completed, the Java program will automatically read the library file according to the address and immediately execute the external code. The present invention encapsulates the initialization module and the collection module as external code into the library file, and uses the JavaAgent technology to make the target Java process execute the logic of the initialization module and the collection module. First, the initialization module is executed. Similar to the initialization modules of C / C++ and Python, the Java process will save the symbol table. In addition, the DWARF data in the ELF file will also be saved, which stores the process stack frame information. Then, a signal handler is registered and perf is bound. So far, the initialization of the Java process has been completed.

[0112] After all the initialization work is completed, perf is started through the system call ioctl. Ioctl needs to pass in the file descriptor generated by perf_event_open and the start flag of perf. When the time for the target process to consume cycles reaches the threshold, perf sampling is triggered, and at this time, it will automatically enter the collection module to read the call stack of the target process; the threshold refers to the reciprocal of the custom sampling frequency.

[0113] 2. Collection Module

[0114] The collection module is triggered to run during perf sampling. According to the settings in perf_event_open, perf will automatically go to sleep and stop counting cycles after the collection is completed. Then the collection module will perform the following operations in sequence: read the target call stack, read the stack frame symbols, and save the data. Among them, different methods will be adopted for the first two steps according to different programming languages. Finally, perf is restarted to prepare for the next collection.

[0115] (1) Read the Call Stack

[0116] Different programming languages have different operating mechanisms, so they have different call stack compositions. For example Figure 2As shown, the C / C++ program can run directly after compilation, and the call stack consists only of C / C++ symbols. The Python program needs to run through the Python interpreter. During the running process, the Python interpreter interprets the Python code into executable code. The call stack of the Python program consists of the Python program stack and the Python interpreter stack. The Python interpreter itself is written in C / C++. The Java program needs to run through the Java virtual machine. The Java code is interpreted into executable code by the Java virtual machine. The call stack of the Java program consists of the Java program stack and the Java virtual machine stack. The Java virtual machine itself is written in C / C++.

[0117] Therefore, the call stacks of both Python programs and Java programs consist of at least two parts: the underlying C / C++ stack and the upper-layer Python or Java stack. The present invention can simultaneously count the hot symbols of multiple target processes. Therefore, it is necessary to call perf_event_open multiple times to create multiple file descriptors and buffers. The present invention monitors all monitored file descriptors by passing them into epoll through a system call. When perf writes the collected data into one of the buffers, the corresponding file descriptor is awakened and thus captured and returned by epoll. The acquisition module finds the corresponding process ID and its programming language according to the file descriptor returned by epoll, and reads the call stack using different methods. The call stacks of different programming languages are as Figure 2 shown: a) Reading the C / C++ call stack

[0118] If it is determined that the current process is a C / C++ program, use the ptrace system call to suspend the process; then use the method in the libunwind.so library to read the memory addresses of all stack frames in the call stack.

[0119] The ptrace is a system call for process tracing, usually used by tools such as debuggers to monitor and control processes. It allows one process (called the tracer) to observe and manipulate the execution of another process (called the traced process).

[0120] The libunwind.so is a library for obtaining program call stack information. It provides the function of backtracking the call stack and can obtain the call stack information of the currently executing code, including function call relationships, function names, etc.

[0121] b) Reading the Python call stack

[0122] The reading of the Python process call stack consists of two parts: C / C++ and Python. Among them, the C / C++ stack (Python interpreter stack) is composed of symbols of the Python interpreter, and the Python program stack is composed of symbols in the Python program. If it is determined that the current process is a Python program, the acquisition module reads the Python program stack and the Python interpreter stack composed of C / C++ respectively: using the global variables read by the initialization module, reads the stack frame objects of the process one by one to complete the reading of the Python call stack; the method of reading the C / C++ call stack is the same as before, pauses the process through ptrace, and reads the call stack through libunwind.so. The Python call stack is spliced with the C / C++ stack to form a complete Python process call stack.

[0123] c) Reading the Java call stack

[0124] Different from C / C++ or Python programs, the operation of reading the Java process call stack is implemented through the signal mechanism of the operating system. In the Linux system, when a process receives a signal, it will interrupt the current process and then call the signal handler. A signal is a way of communication between processes for transmitting messages. In the initialization stage, the target Java process registers a signal handler for the signal SIGPROF by running external code, and at the same time sets through perf_event_open that when perf collects data, a SIGPROF signal will be sent to the target process. The signal handler is a special code block responsible for reacting to the specified signal. After the process registers a handler for the specified signal, it will enter this handler every time it receives the signal. So after the initialization module is completed, every time the target Java process receives the SIGPROF signal sent by perf, it will run the corresponding handler. In addition, the signal handler can also read the values of the instruction pointer (IP), stack pointer (SP), and frame pointer (FP) registers at the process interruption position, and these three values will be used when reading the call stack of the Java process.

[0125] The operating system maintains the call stack structure through the SP and FP registers. The SP register records the termination address of the top of the call stack, and the FP records the starting address of the top of the call stack. The IP register saves the position of the code currently running by the process. It should be noted that the SP and FP registers here are general terms, and their names in the x86_64 and aarch64 architectures are the RBP register, RSP register and x29 register, SP register respectively.

[0126] The hot spot collection of C / C++ and Python programs is a target / profiler model, which is connected in series through perf, such as Figure 3When perf samples, the profiler pauses the target process, reads its call stack, and then the target continues to run. For hotspot collection of Java programs, since the signal handler has been registered in the initialization module through the JavaAgent technology, when the target Java process receives a perf signal, the signal handler is automatically run to read its own call stack. This mode is equivalent to integrating the profiler with the target, as Figure 4 The reading of the Java process call stack includes two parts: C / C++ and Java. Among them, the C / C++ stack (Java virtual machine stack) consists of symbols of the Java virtual machine, and the Java program stack consists of symbols in the Java program. The signal handler includes operations to read the call stack and stack frame symbols. Among them, the Java program stack is read through functions in the libjvm.so library, and the function returns all stack frame IDs of the call stack of the current process. Thus, the reading of the Java program stack is completed.

[0127] Use DWARF data and virtual machine global variables to read the C / C++ part of the call stack (Java virtual machine stack). Among them, the virtual machine global variables store the address range of the current virtual machine code, which is also the range of the C / C++ stack; DWARF is a type of debugging data stored in the ELF file. It associates the positions of all codes in the program with SP and FP. Therefore, by reading the IP, SP, and FP registers of the process at any time, the entire call stack can be read through DWARF.

[0128] When the Java program runs, Java code is translated into bytecode and runs on the CPU through the virtual machine. At the same time, the SP and FP registers maintain the call stack of the bytecode, that is, the Java stack. Therefore, to use IP, SP, and FP to read the C / C++ call stack, the three need to be rolled back to the C / C++ stack range. The rollback operation starts from the initial values of IP, SP, and FP in the signal handler. Since the virtual machine global variables store the offset values of IP, SP, and FP for each Java stack frame, using the offset values, continuous addition and subtraction operations are performed on the initial values of the three until the IP falls within the virtual machine code range. At this time, SP and FP also point to the top layer of the C / C++ call stack. Next, use the data in DWARF to read the C / C++ call stack. As Figure 5 , the Java program stack and the C / C++ stack (Java virtual machine stack) are spliced to form the complete Java process call stack.

[0129] (2) Reading stack frame symbols

[0130] a) Reading C / C++ stack frame symbols

[0131] The memory address of the stack frame read in the previous step is combined with the symbol table saved in the initialization module to read all symbols corresponding to the stack frames and the source files where the symbols are located. Thus, all data reading tasks in a single collection are completed. Then, ptrace is called again, and the process continues to run.

[0132] b) Read Python stack frame symbols

[0133] The stack frame object read in the previous step can be further parsed. The object contains the symbols corresponding to the code blocks of the current stack frame and the source files where they are located, which are obtained through the process_vm_readv system call, completing the symbol reading of the Python stack frame. The method for reading C / C++ stack frame symbols is the same as that for C / C++ programs. Thus, all data reading tasks in a single collection are completed. Finally, ptrace is called again, and the process continues to run.

[0134] c) Read Java stack frame symbols

[0135] The Java stack frame ID read in the previous step is further parsed through the Java Virtual Machine Tool Interface (JVMTI) to obtain the symbols of the stack frames in the call stack and the source files where the symbols are located, completing the symbol reading of the Java stack frame. The method for reading C / C++ stack frame symbols is the same as that for C / C++ programs. Thus, all data reading tasks in a single collection are completed. The signal handler exits, and the process continues to run.

[0136] (3) Data saving

[0137] The data read is saved. When saving, in addition to the symbols of the call stack and the source files where the symbols are located, metadata such as the process ID and collection time are also saved together for use by the subsequent data processing module.

[0138] (4) Restart perf again through ioctl

[0139] The file descriptor corresponding to perf and the start flag are passed into the system call ioctl to restart perf, and the above collection process is looped.

[0140] 3. Data processing module

[0141] After the parsing is completed, for C / C++ and Python processes, unbind the previously bound perf. For Java processes, use the JavaAgent technology to notify them to unbind the perf. Statistically collect all call stack data in the collection module. According to the triggering principle of perf sampling, generally, the more cycles the target process consumes, the more times it is collected, and the more call stacks of this process in the collection module. The data processing module first statistically analyzes the hot symbols. At any given moment, only the top symbol on the stack of a process consumes cycles. Therefore, summarize the top symbols on the call stacks of all processes and count the number of times they appear. The more times, the "hotter" the symbol is. Thus, obtain the sorted list of hot symbols in a multi-programming language environment. At the same time, display the call stacks related to all symbols and the proportion of cycles consumed by this symbol in different processes, so as to provide more runtime information of the symbols for the optimization personnel.

[0142] The call stack reflects the call relationship between symbols. One symbol may be called by multiple other symbols. Therefore, the present invention not only statistically analyzes the hot symbols, but also counts the call stacks of the hot symbols, in order to provide better optimization suggestions for programmers.

[0143] Finally, the present invention will output the sorted graph of hot symbols, the call stacks of hot symbols, and the flame graph. In a data visualization manner, intuitively display the cycles consumption of hot symbols and their runtime background for the optimization personnel.

[0144] Embodiment 1: Run directly in a single containerless server for parsing the global hot symbols of the server

[0145] In a server without running containers, multiple processes are running simultaneously. These processes include Java applications, Python applications, and C / C++ applications. In order to alleviate problems such as insufficient CPU resources and reduced process response time, it is now necessary to parse the hot symbols of the processes running on the system.

[0146] The present invention can be directly started after being deployed on the current server. When starting, specify the process numbers within the target range, the parsing time, and the sampling frequency, and then run. The process of running the present invention in a single containerless server is as Figure 7 , first initialize the module to run, which will identify the programming language of the target process, save the symbol table of the target process, and bind perf. If the target process is a Java program, complete the initialization through the Java Agent technology. Then start perf. After perf completes data collection, it will automatically go to sleep, and at the same time trigger the collection module to run. The collection module will read the call stack and stack frame symbols, save the relevant data, and start perf again. After the parsing is completed, unbind perf. The data processing module will statistically analyze the hot symbols and their call stacks and perform visual output. The detailed statistical process is as Figure 6, first find all the call stacks of the same hot symbol, then compare the stack frame symbols one by one, merge the call stacks with exactly the same stack frame symbols, accumulate their sampling counts, and finally calculate the call counts of the hot symbol in different call stacks. During this process, no operation needs to be performed on the target process, and the present invention will not affect its original execution progress. Thus, the parsing work of hot symbols in the hybrid environment on a single server is completed.

[0147] If existing tools or technologies are used, the task of parsing global hot symbols in a single containerless server cannot be completed. The reason is that in a server with a multi-programming language environment, existing tools or technologies cannot perform hot symbol parsing on multiple processes simultaneously with a unified hot measurement standard, so accurate global hot symbols cannot be counted.

[0148] Embodiment 2: Run in a server cluster for parsing hot symbols of the cluster

[0149] The present invention is not only applicable to the system environment of a single server, but also can be distributedly deployed within a computing cluster, and each server in the cluster contains an instance of the present invention. In this way, not only can the hot symbols within a single server be parsed, but also the hot symbols in the cluster can be parsed.

[0150] Each server is a node, and any node can be used as the master node to start the present invention, and the remaining nodes are slave nodes. The master node can specify the slave nodes to be analyzed for hot symbols when starting. The process of parsing hot symbols of the server cluster is as follows Figure 8 , after receiving the signal from the master node, the target slave node starts from the initialization module, starts perf, goes through the acquisition module, and generates a data file. The data file is the original call stack data saved in the acquisition module. The parsing process in a single slave node is similar to that in Embodiment 1, except that the data processing module does not run in the slave node. After the parsing is completed, the master node sends a signal to all target slave nodes to notify them to stop parsing, unbind perf, and transfer the data file to the master node. The master node aggregates all the data files, and after passing through the data processing module, performs visual output. Thus, the parsing work of hot symbols in the hybrid environment in the computing cluster is completed.

[0151] Currently, there are no available tools or technologies that can perform hot symbol parsing in a server cluster with a multi-programming language environment. Embodiment 3: Run in a single containerized server for parsing hot symbols of the target container

[0152] A server running containers, and multiple processes are running in each container. These processes include Java applications, Python applications, and C / C++ applications. To alleviate problems such as insufficient CPU resources and reduced process response time, it is now necessary to perform hot symbol parsing on multiple target containers running in the system.

[0153] The present invention can be directly started after the current server deployment is completed. When starting, specify the container to be parsed, the parsing time, and the sampling frequency, and then run. The process of running the present invention in a single server with containers is as follows Figure 9 , first, initialize the module to run, save the symbol table, and analyze the programming languages of all processes in the container. Then, bind perf. After perf is started, it will automatically trigger the operation of the collection module to collect the call stacks of all processes in the container during the parsing period. After the parsing of all containers is completed, unbind perf. The data processing module collects the data files in all target containers, uniformly processes them, and performs visual output. Thus, the work of hybrid environment hot symbol parsing for target containers in a single server is completed.

[0154] Currently, there is no available tool or technology that can implement hot symbol parsing in a single server with containers.

[0155] Example 4: Running in a single server for performance anomaly detection and diagnosis

[0156] The hot symbol parsing method provided by the present invention can not only be used to solve the problem of too high CPU usage rate, but also provide a direction for judging the abnormal state of a process. This embodiment is applicable whether there are containers running in the server or not.

[0157] In a single server, multiple processes are running simultaneously. These processes include C / C++ processes, Java processes, and Python processes. The log of a certain process shows that its progress has stalled, but it is observed that its CPU usage rate is very high. To further understand the status of this process and find out where it has stalled, hot symbol parsing is now carried out.

[0158] The present invention can be directly started after the current server deployment is completed. When starting, specify the target process ID, the parsing time, and the sampling frequency, and then run. The process of running the present invention in a single server is as follows Figure 10 , first, initialize the module to run, save the symbol table, and analyze the programming language of the target process. Then, bind perf. After perf is started, it will automatically trigger the operation of the collection module to collect the call stack of the target process during the parsing period. After the parsing is completed, unbind perf. The data processing module will process the collected data and perform visual output. During this period, no operation needs to be performed on the target process, and the present invention will not affect its original execution progress. By observing the hot symbols and the call stacks where the symbols are located in the output file, it can be known where the process has stalled, initially judge whether the process is in an abnormal state, and provide a direction for analyzing the cause of the abnormality.

[0159] Existing tools or technologies can also be used to detect and diagnose performance anomalies in a single server. First, it is necessary to identify the programming language of the stalled process, and then start the corresponding parsing tool to complete the detection and diagnosis of performance anomalies. However, if the programming language of the stalled process cannot be identified, the task of detecting and diagnosing performance anomalies cannot be completed using existing tools or technologies.

[0160] References

[0161] [1]JProfiler.https: / / www.ej-technologies.com / resources / jprofiler / help / doc / JProfiler.pdf

[0162] [2]JFR.https: / / docs.oracle.com / javacomponents / jmc-5-4 / jfr-runtime-guide / about.htm#JFRUH170[3]py-spy.https: / / github.com / benfred / py-spy

[0163] [4]cProfile.https: / / docs.python.org / 3 / library / profile.html

[0164] The protection scope of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the inventive concept, changes and advantages that can be conceived by those skilled in the art are included in the present invention, and the scope of protection is defined by the appended claims.

Claims

1. A hotspot symbol parsing method supporting multiple programming language environments, characterized in that, The method includes the following steps: Step 1, process initialization: Save all process symbol tables within the target range, determine the programming language of the target process, perform initialization operations for different programming languages respectively, bind perf to the target process, and set the sampling frequency; Step 2, data collection: Sample according to the sampling frequency set in Step 1. After sampling is completed, stop perf, read the call stack and stack frame symbols of the target process, save the read data, and then restart perf for the next sampling; Step 3, data processing and analysis: Unbind from perf, count the call stack data obtained in data collection in Step 2, and perform analysis and visualization of hot symbols after data processing.

2. The parsing method according to claim 1, wherein Step 1 further includes the following steps: Step 1.1, save the process symbol table: Traverse the memory snapshots of all processes, find all ELF files related to the process, and then parse and save the symbol table information in the ELF files one by one; when the symbol table is not saved in the ELF file, search for the backup symbol table in the system according to the build information and debug information of the process; Step 1.2, determine the programming language of the target process: Parse the ELF file corresponding to the target process, traverse the symbol table of the target process, and match and compare the symbols in the target process symbol table with the process symbol tables of different programming languages saved in Step 1.1 one by one. Based on whether there are symbols in the target process symbol table that match the process symbol tables of different programming languages, preliminarily determine the programming language type of the target process; Step 1.3, initialization operation: If the programming language of the target process determined in Step 1.2 is Python, parse the ELF file of the target process, find the memory address of the global variable, and obtain the runtime data of the variable; or, If the programming language of the target process determined in Step 1.2 is Java, let the target process run external code, save the DWARF data in the ELF file, and register a signal handler; or, If the programming language of the target process determined in Step 1.2 is C / C++, the operation in Step 1.2 is sufficient to meet the initialization requirements and no other operations are required; Step 1.4, bind perf to the target process and set the sampling frequency.

3. The parsing method according to claim 2, characterized in that In Step 1.1, the process symbol table records the mapping relationship between program entities including function names and global variable names and memory addresses; Both the build information and debug information of the process contain the symbol table of the corresponding program; when the symbol table of the ELF file itself is stripped, the build information and debug information files serve as the source of the backup symbol table; Identify and obtain the process ELF file through the / proc / [pid] / maps file; the process ELF file contains the memory snapshot of the current process; the memory snapshot contains the files read and loaded into memory by the target process and the file usage permissions of the files; the file usage permissions include three parts, readable, writable, and executable; traverse the memory snapshot, and for files with executable permissions, determine them as ELF files; Each program entity in the symbol table, including symbols and global variables, has a corresponding memory address, which represents the memory storage location of the program entity during program execution. The program entity and the address are saved in a one-to-one correspondence. For C / C++ programs, during the call stack reading phase, the memory addresses of each stack frame are saved. During the stack frame symbol reading phase, the stack frame memory addresses are compared with the addresses in the symbol table, and the symbols with the same addresses are taken as the symbols corresponding to the stack frames. and / or, In step 1.2, when the target process belongs to a Java or Python program, double verification is used for further judgment: when it is initially determined that the target process belongs to a Java program, search whether the socket file created according to the rules exists in the file directory address in the Java system file directory. If it exists, it is further determined that the target process belongs to a Java program. or, when it is initially determined that the target process belongs to a Python program, start the executable file of the target process. If a string matching the Python output format can be output, it is further determined that the target process belongs to a Python program; or, when the target process belongs to a C / C++ program, parse its ELF file, build information, and debug information to obtain the symbol table data of the target process.

4. The parsing method according to claim 2, wherein In step 1.3, for the target process, corresponding initialization operations are taken according to the programming language it uses: when it is determined in step 1.2 that the current process is a Python program, read the global variables PyInterpreterState, PyThreadState, PyFrameObject, and PyCodeObject respectively. The global variables save the interpreter state data, Python thread state data, Python stack frame state data, and Python stack frame code object data of the current Python process, which are used to read the call stack and stack frame symbols to analyze the execution state and behavior of the Python program; or, when it is determined in step 1.2 that the current process is a Java program, read the DWARF data in the ELF file; and / or, In step 1.4, the perf parameters target process PID, sampling frequency, perf_event wake-up times, and perf_event response method are passed into the system call function perf_event_open to bind the perf_event to the target process and implement the statistics of cycles of the target process.

5. The parsing method according to claim 1, wherein Step two further includes the following steps: Step 2.1: Sample the data according to the sampling frequency set in step one, and stop the statistics of cycles by perf after sampling is completed. Step 2.2: Read the call stack: if it is determined that the target process is a C / C++ program, use the ptrace system call to suspend the process. After that, use the method in the libunwind.so library to read the memory addresses of all stack frames in the call stack; or, If it is determined that the target process is a Python program, the Python program stack and the Python interpreter stack composed of C / C++ are read respectively: using global variables, the stack frame objects of the process are read one by one to complete the reading of the Python program stack; the target process is paused through ptrace, and the Python interpreter stack is read through libunwind.so; after the Python program stack and the Python interpreter stack are concatenated, a complete Python process call stack is formed; or, If it is determined that the target process is a Java program, the Java process call stack is implemented through a signal handler; the information of the Java program stack is read using the signal handler, read through functions in the libjvm.so library, and the function returns all stack frame IDs of the call stack of the current process; then the Java virtual machine stack composed of C / C++ is read through DWARF data and virtual machine global variables, and finally the Java program stack and the Java virtual machine stack are concatenated into a complete Java process call stack; Step 2.3, Read stack frame symbols: If it is determined that the target process is a C / C++ program, the stack frame memory addresses read in Step 2.2 are combined with the symbol table saved in Step 1 to read all symbols corresponding to the stack frames and the source files where the symbols are located; or, If it is determined that the target process is a Python program, the stack frame objects in Step 2.2 are parsed. The stack frame objects contain the symbols corresponding to the code blocks of the current stack frame and the source files where they are located, obtained through the process_vm_readv system call, to complete the symbol reading of the Python stack frames; read the Python interpreter stack frame symbols in the Python program; or, If it is determined that the target process is a Java program, the stack frame IDs obtained in Step 2.2 are parsed to obtain the stack frame symbols and the source files where the symbols are located in the call stack, to complete the symbol reading of the Java program stack frames; Step 2.4, Save the read data, where the data includes all symbols in the call stack, the source files where the symbols are located, as well as metadata such as the process ID and the collection time.

6. The parsing method according to claim 5, wherein In Step 2.1, perf samples at the set sampling frequency. After completing data collection, it will automatically go to sleep until the subsequent call stack reading, stack frame symbol reading, and data saving work are completed, then perf is awakened to perform the next sampling and re - count cycles; and / or, In Step 2.2, the call stack of the C / C++ program is only composed of C / C++ symbols, the call stack of the Python program is composed of the Python program stack and the Python interpreter stack, and the call stack of the Java program is composed of the Java program stack and the Java virtual machine stack; and / or, In step 2.3, the call stacks of the C / C++ program, the Python program, and the Java program all contain C / C++ symbols, so the C / C++ symbol resolution method is used in all cases; for reading the stack frame symbols of Python and Java programs, the stack frame IDs obtained in step 2.2 are retrieved and parsed through the process_vm_readv system call to obtain the stack frame symbols in the call stack and the source files where the symbols are located to obtain Python and Java symbols; and / or, In step 2.4, the saved metadata serves as the data source for subsequent hotspot statistics.

7. The parsing method according to claim 1, characterized in that Step three further includes the following steps: Step 3.1: Unbind the perf according to the programming language type of the target process. Step 3.2: Statistically analyze all the call stack data in step two, summarize the top symbols of the process call stacks for each process, count the number of occurrences of the symbols, and statistically determine the hotspot symbols. Step 3.3: Statistically analyze the call stacks of the hotspot symbols, find all the call stacks of the same hotspot symbol, compare the stack frame symbols one by one, merge the call stacks with exactly the same stack frame symbols, accumulate their sampling times, and calculate the call times of the hotspot symbols in different call stacks. Step 3.4: Generate and output visual hotspot symbol sorting graphs, hotspot symbol call stacks, and flame graphs.

8. The parsing method according to claim 7, wherein, In step 3.1, different methods are adopted to unbind the perf according to the different programming language types used by the target process; for Java programs, run external code to stop the perf operation and unbind it; for Python and C / C++ programs, directly close the perf through system call functions to unbind it; and / or, In step 3.2, count the number of occurrences of the top symbols in the call stack, sort the hotspot symbols according to the number of occurrences, and the more times a symbol appears, the more cycles it consumes; and / or, In step 3.3, for the top symbols obtained in step 3.2, count the number of times they appear in different call stacks; and / or, In step 3.4, sort and display them according to the number of cycles consumed by the symbols from most to least, and at the same time display the call stacks involved in each symbol and the proportion of cycle consumption in different processes; generate a flame graph based on the collected data to visually display the analysis results of the hotspot symbols.

9. An analysis system for implementing the analysis method according to any one of claims 1-8, characterized in that, The parsing system includes: an initialization module, a collection module, and a data processing module; The initialization module is used to initialize the process, including saving the symbol table of the target process, determining the programming language of the target process, binding the perf, and setting the sampling frequency; The collection module samples at the set frequency and automatically adopts different processing methods according to different programming languages to collect and save data; The data processing module processes and analyzes the collected symbol data and visually displays the analysis results.

10. The parsing method according to any one of claims 1-8, or the parsing system according to claim 9, is applied in hotspot symbol parsing in a multi-programming language environment, hotspot symbol parsing in a server cluster, hotspot symbol parsing of a server including containers, performance anomaly detection and diagnosis.

Images