A method for optimizing java code layout without interrupting application services

Abstract

This invention discloses a Java code layout optimization method that does not interrupt application services. The method includes: acquiring front-end bottleneck data of the application; when it is determined that code layout optimization is needed based on the front-end bottleneck data, collecting the dynamic control flow graph of the application service; calculating a new code layout based on the dynamic control flow graph; generating new code according to the new code layout; correcting relocation information in all code, that is, redirecting function calls and data references pointing to the old code to the new code. During the correction process, memory barriers and cache maintenance instructions are used to ensure the atomicity and consistency of the modification operations, so as to achieve uninterrupted application services; only reclaiming the memory space used by the old code that is no longer executed to store the new code generated in the next optimization, retaining the function code still on the call stack, so that most of the memory space can be reclaimed without interrupting application services. This method effectively alleviates the processor front-end bottleneck of Java applications and improves application performance.

Description

Technical Field

[0001] This invention belongs to the field of code layout optimization technology, specifically relating to a Java code layout optimization method that does not interrupt application services. Background Technology

[0002] Due to its advantages in development efficiency, memory safety, application ecosystem, and portability, Java is widely used to write data center applications. Data center Java applications typically contain deep and complex software stacks, such as web frameworks, networking, storage, logging, monitoring, middleware, and so on. This results in application codebases that are extremely large, sometimes reaching hundreds of megabytes, far exceeding the typical size of the L1 instruction cache and L1 instruction TLB. This massive codebase leads to frequent cache misses and TLB misses, causing significant pipeline front-end stalls. Worse still, even the most advanced out-of-order execution mechanisms cannot hide these pipeline front-end stalls, making them a severe performance bottleneck (front-end bottleneck) for data center Java applications.

[0003] Feedback-guided code layout optimization (FDCLO) is an effective method for optimizing front-end bottlenecks. FDCLO collects the control flow characteristics of the application's execution and groups code blocks along hot control flow paths together. FDCLO increases the probability that branch source and destination addresses are in the same cache block or memory page, thereby reducing instruction cache misses and instruction TLB misses, thus improving performance. Based on the granularity of code reordering, FDCLO can be divided into basic block reordering and function reordering. Basic block reordering adjusts the layout of basic blocks within a function at the basic block level. Function reordering adjusts the layout of functions in a binary at the function level. While FDCLO can effectively optimize front-end bottlenecks, existing FDCLO solutions face problems such as lack of support for Java applications, lack of support for continuous optimization, and interruption of application execution during the optimization process, making them unsuitable for large-scale data center Java applications. Details are as follows.

[0004] BOLT (Binary Optimization and Layout Tool) is a binary optimization tool developed by Meta and is now integrated into the LLVM repository. BOLT is an offline FDCLO solution. It acquires the application's control flow characteristics in an offline environment and generates an optimized binary program, which is then deployed online by operations personnel. BOLT utilizes hardware sampling capabilities to obtain the application's control flow graph, such as SPE on ARM platforms or LBR on x86 platforms. Because low-level code layout optimization is required, control flow graphs obtained using hardware sampling have better accuracy and matching. OCOLOS builds upon BOLT to propose a continuously optimizing online code reordering framework. OCOLOS's core principle is to dynamically monitor the application's front-end bottlenecks. When the front-end bottleneck increases, it calls BOLT to generate an optimized binary program and switches to the new version of the code via hot code updates. Both BOLT and OCOLOS offer good optimization effects for applications, but they only support applications written in native languages ​​like C and C++, and not Java applications. For Java applications, source code is first compiled into high-level virtual machine instructions called bytecode, rather than directly into executable machine code. Then, the JVM compiles the bytecode into executable machine code after the application starts. Therefore, the code layout of a Java application is dynamically determined by the JVM at runtime, unlike applications written in native languages ​​which are determined at link time.

[0005] Dynamic Code Management (DCM) is a code layout optimization framework for Java applications. DCM is the closest technical solution to this invention. DCM mainly includes the following key steps: 1) Obtaining a dynamic control flow graph using instrumentation; 2) Pausing all Java application threads; 3) Calculating the optimized code layout; 4) Allocating new memory space to store the optimized code; 5) Moving the function code to the newly allocated memory space and repairing relocation information and the corresponding metadata; 6) Updating the call stack of each thread, modifying the return address on the call stack to the new address of the function, and updating the register containing the code address; 7) Restarting all Java application threads. The shortcomings of DCM are: 1) The instrumentation method for obtaining the control flow graph interferes too much with the application, causing a significant drop in application performance during the acquisition process; 2) It requires interrupting all application threads to perform optimization, which can cause timeouts when processing requests; 3) It only supports function-level code reordering, losing most optimization opportunities. Based on these three points, DCM cannot be applied to large-scale data center Java applications. Summary of the Invention

[0006] In view of the above, the purpose of this invention is to provide a Java code layout optimization method and apparatus that does not interrupt application services, effectively alleviating the processor front-end bottleneck of Java applications, improving application performance, and solving the problem of application request processing timeouts caused by DCM through a mechanism that does not interrupt application service execution. It also achieves code reordering at the basic block and function levels, with optimization effects superior to DCM.

[0007] To achieve the above-mentioned objectives, an embodiment provides a Java code layout optimization method that does not interrupt application services, comprising the following steps:

[0008] Obtain front-end bottleneck data of the application service. When it is determined that code layout optimization needs to be performed based on the front-end bottleneck data, collect the dynamic control flow graph of the application service. The dynamic control flow graph includes the dynamic control flow graph of the basic block inside each function and the dynamic control flow graph of calls between global functions.

[0009] The new code layout is calculated based on the dynamic control flow graph, and the new code layout includes the new code layout of the basic blocks inside each function and the new code layout of the global function.

[0010] Generate new code according to the new code layout;

[0011] Correct the relocation information in all code, redirecting calls and data references that pointed to the old code to the new code. During the correction process, memory barriers and cache maintenance instructions are used to ensure the atomicity and consistency of the modification operations, so as to achieve uninterrupted application services.

[0012] The memory space used by old code that is no longer being executed is reclaimed to store the code generated in the next optimization, while the function code that is still on the call stack is retained, so that a portion of the memory space can be reclaimed without interrupting the application service.

[0013] In one embodiment, the processor's hardware performance monitoring unit collects front-end bottleneck data of the application service to be optimized within a fixed time period. When the difference between two adjacent collections of front-end bottleneck data is greater than a set threshold, it is considered that code layout optimization operation needs to be performed.

[0014] In one embodiment, the dynamic control flow graph of the data acquisition application service includes:

[0015] The sample branch instruction call data is used, and each branch instruction contains a source address and a destination address. The function with the source address and destination address is found. If the source address and destination address belong to the same function, the dynamic control flow graph of the basic block inside that function is updated. If the source address and destination address do not belong to the same function, the dynamic control flow graph of calls between global functions is updated.

[0016] In one embodiment, in the dynamic control flow graph of the basic blocks within each function, nodes represent basic blocks, edges represent the jump relationships between basic blocks, and the weight of the edge represents the number of jumps.

[0017] In one embodiment, in the dynamic control flow graph of global function calls, nodes represent functions, edges represent the call relationships between functions, and the weight of an edge represents the number of times the corresponding function call occurs.

[0018] In one embodiment, calculating the new code layout based on the dynamic control flow graph includes:

[0019] The basic block rearrangement algorithm is used to rearrange the basic blocks inside each function based on the dynamic control flow graph of the input function, and outputs a new code layout for the basic blocks inside each function. The basic block rearrangement algorithm includes the ExtTSP algorithm.

[0020] In one embodiment, calculating the new code layout based on the dynamic control flow graph includes:

[0021] A function reordering algorithm is used to reorder the dynamic control flow graph of calls between input global functions and output a new code layout for the global functions. The function reordering algorithm includes the Call-Chain Clustering algorithm.

[0022] In one embodiment, generating new code according to the new code layout includes:

[0023] First, allocate a temporary cache space to store the new code for each function; traverse the global function code layout list, processing one function at a time; generate new code for each function based on the new code layout of the basic blocks inside each function and save it to the temporary cache space; allocate memory space for the function in the code memory space, and copy the new code in the temporary cache space to the newly allocated memory space; save a mapping table of the old and new code addresses of the same function, which is used for subsequent address relocation.

[0024] In one embodiment, the specific process for correcting relocation information in all code is as follows:

[0025] Based on the mapping table of addresses between the old and new code, the destination addresses of all function calls in the new code are modified to the new addresses; based on the mapping table of addresses between the old and new code, the destination addresses of all function calls in the old code are modified to the new addresses.

[0026] In one embodiment, the reclaiming of memory space used only by old code that is no longer executed includes:

[0027] Sample the call stack of each thread in the application service and parse the old function code on the call stack of each thread; find the union of the old function code on the call stack of each thread, and the union is the old code that needs to be retained; traverse the entire old code space and delete the old code other than the old code that needs to be retained.

[0028] Compared with the prior art, the beneficial effects of the present invention include at least the following:

[0029] Obtaining the dynamic control flow graph of a Java application service through sampling has the advantage of minimal interference with application performance. Furthermore, the optimization process does not require interrupting application execution, thus minimizing the impact on application performance. These two points allow the method of this invention to be applied to Java applications in data centers without causing timeouts when processing user requests. In addition, the method of this invention considers both basic block code reordering and function code reordering, resulting in better optimization effects. Attached Figure Description

[0030] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0031] Figure 1 This is a flowchart of the Java code layout optimization method for non-interrupting application services provided in the embodiment;

[0032] Figure 2 This is a code generation diagram provided in the embodiment;

[0033] Figure 3 This is a diagram showing the function call history before the destination address of the function call for updating code, provided in the embodiment.

[0034] Figure 4 This is a diagram showing the function call sequence after the destination address of the function call for updating code, provided in the embodiment.

[0035] Figure 5 This is a schematic diagram of old code recycling provided in the embodiment. Detailed Implementation

[0036] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the scope of protection of this invention.

[0037] Figure 1This is a flowchart of a Java code layout optimization method that does not interrupt application services, as provided in this embodiment. Figure 1 As shown, the Java code layout optimization method provided in this embodiment includes the following steps:

[0038] Step 1: Obtain front-end bottleneck data for the application service.

[0039] In this embodiment, the processor's hardware performance monitoring unit collects front-end bottleneck data P of the application service to be optimized within a fixed time period at fixed intervals. i Specifically, the percentage of pipeline stalls caused by processor front-end bottlenecks is periodically obtained through the Linux Perf interface. Based on this percentage of pipeline stalls, front-end bottleneck data P is obtained. i .

[0040] Step 2: When it is determined that code layout optimization needs to be performed based on the front-end bottleneck data, collect the dynamic control flow graph of the application service.

[0041] In this embodiment, it is assumed that the previously collected front-end bottleneck data is P. i-1 Determine the bottleneck data P between two consecutive data collections. i -P i-1 If the difference is greater than the set threshold T, it is considered that code layout optimization operation needs to be performed, that is, steps 2-6 are executed; otherwise, monitoring continues, that is, step 1 is executed.

[0042] When code layout optimization is required, the dynamic control flow graph of the application service is collected. This dynamic control flow graph serves as input for calculating the new code layout algorithm. There are two types of dynamic control flow graphs: one is the dynamic control flow graph of the basic blocks within each function, and the other is the dynamic control flow graph of calls between global functions.

[0043] Branch instructions are sampled using hardware-assisted sampling, specifically through the SPE function on the ARM platform or the LBR function on the Intel platform. A branch instruction contains a source address and a destination address. Based on this branch instruction, the dynamic control flow graph is collected and updated, including:

[0044] The process involves searching for the functions containing the source and destination addresses of branch instructions. If the source and destination addresses belong to the same function, the dynamic control flow graph of the basic blocks within that function is updated. In this dynamic control flow graph, nodes represent basic blocks, edges represent jump relationships between basic blocks, and the weight of the edge represents the number of jumps. If the source and destination addresses do not belong to the same function, it indicates a function call branch instruction. In this case, the dynamic control flow graph of global function calls is updated. In this dynamic control flow graph, nodes represent functions, edges represent call relationships between functions, and the weight of the edge represents the number of times the corresponding function call occurs.

[0045] Step 3: Calculate the new code layout based on the dynamic control flow graph.

[0046] In this embodiment, the new code layout includes the new code layout for each basic block within a function and the new code layout for global functions. The process of calculating the new code layout includes:

[0047] Taking the dynamic control flow graph of the basic blocks inside each function as input, the new code layout of the basic blocks inside each function is output through the basic block reordering algorithm. The basic block reordering algorithm can be the ExtTSP algorithm.

[0048] Taking the dynamic control flow graph of global function calls as input, a new code layout for the global functions is output through a function reordering algorithm. This function reordering algorithm can be Call-Chain Clustering (C... 3 )algorithm.

[0049] Step 4: Generate new code according to the new code layout.

[0050] In this embodiment, the code for each function is generated in the new memory space according to the function order, and the code for each function is generated sequentially based on the layout of the basic blocks inside each function. For example... Figure 2 As shown, the specific process includes:

[0051] First, allocate a temporary cache space to store the new code for each function; traverse the global function code layout list, processing one function at a time; call the compiler to generate new code for each function based on the new code layout of the basic blocks inside each function and save it to the temporary cache space; allocate memory space for the function in the code memory space, and copy the new code in the temporary cache space to the newly allocated memory space; save a mapping table of the old and new addresses of the same function, which is used for subsequent address relocation.

[0052] Step 5: Correct the relocation information in all code.

[0053] In this embodiment, the calls and data references pointing to the old code are redirected to the new code. This means that the destination addresses of all function calls in both the old and new code are updated to the addresses of the new code, thereby enabling the application to switch to executing the new code. Figure 3 This is to update the function call information of the program before the operation is executed. Figure 4 This involves updating the function call history of the program after the update operation. During the correction process, memory barriers and cache maintenance instructions are used to ensure the atomicity and consistency of the modification operations, so as to avoid interrupting application services. The specific process includes:

[0054] Based on the mapping table of old and new code addresses, the destination address of function calls in the new code is modified to the new address; based on the mapping table of old and new code addresses, the destination address of function calls in the old code is modified to the new address.

[0055] Step 6: Reclaim the memory space used by old code that is no longer being executed.

[0056] In this embodiment, memory space occupied by old code that is no longer executed is reclaimed. However, old code still on the application thread call stack cannot be deleted because the program may re-execute it via a return instruction. Specifically, as shown below... Figure 5 As shown, the steps are as follows:

[0057] Sample the call stack of each thread in the application service and parse the old function code on the call stack of each thread; find the union of the old function code on the call stack of each thread, and the union is the old code that needs to be retained; traverse the entire old code space and delete the old code other than the old code that needs to be retained.

[0058] The specific embodiments described above illustrate the technical solution and beneficial effects of the present invention in detail. It should be understood that the above description is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, additions, and equivalent substitutions made within the scope of the principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A Java code layout optimization method that does not interrupt application services, characterized in that, Includes the following steps: Obtain front-end bottleneck data of the application service. When it is determined that code layout optimization needs to be performed based on the front-end bottleneck data, collect the dynamic control flow graph of the application service. The dynamic control flow graph includes the dynamic control flow graph of the basic block inside each function and the dynamic control flow graph of calls between global functions. The new code layout is calculated based on the dynamic control flow graph, and the new code layout includes the new code layout of the basic blocks inside each function and the new code layout of the global function. Generate new code according to the new code layout; Correct the relocation information in all code, redirecting calls and data references that pointed to the old code to the new code. During the correction process, memory barriers and cache maintenance instructions are used to ensure the atomicity and consistency of the modification operations, so as to achieve uninterrupted application services. The memory space used by old code that is no longer being executed is reclaimed to store the code generated in the next optimization, while the function code that is still on the call stack is retained, so that a portion of the memory space can be reclaimed without interrupting the application service.

2. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The processor's hardware performance monitoring unit collects front-end bottleneck data of the application service to be optimized within a fixed time period. When the difference between two consecutive collections of front-end bottleneck data exceeds a set threshold, it is considered that code layout optimization operation needs to be performed.

3. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The dynamic control flow graph of the data acquisition application service includes: The sample branch instruction call data is used, and each branch instruction contains a source address and a destination address. The function with the source address and destination address is found. If the source address and destination address belong to the same function, the dynamic control flow graph of the basic block inside that function is updated. If the source address and destination address do not belong to the same function, the dynamic control flow graph of calls between global functions is updated.

4. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, In the dynamic control flow graph of each function's basic blocks, nodes represent basic blocks, edges represent the jump relationships between basic blocks, and the weight of the edge represents the number of jumps.

5. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, In the dynamic control flow graph of global function calls, nodes represent functions, edges represent the call relationships between functions, and the weight of an edge represents the number of times the corresponding function call occurs.

6. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The calculation of the new code layout based on the dynamic control flow graph includes: The basic block rearrangement algorithm is used to rearrange the dynamic control flow graph of the basic blocks inside each function and output a new code layout for the basic blocks inside each function. The basic block rearrangement algorithm includes the ExtTSP algorithm.

7. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The calculation of the new code layout based on the dynamic control flow graph includes: A function reordering algorithm is used to reorder the dynamic control flow graph of calls between input global functions and output a new code layout for the global functions. The function reordering algorithm includes the Call-Chain Clustering algorithm.

8. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The process of generating new code according to the new code layout includes: First, allocate a temporary cache space to store the new code for each function; traverse the global function code layout list, processing one function at a time; generate new code for each function based on the new code layout of the basic blocks inside each function and save it to the temporary cache space; allocate memory space for the function in the code memory space, and copy the new code in the temporary cache space to the newly allocated memory space; save a mapping table of the old and new code addresses of the same function, which is used for subsequent address relocation.

9. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The specific process for correcting the relocation information in all code is as follows: Based on the mapping table of addresses between the old and new code, the destination addresses of all function calls in the new code are modified to the new addresses; based on the mapping table of addresses between the old and new code, the destination addresses of all function calls in the old code are modified to the new addresses.

10. The Java code layout optimization method for non-interrupted application services according to claim 1, characterized in that, The reclaiming of memory space used by old code that is no longer being executed includes: Sample the call stack of each thread in the application service and parse the old function code on the call stack of each thread; find the union of the old function code on the call stack of each thread, and the union is the old code that needs to be retained; traverse the entire old code space and delete the old code other than the old code that needs to be retained.

Images