Online binary continuous optimization system and method supporting multi-instruction-set architecture and application
Patent Information
- Application Number
- CN202511557024.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-10-29
- Publication Date
- 2026-02-06
Smart Images
Figure FT_1 
Figure FT_2 
Figure FT_3
Abstract
Description
Technical Field
[0001] This invention belongs to the field of compiler optimization technology, and specifically relates to an online binary continuous optimization system, method and application that supports multi-instruction set architecture. Background Technology
[0002] Software application performance optimization aims to improve its operating efficiency on target hardware platforms. Based on the execution stage in which the optimization occurs, existing optimization techniques can be divided into static optimization and dynamic optimization.
[0003] Static optimization refers to the offline process before program execution, where a pre-compiler (Ahead-of-Time, AOT) and linker analyze and transform the program's source code, intermediate representation (IR), or binary file to generate a more efficient sequence of machine instructions. An advanced static optimization technique is Profile-Guided Optimization (PGO), which utilizes program runtime profile information. [1] The implementation of this technique typically involves: first, instrumenting and executing the target program to collect runtime behavior data under typical loads, such as function call frequency and branch jump probability, forming a profiling data file; then, during the second compilation or linking process, using this profiling data file as input to guide the optimizer in making decisions more consistent with the actual runtime scenario, such as code layout optimization and function inlining. However, PGO and all static optimization methods have some inherent technical limitations: the profiling data upon which the optimization decisions rely is based on historical data. [2] Once an optimization decision is made, it is embedded in the final generated binary file. If the program's runtime behavior after actual deployment (e.g., due to changes in input data distribution, operating environment, or business logic) deviates from this historical profile data, the previous optimizations based on outdated information may become ineffective, thus failing to achieve the desired performance improvement.
[0004] Dynamic optimization refers to the optimization of program execution by a runtime system during the online phase of program runtime. Dynamic optimization mainly generates or modifies machine code dynamically by monitoring program behavior in real time. Its mainstream implementation methods include: (1) Just-in-Time (JIT) compilation systems managed by the runtime. [3]For example, the Java Virtual Machine (JVM) or the JavaScript V8 engine, which compiles platform-independent bytecode into optimized native machine code at runtime; (2) Dynamic Binary Optimization / Translation (DBT / DBO) system [4] It intercepts raw binary instructions, elevates them to the internal IR layer for analysis and optimization, and then regenerates efficient machine code for execution. A significant advantage of dynamic optimization techniques lies in their adaptability, enabling them to capture the program's most realistic operational characteristics and optimize accordingly. However, this adaptability comes at the cost of significant runtime overhead. These overheads primarily include: the persistent overhead of monitoring program behavior, the computational resource overhead required to execute just-in-time (JIT) compilation and optimization algorithms, and the memory overhead of storing the optimized code.
[0005] To reduce the aforementioned overhead, the industry has proposed a lightweight online binary optimizer. [5] Such solutions are often "one-off optimizations," typically involving hotspot identification and modification only during program startup or a fixed phase. This design assumes that the core behavior of the program stabilizes after the initial phase, but this assumption is increasingly inapplicable in modern software applications. Once the initial optimization phase ends, it also faces the technical dilemma of rigid optimization decisions and an inability to adapt to subsequent behavioral changes.
[0006] Modern software systems, especially long-running services, exhibit significant dynamism and continuity in their operation. [6] Such applications need to run continuously for days or even months. During this time, their workload continuously evolves due to changes in external inputs, internal state transitions, and functional iterations, leading to constant changes in their computational hotspots, branching behaviors, and data access patterns. In this context, any fixed optimization strategy based on a single time slice will inevitably become suboptimal or even ineffective over time. Therefore, to ensure that such long-cycle applications can maintain high performance, a real-time, continuous adaptive optimization capability has become a critical technical requirement. This requires an optimization system to have the ability to monitor behavioral changes in real time and automatically throughout the entire program's lifecycle, and to promptly apply new optimization strategies that better match the current behavior.
[0007] There is a clear technological gap in this field: the lack of a systematic solution that effectively combines the three core characteristics of general-purpose online binary optimization capabilities, continuous adaptive optimization, and acceptable low runtime overhead. While offline PGO is technically mature, its static nature prevents it from meeting the requirements of online continuity. JIT systems built into language-specific runtimes, although achieving continuous optimization, are deeply coupled with the specific language ecosystem, and their high overhead makes them unsuitable as a solution for general-purpose native binary programs. Existing lightweight online optimizers, due to their "one-off" design philosophy, do not provide true continuous iterative optimization capabilities. Therefore, a new online binary continuous optimization solution is urgently needed: one that, with acceptable low runtime overhead, introduces the advantages of PGO's "analysis-guided optimization" into online scenarios, enabling continuous and repeated updates and optimizations based on the latest analysis information throughout the program's lifecycle, while minimizing coupling with specific instruction sets to improve cross-platform capabilities. Summary of the Invention
[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide an adaptive, low-overhead, cross-platform online binary optimization system, method, and application that supports continuous optimization.
[0009] The technical solution of the present invention includes the following:
[0010] An online binary continuous optimization system supporting a multi-instruction set architecture includes: a program behavior information acquisition module, an offline binary optimization module, an online code replacement module, an old version code recycling module, and a program stage change detection and decision-making module;
[0011] The program behavior information acquisition module achieves general runtime information acquisition through an adaptive dual-mode design;
[0012] The offline binary optimization module converts the collected dynamic information into an intermediate representation and performs an optimization pass on the intermediate representation;
[0013] The online code replacement module controls the start and stop of the process and replaces the optimized segments;
[0014] The old version code recycling module detects and recycles old version code through a runtime memory management mechanism;
[0015] The program stage change detection and decision-making module monitors changes in program performance indicators, identifies stage-specific behavior switching, and achieves continuous adaptive optimization closed loop.
[0016] The present invention also provides an optimization method implemented using the above system, comprising:
[0017] Step 1: Collect program behavior information of the target program: When the processor supports hardware branch recording, program control flow information is collected through the hardware performance monitoring unit; or, when hardware support is lacking, the control flow graph is reconstructed and program behavior information is generated through system interrupt sampling and path inference algorithms.
[0018] Step 2: Perform offline binary optimization: Input the collected program behavior information and the target program binary file into the general binary optimization engine, disassemble them into an intermediate representation, and perform a weight-based optimization pass to generate the optimized target executable file;
[0019] Step 3: Perform online code replacement: Pause the execution of the target program, compare the differences between the original code and the optimized code, extract the optimized segments and insert them into the memory reserved area, and correct and redirect the addresses pointing to the old code in the call stack of each thread;
[0020] Step 4: Perform old version code reclamation: Scan the context of each thread while the system is in a global pause state to identify and release memory areas of unreferenced old version code;
[0021] Step 5: Based on the analysis results of the program stage change detection and decision-making module, the loop execution of steps one to four is automatically triggered to achieve continuous optimization throughout the program's entire lifecycle.
[0022] The present invention also provides applications of the above-mentioned system or method in automatically performing program behavior acquisition, binary optimization and online code replacement at different operating stages to achieve continuous performance adaptive optimization.
[0023] The optimization system or method of this invention can solve the following problems:
[0024] Achieving low-overhead and platform-independent program behavior acquisition. The effectiveness of dynamic optimization depends on accurate observation of program runtime behavior, especially the statistics of control flow transitions. A mainstream existing acquisition method relies on hardware performance events provided by specific processors, such as Intel's LBR (Last Branch Record), AMD's LbrExtV2 (Last Branch Record Extension Version 2), or ARM's BRBE (Branch Record Buffer Extension). However, these hardware features are related to whether and how the underlying hardware provides these specific functions. Another common approach is to use software instrumentation techniques with higher runtime overhead, but the performance loss introduced is unacceptable in many production environments. Therefore, the first technical problem this invention aims to solve is: how to design a lightweight, portable dynamic information acquisition mechanism with minimal impact on key performance indicators of the target load. This mechanism should be able to efficiently collect key program behavior information of the application load with low overhead, primarily program control execution flow information, but not limited to program control flow information, including microarchitectural data during program runtime, without forcibly relying on specific hardware branch record features, thus providing reliable data support for cross-platform adaptive optimization.
[0025] This invention achieves intelligent aggregation of multi-version optimization information. The optimization system in this invention accumulates multiple sets of profiling data for different behavioral stages of the program. If optimization decisions rely solely on the most recently collected profiling data representing a single run, the randomness or incompleteness of this data may lead to biased decisions. Therefore, this invention designs a multi-version optimization information collection and aggregation algorithm capable of collecting multiple runtime information. When the decision engine identifies that the current program has entered a known historical behavioral stage, this algorithm can retrieve and aggregate all historical profiling data previously collected when the program was in the same stage. Sampling information that interferes with or negatively impacts the final program code optimization is defined as noise. By aggregating program behavior information from multiple runs at the same stage, the noise weight is reduced, and then statistical noise and / or transient anomalies from a single run are filtered out, thereby revealing the most fundamental performance bottleneck at that behavioral stage. Ultimately, this will provide the offline optimization module with more reliable and comprehensive profiling data, leading to more accurate and effective optimization decisions.
[0026] The analytical data is mainly represented in the following data format:
[0027] <source_location> -><destination_location> <count><mispredicted_count>
[0028] Each line can represent a control flow transfer and the frequency of execution.
[0029] Build a portable and scalable general-purpose binary optimization engine. The core of offline binary optimization lies in the deep analysis and transformation of the input binary file to improve its performance characteristics. A mainstream approach is to develop optimization tools for a specific instruction set architecture (ISA) (e.g., x86-64). These tools are often monolithic in design, deeply coupling their binary parsing, program analysis, and optimization transformation logic with the specific ISA. While this design can achieve good optimization results on the target platform, its portability and versatility are severely limited. When it is necessary to extend optimization capabilities to a new ISA (e.g., AArch64 or RISC-V), almost the entire toolchain needs to be rewritten, resulting in huge and repetitive engineering development costs. Therefore, this invention requires the design of an offline binary optimization engine with a modular, layered architecture. This engine can effectively decouple platform-dependent binary processing from platform-independent core optimization algorithms. By raising the inputs of different ISAs to a unified intermediate representation (IR) similar to, but not limited to, LLVM MCINST IR, and performing general optimization passes on this IR, including but not limited to function rearrangement, basic block rearrangement, equivalent code folding, and process link table optimization, the engine is finally downgraded to an executable file that matches the original target platform. This achieves efficient reuse and consistency of optimization capabilities across multiple platforms.
[0030] Achieving secure and efficient hot-swapping of online binary code. After generating optimized binary code snippets, the core challenge lies in seamlessly and securely applying them to a running, active, multi-threaded target program. This requires the system described in this invention to have a reliable global state control mechanism for application load, capable of synchronously pausing and resuming all threads at any time using the ptrace tool based on actual runtime conditions; it needs efficient code management and layout strategies to support multiple, incremental code updates without causing memory corruption; most importantly, it must be able to handle complex historical execution contexts of application loads, preventing thread crashes due to dangling pointers after execution resumption through precise call stack repair; specifically, when analyzing the call graph of the currently running code, for the optimized target function (or module), it automatically detects and locates all direct or indirect call paths, and updates the call nodes to point to the optimized target function (or module) to maintain the consistency and correctness of the call relationships. Therefore, this invention aims to build a complete, secure, and efficient online code replacement framework that can safely replace the original code with optimized binary code fragments at runtime and ensure that the execution state of all threads is correctly restored after the replacement is completed, thereby realizing the transition from the original code to the optimized code.
[0031] Managing multiple versions of code and execution states during continuous optimization. Achieving continuous optimization of application load is the core innovation of this invention, as the hotspots of program runtime behavior are constantly evolving, and a single optimization cannot adapt to these dynamic changes. However, a direct consequence of continuous optimization is that multiple versions of code (including the original code and previous optimized versions) will inevitably coexist in the address space of the target program. This coexistence of multiple versions poses a severe challenge to the correct execution of the program and resource management. Existing technologies typically lack a systematic solution to handle this complex situation. On the one hand, the safety and correctness of control flow transfers between different code versions must be ensured; that is, the transition from any old version to a new version must be reliable to maintain the global consistency of the program state. On the other hand, as optimization continues, a large number of replaced and obsolete code versions will accumulate in memory. If not managed, this will lead to memory leaks, ultimately affecting the long-term stability of the system. Therefore, this invention requires the design of a robust code version management and state consistency model. The model must be able to support multiple optimizations and replacements of the same code region. While ensuring the correct transition of program state when multiple versions coexist, it must also be able to safely identify and reclaim obsolete code versions that are no longer referenced by any active execution path. This will enable continuous adaptive optimization while ensuring the long-term stable operation of the system. The identification and reclamation functions can be triggered manually and / or automatically, and can be executed periodically or irregularly according to the relevant process and preset start time points.
[0032] This intelligent decision-making mechanism identifies and initiates an optimization process by setting preset thresholds and duration criteria and comparing them. When these thresholds and durations are exceeded, it's recognized as a staged change in the program. Existing optimization processes have limited automation and intelligence. Traditional application load optimization processes require manual triggering by relevant personnel based on external monitoring alerts or experience. This approach is not only untimely but also heavily reliant on human experience, failing to accurately capture changes in the program's internal behavior patterns. Even some so-called dynamic optimizations passively respond based on simple, fixed hotspot thresholds, making them unable to effectively identify continuous behavior pattern switching caused by external load or internal state transitions. They may even make unnecessary or negative optimization adjustments to temporary performance fluctuations.
[0033] The beneficial effects of this invention include: By directly analyzing and optimizing binary files at runtime, this invention solves the fundamental problem of distorted analysis information and poor optimization results caused by inconsistencies between the analysis environment and the production environment in previous PGO technologies. The program behavior information acquisition module of this invention achieves low-overhead, accurate acquisition using processor-provided hardware performance events when hardware support is available. Even without hardware support, it can still generate high-precision side information through sampling and path inference algorithms based on system events. This ensures that subsequent optimizations are based on the most realistic execution hotspots of the production load, regardless of the platform, thereby achieving more accurate and effective code optimization, significantly improving instruction cache locality, and further enhancing program performance.
[0034] This invention proposes an online code replacement scheme aimed at addressing the challenge of downtime for updates or redeployment commonly faced by large applications after optimization. Its core technological breakthrough lies in achieving precise repair of the call stacks of all threads: after online code replacement, this invention can traverse and correct every return address and function pointer pointing to the old code region. This refined execution context synchronization mechanism is key to achieving application optimization without restarting. It fundamentally guarantees that even in highly concurrent and complex scenarios, the program state maintains strict consistency at the moment of version switching and afterwards, thereby significantly improving the security and feasibility of online binary optimization.
[0035] This invention demonstrates particularly significant advantages when handling long-running services with complex and dynamically changing behavior patterns. Taking a large MySQL database service deployed on a server cluster as an example, during peak business periods, it may be in a high-concurrency mixed read / write phase; when generating reports, it switches to a read-only intensive analysis phase; and when performing data migration or batch import, it enters a write-only intensive phase. For such applications, traditional static optimization can only generate optimization code for a specific phase, failing to consider all behavior patterns, resulting in poor performance improvements in certain phases. This invention, through a macro-level behavior detection and decision engine, can autonomously identify such macro-level phase transitions. Once a change is detected, the system automatically triggers a new round of optimization. In one specific implementation, the system of this invention can reduce the average latency of batch commits in the read-only intensive analysis phase by approximately 31%, and the average transaction processing latency in the high-concurrency mixed read / write phase by approximately 37%. Attached Figure Description
[0036] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0037] Figure 1 This is a schematic diagram illustrating the implementation principle of the program behavior information collection module of this invention.
[0038] Figure 2 This is a schematic diagram illustrating the implementation principle of the offline binary optimization module of this invention.
[0039] Figure 3 This is a schematic diagram illustrating the implementation principle of the online code replacement module of this invention.
[0040] Figure 4 This is a schematic diagram illustrating the implementation principle of the old version of the code recycling module of this invention.
[0041] Figure 5 This is a schematic diagram illustrating the implementation principle of the program stage change detection and decision-making module of this invention.
[0042] Figure 6 This is an implementation scheme architecture diagram of embodiments 1 and 2 of the present invention.
[0043] Figure 7 This is an implementation scheme architecture diagram of embodiments 3 and 4 of the present invention.
[0044] Figure 8 This is an implementation scheme architecture diagram of embodiments 5 and 6 of the present invention.
[0045] Figure 9 This is an implementation scheme architecture diagram of embodiments 7 and 8 of the present invention. Detailed Implementation
[0046] The present invention will be further described in detail below with reference to the specific embodiments and accompanying drawings. Except for the contents specifically mentioned below, the processes, conditions, and experimental methods for implementing the present invention are all common knowledge and general knowledge in the art, and the present invention does not have any particular limitations.
[0047] This invention provides a low-overhead, cross-platform online binary continuous optimization system. The system mainly consists of five core modules that work together to achieve a continuous optimization closed loop from program behavior information collection and offline binary optimization to online code replacement. The program behavior information collection module acquires dynamic program behavior in an adaptive manner; the offline binary optimization module uses this information to perform deep rewriting and optimization of the binary code; the online code replacement module hot-deploys the optimization results to the running process without the program's awareness, and the old version code recycling module ensures long-term memory robustness; the program stage change detection and decision-making module analyzes the target program and autonomously triggers the optimization process based on the analysis results.
[0048] In one specific implementation, for example on MySQL, the program behavior information collection module described in this invention will affect the performance of MySQL during operation, by about 10%-20%. However, since the collection time is short and only takes about 5 seconds, the performance may be improved by 30%+ after optimization. When performing corresponding hot code replacement, the replacement process can be completed in milliseconds (about 300ms), which is basically imperceptible.
[0049] Specifically,
[0050] a. Program behavior information collection module
[0051] The program behavior information acquisition module of this invention adopts an adaptive dual-mode design, ensuring a unified format for program behavior information output across different platforms, thereby achieving efficient and platform-independent runtime information acquisition. On platforms supporting processor hardware-level control flow tracing mechanisms, this module directly configures and utilizes the processor's hardware performance monitoring unit to capture accurate program behavior information (program control execution flow information) with low overhead. By performing simple processing on each piece of program behavior information, high-precision edge weight information is formed, which serves as the program behavior information. On platforms lacking such hardware support, the module switches to a general sampling mode based on system events. It periodically performs interrupt sampling at any point in program execution, forming a series of discrete execution location samples. Subsequently, the module's built-in path inference algorithms, including hotspot-based algorithms and iterative solution-based algorithms, analyze these discrete sample point sequences to reconstruct the program's control flow graph, thereby inferring key edge weight information as program behavior information, ultimately providing data support for optimization in a platform-independent manner.
[0052] This module also includes a multi-version information intelligent aggregation function to address the potential for randomness and incompleteness in information collected in a single instance. When the program stage change detection and decision-making module identifies that the current program is in a known historical behavior stage, this module's multi-version information intelligent aggregation function will retrieve and aggregate all historical program behavior information collected when the program was in the same stage, outputting a more reliable and comprehensive set of program behavior information.
[0053] The workflow of this module is as follows: Figure 1 As shown, after the target load starts, there is a warm-up period, after which the program behavior information acquisition module of this invention will operate. First, the hardware detector identifies whether the current processor has hardware registers that support recording program behavior information through the operating system-level interface. When it detects the presence of such registers, this invention will use execution path 1, directly acquiring program behavior information using the hardware performance monitoring unit. When no such registers exist, this invention will use execution path 2, interrupting the sampling of the current program and forming a series of execution hotspot samples. Then, an algorithm is used to reconstruct the program control flow graph, generating program behavior information inferred from the algorithm. The program stage change detection and decision module identifies the current program behavior stage and retrieves all historical program behavior information matching the current stage, aggregating them with the information collected this time. Finally, this invention stores the multi-version fused program behavior information for subsequent input to the offline binary optimization module.
[0054] b. Offline binary optimization module
[0055] The offline binary optimization module is a general-purpose optimization engine supporting multiple instruction set architectures, responsible for transforming collected dynamic information into actual performance improvements. The core architecture of this module includes a multi-target frontend and a unified optimization backend. For inputs from different instruction set architectures (including but not limited to x86-64, AArch64, etc.), the corresponding frontend is responsible for parsing the binary files, disassembling them, and converting them into a unified, platform-independent intermediate representation (IR). All optimization work is performed on top of the IR. The optimization backend uses the side information provided by the program behavior information acquisition module as weights to perform weighted analysis on the IR representing the program control flow, and then executes a series of Passes (optimization passes, i.e., a process of traversing and transforming the program IR to achieve a specific optimization goal) to perform code transformations. The executed Passes include, but are not limited to, heat-based code layout aimed at improving instruction cache locality, process inlining aimed at reducing function call overhead, and other instruction-level optimizations. After this series of Pass optimizations, the overall code structure is adjusted so that frequently executed code paths are contiguous in runtime memory addresses in the final generated target code. After optimization, the backend will regenerate the optimized IR into an executable file for the target instruction set architecture, ready for deployment.
[0056] The workflow of this module is as follows: Figure 2 As shown, the offline binary optimization module of the present invention takes the target payload executable binary file and program behavior information as input. Then, the present invention will disassemble the target payload into a platform-independent intermediate representation by a disassembler according to the instruction set of the current processor. The code optimizer will optimize this by using the edge weight information in the program behavior information. Finally, the present invention will regenerate the executable file of the target instruction set architecture by a code generator.
[0057] c. Online code replacement module
[0058] The online code replacement module is the core of the online optimization process. This module has a process controller that sends signals to the target process to coordinate entering and exiting a global pause state, ensuring the availability of subsequent modifications. During the program pause, the module can extract optimized segments by comparing the old and new code and insert them into reserved memory areas. Furthermore, it actively traverses the call stacks of all threads, precisely locating and correcting all return addresses and function pointers pointing to the old code areas, redirecting them to the corresponding entry points of the newly optimized code.
[0059] The workflow of this module is as follows: Figure 3 As shown, after the target load starts, the program behavior information module of this invention collects the program behavior information. This information, along with the executable binary file of the target load, is input to the offline binary optimization module to generate a new binary file with better performance characteristics. This invention accurately extracts the optimized code segments by comparing the differences between the new code and the original code, and places these code segments in a pre-planned memory area dedicated to storing the optimized code. Then, this invention finds and modifies all return addresses and function pointers pointing to the old version functions, redirecting them to the corresponding entry points of the new optimized code. Finally, after briefly pausing the target program, this invention inserts the optimized code segments and modified call relationships into the memory area, and then resumes the target program, which will then run the code optimized by this invention.
[0060] d. Old version code recycling module
[0061] The core objective of the legacy code recycling module of this invention is to ensure the long-term stability and memory efficiency of the system after multiple continuous optimizations. The continuous optimization supported by this invention directly results in multiple versions of code existing simultaneously in the address space of the target program. For example, after an optimization process is completed, the original code, although no longer the main execution path, may still be referenced by some execution contexts; when subsequent program behavior changes and the system generates and deploys new optimized versions of code, the previous optimized code versions may become "obsolete" historical code. Without effective lifecycle management of these continuously generated obsolete versions of code, memory leaks will occur. The legacy code recycling module of this invention provides a safe and reliable runtime memory management mechanism.
[0062] The workflow of this module is as follows: Figure 4 As shown, it executes its garbage collection logic while the target program is in a globally paused state. Its core lies in constructing a snapshot of all code regions directly or indirectly referenced by active execution paths by performing a comprehensive scan of the runtime execution context of all threads (including but not limited to the call stack and register contents). By comparing this "active reference snapshot" with a list recording all historical code versions (including their address ranges and "deprecated" states), the system can accurately identify deprecated code versions that are no longer referenced by any active execution context. For such identified, safely reclaimable versions, the system performs the corresponding memory reclamation operation (e.g., calling operating system interfaces to release their memory) and resumes the execution of the target program after all operations are complete. This mechanism ensures that cleanup is only performed after confirming that no pointers or return addresses point to an older version, thus achieving efficient memory reuse while guaranteeing the highest system stability.
[0063] e. Program Phase Change Detection and Decision Module
[0064] This module analyzes the macroscopic operational status of the target program and, based on the analysis results, autonomously triggers an optimization process from program behavior information collection, offline binary optimization, to online code replacement. This module includes a built-in data acquisition module, independent of the aforementioned program behavior information collection module. This module works by monitoring one or more preset key performance indicators that reflect the overall behavior pattern of the program. These key performance indicators may include: microarchitecture-level indicators provided by processor performance counters (including but not limited to cache miss rate, instructions per cycle, etc.); operating system kernel statistics (including but not limited to CPU utilization, memory and cache status, I / O latency, etc.); and application-defined characteristic indicators (including but not limited to read / write operation ratio, concurrency, etc.). This module analyzes the time-series data of these key performance indicators to identify phased changes during program execution. When a phased change under a preset condition is detected, this module automatically assigns an optimization task, which includes the collection of behavioral characteristic information for the current phase and instructions to initiate the optimization process.
[0065] The workflow of this module is as follows: Figure 5 As shown, the program stage change detection and decision-making module of the present invention collects microarchitecture data, operating system kernel statistics data, and application performance data during the operation of the target program, integrates them into a complete set of status information data, and persists it. This data is then used to extract key performance indicator data through data feature extraction and to perform data analysis through a behavior pattern analysis engine to identify stage changes during program operation. When a stage change under a preset condition is detected, this module will execute path 1 to trigger an optimization decision; otherwise, it will execute path 2 to continue monitoring the operation of the target program.
[0066] To further illustrate these beneficial effects, consider a representative application example: After deploying this invention, the system automatically applies an optimized version of code for each stage to improve its core performance at that stage. During the write-intensive stage, the system deploys a version of code optimized for batch data loading, reducing the average latency of batch commits by approximately 31%. When the system detects a switch to the read-intensive analysis stage, it automatically deploys a new version of code optimized for complex analytical queries, reducing the response latency of analytical queries by approximately 33%. Subsequently, when the system detects a switch to the high-concurrency mixed read-write stage, it deploys another version of code optimized for high-concurrency transaction processing, reducing the average transaction processing latency by approximately 37%.
[0067] This continuous, closed-loop optimization process is completed entirely through an online code replacement module, without requiring a database service restart. This ensures that the database always runs the version of code best suited to its current workload characteristics throughout the entire runtime. This not only significantly improves the overall performance and resource utilization of the application but also transforms performance tuning from a static, offline activity into a dynamic process that is synchronized with application behavior in real time, significantly enhancing the automation and adaptability of complex systems in performance management.
[0068] In the following embodiments, the platform and workload used are defined as follows:
[0069] Type A processors: refer to processors that support advanced hardware performance events such as recording program behavior information, including but not limited to Xeon processors, Core processors, and Pentium processors based on the Intel 64 instruction set architecture.
[0070] Type B processor: refers to a processor that does not have the aforementioned advanced hardware features, including but not limited to processors based on various ARM, RISC-V or MIPS instruction set architectures.
[0071] Computing platform: refers to a complete computing environment built on a type A or type B processor. It should be understood that the computing platform can be a server, desktop computer, mobile device, edge computing node, or any other form of computing device; this invention does not specifically limit it.
[0072] Target workload: refers to a binary application to be optimized that runs on the computing platform.
[0073] Homogeneous computing cluster: refers to a cluster of computing platforms that are identical in processor model, hardware configuration and target workload.
[0074] Example 1:
[0075] Fully autonomous online continuous optimization on a single computing platform equipped with hardware registers for recording program behavior information. This embodiment describes the adaptive application mode of the present invention on a single computing platform employing a Type A processor. This computing platform has hardware registers for recording program behavior information. Embodiment 1 aims to enable a program macro-behavior detection and decision-making module to autonomously identify changes in program behavior stages and complete the online continuous optimization process on a single computing platform. The optimization process includes the collection of program behavior information, binary optimization based on the post-link optimizer, and online deployment of the optimized workload, and can periodically and automatically reclaim invalid memory space.
[0076] The execution flow of this embodiment is as follows: Figure 6 As shown. First, after the system of this invention starts and is attached to the process of the target workload, the program macro-behavior detection and decision-making module monitors the various performance indicators of the process throughout. Once a change in program behavior is detected, the optimization process is automatically triggered. At this time, the program behavior information acquisition module, after confirming platform support through its internal hardware detector, starts a low-overhead acquisition mode based on hardware performance events. Then, the acquired program behavior information data, along with the executable binary file of the target workload, is passed to the offline binary optimization module to generate a new binary file with better performance characteristics for the optimized application workload. By comparing the differences between the new code and the original code, the optimized code fragments are accurately extracted and placed in a pre-planned memory area dedicated to storing the optimized code. Finally, after a brief pause of the target program, the optimized code fragments and the modified call relationships are inserted into the memory area, and the target program is restored. Then, the optimized code of this invention will run. At the same time, the independent old version code recycling module periodically scans the current call stack and register status to generate a call stack function information file. By comparing the details of each optimized version, invalid memory space is identified and released.
[0077] Example 2:
[0078] This embodiment describes the adaptive application mode of the present invention on a single computing platform using a B-type processor. This computing platform lacks hardware registers for recording program behavior information. Embodiment 2 aims to enable a fully autonomous online continuous optimization process on a single computing platform, where a macroscopic program behavior detection and decision-making module can autonomously identify changes in program behavior stages and complete the online continuous optimization process. The optimization process includes collecting program behavior information, performing binary optimization based on the post-link optimizer, and deploying the optimized workload online, while automatically reclaiming invalid memory space.
[0079] The execution flow of this embodiment is as follows: Figure 6 As shown, the overall process is similar to that of Example 1. The key difference lies in the working method of the program behavior information acquisition module. This module's hardware detector detects that the platform does not support advanced hardware features, therefore automatically switching to a software sampling mode based on periodic system interruptions, and inferring edge information through its built-in algorithm. The decision-making module can still autonomously trigger program behavior information acquisition, offline binary optimization, and online code replacement based on various performance indicators. A separate old version code recycling module also periodically ensures system memory health.
[0080] Example 3:
[0081] This embodiment describes a manually triggered online continuous optimization process on a single computing platform equipped with hardware registers for recording program behavior information. The platform features hardware registers for recording program behavior information. Embodiment 3 aims to complete the online continuous optimization process on a single computing platform by manually identifying changes in program behavior stages. The optimization process includes collecting program behavior information, performing binary optimization based on the post-link optimizer, and deploying the optimized workload online, while automatically reclaiming invalid memory space.
[0082] The execution flow of this embodiment is as follows: Figure 7 As shown. In this mode, the system's macro-level program behavior detection and decision-making module is configured to be inactive. Operations personnel or developers can initiate an optimization process multiple times at any time through the control interface provided by this system. This command will activate the program behavior information collection module, the offline binary optimization module, and the online code replacement module, performing an on-demand performance optimization. Simultaneously, the independent old version code recycling module continues to run periodically in the background.
[0083] Example 4:
[0084] This embodiment describes a manually triggered online continuous optimization process on a single computing platform that lacks hardware registers for recording program behavior information. The application mode of this invention is manually controlled on a single computing platform using a Type B processor. This platform does not have hardware registers for recording program behavior information. Embodiment 4 aims to complete the online continuous optimization process on a single computing platform by manually identifying changes in program behavior stages. The optimization process includes collecting program behavior information, performing binary optimization based on the post-link optimizer, and deploying the optimized workload online, while automatically reclaiming invalid memory space.
[0085] The execution flow of this embodiment is as follows: Figure 7 As shown, the process is similar to that of Example 3, with the decision-making module also inactive and optimization triggered manually. The key difference is that the program behavior information collection module on this platform uses a software sampling mode based on periodic system interruptions to collect data. Upon receiving a manual instruction, the program behavior information collection module, offline binary optimization module, and online code replacement module are activated to complete the optimization deployment. An independent old version code recycling module also runs periodically in the background.
[0086] Example 5:
[0087] This invention enables fully autonomous online continuous optimization on a homogeneous computing cluster equipped with hardware registers for recording program behavior information. This embodiment describes the distributed application mode of the invention in a homogeneous computing cluster composed of multiple computing platforms using Type-A processors. This computing platform has hardware registers for recording program behavior information. Embodiment 5 aims to enable a program macro-behavior detection and decision-making module to autonomously identify changes in program behavior stages and complete the online continuous optimization process on a single computing platform, while simultaneously deploying the optimized workloads online on multiple computing platforms. This achieves performance improvements for running application workloads on multiple computing platforms while only one computing platform bears the additional performance overhead during the optimization process.
[0088] The execution flow of this embodiment is as follows: Figure 8 As shown, the system of this invention is deployed in a fully autonomous mode on all nodes of the cluster. The decision-making module on the master node independently or collaboratively monitors the behavior patterns of local instances. When optimization conditions are met, the program behavior information collection module and the offline binary optimization module on the master node are triggered to generate optimized code. This optimized code can then be distributed to other nodes in the cluster, and the online code replacement module is deployed on multiple nodes. An independent old version code recycling module runs periodically on each node.
[0089] Example 6:
[0090] This embodiment describes a fully autonomous online continuous optimization on a homogeneous computing cluster that lacks hardware registers for recording program behavior information. The distributed application mode of this invention is described in a homogeneous computing cluster composed of multiple computing platforms using Type-B processors. This computing platform does not have hardware registers for recording program behavior information. Embodiment 6 aims to enable a program macro-behavior detection and decision-making module to autonomously identify changes in program behavior stages and complete the online continuous optimization process on a single computing platform, while simultaneously deploying the optimized workload online on multiple computing platforms. This achieves performance improvements for running application workloads on multiple computing platforms while only one computing platform bears the additional performance overhead during the optimization process.
[0091] The execution flow of this embodiment is as follows: Figure 8 As shown, its distributed collaboration logic is similar to that of Example 5, also employing a fully autonomous mode. The key difference lies in the fact that the program behavior information collection modules on the master node in the cluster all operate using a software sampling mode based on periodic system interruptions. The subsequent autonomous decision-making, distributed optimization and deployment, and periodic code recycling processes remain consistent with Example 5.
[0092] Example 7:
[0093] This embodiment describes a centralized, manually controlled optimization strategy in a homogeneous computing cluster consisting of multiple computing platforms employing Type-A processors. The computing platforms have hardware registers for recording program behavior information. Embodiment 7 aims to complete the online continuous optimization process on a single computing platform by manually identifying changes in program behavior stages, and then deploy the optimized workloads online across multiple computing platforms. This achieves performance improvements for running application workloads on multiple computing platforms while only one computing platform bears the additional performance overhead of the optimization process.
[0094] The execution flow of this embodiment is as follows: Figure 9 As shown, operations personnel select a computing platform in the cluster as the master optimization node and manually trigger the information collection and offline optimization process on that node to generate an optimized binary code. This optimized code is then distributed to all other slave deployment nodes in the cluster. Finally, the online code replacement module on all nodes is triggered, achieving synchronous performance upgrades for the same workload across the entire cluster. A separate old version code recycling module runs periodically on each node.
[0095] Example 8:
[0096] This embodiment describes a manually triggered online continuous optimization on a homogeneous computing cluster that lacks hardware registers for recording program behavior information. The centralized, manually controlled optimization strategy of this invention is implemented in a homogeneous computing cluster composed of multiple computing platforms employing Type-B processors. This computing platform lacks hardware registers for recording program behavior information. Embodiment 8 aims to complete the online continuous optimization process on a single computing platform by manually identifying changes in program behavior stages, and then deploy the optimized workload online across multiple computing platforms. This achieves performance improvements for running application workloads on multiple computing platforms while only one computing platform bears the additional performance overhead during the optimization process.
[0097] The execution flow of this embodiment is as follows: Figure 9 As shown, the process is similar to that of Example 7, also triggered manually. The key difference is that the computing platform, which serves as the main optimization node, uses a software sampling mode based on periodic system interruptions to generate program behavior information through its program behavior information acquisition module. The subsequent offline optimization, optimized code distribution, and synchronized code replacement processes on all nodes remain consistent with Example 7. An independent old version code recycling module also runs periodically on each node.
[0098] References
[0099] [1] Dehao Chen, David Xinliang Li, and Tipp Moseley. 2016. AutoFDO:automatic feedback-directed optimization for warehouse-scale applications. InProceedings of the 2016 International Symposium on Code Generation andOptimization (CGO '16). Association for Computing Machinery, New York, NY,USA, 12–23. https: / / doi.org / 10.1145 / 2854038.2854044
[0100] [2] Maksim Panchenko, Rafael Auler, Bill Nell, and Guilherme Ottoni.2019. BOLT: a practical binary optimizer for data centers and beyond. InProceedings of the 2019 IEEE / ACM International Symposium on Code Generationand Optimization (CGO 2019). IEEE Press, 2–14.
[0101] [3] John Aycock. 2003. A brief history of just-in-time. ACM Comput.Surv. 35, 2 (June 2003), 97–113. https: / / doi.org / 10.1145 / 857076.857077
[0102] [4] B. Hawkins, B. Demsky, D. Bruening and Q. Zhao, "Optimizingbinary translation of dynamically generated code," 2015 IEEE / ACMInternational Symposium on Code Generation and Optimization (CGO), SanFrancisco, CA, USA, 2015, pp. 68-78, doi: 10.1109 / CGO.2015.7054188.
[0103] [5] Y. Zhang, TA Khan, G. Pokam, B. Kasikci, H. Litz and J.Devietti, "OCOLOS: Online COde Layout OptimizationS," 2022 55th IEEE / ACMInternational Symposium on Microarchitecture (MICRO), Chicago, IL, USA, 2022, pp. 530-545, doi: 10.1109 / MICRO56248.2022.00045.
[0104] [6] Timothy Sherwood, Erez Perelman, Greg Hamerly, and Brad Calder. 2002. Automatically characterizing large scale program behavior. SIGPLAN Not. 37, 10 (October 2002), 45–57. https: / / doi.org / 10.1145 / 605432.605403
[0105] The scope of protection of this invention is not limited to the above embodiments. Any variations and advantages that can be conceived by those skilled in the art without departing from the spirit and scope of this invention are included in this invention and are protected by the appended claims.< / count>
Claims
1. An online binary continuous optimization system supporting multi-instruction set architecture, characterized in that, include: The module includes a program behavior information collection module, an offline binary optimization module, an online code replacement module, an old version code recycling module, and a program stage change detection and decision-making module. The program behavior information acquisition module achieves general runtime information acquisition through an adaptive dual-mode design; The offline binary optimization module converts the collected dynamic information into an intermediate representation and performs an optimization pass on the intermediate representation; The online code replacement module controls the start and stop of the process and replaces the optimized segments; The old version code recycling module detects and recycles old version code through a runtime memory management mechanism; The program stage change detection and decision-making module monitors changes in program performance indicators, identifies stage-specific behavior switching, and achieves continuous adaptive optimization closed loop.
2. The system as described in claim 1, characterized in that, The program behavior information acquisition module further includes a hardware detector; wherein... The hardware detector automatically identifies whether the current processor supports hardware registers for recording branch events during the program startup phase, and switches the sampling mode accordingly. And / or, The program behavior information collection module retrieves and aggregates all historical program behavior information collected when the program was previously in the same stage.
3. The system as described in claim 2, characterized in that, When the platform has hardware registers that support recording program behavior information, the hardware performance monitoring unit can be used to directly obtain the program behavior information. And / or, When no hardware registers are available to record program behavior information, the current program is sampled and a series of execution hotspot samples are formed. The program control flow graph is then reconstructed, and program behavior information inferred from the algorithm is generated.
4. The system as described in claim 1, characterized in that, The program behavior information collection module aggregates information from multiple versions, retrieves historical program behavior data that matches the current stage, integrates program behavior information from multiple runs, and filters statistical noise and / or transient anomalies from a single run.
5. The system as described in claim 1, characterized in that, The offline binary optimization module includes a multi-target front-end and a unified optimization back-end; wherein, the front-end is responsible for parsing, disassembling, and converting binary files input from different instruction set architectures into a general intermediate representation; the optimization back-end performs optimization passes on the intermediate representation to optimize the code; And / or, The optimization backend uses the edge information provided by the program behavior information collection module as weights to perform weighted analysis on the intermediate representation representing the program control flow, and performs a series of optimization passes on the intermediate representation to transform the code; And / or, The optimization includes popularity-based code layout and process inlining.
6. The system as described in claim 1, characterized in that, The online code replacement module coordinates and controls the program process through a process controller. And / or, Locate and modify all return addresses and function pointers that point to the old version of functions, and redirect them to the corresponding entry points of the new optimized code to ensure program consistency.
7. The system as described in claim 1, characterized in that, The old version code recycling module generates a snapshot of all active code regions in the current execution context, records the address range and status identifier of historical code versions, and releases the memory resources of obsolete code after confirming that there are no active references.
8. The system as described in claim 1, characterized in that, The program phase change detection and decision-making module further includes a data acquisition module, which collects processor performance counter data, operating system statistics data and application-defined indicator data to construct and store a state information dataset, and identifies phased pattern changes through a behavior pattern analysis engine.
9. A method for online continuous binary optimization supporting multi-instruction set architectures, characterized in that, The method, applied to the system as described in any one of claims 1-8, includes the following steps: Step 1: Collect program behavior information of the target program: When the processor supports hardware branch recording, program control flow information is collected through the hardware performance monitoring unit; or, when hardware support is lacking, the control flow graph is reconstructed and program behavior information is generated through system interrupt sampling and path inference algorithms. Step 2: Perform offline binary optimization: Input the collected program behavior information and the target program binary file into the general binary optimization engine, disassemble them into an intermediate representation, and perform a weight-based optimization pass to generate the optimized target executable file; Step 3: Perform online code replacement: Pause the execution of the target program, compare the differences between the original code and the optimized code, extract the optimized segments and insert them into the memory reserved area, and correct and redirect the addresses pointing to the old code in the call stack of each thread; Step 4: Perform old version code reclamation: Scan the context of each thread while the system is in a global pause state to identify and release memory areas of unreferenced old version code; Step 5: Based on the analysis results of the program stage change detection and decision-making module, the loop execution of steps one to four is automatically triggered to achieve continuous optimization throughout the program's entire lifecycle.
10. The system as described in any one of claims 1-8, or the method as described in claim 9, automatically performs program behavior acquisition, binary optimization, and online code replacement at different operating stages to achieve continuous performance adaptive optimization.