A Noise-Introduced Benchmarking Framework and Adaptive Optimization Platform and Its Working Method

Abstract

This invention relates to a noise-introducing benchmark framework and adaptive optimization platform, and their working methods, comprising: a benchmark framework: controlling noise through methods such as loading and unloading kernel modules, changing system parameters, CPU frequency locking, cache interference suppression, process scheduling restrictions, and resource quota restrictions; and an adaptive optimization platform: acquiring underlying data collection architecture, cache, memory, core count, die, and V instructions through the platform's native interface, binding them with specific hardware features and architectures to obtain optimization templates to form an optimization baseline for a specific hardware architecture, constructing a compiler optimization space, capturing the compiler's set of compilation options, and forming optimization strategies from each compilation option; during testing, adding flags and location information to each option, and the optimization framework using test feedback to adjust and select the best combination in the optimization space by calling different optimization strategies.

Description

Technical Field

[0001] The present invention relates to the field of embedded benchmark testing and adaptive optimization, and particularly relates to a benchmark testing framework and an adaptive optimization platform introducing noise and their working methods. Background Art

[0002] The technological evolution context of compilation optimization testing frameworks can be traced back to the gradual formation of the standardized performance benchmark testing system in the 1980s. In 1988, SPEC (Standard Performance Evaluation Corporation) was formally established, and in 1989, it launched the industry's first cross-platform central processing unit performance evaluation suite SPEC89, providing a standardized benchmark for quantitatively evaluating the effects of compiler optimizations. In the 1990s, with the complication of compiler optimization options such as GCC and Intel ICC, the academic community began to build automated testing systems. For example, the SUIF Explorer system developed by the University of Wisconsin in 1995 explored the effects of compilation optimizations by combining basic optimization options such as loop unrolling and vectorization, but its test cases still relied on manual writing. During the same period, the idea of adaptive optimization germinated in the runtime field: The Self language launched by Sun Microsystems in 1990 first implemented an immediate compilation dynamic optimization mechanism based on the method call frequency, and the Daisy project launched by IBM in 1997 built a cross-platform adaptive execution framework through VLIW binary translation technology, laying a technical foundation for hardware abstraction layer optimizations.

[0003] In the early 21st century, the two technical routes accelerated their integrated development. Intel launched the Pin dynamic instrumentation framework in 2002, which supports collecting microarchitecture performance data such as branch prediction and cache hit rate during program runtime, driving the development of feedback-directed optimization (FDO) technology. Subsequently, the LLVM project introduced the profile-guided optimization (PGO) mechanism in 2006, feeding runtime performance data back to the static compilation stage, and initially constructing a closed-loop system of "compilation - testing - optimization". However, the limitations of traditional benchmark testing gradually emerged: It is difficult to simulate interference factors in real application scenarios in an idealized testing environment. Research on data center loads by Google in 2010 showed that noise factors such as CPU cache contention and memory bandwidth fluctuations can cause program execution performance to fluctuate by more than 30%, thus giving rise to the technical requirement for noise-aware testing. In 2017, the Chronos framework developed by the Swiss Federal Institute of Technology in Zurich first introduced a controllable noise injection mechanism (such as random delay injection) in compilation testing, but it has not yet achieved linkage control with optimization decisions.

[0004] The compiler option system experienced explosive growth in the early 21st century due to the increasing complexity of hardware architectures. Intel's ICC compiler, launched in 1999, added dedicated architecture optimization options for the Pentium III processor's SSE instruction set, ushering in a new era of CPU microarchitecture adaptation optimization. The LLVM project, launched in 2003, further developed a modular option management system, decomposing optimization logic into independent Pass units (such as the -loop-reduce loop reduction module), supporting dynamic combination of optimization strategies. A key technological breakthrough occurred in 2005: GCC version 4.0 introduced link-time optimization (-flto) technology, overcoming the limitations of single-file optimization; Intel simultaneously proposed inter-procedure optimization (-ipo) technology, enabling cross-module function inlining and constant propagation. By this time, mainstream compilers offered over a thousand optimization options, but option conflicts became increasingly prominent—AMD research data showed that the combination of the -funroll-loops and -ftree-vectorize options on Opteron processors could lead to a 23% performance degradation.

[0005] Current technological development faces two core challenges: insufficient noise robustness and the lack of end-to-end collaborative mechanisms. Most existing frameworks, such as TVM / Ansor, rely on idealized testing environments, leading to performance degradation of optimization strategies in real-world applications. Meanwhile, post-linking optimization tools like BOLT lack collaborative mechanisms with the compiler frontend and runtime system. Cutting-edge research is shifting towards building a unified optimization platform: training reinforcement learning optimization strategies through noise-simulated injection tests (such as simulating resource contention scenarios in cloud environments), achieving collaborative control between full-process optimization at link time (FLTO) and BOLT post-linking instruction rearrangement, and promoting cross-layer adaptive optimization from compilation to execution. This technological direction is particularly critical for open architectures like RISC-V—their fragmented hardware ecosystem requires optimization platforms to have cross-platform awareness capabilities, while the complex noise environment of edge computing scenarios urgently needs robust protection mechanisms.

[0006] In the field of intelligent compiler optimization, international research, represented by TVM / Ansor, achieves CPU / GPU collaborative optimization of tensor programs through automated scheduling, supporting operator fusion and data layout transformation. However, these frameworks rely on idealized testing environments and lack robust design against real-world noise interference. Microsoft's CompilerGym models compiler option selection as a reinforcement learning problem, but its reward function does not fully consider environmental interference factors, leading to insufficient stability of optimization strategies in complex scenarios. Regarding noise-aware testing, while Google Data Center Benchmark simulates CPU cache contention and memory fluctuations, it is not integrated with the compiler optimization process. The hardware fuzzing tool ChatFuzz uses large language models to generate instruction sequences and combines reinforcement learning to detect vulnerabilities in RISC-V processors (achieving 75% condition coverage, 34.6 times faster than traditional tools), but its research focus is on hardware security rather than software optimization.

[0007] Several breakthroughs have been achieved in domestic research: Huawei's MindSpore integrates dynamic graph computing fusion technology, achieving adaptive optimization for the Ascend chip, but cross-platform support is not yet available; the CodeV-R1 framework developed by the Chinese Academy of Sciences proposes an optimized architecture combining "knowledge distillation and reinforcement learning," verifying Verilog functional equivalence through an automated testing platform, achieving a 72.9% pass rate in the RISC-V hardware description generation task; however, its high computational cost limits its large-scale application. Significant progress has also been made in noise robustness enhancement technology. Huazhong University of Science and Technology proposed a Soft-GNN model that introduces an adaptive sample weight mechanism, improving the performance of graph neural networks by dynamically filtering noisy data (while simultaneously achieving performance gains in noise-free scenarios); Hanko Electronics' publicly disclosed patented technology achieves scene adaptive optimization of audio terminal devices by constructing an environmental noise distribution field model and dynamic suppression strategies, providing a technical reference for industrial-grade noise injection testing.

[0008] Current technological development exhibits a trend of multi-field integration: photonic chip research at Shanghai Jiao Tong University, reinforcement learning applications by Huawei and the Chinese Academy of Sciences, and noise modeling technology from Huazhong University of Science and Technology are gradually penetrating the field of compiler optimization. However, the end-to-end collaborative mechanism between static compiler optimization and runtime adaptive optimization still needs improvement. Existing research has two major gaps: First, mainstream optimization frameworks (such as Ansor) generally fail to adequately model noise factors in real-world operating environments, leading to performance degradation after optimization strategies are deployed. Second, existing tools (such as AutoKernel) are limited by specific hardware architectures, exhibiting significantly insufficient cross-platform adaptability to open architectures such as RISC-V. Performance evaluation tools are software tools used to measure and evaluate the performance of computer systems. They generate performance metrics by running a series of predefined test programs or benchmarks, helping users understand and compare the performance of different systems. Performance evaluation tools play a crucial role in hardware development, system optimization, and procurement decisions. Performance evaluation tools typically contain a standardized set of test programs designed to simulate various real-world application workloads, specifically for evaluating the performance of processors, memory, and compilers. They include multiple computationally intensive test programs covering different areas such as scientific computing, image processing, and compiler optimization. By running these tests, the tool can generate a set of performance scores that can be used to compare the performance of different systems. A key role of performance evaluation tools is to assist hardware manufacturers and system integrators in design verification and performance optimization. During hardware development, manufacturers can use performance evaluation tools to test the performance of new processors or systems, identify and correct performance bottlenecks, and improve system efficiency. Simultaneously, these tools can also be used for system integration and configuration optimization, ensuring that the system achieves optimal performance under different workloads.

[0009] However, for developers, compilers and testing frameworks, as the core infrastructure for building high-performance software systems, are facing a dual challenge in their technological evolution. Traditional compiler optimization mechanisms often tune parameters based on idealized benchmark testing scenarios, neglecting environmental noise commonly found in real-world production environments—including interference factors such as resource contention, background process scheduling, and network jitter. This often leads to performance degradation or even functional abnormalities in optimized programs during actual deployment. At the same time, contemporary computing hardware environments exhibit diverse and heterogeneous characteristics: the coexistence of instruction set architectures such as x86, ARM, and RISC-V, the differentiated design of core configurations, cache architectures, and memory subsystems, coupled with the complex differentiation of application scenarios—from edge computing devices to data center servers, covering diverse workloads such as compute-intensive and memory-sensitive workloads—all of these necessitate precise architecture adaptation and dynamic strategy adjustment in compiler optimization techniques. Summary of the Invention

[0010] To address the shortcomings of existing technologies, this invention provides a benchmark testing framework and adaptive optimization platform that introduces noise.

[0011] This invention proposes and constructs a comprehensive platform integrating a "noise injection benchmark testing framework" and an "adaptive optimization engine." The core innovations of this platform are: First, by building a benchmark testing framework capable of implanting controllable noise, it achieves precise control over dynamic interference factors in the actual operating environment, ensuring that robustness considerations are fully integrated into the optimization strategy from the design stage, and guaranteeing that the program can clearly expose performance bottlenecks and optimization space in complex and ever-changing real-world scenarios. Second, the platform uses hardware information detection components to accurately acquire the architectural characteristics of the target platform (such as CPU microarchitecture, cache configuration, and memory bandwidth parameters), while combining dynamic and static in-depth analysis of the application (covering code structure, hot function distribution, and data access patterns) and real-time runtime data acquisition (such as cache invalidation statistics and branch record information). This drives the intelligent optimization engine (integrating LLVM / GCC compilation parameters, full-program linking optimization FLTO, and Pass optimization units) and the runtime adaptive engine (including BOLT post-linking optimization and JIT just-in-time compilation) to dynamically generate and adjust the optimal compilation and execution strategy. This mechanism can provide highly customized optimization solutions based on different hardware characteristics (such as enabling prefetch optimization and memory alignment strategies for large cache architectures) and application types (such as optimizing data access patterns for memory-intensive applications), significantly improving performance in cross-platform scenarios and expanding the applicability of the software. Particularly noteworthy is the platform's deep integration of advanced technologies such as full-program linking optimization, combined with post-linking stage optimization, constructing a full lifecycle performance optimization system from compile time, linking time to runtime, capable of meeting diverse application needs from high-performance computing systems to resource-constrained embedded devices.

[0012] Terminology Explanation: 1. Noise: Random environmental factors that interfere with the consistency of performance metrics. It can cause significantly different performance results (such as execution time fluctuations, abnormal resource usage, etc.) when the same test case is run multiple times under the same configuration, thus masking the true performance of the system. (Examples include hardware resource contention and software environment interference.) 2. Kernel: The kernel is an operating system core developed based on the open-source model. As an intermediate layer connecting hardware and upper-level software, it is responsible for managing the underlying resources of the computer system and providing basic services.

[0013] 3. Post-link Optimization: Post-link optimization is a further optimization process performed on the binary code after the executable file or library is generated during the linking stage in the software compilation and deployment process. Its core goal is to improve program execution efficiency, reduce binary size, or enhance security by adjusting code structure, data layout, or instruction set characteristics, while maintaining functional consistency.

[0014] 4. Instruction Set Extensions: Instruction set extensions refer to the addition of extra instruction set functionality or features to the instruction set architecture (ISA) of a computer processor. These extensions can provide new instructions or functions to enhance the processor's performance, functionality, or capabilities, while enabling it to perform specific types of tasks more efficiently. Instruction set extensions can be introduced and defined by processor designers, chip manufacturers, or standardization organizations.

[0015] 5. Compiler Optimization: Compiler optimization refers to the process by which a compiler improves the execution efficiency and performance of the generated target code by converting high-level language code (such as C, C++, Java, etc.) into machine code for the target platform through a series of analysis and transformation techniques. The main goal of compiler optimization is to make the generated target code execute faster, consume less memory, and have a higher energy efficiency. It mainly includes syntax analysis and semantic analysis, intermediate code generation, and optimization analysis. Compiler optimization can not only improve the execution efficiency of the program, but also improve the resource utilization efficiency and energy consumption of the program, which is crucial for software performance and response speed, especially in large-scale and real-time systems.

[0016] 6. Performance Evaluation Tools: Performance evaluation tools are software tools used to evaluate the performance of computer systems, applications, or specific tasks. They provide various ways to collect, analyze, and report system performance data under different conditions, helping developers, system administrators, or researchers understand the system's operating status, bottlenecks, and areas for improvement.

[0017] 7. Static Compilation: Static compilation refers to the process of compiling all necessary libraries and dependencies into the final executable file during the program's compilation phase. After static compilation, the generated executable file contains all the program's code and dependent libraries, so it does not require external dynamic link libraries at runtime.

[0018] The technical solution of the present invention is as follows: A benchmarking framework and adaptive optimization platform that introduces noise, comprising: Benchmarking framework: Noise is controlled by loading and unloading kernel modules, changing system parameters, locking CPU frequency, suppressing cache interference, limiting process scheduling, and limiting resource quotas, so as to collect real performance test data of the program under controllable noise. Adaptive optimization platform: It obtains underlying data collection architecture, cache, memory, core count, die, and V instruction through the platform's native interface, binds it with specific hardware features and architecture, and obtains optimization templates to form the optimization baseline for specific hardware architectures. By constructing a compilation optimization space, it captures the compiler's set of compilation options and forms optimization strategies for each compilation option. During testing, it adds flags and location information to each option, and the optimization framework uses test feedback to adjust and select the best combination in the optimization space by calling different optimization strategies.

[0019] The above-mentioned benchmarking framework with introduced noise and the working method of the adaptive optimization platform include: Step 1: The startup process of the benchmark framework is as follows: Command line parameter parsing: Users specify runtime parameters and configuration files via the command line. The benchmark framework supports a variety of parameters and options for configuring the benchmark framework's runtime environment, test load, runtime plugins, and log file storage location. System initialization: When the benchmark framework starts executing, initialization operations are performed, including: loading kernel modules, loading and configuring devices, loading test loads, and loading performance testing tools; Test begins: Once all initialization and configuration are complete, the benchmark framework begins executing tasks; Performance statistics and output: During the test, the benchmark framework collects and records various performance data, including instruction execution count, cache hit rate, power consumption, and cycle count; users can obtain this performance statistics as needed or generate reports after the test. Test termination and result output: When the test reaches the preset termination conditions, the benchmark testing framework ends the test and outputs the final test results, including: performance statistics, log information, and memory dump. Step 2: The startup process of the adaptive optimization platform is as follows: Adaptive optimization platform initialization: Capture hardware information of the target platform and program performance information during testing; Adaptive optimization: By using a search algorithm to search for the combination of compiler options that achieves the minimum execution time in the target platform and target compiler options, the optimal combination of compiler options can be obtained within a specified time, and then integrated into the linker optimization, i.e., the obtained combination of compiler options is executed. Post-link optimization: Optimize binary code layout using performance data and hardware feedback during testing. Treat each function as a cluster, and group binary clusters together based on relevance and popularity. Relevance refers to function call relationships, including being called or calling; high call frequency means greater than average. Optimization Verification: The optimized code, completed using the adaptive optimization platform, will be tested again using the testing framework in step 1 to verify the effectiveness of the optimization; if the performance of the optimized program is higher than that before optimization, that is, if the execution time of the optimized program is shorter than that before optimization, then the optimization is complete; Introduce noise and repeat the test: Under different noise conditions, repeat steps 1 and 2. If the performance of the optimized program is higher than that before optimization, the optimization is complete.

[0020] According to a preferred embodiment of the present invention, the benchmark testing framework is constructed with hardware awareness and automation processes as its core, and adopts a layered architecture to achieve end-to-end optimization. The hardware probe layer obtains platform characteristics through a multi-protocol adaptation mechanism.

[0021] Furthermore, noise control achieves precise regulation through a hierarchical strategy; In the harmful noise cancellation layer, physical isolation and system standardization technologies are used: resource contention is eliminated by processor core binding and multi-threading is disabled, the processor operating frequency is fixed, and cache layout deviations are broken by memory address space randomization; hyper-threading is disabled, CPU cores are bound, and dynamic frequency scaling is disabled. In the beneficial noise injection layer, dynamic simulation is implemented based on the characteristics of the application scenario: processor frequency fluctuation simulation is introduced for computationally intensive scenarios, cache contention pressure is injected for memory-intensive scenarios, and resource contention in the actual operating environment is simulated through resource quota control; hyper-threading is enabled; and high frequency is fixed in performance mode.

[0022] Further preferred, the hardware information of the target platform includes processor core information, memory and storage information, heterogeneous computing component information, and other hardware characteristics.

[0023] Further preferred, the program performance information during testing includes execution efficiency indicators, hardware resource utilization indicators, compiler intermediate representation related indicators, and energy consumption and stability indicators.

[0024] Further preferred, if the user specifies a configuration file, the configuration file is loaded after system initialization: the benchmark framework loads and parses the configuration file to directly obtain system configuration information and test environment configuration.

[0025] Further preferably, if it is necessary to run specific workloads, which refer to applications or test programs, the workloads are loaded after system initialization: the benchmark framework loads the executable files or images of these workloads and sets up the running environment.

[0026] A further preferred option is that the preset termination condition is: the test time is reached, or the workload is completed.

[0027] The beneficial effects of this invention are as follows: 1. Capture hardware information of the target platform, build a hardware information database, and provide optimization templates when optimizing code on the same platform again.

[0028] 2. Adaptive code optimization on the target platform saves developers optimization time and reduces the specialized knowledge required by developers.

[0029] 3. Introduce advanced noise reduction operations by compiling and importing kernel modules to customize the test environment during testing (disabling cache, disabling massive pages, changing scheduling algorithms, changing CPU scheduling modes, and changing the CPU affinity of test tasks) to test and verify the performance and bottlenecks of the test load under different environments. Attached Figure Description

[0030] Figure 1 A schematic diagram of a benchmark testing framework that introduces noise and an adaptive optimization platform; Detailed Implementation

[0031] The present invention will be further defined below with reference to the accompanying drawings and embodiments, but is not limited thereto. Example 1

[0032] A benchmark framework that introduces noise and an adaptive optimization platform, such as Figure 1 As shown, Figure 1 The description covers the process of generating an Injection File (IR) from source code using a compiler search algorithm to obtain the optimal combination of compilation options. This IR is then compiled into an object file module, optimized using LTO compilation options, resulting in an object file and an executable file. During executable execution, program information obtained from perf testing is fed back to post-linking optimizations to further refine the executable. The benchmark framework, which introduces noise, includes CPU frequency locking, cache interference suppression, and resource quota limits. It includes: Benchmarking framework: Noise is controlled by loading and unloading kernel modules, changing system parameters, locking CPU frequency, suppressing cache interference, limiting process scheduling, and limiting resource quotas, so as to collect real performance test data of the program under controllable noise. How to change system parameters: 1. Disable large pages (including transparent large pages and standard large pages): sudo echo never> / sys / kernel / mm / transparent_hugepage / enabled sudo echo never> / sys / kernel / mm / transparent_hugepage / defrag sudo sysctl -w vm.nr_hugepages=0 2. Disable virtual memory: sudo swapoff -a 3. Disable cache: sudo mount -o remount,noatime,nodiratime / (Remount the root directory (example), disable access time logging) 4. NUMA Strategy Bind a process to a specific CPU and memory node (e.g., numactl --cpunodebind=0 --membind=0 . / program). 5. CPU scheduling policies (process priority and scheduling) chrt -f<priority>-p <pid>Set to SCHED_FIFO, priority 1-99 (higher value, higher priority). chrt -r<priority>-p <pid>Set to SCHED_RR chrt -o 0 -p <pid>Restore to the default SCHED_OTHER Change to NOOP scheduler: `echo noop | sudo tee / sys / block / sda / queue / scheduler` 6. Memory management parameters Adjust swap usage preferences (e.g., sudo sysctl -w vm.swappiness=10, the lower the value, the more likely it is to use physical memory). Control overcommitted memory allocation policies (e.g., sudo sysctl -w vm.overcommit_memory=2, 0=default, 1=allow overcommitted allocation, 2=strict check). Limit the maximum number of memory maps per process (e.g., sudo sysctl -w vm.max_map_count=262144, default 65530, multi-threaded programs may need to increase this number). CPU frequency lock: 1. View the list of available frequencies cat / sys / devices / system / cpu / cpu0 / cpufreq / scaling_available_frequencies 2. Select the highest frequency and set it (taking cpu0 as an example). sudo sh -c "echo <maximum frequency value>> / sys / devices / system / cpu / cpu0 / cpufreq / scaling_max_freq" sudo sh -c "echo <maximum frequency value>> / sys / devices / system / cpu / cpu0 / cpufreq / scaling_min_freq" 3. Perform the same operation on all CPUs. for cpu in / sys / devices / system / cpu / cpu* / cpufreq / ; do sudo sh -c "echo <maximum frequency value>>${cpu}scaling_max_freq" sudo sh -c "echo <highest frequency value>>${cpu}scaling_min_freq" done 4. Set all cores to performance mode. for cpu in / sys / devices / system / cpu / cpu* / cpufreq / scaling_governor; do echo performance | sudo tee $cpu done Adaptive optimization platform: This platform acquires low-level data (such as Linux's ' / proc / cpuinfo / ', ' / sys / devices' and Windows' WMI) through native platform interfaces. It collects data on architecture, caches (L1 / L2 / L3), memory, core count, die count, and V instructions. This data is then bound to specific hardware characteristics and architecture to obtain optimization templates, forming a baseline for optimization specific to the hardware architecture (e.g., for floating-point intensive programs on the S5000C, the GCC parameters are -O3, -march=armv8-a+crc+fp+crypto, -mtune=cortex-a72-static, -ffast-math, -funroll-loops, -fomit-frame-pointer, -fprefetch-loop-arrays). The `-finline-functions`, `-ftree-vectorize`, `-faggressive-loop-optimizations`, `-DAARCH64_QUADMATH`, `-flto`, `-flto-partition=one`, `-fipa-struct-reorg`, and `-ftree-vect-analyze-slp-group` construct a compiler optimization space, capturing the compiler's set of compilation options and forming optimization strategies for each option (the parameter option space is [min~max], and the space for other compilation options is [on / off]). During testing, a flag (performance speedup ratio) and location information are added to each option. The optimization framework uses test feedback to adjust and select the best combination in the optimization space by calling different optimization strategies. Example 2

[0033] The working method of the benchmark testing framework and adaptive optimization platform that introduces noise as described in Example 1 includes: Step 1: The startup process of the benchmark framework is as follows: Command line parameter parsing: Users specify runtime parameters and configuration files via the command line. The benchmark framework supports a variety of parameters and options for configuring the benchmark framework's runtime environment, test load, runtime plugins, and log file storage location. System initialization: When the benchmark framework starts executing, initialization operations are performed, including: loading kernel modules, loading and configuring devices, loading test loads, and loading performance testing tools; Test begins: Once all initialization and configuration are complete, the benchmark framework begins executing tasks; Performance statistics and output: During the test, the benchmark framework collects and records various performance data, including instruction execution count, cache hit rate, power consumption, and cycle count; users can obtain this performance statistics as needed or generate reports after the test. Test termination and result output: When the test reaches the preset termination conditions, the benchmark testing framework ends the test and outputs the final test results, including: performance statistics, log information, and memory dump; performance statistics include the number of instructions executed, cache hit rate, power consumption, and cycle count; program running information is dumped to the hard disk as a log.

[0034] Step 2: The startup process of the adaptive optimization platform is as follows: Adaptive optimization platform initialization: Capture hardware information of the target platform and program performance information during testing; Adaptive optimization: By using a search algorithm to search for the combination of compiler options that achieves the minimum execution time (i.e., the best performance) among the target platform and target compiler options, the optimal combination of compiler options can be obtained within a specified time, and then integrated and optimized during linking, i.e., the obtained combination of compiler options is executed. Post-link optimization: Utilize performance data and hardware feedback during testing (such as control flow information: function call relationships, branch jump frequency, execution frequency data: including the number of times functions, basic blocks, and instructions are executed, and call graph information: the call relationships and call counts between functions) to optimize binary code layout. Treat each function as a cluster, and group binary clusters together based on relevance and popularity. Relevance refers to function call relationships, including being called or calling; high call frequency means greater than the average. Post-link optimization utilizes performance monitoring tools to collect CPU runtime event data, providing fundamental support for performance bottleneck analysis. During the data collection phase, hardware event monitoring mechanisms record runtime state information such as high-frequency executed functions, data access patterns, cache miss events, branch records, and instruction cycles. After collection, dedicated analysis tools parse the data, combining real-time monitoring with in-depth report generation to pinpoint performance bottlenecks from dimensions such as instruction cycle efficiency and cache utilization. The analyzed performance characteristic data is integrated to form an optimization feedback file, directly driving intermediate representation optimization and binary code layout adjustments during the compilation phase. By constructing a closed-loop process of "data collection - bottleneck analysis - compilation optimization," iterative improvements in program execution performance are achieved.

[0035] Optimization Verification: The optimized code, completed using the adaptive optimization platform, will be tested again using the testing framework in step 1 to verify the effectiveness of the optimization; if the performance of the optimized program is higher than that before optimization, that is, if the execution time of the optimized program is shorter than that before optimization, then the optimization is complete; Introducing noise and repeated testing: Under varying noise levels, steps 1 and 2 are executed repeatedly. Optimization ends when the optimized program performs better than before. This provides a more comprehensive and accurate measure of the program's performance and bottlenecks.

[0036] The entire adaptive optimization framework is driven by a hardware feature library and a runtime feedback mechanism. It dynamically acquires hardware microarchitecture features and builds real-time profiles through cross-platform probe components. The optimization engine employs a two-layer collaborative optimization architecture: the bottom layer, based on a mapping model between hardware features and application load, automatically generates suitable combinations of compilation options and efficiently searches the optimization space using intelligent search algorithms; the upper layer, combined with runtime performance feedback data, links full-program linking optimization and post-linking code layout adjustments to achieve dynamic adaptation between compilation strategies and the execution environment. This framework integrates the entire optimization chain—compilation option optimization, link-time global optimization, and post-linking instruction reordering—to form a closed-loop optimization system from hardware feature awareness to execution strategy adjustment, achieving comprehensive performance improvements.

[0037] The benchmark framework is built around hardware awareness and automated processes, employing a layered architecture to achieve end-to-end optimization. The hardware probe layer acquires platform characteristics through a multi-protocol adaptation mechanism. It parses cache topology and hardware configuration on different architecture platforms and constructs hotspot function profiles by combining runtime performance event collection. The testing system uses orthogonal experimental design for compiler optimization options, compressing high-dimensional option combinations into a core test set, forming a closed-loop workflow of "hardware detection - option generation - compilation - performance collection - feedback". The evaluation system establishes indicator models from multiple dimensions such as performance efficiency and resource overhead to achieve quantitative analysis of optimization effects.

[0038] The noise control system follows the principle of "eliminating interference noise and controllably injecting simulated noise," and achieves precise control through a layered strategy. In the harmful noise cancellation layer, physical isolation and system standardization technologies are employed: resource contention is eliminated by processor core binding and multi-threading is disabled, the processor operating frequency is fixed to avoid dynamic fluctuations, and cache layout deviations are broken by memory address space randomization; hyper-threading is disabled, CPU cores are bound, and dynamic frequency scaling is disabled. In the beneficial noise injection layer, dynamic simulation is implemented based on application scenario characteristics: processor frequency fluctuation simulation is introduced for computationally intensive scenarios, cache contention pressure is injected for memory-intensive scenarios, and resource contention in the actual operating environment is simulated through resource quota control; hyper-threading is enabled; and high frequencies are fixed in the performance mode. The key innovation lies in the resource isolation mechanism between monitoring and the program under test. Hierarchical resource control decouples noise injection from performance monitoring, ensuring the accuracy of test data and the effective exposure of optimization space.

[0039] The hardware information of the target platform includes processor core information, memory and storage information, heterogeneous computing component information, and other hardware characteristics.

[0040] 1. Processor core information: CPU core count (physical cores / logical cores), architecture (x86_64 / ARMv8-A / RISC-V), instruction set support (e.g., whether it supports SIMD instructions such as AVX-512, NEON, and SVE, and whether it supports hardware features related to branch prediction optimization), cache levels and size (L1 / L2 / L3 cache capacity, cache line size, cache associativity), clock speed and turbo frequency range; 2. Memory and storage information: memory capacity, memory bandwidth (such as the theoretical bandwidth of DDR5-4800), number of memory channels, storage media type (SSD / HDD), and IO read / write speed; 3. Heterogeneous computing component information (if present): GPU model, number of stream processors, memory capacity and bandwidth, supported computing architecture (such as NVIDIA's Ampere, AMD's RDNA 3). 4. Other hardware characteristics: Whether hardware virtualization, out-of-order execution, hyper-threading technology are supported, and bus interface type (such as PCIe 4.0 / 5.0), etc. These factors directly affect the choice of optimization pass (for example, when the hardware supports AVX-512, the Loop Vectorize Pass can preferentially use a 512-bit vector width; if the memory bandwidth is low, it needs to be paired with a memory optimization pass to avoid bottlenecks).

[0041] The program performance information during testing includes execution efficiency metrics, hardware resource utilization metrics, compiler intermediate representation (IR) related metrics, and energy consumption and stability metrics.

[0042] 1. Execution efficiency metrics: total program execution time (wall-clock time), CPU user-mode time, kernel-mode time, and the percentage of execution time for critical functions / code blocks (such as loop bodies and hot branches); 2. Hardware resource utilization metrics: CPU utilization (single-core / multi-core average), cache hit rate (number of hits / misses and ratios of L1 instruction cache / L1 data cache / L2 / L3 cache), memory access latency (average load / store latency), SIMD instruction utilization (e.g., percentage of AVX instruction execution), branch prediction accuracy (number of branch hits / total number of branches). 3. Compiler Intermediate Representation (IR) related metrics: the total number of instructions in the pre-optimized program's IR (such as the number of arithmetic instructions, memory instructions, and control flow instructions), the depth of nested loops, the number of basic blocks, and the function call level; 4. Energy consumption and stability indicators: CPU power consumption and temperature during program operation, whether there are memory leaks or instruction anomalies, etc. This information needs to be collected multiple times during the testing phase (such as cold start test, load test) and averaged to reduce errors. It is then used as the core feature of the observation space to input reinforcement learning model, and is also used to calculate rewards (such as generating reward values ​​based on indicators such as "the improvement of cache hit rate before and after optimization" and "the proportion of reduction in execution time"). If the user specifies a configuration file, the configuration file will be loaded after the system initializes: the benchmark framework loads and parses the configuration file to directly obtain system configuration information and test environment configuration.

[0043] If a specific workload needs to be run, which refers to an application or test program, the workload is loaded after system initialization: the benchmark framework loads the executable files or images of these workloads and sets up the runtime environment.

[0044] The preset termination condition refers to: reaching the test time or completing the workload. The specific implementation method of this embodiment is as follows:

[0045] 1. Create a hardware information collection probe: The probe can run on different platforms to collect hardware-related information, such as CPU architecture (x86, ARM, etc.), number of cores, cache size (L1, L2, L3 cache), memory bandwidth, etc.

[0046] 2. Hardware Feature Library: Composed of architecture information, cache information, memory, cores, dies, and instruction sets. 3. Test Program Feature Collection: Given a test load, analyze the code structure, function call graph, data access pattern, etc. of the application. Use the performance analysis tool perf to collect dynamic information such as hot functions, memory access frequency, Cache Misses, and LBR during program execution.

[0047] 4. Adaptive Optimization Platform: Select a compiler (GCC or LLVM) based on the collected hardware information and application features. Configure the optimization options of the compiler for different architectures and hardware characteristics. Establish an optimization strategy library, select appropriate optimization strategies according to different hardware and application features, and use search algorithms such as the annealing algorithm, greedy algorithm, and differential evolution algorithm to screen the order and types of optimization options, establish an optimization library, and create a mapping table of hardware information and optimization options.

[0048] 5. Noise and Noise Introduction: For the test load, different test environments can be enabled on the test platform, including disabling cache, disabling large pages, changing the scheduling algorithm, changing the CPU scheduling mode, and changing the CPU affinity of the test task; test the program performance in different environments to complete a comprehensive performance evaluation of the test load.

[0049] 6. Test Data Comparison Method: When the data n is less than 3, use the arithmetic mean; when 3 < n < 20, create a box plot to represent it; when n > 20, use the Mahalanobis distance.

[0050] OpenBlas improves the average performance of the test program by more than 5% through the above benchmark test framework for introducing noise and the optimization strategy of the adaptive optimization platform.< / pid> < / pid> < / pid>

Claims

1. A benchmark testing framework and adaptive optimization platform that introduces noise, characterized in that, include: Benchmarking framework: Noise is controlled by loading and unloading kernel modules, changing system parameters, locking CPU frequency, suppressing cache interference, limiting process scheduling, and limiting resource quotas, so as to collect real performance test data of the program under controllable noise. Adaptive optimization platform: It obtains underlying data collection architecture, cache, memory, core count, die, and V instruction through the platform's native interface, binds it with specific hardware characteristics and architecture, and obtains optimization templates to form the optimization baseline for specific hardware architectures. By constructing a compilation optimization space, it captures the compiler's set of compilation options and forms optimization strategies for each compilation option. During testing, a flag and location information are added to each option. The optimization framework uses test feedback to adjust and select the best combination in the optimization space by calling different optimization strategies. The hardware microarchitecture features are dynamically acquired through cross-platform probe components and a real-time profile is built. The optimization engine adopts a two-layer collaborative optimization architecture: the bottom layer is based on a mapping model between hardware features and application load, which automatically generates a combination of suitable compilation options and efficiently seeks optimization in the optimization space through intelligent search algorithms. Noise control achieves precise regulation through a hierarchical strategy; In the harmful noise cancellation layer, physical isolation and system standardization technologies are used: resource contention is eliminated by processor core binding and multi-threading is disabled, the processor operating frequency is fixed, and cache layout deviations are broken by memory address space randomization; hyper-threading is disabled, CPU cores are bound, and dynamic frequency scaling is disabled. In the beneficial noise injection layer, dynamic simulation is implemented based on the characteristics of the application scenario: processor frequency fluctuation simulation is introduced for computationally intensive scenarios, cache contention pressure is injected for memory-intensive scenarios, and resource contention in the actual operating environment is simulated through resource quota control. Enable hyper-threading; high frequency in fixed performance mode.

2. The working method of the noise-introduced benchmark testing framework and adaptive optimization platform as described in claim 1, characterized in that, include: Step 1: The startup process of the benchmark framework is as follows: Command line parameter parsing: Users specify runtime parameters and configuration files via the command line. The benchmark framework supports a variety of parameters and options for configuring the benchmark framework's runtime environment, test load, runtime plugins, and log file storage location. System initialization: When the benchmark framework starts executing, initialization operations are performed, including: loading kernel modules, loading and configuring devices, loading test loads, and loading performance testing tools; Test begins: Once all initialization and configuration are complete, the benchmark framework begins executing tasks; Performance statistics and output: During the test, the benchmark framework collects and records various performance data, including instruction execution count, cache hit rate, power consumption, and cycle count; users can obtain this performance statistics as needed or generate reports after the test. Test termination and result output: When the test reaches the preset termination conditions, the benchmark testing framework ends the test and outputs the final test results, including: performance statistics, log information, and memory dump. Step 2: The startup process of the adaptive optimization platform is as follows: Adaptive optimization platform initialization: Capture hardware information of the target platform and program performance information during testing; Adaptive optimization: By using a search algorithm to search for the combination of compiler options that achieves the minimum execution time in the target platform and target compiler options, the optimal combination of compiler options can be obtained within a specified time, and then integrated into the linker optimization, i.e., the obtained combination of compiler options is executed. Post-link optimization: Optimize binary code layout using performance data and hardware feedback during testing. Treat each function as a cluster, and group binary clusters together based on relevance and popularity. Relevance refers to function call relationships, including being called or calling; high call frequency means greater than average. Optimization Verification: The optimized code, completed using the adaptive optimization platform, will be tested again using the testing framework in step 1 to verify the effectiveness of the optimization; if the performance of the optimized program is higher than that before optimization, that is, if the execution time of the optimized program is shorter than that before optimization, then the optimization is complete; Introduce noise and repeat the test: Under different noise conditions, repeat steps 1 and 2. If the performance of the optimized program is higher than that before optimization, the optimization is complete.

3. The working method of the benchmark testing framework and adaptive optimization platform with introduced noise according to claim 2, characterized in that, The benchmark testing framework is built around hardware awareness and automated processes, and adopts a layered architecture to achieve end-to-end optimization. The hardware probe layer obtains platform characteristics through a multi-protocol adaptation mechanism.

4. The working method of the benchmark testing framework and adaptive optimization platform with introduced noise according to claim 2, characterized in that, The hardware information of the target platform includes processor core information, memory and storage information, heterogeneous computing component information, and other hardware characteristics.

5. The working method of the benchmark testing framework and adaptive optimization platform with introduced noise according to claim 2, characterized in that, The program performance information during testing includes execution efficiency metrics, hardware resource utilization metrics, compiler intermediate representation metrics, and energy consumption and stability metrics.

6. The working method of the benchmark testing framework and adaptive optimization platform with introduced noise according to claim 2, characterized in that, If the user specifies a configuration file, the configuration file will be loaded after the system initializes: the benchmark framework loads and parses the configuration file to directly obtain system configuration information and test environment configuration.

7. The working method of the benchmark testing framework and adaptive optimization platform with introduced noise according to claim 2, characterized in that, If a specific workload needs to be run, which refers to an application or test program, the workload is loaded after system initialization: the benchmark framework loads the executable files or images of these workloads and sets up the runtime environment.

8. The working method of the noise-introduced benchmark testing framework and adaptive optimization platform according to any one of claims 2-7, characterized in that, The preset termination condition is: the test time is reached, or the workload is completed.

Images