Parameter adjustment method and device for testing software, equipment, medium and program product

Abstract

The application provides a parameter adjustment method and device for testing software, equipment, a medium and a program product. The method comprises the following steps: determining optimal parameter configurations of a plurality of software parameters of the testing software when a plurality of performance indexes reach optimal values, determining a linear variable which is a software parameter related to chip operation and having a linear relationship with the plurality of performance indexes, determining first characteristic values of the plurality of performance indexes when the linear variable reaches the optimal value based on the optimal parameter configurations and the linear relationship, and generating a benchmark characteristic vector corresponding to the plurality of first characteristic values. The parameter configuration values of the plurality of software parameters are reduced until the similarity between the first characteristic vector determined based on the adjusted parameter configurations and the benchmark characteristic vector is greater than a similarity threshold value, and the adjusted parameter configurations are determined as target parameter configurations, so as to reduce the software parameters of the testing software. When the testing software performs performance testing on the chip based on the target parameter configurations, the occupation of resources in time and space is reduced.

Description

Technical Field

[0001] This application relates to the field of software program technology, and in particular to a method, apparatus, device, medium and program product for adjusting parameters of testing software. Background Technology

[0002] In the early stages of chip function and performance verification, the development of central processing units and system-on-a-chips requires a simulation environment for support because there are no physical products available.

[0003] Some software programs requiring verification need to run on an operating system. In this case, a HAP or EMU environment is needed to load the operating system, benchmarks, workloads, and other environments of the software for actual execution to verify the chip's functionality and performance. The biggest problem with using a HAP or EMU environment for performance testing is the slow system speed. Running some benchmarks or workloads can consume a lot of time, sometimes even making them nearly impossible tasks. Therefore, software slicing techniques are typically used to "prune" the benchmark or workload software programs.

[0004] However, using software slicing technology may result in extremely high fragmentation of the slices, which will increase the running time and overall complexity, or even make it impossible to run. Summary of the Invention

[0005] In view of the above, embodiments of this application provide a method, apparatus, device, medium, and program product for adjusting parameters of testing software, so as to at least solve or alleviate the above problems.

[0006] According to a first aspect of the present application, a method for adjusting parameters of testing software is provided. The testing software is used to test a chip. The method includes: determining the parameter configuration of multiple software parameters of the testing software when multiple performance indicators measured by the testing software reach their optimal values, thereby obtaining an optimal parameter configuration, wherein the multiple performance indicators include at least hardware performance indicators for optimizing the chip; identifying software parameters among the multiple software parameters that have a linear relationship with the multiple performance indicators and are related to the operation of the chip as linear variables; determining the optimal value of the linear variables from the optimal parameter configuration, and based on the linear relationship, determining a first feature value of the multiple performance indicators when the optimal value of the linear variables is obtained, and generating a benchmark feature vector based on multiple first feature values; adjusting the parameter configuration of the multiple software parameters to reduce the parameter configuration value of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold; and determining the adjusted parameter configuration as the target parameter configuration of the multiple software parameters, so that the testing software performs performance testing on the chip based on the target parameter configuration.

[0007] According to a second aspect of the embodiments of this application, a parameter adjustment device for testing software is provided. The testing software is used to test a chip. The device includes: a first determining module, configured to determine the parameter configuration of multiple software parameters of the testing software when multiple performance indicators measured by the testing software reach optimal values, thereby obtaining an optimal parameter configuration, wherein the multiple performance indicators at least include hardware performance indicators for optimizing the chip; a second determining module, configured to determine software parameters among the multiple software parameters that have a linear relationship with the multiple performance indicators and are related to the operation of the chip as linear variables; and a generating module, configured to determine the linear variables from the optimal parameter configuration. The optimal value of the variable is determined, and based on the linear relationship, a first feature value of the multiple performance indicators is determined when the optimal value of the linear variable is obtained. A benchmark feature vector is generated based on the multiple first feature values. An adjustment module is used to adjust the parameter configuration of the multiple software parameters to reduce the parameter configuration value of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold. The adjustment module is also used to determine the adjusted parameter configuration as the target parameter configuration of the multiple software parameters so that the test software performs performance testing on the chip based on the target parameter configuration.

[0008] According to a third aspect of the present application, an electronic device is provided, comprising: a processor, a memory, a communication interface, and a communication bus, wherein the processor, the memory, and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, wherein the executable instruction causes the processor to execute a parameter adjustment method of the test software as described in the first aspect.

[0009] According to a fourth aspect of the present application, a computer storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, it implements the parameter adjustment method of the test software as described in the first aspect.

[0010] According to a fifth aspect of the embodiments of this application, a computer program product is provided, the computer program product including computer instructions that instruct a computer device to execute a parameter adjustment method for the test software as described in the first aspect.

[0011] According to the parameter adjustment method of the test software provided in this application embodiment, the parameter configuration of multiple software parameters of the test software is determined when multiple performance indicators measured by the test software reach their optimal values. The optimal parameter configuration is obtained, and software parameters that have a linear relationship with multiple performance indicators and are related to chip operation are identified as linear variables. The optimal value of the linear variable is determined from the optimal parameter configuration, and the first feature value of multiple performance indicators is determined based on the linear relationship when the optimal value of the linear variable is obtained. A benchmark feature vector is generated based on the multiple first feature values. Then, the parameter configuration of multiple software parameters is adjusted to reduce the parameter configuration value of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold. The adjusted parameter configuration is determined as the target parameter configuration of multiple software parameters, thereby achieving the purpose of reducing the software parameters of the test software. This reduces the resource consumption in both time and space when the test software performs performance testing on the chip based on the target parameter configuration, such as reducing the consumption of processor time and space resources. Compared with slicing technology, this technical solution avoids the problem of high fragmentation caused by slicing, which increases runtime and complexity. Attached Figure Description

[0012] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings.

[0013] Figure 1This is a flowchart of a parameter adjustment method for testing software provided in an exemplary embodiment of this application;

[0014] Figure 2 This is a schematic diagram of a performance relationship curve provided in an exemplary embodiment of this application;

[0015] Figure 3 This is a flowchart of a parameter adjustment method for testing software provided in another exemplary embodiment of this application;

[0016] Figure 4 This is a block diagram of a parameter adjustment device for test software provided in an exemplary embodiment of this application;

[0017] Figure 5 This is a structural diagram of an electronic device provided in an exemplary embodiment of this application. Detailed Implementation

[0018] The present application is described below based on embodiments, but it is not limited to these embodiments. In the detailed description of the present application below, certain specific details are described in detail. Those skilled in the art can fully understand the present application without these details. To avoid obscuring the essence of the present application, well-known methods, processes, and flows are not described in detail. Furthermore, the accompanying drawings are not necessarily drawn to scale.

[0019] In the early stages of chip functionality and performance verification during the development of Central Processing Units (CPUs) and System-on-Chip (SoCs), a simulation environment is needed to support the verification of chip functions and performance since there are no physical products available. Performance testing (benchmarks) of the chip is conducted in a partially bare-metal environment in the front-end environment, such as memory copy performance testing (memcpy), core benchmark testing (coremark), and integer performance benchmark testing (dhystone). After completing the benchmarks, subsequent stages require the addition of an operating system to verify the functionality and performance of more benchmarks and workloads that can only run on the operating system. At this point, a HAP or EMU environment is needed to load the software's operating system, benchmarks, workloads, and other environments for actual execution. The biggest problem with using Haps or EMU environments for performance testing is the slow system speed. Running benchmarks or workloads takes a lot of time, and some tasks are even nearly impossible. For example, the commonly used CPU benchmark spec2006int takes about 45 minutes to complete once on a physical product. If run on Haps, it would take about a week to complete. Furthermore, due to the nature of Haps, the data obtained from running on Haps is not accurate. If run on a precise EMU system, this time will be even longer, possibly several months or even more than half a year. This cycle is unacceptable for chip development. Among them, Emulator is a hardware simulator used to verify the CPU's functionality and performance before the actual hardware is fabricated; it is abbreviated as EMU. Haps is a hardware platform, which stands for High-performance ASIC Prototyping System. Its core function is to verify the hardware architecture performance, software compatibility, and system-level interaction before the chip is fabricated. It is especially suitable for verifying high-speed interfaces, complex computing modules (such as AI accelerators and GPUs), and real-time systems.

[0020] To address the aforementioned difficulties, the common practice in the industry is to effectively "tailor" the benchmark or workload software program so that the required runtime and operating time for running on a precise simulation platform like EMU are feasible for chip development.

[0021] The "trimming" mentioned here commonly refers to software slicing. Software slicing is a program analysis and decomposition technique. Its core is to extract the main features of a complex program and form "subroutine fragments" (also called "slices"), ignoring other irrelevant code, thereby simplifying the program analysis process.

[0022] While the specific implementations of this technology may differ, the core idea remains the same: to obtain "slices" through technical means, then run them on the EMU platform to infer the actual benchmark performance values. However, regardless of the specific implementation method, slices based on program hotspots are required. This necessitates that the program has concentrated hotspots; otherwise, the slices will be highly fragmented, increasing runtime and overall complexity, or even rendering the program unusable.

[0023] Please refer to Figure 1 The diagram illustrates a flowchart of a parameter adjustment method for testing software provided in an exemplary embodiment of this application. The method includes:

[0024] Step 101: Determine the parameter configuration of multiple software parameters of the test software when the multiple performance indicators measured by the test software reach their optimal values, and obtain the optimal parameter configuration.

[0025] This method can be applied to hardware-assisted verification platforms, such as HAPs or EMU.

[0026] This testing software is used to test the chip. Running the testing software on the platform allows for chip testing. For example, this testing software can be "infrastructure software" and / or "internet application software," such as MySQL, Redis, Nginx, MongoDB, and other core foundational software used to build and support internet applications.

[0027] For example, the parameter set of the test software contains multiple software parameters of the test software. Each test software has its own parameter set, and there is overlap between the parameter sets of different test software, that is, the parameter sets of two test software can be the same; the parameter sets of two test software can also have some software parameters that are the same and some software parameters that are different. Generally, software parameters include, but are not limited to: concurrency, test data size, and data changes in each dimension of multi-dimensional data.

[0028] The platform runs the testing software and determines the optimal configuration of multiple software parameters when the multiple performance indicators measured by the testing software reach their optimal values. In other words, it finds the maximum performance point of the testing software. The maximum performance point refers to a specific and optimized software configuration state. The optimal parameter configuration of multiple software parameters of the testing software is obtained at the maximum performance point.

[0029] Optionally, the platform can determine the optimal configuration of various software parameters when multiple performance indicators reach their best values ​​by stress testing the testing software. For example, based on the software parameter configuration requirements of the testing software, the platform can perform stress testing (referred to as "stress testing") on the testing software to obtain a set of multiple performance indicators, which includes the characteristic value set of each performance indicator.

[0030] The multiple performance metrics include at least the hardware performance metrics used to optimize the chip. Optionally, these multiple hardware performance metrics include at least two of the following:

[0031] Processor execution efficiency, for example, is reflected by the number of instructions per clock cycle (IPC);

[0032] Front-end pressure during processor execution, for example, can be reflected by front-end blocking (frontend_bound);

[0033] Backend pressure during processor execution can be reflected, for example, by using backend blocking (backend_bound) to indicate the backend pressure during CPU execution.

[0034] The overhead caused by speculative prediction failures during processor execution can be reflected by using a bad_speculation bound.

[0035] Branch miss rate during processor execution;

[0036] The L1 cache miss rate (l1_icache_miss) of the processor.

[0037] The L1 cache miss rate (l1_dcache_miss) performed by the processor.

[0038] The L2 cache miss rate performed by the processor.

[0039] The L1 instruction translation back buffer miss rate (l1_itlb_miss) executed by the processor.

[0040] The L1 data translation back buffer miss rate (l1_dtlb_miss) performed by the processor.

[0041] The processor's secondary translation back buffer miss rate (l2_tlb_miss).

[0042] Optionally, the performance metrics may also include: software performance metrics used to maintain the program behavior of the test software, such as the percentage of hot functions during program runtime, to ensure that the program behavior of the test software before parameter adjustment is the same as or similar to the program behavior of the test software after parameter adjustment.

[0043] Step 102: Among the multiple software parameters, those that have a linear relationship with multiple performance indicators and are related to chip operation are identified as linear variables.

[0044] The platform determines the relationship between various software parameters and their related performance indicators. The performance indicators related to the software parameters refer to the performance indicators affected by those software parameters.

[0045] If a software parameter exhibits a linear relationship with all of the multiple performance metrics, and this software parameter is related to chip operation, then this software parameter is defined as a linear variable. The linear relationship described above expresses a linear relationship between the performance metrics of the test software and the configured values ​​of the software parameters.

[0046] Optionally, the linear variable includes one of the following: processor utilization in user mode during program execution (process_user%); processor utilization in kernel mode during program execution (process_system%); processor utilization in process execution (process%); and the number of processor cores used, such as the number of CPU cores bound to critical programs.

[0047] For example, a linear variable is unique and is determined from multiple linear relationships. Specifically, the platform identifies a linear variable that meets the following two conditions: first, a performance characteristic exists within the linear relationship corresponding to the linear variable; second, an actual performance bottleneck exists within the linear relationship corresponding to the linear variable.

[0048] like Figure 2It shows five relationship curves: Curve 11 is the relationship between the kernel-mode CPU utilization of the redis-server main thread and the number of CPUs used by redis-server; Curve 12 is the relationship between the QPS performance of redis and the number of CPUs used by redis-server; Curve 13 is the relationship between the CPU utilization of the redis-server IO thread and the number of CPUs used by redis-server; Curve 14 is the relationship between the user-mode CPU utilization of the redis-server main thread and the number of CPUs used by redis-server; and Curve 15 is the relationship between the CPU utilization of the redis-server main thread and the number of CPUs used by redis-server. The X-axis represents the number of CPUs used by redis-server, and the Y-axis is a normalized representation of each performance indicator. For example, the X-axis of curve 11 represents the number of CPUs used by redis-server, and the Y-axis is a normalized representation of the kernel-mode CPU utilization of the redis-server main thread.

[0049] Clearly, curves 13 and 14 exhibit a linear relationship. Further analysis of the variable "number of CPUs used by redis-server" corresponding to these two curves determines whether it meets the two conditions mentioned above. For example, the performance metric "CPU utilization of redis-server's IO threads" increases linearly with the number of CPUs. Once a certain level is reached, even if the number of CPUs increases further, this performance metric will not increase further. Therefore, it can be concluded that these are the actual performance bottlenecks and exhibit obvious performance characteristics. Thus, the number of CPUs can be identified as a linear variable.

[0050] For example, the platform can also define program-related variables as linear variables, such as the CPU utilization of IOThread and process_user%; where the CPU utilization of IOThread is a variable related to external stimulus pressure, while process_user% is a variable related to the program itself; therefore, process_user% is used as a linear variable.

[0051] Step 103: Determine the optimal value of the linear variable from the optimal parameter configuration, and based on the linear relationship, determine the first characteristic value of multiple performance indicators when the optimal value of the linear variable is obtained, and generate a benchmark feature vector based on the multiple first characteristic values.

[0052] The platform determines the optimal value of the linear variable from the optimal parameter configuration, and then determines the first characteristic value of the performance index when the linear variable obtains the optimal value from the linear relationship curve between the linear variable and the performance index. There are multiple linear relationships between the linear variable and multiple performance indexes, thus obtaining multiple first characteristic values, and forming a benchmark characteristic vector from the multiple first characteristic values.

[0053] For example, the platform obtains the first feature value corresponding to each of the multiple performance indicators when the optimal parameter configuration is obtained, obtains multiple first feature values, and then generates a benchmark feature vector based on the multiple first feature values, that is, the feature vector at the maximum performance point is used as the benchmark feature vector.

[0054] Optionally, the process of generating the benchmark feature vector includes: forming a feature sequence from multiple first feature values, and adding each feature value in the feature sequence to a preset parameter value to form a one-dimensional benchmark feature vector. For example, the preset parameter value can be 1, meaning that after forming the feature sequence from multiple first feature values, each feature value in the feature sequence is incremented by 1 to obtain the one-dimensional benchmark feature vector. In this way, by increasing the preset parameter value, each feature is strengthened, increasing the proportion of the feature in subsequent comparisons to offset the errors caused by the distortion of certain features during subsequent parameter adjustments.

[0055] Step 104: Adjust the parameter configuration of multiple software parameters to reduce the parameter configuration values ​​of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than the similarity threshold.

[0056] The platform repeats the following steps until the similarity is greater than the similarity threshold: adjust the parameter configuration of multiple software parameters to obtain the adjusted parameter configuration; determine the second feature values ​​of multiple performance indicators of the test software based on the adjusted parameter configuration, generate a first feature vector based on the multiple second feature values, and if the similarity is less than or equal to the similarity threshold, return to the step of adjusting the parameter configuration of multiple software parameters.

[0057] That is, the platform adjusts the parameter configuration of multiple software parameters to reduce the parameter configuration values, resulting in the adjusted parameter configuration; the test software is run based on the adjusted parameter configuration to determine multiple second feature values ​​corresponding to multiple performance indicators, and a first feature vector is generated based on the multiple second feature values, that is, the multiple second feature values ​​are arranged into a sequence to obtain the first feature vector. The similarity between the first feature vector and the benchmark feature vector is calculated. If the similarity is less than or equal to the similarity threshold, the parameter configuration is readjusted, and the similarity between the first feature vector corresponding to the readjusted parameter configuration and the benchmark feature vector is calculated until the similarity is greater than the similarity threshold; if the similarity is greater than the similarity threshold, step 105 is executed.

[0058] Among the various software parameters of the testing software, there may be parameters that do not need adjustment. For example, some software parameters are unrelated to the resource requirements for software operation, and their operation is unaffected by changes in the configuration of other software parameters. The platform can filter out these parameters that require adjustment, thus reducing the computational load of the entire parameter configuration adjustment process by eliminating the need to adjust all software parameters. Optionally, when adjusting the configuration of multiple software parameters, the platform first identifies the software parameters that affect the resource requirements for the testing software, obtaining multiple adjustable parameters; then, it adjusts the configuration of these adjustable parameters.

[0059] For example, the similarity between feature vectors can be represented by one of the following: standard deviation, squared difference, Manhattan distance, etc.

[0060] Step 105: Determine the adjusted parameter configuration as the target parameter configuration for multiple software parameters so that the test software can perform performance testing on the chip based on the target parameter configuration.

[0061] The platform will determine the adjusted parameter configuration as the target parameter configuration for multiple software parameters. Subsequently, the platform will run the test software based on the target parameter configuration to perform performance tests on the chip.

[0062] In summary, the parameter adjustment method for the testing software provided in this embodiment determines the parameter configuration of multiple software parameters when the multiple performance indicators measured by the testing software reach their optimal values. This optimal parameter configuration is obtained by identifying software parameters that have a linear relationship with the multiple performance indicators and are related to chip operation as linear variables. The optimal value of the linear variables is determined from the optimal parameter configuration, and the first characteristic value of the multiple performance indicators is determined based on the linear relationship when the optimal value of the linear variables is obtained. A benchmark feature vector is generated based on the multiple first characteristic values. Then, the parameter configuration of the multiple software parameters is adjusted to reduce the parameter configuration values ​​until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold. The adjusted parameter configuration is then determined as the target parameter configuration for the multiple software parameters, thereby reducing the software parameters of the testing software. This reduces the resource consumption in both time and space when the testing software performs performance testing on the chip based on the target parameter configuration, such as reducing the consumption of processor time and space resources. Compared to slicing technology, this technical solution avoids the problem of high fragmentation caused by slicing, which increases runtime and complexity.

[0063] In some alternative implementations, when the test software has many parameters, the weight of a particular software parameter can be determined first. This allows for more accurate adjustment of the software parameters based on the proportions among the various parameters. Figure 3 As shown, the process of adjusting parameters based on this method is as follows:

[0064] Step 201: Determine the parameter configuration of multiple software parameters of the test software when the multiple performance indicators measured by the test software reach their optimal values, and obtain the optimal parameter configuration.

[0065] Step 202: Among the multiple software parameters, those that have a linear relationship with multiple performance indicators and are related to chip operation are identified as linear variables.

[0066] Step 203: Determine the optimal value of the linear variable from the optimal parameter configuration, and based on the linear relationship, determine the first characteristic value of multiple performance indicators when the optimal value of the linear variable is obtained, and generate a benchmark feature vector based on the multiple first characteristic values.

[0067] For detailed implementation of steps 201 to 203, please refer to the content of steps 101 to 103, which will not be repeated here.

[0068] Step 204: Train the weight vectors corresponding to multiple software parameters to obtain the target weight vector.

[0069] The platform has a preset weight vector. The platform combines multiple software parameters into a parameter vector; based on the preset weight vector, the parameter vector is adjusted to obtain the adjusted parameter vector; the third feature value of multiple performance indicators of the test software running based on the adjusted parameter vector is determined, and a second feature vector is generated based on multiple third feature values, for example, by forming a sequence of multiple third feature values ​​to obtain the second feature vector; based on the deviation between the second feature vector and the benchmark feature vector, the weight vector is adjusted to obtain the target weight vector.

[0070] Optionally, based on the deviation between the second feature vector and the reference feature vector, the weight vector is adjusted to obtain the target weight vector, including: if the deviation is greater than the deviation threshold, the following steps are repeated: adjusting the weight vector based on the deviation to obtain the adjusted weight vector; adjusting the parameter vector based on the adjusted weight vector to obtain the adjusted parameter vector; regenerating the second feature vector based on the adjusted parameter vector; redetermining the deviation based on the second feature vector and the reference feature vector; if the deviation is less than or equal to the deviation threshold, the adjusted weight vector is determined as the target weight vector.

[0071] Optionally, the platform determines the software parameters affecting the amount of resources required for the test software to run from multiple software parameters, obtaining multiple adjustable parameters; the platform then adjusts the parameter configurations of these adjustable parameters. Accordingly, the platform assembles these adjustable parameters into a parameter vector; based on a preset weight vector, it adjusts the parameter vector to obtain an adjusted parameter vector; it determines the third characteristic value of multiple performance indicators when the test software runs based on the adjusted parameter vector, and generates a second characteristic vector based on these third characteristic values; based on the deviation between the second characteristic vector and the benchmark characteristic vector, it adjusts the weight vector to obtain a target weight vector. For example, if the deviation is greater than a deviation threshold, the following steps are repeated: adjusting the weight vector based on the deviation to obtain an adjusted weight vector; adjusting the parameter vector based on the adjusted weight vector to obtain an adjusted parameter vector; regenerating the second characteristic vector based on the adjusted parameter vector; and re-determining the deviation based on the second characteristic vector and the benchmark characteristic vector; if the deviation is less than or equal to the deviation threshold, the adjusted weight vector is determined as the target weight vector.

[0072] The generation of the second feature vector can refer to the generation of the first feature vector.

[0073] Step 205: Based on the target weight vector, adjust the parameter configuration of multiple software parameters to reduce the parameter configuration values ​​of the software parameters, and determine the first feature vector based on the adjusted parameter configuration.

[0074] The platform repeats the following steps until the similarity is greater than the similarity threshold: Based on the target weight vector, the parameter configuration of multiple software parameters is adjusted to obtain the adjusted parameter configuration; the second feature values ​​of multiple performance indicators of the test software are determined based on the adjusted parameter configuration, and a first feature vector is generated based on the multiple second feature values. If the similarity is less than or equal to the similarity threshold, the platform returns to the step of adjusting the parameter configuration of multiple software parameters.

[0075] Optionally, the platform adjusts the parameter configuration of multiple adjustable parameters based on the target weight vector.

[0076] Step 206: Determine whether the similarity between the first feature vector and the baseline feature vector is greater than the similarity threshold.

[0077] If the platform determines that the above similarity is less than or equal to the similarity threshold, it returns to step 205; if the platform determines that the above similarity is greater than the similarity threshold, it proceeds to step 207.

[0078] Step 207: Determine the adjusted parameter configuration as the target parameter configuration for multiple software parameters so that the test software can perform performance testing on the chip based on the target parameter configuration.

[0079] For a detailed explanation of the implementation process of step 207, please refer to the content of step 105, which will not be repeated here.

[0080] In summary, the parameter adjustment method for the testing software provided in this embodiment trains the weight vectors of multiple software parameters before adjusting the parameter configuration. It adjusts the weight of each parameter, increasing the weight of parameters with small deviations and decreasing the weight of parameters with large deviations. This method aims to determine the degree of influence of each parameter on performance indicators and the degree of influence on the deviation between feature vectors. This ensures higher accuracy in each parameter configuration adjustment process, allowing for faster identification of the target parameter configuration and improving the efficiency of parameter adjustment for the testing software.

[0081] The parameter adjustment method for test software provided in this application analyzes the functions and characteristics of the test software, extracts its operational features, and adjusts the software's operational parameters while ensuring that the operational features remain unchanged or similar. This significantly reduces the resources required for the test software to run (such as the number of CPUs, memory space, and running time), thereby enabling it to run directly in simulation environments such as EMU. The parameter adjustment in this method is equivalent to "pruning" the program. The entire "pruning" process is concise and clear, and the mapping result before pruning is clear. It does not distinguish between user mode, kernel mode, single-threaded, multi-threaded, or multi-process factors. It has no restrictions on the internal software architecture of the program, such as multi-threaded / multi-process, and effectively solves the problem of insufficient accuracy in mainstream slicing techniques for multi-threaded and multi-process software.

[0082] Figure 4 This is a block diagram of a parameter adjustment device for testing software provided in an embodiment of this application. The testing software is used to test chips, and the device includes:

[0083] The first determining module 301 is used to determine the parameter configuration of multiple software parameters of the test software when the multiple performance indicators measured by the test software reach their optimal values, thereby obtaining the optimal parameter configuration. The multiple performance indicators include at least the hardware performance indicators used to optimize the chip. The second determining module 302 is used to determine the software parameters among the multiple software parameters that have a linear relationship with the multiple performance indicators and are related to chip operation as linear variables. The generating module 303 is used to determine the optimal value of the linear variables from the optimal parameter configuration, and based on the linear relationship, determine the first characteristic value of the multiple performance indicators when the optimal value of the linear variables is obtained, and generate a benchmark feature vector based on the multiple first characteristic values. The adjusting module 304 is used to adjust the parameter configuration of the multiple software parameters to reduce the parameter configuration values ​​of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold. The adjusting module 304 is also used to determine the adjusted parameter configuration as the target parameter configuration for the multiple software parameters, so that the test software performs performance testing on the chip based on the target parameter configuration.

[0084] In some optional implementations, the adjustment module 304 is used to repeatedly perform the following steps until the similarity is greater than the similarity threshold: adjust the parameter configuration of multiple software parameters to obtain the adjusted parameter configuration; determine the second feature values ​​of multiple performance indicators of the test software running based on the adjusted parameter configuration; generate a first feature vector based on the multiple second feature values; if the similarity is less than or equal to the similarity threshold, return to the step of adjusting the parameter configuration of multiple software parameters.

[0085] In some optional implementations, adjustment module 304 is used to determine the software parameters that affect the amount of resources required for the test software to run from multiple software parameters, obtain multiple adjustable parameters, and adjust the parameter configuration of the multiple adjustable parameters.

[0086] In some optional implementations, the adjustment module 304 is also used to: 1) compose multiple adjustable parameters into a parameter vector before adjusting the parameter configuration of multiple adjustable parameters; 2) adjust the parameter vector based on a preset weight vector to obtain an adjusted parameter vector; 3) determine the third feature value of multiple performance indicators when the test software runs based on the adjusted parameter vector; 4) generate a second feature vector based on the multiple third feature values; 5) adjust the weight vector based on the deviation between the second feature vector and the benchmark feature vector to obtain a target weight vector; and 6) adjust multiple adjustable parameters based on the target weight vector.

[0087] In some optional implementations, the adjustment module 304 is also used to repeat the following steps if the deviation is greater than the deviation threshold: adjust the weight vector based on the deviation to obtain the adjusted weight vector; adjust the parameter vector based on the adjusted weight vector to obtain the adjusted parameter vector; regenerate the second feature vector based on the adjusted parameter vector; and redetermine the deviation based on the second feature vector and the benchmark feature vector; if the deviation is less than or equal to the deviation threshold, the adjusted weight vector is determined as the target weight vector.

[0088] In some optional implementations, the generation module 303 is used to form a feature sequence from multiple first feature values, and to add each feature value in the feature sequence to a preset parameter value to form a one-dimensional reference feature vector.

[0089] In some optional implementations, the first determining module 301 is used to determine the parameter configuration of multiple software parameters when multiple performance indicators reach their optimal values ​​by stress testing the test software.

[0090] In some alternative implementations, the linear variables include one of the following: processor utilization in user mode during program execution; processor utilization in kernel mode during program execution; processor utilization during program execution; number of processor cores used.

[0091] In some alternative implementations, multiple performance metrics include at least two of the following: processor execution efficiency; front-end stress during processor execution; back-end stress during processor execution; overhead from speculative prediction failures during processor execution; branch miss rate during processor execution; L1 instruction cache miss rate during processor execution; L1 data cache miss rate during processor execution; L2 cache miss rate during processor execution; L1 instruction translation back buffer miss rate during processor execution; L1 data translation back buffer miss rate during processor execution; and L2 translation back buffer miss rate during processor execution.

[0092] In some alternative implementations, several performance metrics also include the percentage of hot functions during program runtime.

[0093] Figure 5 This is a schematic block diagram of an electronic device provided in an embodiment of this application. Specific embodiments of this application do not limit the specific implementation of the electronic device. Figure 5 As shown, the electronic device may include: a processor 402, a communications interface 404, a memory 406, and a communication bus 408. Wherein:

[0094] The processor 402, communication interface 404, and memory 406 communicate with each other via communication bus 408.

[0095] Communication interface 404 is used for communication with other electronic devices.

[0096] The processor 402 is used to execute program 410, specifically to execute the relevant steps in any of the aforementioned test software parameter adjustment method embodiments.

[0097] Specifically, program 410 may include program code that includes computer operation instructions.

[0098] Processor 402 may be a Central Processing Unit (CPU), an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or they may be processors of different types, such as one or more CPUs and one or more ASICs.

[0099] RISC-V is an open-source instruction set architecture based on the Reduced Instruction Set Computing (RISC) principle. It can be applied to various aspects of microcontrollers and FPGA chips, specifically in areas such as IoT security, industrial control, mobile phones, and personal computers. Because its design considers small size, speed, and low power consumption, it is particularly suitable for modern computing devices such as warehouse-scale cloud computers, high-end mobile phones, and tiny embedded systems. With the rise of AIoT (Artificial Intelligence of Things), the RISC-V instruction set architecture is receiving increasing attention and support and is expected to become the next generation of widely used CPU architecture.

[0100] The computer operation instructions in this application embodiment can be computer operation instructions based on the RISC-V instruction set architecture. Correspondingly, the processor 402 can be designed based on the RISC-V instruction set. Specifically, the processor chip in the electronic device provided in this application embodiment can be a chip designed using the RISC-V instruction set. This chip can execute executable code based on the configured instructions, thereby implementing the parameter adjustment method of the test software in the above embodiment.

[0101] Memory 406 is used to store program 410. Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk storage device.

[0102] Specifically, program 410 can be used to cause processor 402 to execute the parameter adjustment method of the test software in any of the foregoing embodiments.

[0103] The specific implementation of each step in program 410 can be found in the corresponding steps and units described in the aforementioned parameter adjustment method embodiments of any of the testing software, and will not be repeated here. Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the devices and modules described above can be referred to the corresponding process descriptions in the aforementioned method embodiments, and will not be repeated here.

[0104] This application also provides a computer-readable storage medium storing instructions for causing a machine to execute a parameter adjustment method of the test software described herein. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0105] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of this application.

[0106] Storage media embodiments for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0107] This application also provides a computer program product, including computer instructions that instruct a computing device to perform any corresponding operation in the above-described plurality of method embodiments.

[0108] It should be noted that, depending on the implementation needs, the various components / steps described in the embodiments of this application can be broken down into more components / steps, or two or more components / steps or parts of the operation of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of this application.

[0109] The methods described in the embodiments of this application can be implemented in hardware, firmware, or as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk, or magneto-optical disk), or as computer code downloaded over a network that is originally stored in a remote recording medium or a non-transitory machine-readable medium and will be stored in a local recording medium. Thus, the methods described herein can be processed by software stored on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an ASIC or FPGA). It is understood that the computer, processor, microprocessor controller, or programmable hardware includes storage components (e.g., RAM, ROM, flash memory, etc.) capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods described herein. Furthermore, when a general-purpose computer accesses code used to implement the methods shown herein, the execution of the code transforms the general-purpose computer into a dedicated computer for executing the methods shown herein.

[0110] Those skilled in the art will recognize that the units and method steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0111] The above embodiments are only used to illustrate the embodiments of this application, and are not intended to limit the embodiments of this application. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the embodiments of this application. Therefore, all equivalent technical solutions also fall within the scope of the embodiments of this application, and the patent protection scope of the embodiments of this application should be defined by the claims.

Claims

1. A method for adjusting parameters of testing software, the testing software being used to test a chip, the method comprising: The optimal parameter configuration is obtained by determining the parameter configuration of multiple software parameters of the test software when the multiple performance indicators measured by the test software reach their optimal values. The multiple performance indicators include at least the hardware performance indicators used to optimize the chip. Among the multiple software parameters, those that have a linear relationship with the multiple performance indicators and are related to the operation of the chip are determined as linear variables; wherein, the linear variables meet the following conditions: there is a performance characteristic in the linear relationship corresponding to the linear variable, and there is an actual performance bottleneck in the linear relationship corresponding to the linear variable; The optimal value of the linear variable is determined from the optimal parameter configuration, and based on the linear relationship, the first feature value of the multiple performance indicators is determined when the optimal value of the linear variable is obtained, and a benchmark feature vector is generated based on the multiple first feature values. The parameter configurations of the multiple software parameters are adjusted to reduce the parameter configuration values ​​of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than the similarity threshold. The adjusted parameter configuration is determined as the target parameter configuration for the multiple software parameters, so that the test software performs performance testing on the chip based on the target parameter configuration.

2. The method according to claim 1, wherein, The step of adjusting the parameter configuration of the multiple software parameters to reduce the parameter configuration values ​​of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than a similarity threshold includes: Repeat the following steps until the similarity is greater than the similarity threshold: adjust the parameter configuration of the multiple software parameters to obtain the adjusted parameter configuration; determine the second feature value of the multiple performance indicators when the test software runs based on the adjusted parameter configuration, generate a first feature vector based on the multiple second feature values, and if the similarity is less than or equal to the similarity threshold, return to the step of adjusting the parameter configuration of the multiple software parameters.

3. The method according to claim 1, wherein, The adjustment of the parameter configuration of the multiple software parameters includes: From the multiple software parameters, determine the software parameters that affect the amount of resources required for the test software to run, and obtain multiple adjustable parameters; The parameter configurations of the aforementioned adjustable parameters are adjusted.

4. The method according to claim 3, wherein, Before adjusting the parameter configuration of the multiple adjustable parameters, the process includes: forming a parameter vector from the multiple adjustable parameters; adjusting the parameter vector based on a preset weight vector to obtain an adjusted parameter vector; determining the third feature value of the multiple performance indicators when the test software runs based on the adjusted parameter vector; generating a second feature vector based on multiple third feature values; and adjusting the weight vector based on the deviation between the second feature vector and the benchmark feature vector to obtain a target weight vector. The adjustment of the parameter configuration of the plurality of adjustable parameters includes: adjusting the plurality of adjustable parameters based on the target weight vector.

5. The method according to claim 4, wherein, The step of adjusting the weight vector based on the deviation between the second feature vector and the reference feature vector to obtain the target weight vector includes: If the deviation is greater than the deviation threshold, the following steps are repeated: adjust the weight vector based on the deviation to obtain the adjusted weight vector; adjust the parameter vector based on the adjusted weight vector to obtain the adjusted parameter vector; regenerate the second feature vector based on the adjusted parameter vector; and redetermine the deviation based on the second feature vector and the benchmark feature vector. If the deviation is less than or equal to the deviation threshold, then the adjusted weight vector is determined as the target weight vector.

6. The method according to any one of claims 1 to 5, wherein, The step of generating a baseline feature vector based on multiple first feature values ​​includes: Multiple first feature values ​​are combined to form a feature sequence, and each feature value in the feature sequence is added to a preset parameter value to form a one-dimensional reference feature vector.

7. The method according to any one of claims 1 to 5, wherein, The parameter configuration of multiple software parameters of the testing software when determining that the multiple performance indicators measured by the testing software reach their optimal values ​​includes: By performing stress tests on the testing software, the parameter configurations of the various software parameters are determined when the various performance indicators reach their optimal values.

8. The method according to any one of claims 1 to 5, wherein, The linear variable includes one of the following: The processor utilization rate in user mode during program execution; The processor utilization of the kernel executing the program; Processor utilization during program execution; Number of processor cores used.

9. The method according to any one of claims 1 to 5, wherein, The aforementioned multiple performance indicators include at least two of the following: Processor execution efficiency; Front-end pressure during processor execution; Backend pressure during processor execution; The overhead caused by failed speculative predictions during processor execution; Branch miss rate during processor execution; The L1 cache miss rate of the processor; The L1 cache miss rate executed by the processor; The L2 cache miss rate performed by the processor; The processor's Level 1 instruction translation back buffer miss rate; The processor's Level 1 data translation back buffer miss rate; The processor's secondary translation back buffer miss rate.

10. The method according to claim 9, wherein, The aforementioned performance metrics also include the percentage of hot functions during program runtime.

11. A parameter adjustment device for testing software, the testing software being used to test a chip, the device comprising: The first determining module is used to determine the parameter configuration of multiple software parameters of the test software when the multiple performance indicators measured by the test software reach the optimal value, and to obtain the optimal parameter configuration, wherein the multiple performance indicators include at least the hardware performance indicators used to optimize the chip. The second determining module is used to determine the software parameters among the multiple software parameters that have a linear relationship with the multiple performance indicators and are related to the operation of the chip as linear variables; wherein, the linear variables meet the following conditions: there is a performance characteristic in the linear relationship corresponding to the linear variable, and there is an actual performance bottleneck in the linear relationship corresponding to the linear variable; A generation module is used to determine the optimal value of the linear variable from the optimal parameter configuration, and based on the linear relationship, determine the first feature value of the multiple performance indicators when the optimal value of the linear variable is obtained, and generate a benchmark feature vector based on the multiple first feature values. An adjustment module is used to adjust the parameter configuration of the multiple software parameters to reduce the parameter configuration value of the software parameters until the similarity between the first feature vector determined based on the adjusted parameter configuration and the benchmark feature vector is greater than the similarity threshold. The adjustment module is further configured to determine the adjusted parameter configuration as the target parameter configuration of the multiple software parameters, so that the test software performs performance testing on the chip based on the target parameter configuration.

12. An electronic device, comprising: The processor, memory, communication interface, and communication bus are provided, wherein the processor, memory, and communication interface communicate with each other via the communication bus. The memory is used to store at least one executable instruction, which causes the processor to execute the parameter adjustment method of the test software as described in any one of claims 1-10.

13. A computer storage medium storing a computer program, wherein the computer program, when executed by a processor, implements the parameter adjustment method of the test software as described in any one of claims 1-10.

14. A computer program product comprising computer instructions that instruct a computer device to perform a parameter adjustment method of the test software as described in any one of claims 1-10.

Images