Performance evaluation method and device of processor, electronic equipment and storage medium
By dividing the test case set and distributing it to different processing nodes of a multi-core processor for parallel processing, the problem of long processing time in the traditional SPEC CPU2017 Rate test is solved, and efficient processor performance evaluation is achieved.
Patent Information
- Application Number
- CN202511766908.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-27
- Publication Date
- 2026-02-24
AI Technical Summary
The traditional SPEC CPU2017 Rate test method is time-consuming and difficult to efficiently evaluate the performance changes of multi-core processors, especially when verifying compilation optimization parameters or system parameters, which requires repeated execution, resulting in low performance evaluation efficiency.
The test case set is divided into multiple test case groups, which are then assigned to different processing nodes of a processor based on a non-uniform memory access architecture for parallel processing. The computing and storage resources of each processing node are utilized, and the processing core is bound to the test case configuration file to achieve parallel operation.
It significantly shortens testing time, improves the efficiency of processor performance evaluation, makes full use of the computing and storage resources of multi-core processors, and enhances the efficiency and accuracy of evaluation.
Smart Images
Figure CN121560700A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of integrated circuit technology, and in particular to a method, apparatus, electronic device, and storage medium for evaluating processor performance. Background Technology
[0002] The SPEC CPU (Standard Performance Evaluation Corporation CPU) test case set is a core benchmark for measuring processor performance. Among them, the SPEC CPU2017 Rate test mode evaluates the overall throughput performance of the processor by running multiple test cases, and is widely used for performance verification and tuning of servers and high-performance computing platforms.
[0003] However, with multi-core processors becoming mainstream, the traditional SPEC CPU testing method faces a severe efficiency bottleneck. A complete SPEC CPU2017 Rate test typically takes several hours, such as 5 to 8 hours. Furthermore, to verify the impact of different compiler optimization parameters, system parameters, or BIOS (Basic Input Output System) parameters on performance, it is necessary to run SPEC CPU2017 repeatedly, resulting in low performance evaluation efficiency. Summary of the Invention
[0004] In view of this, embodiments of the present invention provide a processor performance evaluation method, apparatus, electronic device, and storage medium, which can improve performance evaluation efficiency.
[0005] In a first aspect, embodiments of the present invention provide a processor performance evaluation method, the method comprising: acquiring architecture information of a processor to be evaluated, the architecture information including processing nodes included in the processor to be evaluated and processing cores included in each processing node; wherein the processor to be evaluated is a processor based on a non-consistent memory access architecture, and the memory corresponding to each processing node is different from each other; dividing each test case in a test case set into at least two test case groups according to the architecture information, each processing node corresponding to each test case group; running the test case set based on the correspondence between each processing node and each test case group to obtain a test score corresponding to each test case, wherein each processing node is used to run its own corresponding test case group; and determining the performance score of the processor to be evaluated based on the test scores of each test case.
[0006] In one implementation, dividing the test cases in the test case set into at least two test case groups based on the architecture information includes: determining the average runtime of each test case based on historical test information of each test case in the test case set; dividing the test cases in the test case set into at least two test case groups based on the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, such that the absolute value of the difference between the runtimes of any two test case groups is less than a preset threshold, wherein the runtime of a test case group is equal to the sum of the average runtimes of each test case in that test case group.
[0007] In one implementation, the step of running the test case set based on the correspondence between each processing node and each test case group to obtain a test score for each test case includes: determining the processing node corresponding to each test case in the test case set according to the correspondence between each processing node and each test case group; binding the test case to the corresponding processing node through the configuration file of each test case; and running each test case based on the configuration file of each test case to obtain a test score for each test case.
[0008] In one implementation, binding the test case to the corresponding processing node through the configuration file of each test case includes: obtaining the core number of the processing core contained in each processing node; and binding the test case to the corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case.
[0009] In one implementation, each processing node includes N processing cores, where N is an integer greater than 1; binding a test case to a corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case includes: inserting N core binding instructions in the configuration file of each test case to bind N test case copies to N processing cores in a one-to-one correspondence; running each test case based on the configuration file of each test case to obtain the test score corresponding to each test case includes: running each test case copy of the test case in parallel on the N processing cores corresponding to the test case based on the configuration file of each test case to obtain the test score corresponding to the test case.
[0010] In one implementation, the step of running each test case based on the configuration file of each test case to obtain the test score corresponding to each test case includes: each processing node runs each test case in a preset order based on the configuration file of its corresponding test cases.
[0011] In one implementation, in each of the test cases that are run sequentially, the next test case is run after the previous test case has finished running and a preset time has elapsed.
[0012] In one implementation, running each test case based on the configuration file of each test case includes: running each test case using the processing node corresponding to each test case and the local memory of the processing node, based on the configuration file of each test case.
[0013] In one implementation, determining the performance score of the processor to be evaluated based on the test scores of each of the test cases includes: calculating the geometric mean of the test scores of each of the test cases to obtain the performance score of the processor to be evaluated.
[0014] In one implementation, the test case set includes the Standard Performance Evaluation Corporation's CPU test case set.
[0015] Secondly, embodiments of the present invention also provide a processor performance evaluation apparatus, the apparatus comprising: an acquisition unit, configured to acquire architecture information of a processor to be evaluated, the architecture information including processing nodes included in the processor to be evaluated and processing cores included in each processing node; wherein the processor to be evaluated is a processor based on a non-consistent memory access architecture, and the memory corresponding to each processing node is different from each other; a partitioning unit, configured to partition each test case in a test case set into at least two test case groups according to the architecture information, each processing node corresponding to each test case group; a running unit, configured to run the test case set based on the correspondence between each processing node and each test case group, and obtain a test score corresponding to each test case, wherein each processing node is used to run its own corresponding test case group; and a determining unit, configured to determine the performance score of the processor to be evaluated based on the test scores of each test case.
[0016] In one embodiment, the partitioning unit is specifically used to: determine the average runtime of each test case based on the historical test information of each test case in the test case set; and divide each test case in the test case set into at least two test case groups based on the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, such that the absolute value of the difference between the group runtimes of any two test case groups is less than a preset threshold, wherein the group runtime of the test case group is equal to the sum of the average runtimes of each test case in the test case group.
[0017] In one embodiment, the running unit includes: a determining module, configured to determine the processing node corresponding to each test case in the test case set according to the correspondence between each processing node and each test case group; a binding module, configured to bind the test case to the corresponding processing node through the configuration file of each test case; and a running module, configured to run each test case based on the configuration file of each test case to obtain the test score corresponding to each test case.
[0018] In one implementation, the binding module is specifically used to: obtain the core number of the processing core contained in each processing node; and bind the test case to the corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case.
[0019] In one implementation, each processing node includes N processing cores, where N is an integer greater than 1; the binding module is specifically used to: insert N core binding instructions into the configuration file of each test case to bind N test case copies to the N processing cores in a one-to-one correspondence; the running module is specifically used to: based on the configuration file of each test case, run each test case copy of the test case in parallel on the N processing cores corresponding to the test case to obtain the test score corresponding to the test case.
[0020] In one implementation, the running module is specifically used to: each processing node runs each test case in a preset order based on its own configuration file for each test case.
[0021] In one implementation, in each of the test cases that are run sequentially, the next test case is run after the previous test case has finished running and a preset time has elapsed.
[0022] In one implementation, the running module is specifically used to: run each test case based on the configuration file of each test case, using the processing node corresponding to each test case and the local memory of the processing node.
[0023] In one implementation, the determining unit is specifically used to: calculate the geometric mean of the test scores of each of the test cases to obtain the performance score of the processor to be evaluated.
[0024] In one implementation, the test case set includes the Standard Performance Evaluation Corporation's CPU test case set.
[0025] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising: a processor and a memory, the processor being electrically connected to the memory; the memory being used to store executable program code; the processor running a program corresponding to the executable program code by reading the executable program code stored in the memory, thereby implementing the processor performance evaluation method provided by any of the embodiments of the present invention.
[0026] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing one or more programs that can be executed by one or more processors to implement any of the processor performance evaluation methods provided in the embodiments of the present invention.
[0027] The processor performance evaluation method, apparatus, electronic device, and storage medium provided in the embodiments of the present invention can acquire the architecture information of the processor to be evaluated, divide each test case in the test case set into at least two test case groups according to the architecture information, and each processing node corresponds one-to-one with each test case group. Based on the correspondence between each processing node and each test case group, the test case set is run to obtain the test score corresponding to each test case, and the performance score of the processor to be evaluated is determined according to the test scores of each test case. Since the processor to be evaluated is a processor based on a non-uniform memory access architecture, the memory corresponding to each processing node is different. By dividing multiple test cases into test case groups and distributing them to different processing nodes, each processing node can make full use of its own computing and storage resources to process multiple test cases in parallel, thereby significantly shortening the testing time and effectively improving the efficiency of performance evaluation. Attached Figure Description
[0028] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0029] Figure 1 A flowchart of a processor performance evaluation method provided for embodiments of the present invention; Figure 2 A detailed flowchart of a processor performance evaluation method provided for embodiments of the present invention; Figure 3 A schematic diagram of a processor performance evaluation device provided in an embodiment of the present invention; Figure 4 A schematic diagram of an electronic device provided as an embodiment of the present invention. Detailed Implementation
[0030] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.
[0031] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.
[0032] In a first aspect, embodiments of the present invention provide a processor performance evaluation method that can improve performance evaluation efficiency.
[0033] like Figure 1 As shown, an embodiment of the present invention provides a processor performance evaluation method, which may include: S11. Obtain the architecture information of the processor to be evaluated. The architecture information includes the processing nodes contained in the processor to be evaluated and the processing cores contained in each processing node. The processor to be evaluated is a processor based on a non-uniform memory access architecture, and the memory corresponding to each processing node is different from each other.
[0034] In this step, the architecture information of the processor to be evaluated can be obtained. Processor architecture information refers to a hardware topology. A processing node can refer to a physical or logical unit in a Non-Uniform Memory Access (NUMA) architecture that has a local memory controller and an independent memory domain, such as a CPU (Central Processing Unit) die. The processing cores contained within each processing node can refer to physical or logical computing units within that processing node that can execute instructions relatively independently.
[0035] In embodiments of the present invention, by calling operating system commands or hardware test sets, information such as the number of all NUMA nodes in the system, the logical cores contained in each node, and the local physical memory address range corresponding to each node can be parsed.
[0036] S12. Based on the architecture information, divide each test case in the test case set into at least two test case groups, and each processing node corresponds one-to-one with each test case group.
[0037] Here, the test case set can refer to a benchmark test suite used to measure the comprehensive performance of the processor. The test case set can include multiple test cases. In this step, based on the architecture information obtained in step S11, the test cases in the test case set can be grouped so that each processing node in the processor to be evaluated corresponds one-to-one with a test case group. For example, in one example, the test case set includes 10 test cases, and the processor to be evaluated includes 4 processing nodes. These 10 test cases can then be divided into 4 test case groups, for example: (Test Case 1, Test Case 3), (Test Case 2, Test Case 8), (Test Case 4, Test Case 5, Test Case 9), and (Test Case 6, Test Case 7, Test Case 10).
[0038] S13. Based on the correspondence between each processing node and each test case group, run the test case set to obtain the test score corresponding to each test case, wherein each processing node is used to run its own corresponding test case group.
[0039] In this step, test case sets can be run and test scores obtained. Specifically, a one-to-one correspondence can be used to run each test case group independently and concurrently on its bound processing node. After each test case group is executed, its test score can be recorded. In embodiments of the present invention, the test scores of the test cases can be used to evaluate the running speed of the test cases on the processor; the faster the running speed, the higher the test score.
[0040] S14. Determine the performance score of the processor to be evaluated based on the test scores of each test case.
[0041] In this step, a performance score for the processor to be evaluated can be determined. For example, a performance score representing the overall performance of the processor can be calculated based on the test scores of each test case using specific statistical algorithms, such as average or weighted average.
[0042] The processor performance evaluation method provided by embodiments of the present invention can obtain the architecture information of the processor to be evaluated, divide each test case in the test case set into at least two test case groups according to the architecture information, and each processing node corresponds one-to-one with each test case group. Based on the correspondence between each processing node and each test case group, the test case set is run to obtain the test score corresponding to each test case, and the performance score of the processor to be evaluated is determined according to the test scores of each test case. Since the processor to be evaluated is a processor based on a non-consistent memory access architecture, the memory corresponding to each processing node is different. By dividing multiple test cases into test case groups and distributing them to different processing nodes, each processing node can make full use of its own computing and storage resources to process multiple test cases in parallel, thereby significantly shortening the testing time and effectively improving the efficiency of performance evaluation.
[0043] Specifically, in one embodiment of the present invention, the test case set can adopt the Standard Performance Evaluation Corporation's CPU Test Case Set (SPEC CPU). For example, SPEC CPU2017 can be used as the test case set for performance evaluation. Specifically, SPEC CPU 2017 can include subsets of intRate (single-threaded integer throughput test) and fprate (multi-threaded floating-point throughput test). The test case set can include multiple test cases; for example, the intRate test case set can include 10 test cases, and the fprate test case set can include 13 test cases.
[0044] After determining the test case set, embodiments of the present invention can obtain the architecture information of the processor to be evaluated according to step S11 and divide it into multiple test case groups. For example, in one embodiment, dividing each test case in the test case set into at least two test case groups according to the architecture information may include: determining the average runtime of each test case based on the historical test information of each test case in the test case set; dividing each test case in the test case set into at least two test case groups according to the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, such that the absolute value of the difference between the group runtimes of any two test case groups is less than a preset threshold, wherein the group runtime of the test case group is equal to the sum of the average runtimes of each test case in the test case group.
[0045] In this embodiment, the test case groups can be divided using a load balancing strategy. Based on the historical runtime or resource consumption of each test case, the total load is distributed approximately equally among the test case groups. This transforms the originally serially executed test tasks into a set of subtasks that can be executed in parallel on multiple processing nodes, thereby accelerating testing. For example, the average runtime of each sub-application (such as 500.perlbench, 502.gcc, etc.) can be obtained based on historical test data. As another example, in a 4-processor-node system, to ensure that the total expected runtime of each test case group is as close as possible, thus guaranteeing that each node can complete computation simultaneously, the 10 intrate sub-applications can be divided into 4 groups. The absolute value threshold of the difference in the runtime between the test case groups is set to either 4 minutes or 10 minutes.
[0046] Specifically, after dividing the test case set into at least two test case groups in step S12, the test case set can be run to obtain a test score for each test case. For example, in one implementation, running the test case set based on the correspondence between each processing node and each test case group to obtain a test score for each test case may include: determining the processing node corresponding to each test case in the test case set according to the correspondence between each processing node and each test case group; binding the test case to the corresponding processing node using the configuration file of each test case; and running each test case based on the configuration file of each test case to obtain a test score for each test case.
[0047] For example, by executing the `lscpu` command, the system is identified as having four processing nodes (Node0, Node1, Node2, and Node3). Simultaneously, the range of CPU core numbers for each node is obtained: Node0: cores 0-15, Node1: cores 16-31, and so on. Each test case can generate a configuration file, binding the test case to a processing node. For instance, test case 500 can be bound to processing node Node0 in its configuration file, and test case 541 can be bound to processing node Node0 in its configuration file, thus binding both test cases 500 and 541 to Node0. Similarly, test cases 502 and 557 can be bound to processing node Node1 in their respective configuration files. In this embodiment, the binding between test cases and processing nodes allows for isolation, enabling test cases to run on the processing node and fully utilizing the parallel potential of the NUMA architecture, thereby improving the efficiency of performance evaluation.
[0048] Understandably, in a NUMA architecture, each DIE is equivalent to a small server, and its performance can be approximated as 1 / N of the overall system performance (where N is the total number of nodes), with each processing node exhibiting essentially the same performance. In this invention, the number of test case replicas X is divided into N parts, allowing each node to run X / N replicas simultaneously. From the perspective of the entire system, the parallel acceleration ratio can reach approximately N times. For example, in a dual-socket server (containing two physical CPUs), each physical CPU typically has 4 physical DIEs, plus another CPU of the same model, resulting in a total of 8 processing nodes (which can be understood as 8 sets of independently allocable L3 cache and local memory resources). In this case, if each processing node runs one replica in parallel, theoretically, an 8x speedup can be achieved. Considering the potential for slight delays due to memory bandwidth or data aggregation in the last few rounds, the actual overall speedup effect can be approximately 4 to 6 times. Furthermore, for a four-socket server (containing four physical CPUs), the number of physical DIEs is very likely to reach 16 or more. Provided that the number of replicas per node matches the memory / cache resources, the speedup ratio can be increased even further.
[0049] Specifically, in one implementation, running each test case based on its configuration file may include: using the processing node corresponding to each test case and the local memory of that processing node to run each test case, based on the configuration file of each test case. In this embodiment, the binding between test cases and processing nodes avoids switching or cross-node access in a multi-core / multi-node environment, thereby achieving more stable and efficient performance evaluation.
[0050] In practical implementation, test cases can be bound by obtaining the core number of the processing core contained in the processing node. For example, in one embodiment, binding the test case to the corresponding processing node through the configuration file of each test case may include: obtaining the core number of the processing core contained in each processing node; and binding the test case to the corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case. Alternatively, operating system tools (such as numactl, taskset) can be used to bind a process or thread to a specific CPU core (and its corresponding local memory) to avoid switching or cross-node access in a multi-core / multi-node environment.
[0051] In specific implementation, when each processing node includes N processing cores, where N is an integer greater than 1, for example, in one embodiment, binding the test case to the corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case may include: inserting N core binding instructions in the configuration file of each test case to bind the N test case copies to the N processing cores in a one-to-one correspondence.
[0052] Accordingly, the step of running each test case based on the configuration file of each test case to obtain the test score corresponding to each test case may include: running each test case copy of the test case in parallel on the N processing cores corresponding to the test case based on the configuration file of each test case to obtain the test score corresponding to the test case.
[0053] Specifically, SPEC replicas can be fixed to the same processing node and processor core using methods such as `numactl --physcpubind`, `taskset`, or `cgroups cpuset`. For example, when the CPU core number of Node0 is 0-15, the following 16 lines of commands can be automatically generated and appended to the end of the configuration file 500.cfg for test case 500: bind0 = numactl --localalloc --physcpubind=0 bind1 = numactl --localalloc --physcpubind=1 ... bind15 = numactl --localalloc --physcpubind=15 When configuring the configuration file, you can create a numactl -C binding process to a specified core. For example, you can append the following command to the configuration file: numactl -C 0 . / exe numactl -C 1 . / exe numactl -C 2 . / exe numactl -C 3 . / exe ... numactl -C 15 . / exe In this way, `numactl -C` binds the process to the specified CPU core, and `--localalloc` ensures that all memory allocations come from the local memory of Node0. When running test case 500, the first replica will use the `bind0` command to bind to CPU core 0, the second replica will use the `bind1` command to bind to CPU core 1, and so on. In the embodiment provided by this invention, each parallel replica only accesses its own physical core and its local memory during execution, and will not compete with other replicas for cache or remote memory across processing nodes. This reduces or avoids competition for all core and memory resources on the same machine, thereby ensuring mutual isolation and stability when multiple replicas run simultaneously.
[0054] In specific implementation, for example, in one embodiment, running each test case based on its configuration file to obtain the test score corresponding to each test case may include: each processing node running each test case sequentially in a preset order based on its own configuration file for each test case. For example, in this embodiment of the invention, after completing intrate, the cache can be cleared and fprate test can be performed to obtain a complete overall floating-point throughput evaluation result.
[0055] For example, in one implementation, in each of the sequentially run test cases, the next test case runs only after the previous test case has finished running and a preset time has elapsed. In this embodiment, waiting a preset time after the end of one implementation before running the next test case avoids excessive resource contention caused by starting too many subprocesses at once.
[0056] Specifically, commands can be separated by "&&sleep 30s&&" to construct parallel execution commands. In this way, for each processing node and its assigned sub-applications, the script will concatenate the run commands (runcpu) of these sub-applications into a single compound command. The concatenated command will be written to a temporary script file and executed in the background, thereby running multiple core-bound replicas in parallel on the same node according to the order of the sub-applications.
[0057] In this embodiment, after running the test case set in step S13 and obtaining the test score corresponding to each test case, the performance score of the processor to be evaluated can be determined. For example, in one implementation, determining the performance score of the processor to be evaluated based on the test scores of each test case may include: calculating the geometric mean of the test scores of each test case to obtain the performance score of the processor to be evaluated.
[0058] In practice, a performance score representing the overall performance of the entire machine can be calculated using specific statistical algorithms. For tests that measure throughput, such as SPEC CPU Rate, the geometric mean method can be used. The geometric mean method first multiplies all test scores and then takes the Mth root of the product (where M is the total number of test cases). In this embodiment of the invention, calculating the performance score of the processor to be evaluated using the geometric mean method can effectively balance the large differences between the scores of each test case, prevent individual extreme values from having an excessive impact on the overall result, and thus obtain a more representative and comparable comprehensive performance index that accurately reflects the overall throughput capability of the processor under multi-task parallel processing. For example, the test scores of sub-application names (such as 500.int) can be extracted sequentially, and the geometric mean of the test scores of all test cases can be calculated to obtain the performance score of the processor to be evaluated, which is then saved in the intrate_score_avg file as the intrate evaluation result of this processor performance evaluation.
[0059] The processor performance evaluation method provided by the embodiments of the present invention will be described in detail below through a specific example.
[0060] like Figure 2 As shown, the processor performance evaluation method provided in the embodiments of the present invention may include: S501. Obtain the architecture information of the processor to be evaluated. The architecture information includes the processing nodes contained in the processor to be evaluated and the processing cores contained in each processing node. The processor to be evaluated is a processor based on a non-uniform memory access architecture, and the memory corresponding to each processing node is different from each other. S502. Based on the historical test information of each test case in the test case set, determine the average runtime of each test case; S503. Based on the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, divide each test case in the test case set into at least two test case groups, so that the absolute value of the difference between the group runtimes of any two test case groups is less than a preset threshold, wherein the group runtime of a test case group is equal to the sum of the average runtimes of each test case in that test case group. S504. Based on the correspondence between each processing node and each test case group, determine the processing node corresponding to each test case in the test case set; S505. Obtain the core number of the processing core contained in each processing node; S506. Insert N core binding instructions into the configuration file of each test case to bind the N test case copies to the N processing cores one by one. S507. Based on the configuration file of each test case, run each test case copy of the test case in parallel on the N processing cores corresponding to the test case to obtain the test score corresponding to each test case. S508. Calculate the geometric mean of the test scores for each test case to obtain the performance score of the processor to be evaluated.
[0061] Secondly, embodiments of the present invention provide a processor performance evaluation apparatus that can improve performance evaluation efficiency.
[0062] like Figure 3 As shown, an embodiment of the present invention provides a processor performance evaluation apparatus, which may include: The acquisition unit 31 is used to acquire the architecture information of the processor to be evaluated. The architecture information includes the processing nodes contained in the processor to be evaluated and the processing cores contained in each processing node. The processor to be evaluated is a processor based on a non-uniform memory access architecture, and the memory corresponding to each processing node is different from each other. The partitioning unit 32 is used to divide each test case in the test case set into at least two test case groups according to the architecture information, and each processing node corresponds one-to-one with each test case group; The running unit 33 is used to run the test case set based on the correspondence between each processing node and each test case group to obtain the test score corresponding to each test case, wherein each processing node is used to run the test case group corresponding to itself; The determining unit 34 is used to determine the performance score of the processor to be evaluated based on the test scores of each of the test cases.
[0063] The processor performance evaluation apparatus provided in the embodiments of the present invention can acquire the architecture information of the processor to be evaluated, divide each test case in the test case set into at least two test case groups according to the architecture information, and each processing node corresponds one-to-one with each test case group. Based on the correspondence between each processing node and each test case group, the test case set is run to obtain the test score corresponding to each test case, and the performance score of the processor to be evaluated is determined according to the test scores of each test case. Since the processor to be evaluated is a processor based on a non-consistent memory access architecture, the memory corresponding to each processing node is different from each other. By dividing multiple test cases into test case groups and distributing them to different processing nodes, each processing node can make full use of its own computing and storage resources and process multiple test cases in parallel, thereby significantly shortening the testing time and effectively improving the performance evaluation efficiency.
[0064] In one embodiment, the partitioning unit 32 is specifically used to: determine the average runtime of each test case based on the historical test information of each test case in the test case set; and divide each test case in the test case set into at least two test case groups based on the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, such that the absolute value of the difference between the group runtimes of any two test case groups is less than a preset threshold, wherein the group runtime of the test case group is equal to the sum of the average runtimes of each test case in the test case group.
[0065] In one embodiment, the running unit 33 includes: a determining module, configured to determine the processing node corresponding to each test case in the test case set according to the correspondence between each processing node and each test case group; a binding module, configured to bind the test case to the corresponding processing node through the configuration file of each test case; and a running module, configured to run each test case based on the configuration file of each test case to obtain the test score corresponding to each test case.
[0066] In one implementation, the binding module is specifically used to: obtain the core number of the processing core contained in each processing node; and bind the test case to the corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case.
[0067] In one implementation, each processing node includes N processing cores, where N is an integer greater than 1; the binding module is specifically used to: insert N core binding instructions into the configuration file of each test case to bind N test case copies to the N processing cores in a one-to-one correspondence; the running module is specifically used to: based on the configuration file of each test case, run each test case copy of the test case in parallel on the N processing cores corresponding to the test case to obtain the test score corresponding to the test case.
[0068] In one implementation, the running module is specifically used to: each processing node runs each test case in a preset order based on its own configuration file for each test case.
[0069] In one implementation, in each of the test cases that are run sequentially, the next test case is run after the previous test case has finished running and a preset time has elapsed.
[0070] In one implementation, the running module is specifically used to: run each test case based on the configuration file of each test case, using the processing node corresponding to each test case and the local memory of the processing node.
[0071] In one implementation, the determining unit 34 is specifically used to: calculate the geometric mean of the test scores of each of the test cases to obtain the performance score of the processor to be evaluated.
[0072] In one implementation, the test case set includes the Standard Performance Evaluation Corporation's CPU test case set.
[0073] Thirdly, embodiments of the present invention also provide an electronic device that can improve performance evaluation efficiency.
[0074] like Figure 4 As shown, the electronic device provided in the embodiments of the present invention may include: a processor 71 and a memory 72, wherein the processor 71 and the memory 72 are electrically connected; the memory 72 is used to store executable program code; the processor 71 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 72, so as to implement any of the processor performance evaluation methods provided in the foregoing embodiments.
[0075] The specific execution process of the above steps by the processor 71, as well as the steps further executed by the processor 71 by running executable program code, can be found in the description of the foregoing embodiments, and will not be repeated here.
[0076] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the performance evaluation method of any of the processors provided in the foregoing embodiments, thus achieving the corresponding technical effects. This has been described in detail above and will not be repeated here.
[0077] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0078] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.
[0079] In particular, the device embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.
[0080] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this invention, the functions of each unit / module can be implemented in one or more software and / or hardware.
[0081] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0082] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.
Claims
1. A method for evaluating the performance of a processor, characterized in that, include: Obtain the architecture information of the processor to be evaluated, which includes the processing nodes contained in the processor to be evaluated and the processing cores contained in each processing node; wherein, the processor to be evaluated is a processor based on a non-uniform memory access architecture, and the memory corresponding to each processing node is different from each other; Based on the architecture information, each test case in the test case set is divided into at least two test case groups, and each processing node corresponds one-to-one with each test case group; Based on the correspondence between each processing node and each test case group, the test case set is run to obtain the test score corresponding to each test case, wherein each processing node is used to run its own corresponding test case group; The performance score of the processor to be evaluated is determined based on the test scores of each test case.
2. The processor performance evaluation method according to claim 1, characterized in that, The step of dividing each test case in the test case set into at least two test case groups based on the architecture information includes: Based on the historical test information of each test case in the test case set, determine the average runtime of each test case; Based on the number of processing nodes contained in the processor to be evaluated and the average runtime of each test case, each test case in the test case set is divided into at least two test case groups, such that the absolute value of the difference between the runtimes of any two test case groups is less than a preset threshold, wherein the runtime of a test case group is equal to the sum of the average runtimes of each test case in that test case group.
3. The processor performance evaluation method according to claim 1, characterized in that, The process of running the test case set based on the correspondence between each processing node and each test case group to obtain the test score for each test case includes: Based on the correspondence between each processing node and each test case group, determine the processing node corresponding to each test case in the test case set; The test case is bound to the corresponding processing node through the configuration file of each test case; Each test case is run based on its configuration file to obtain a test score for each test case.
4. The processor performance evaluation method according to claim 3, characterized in that, The step of binding the test case to the corresponding processing node through the configuration file of each test case includes: Obtain the core number of the processing core contained in each of the processing nodes; By specifying the core number of the processing core to which the test case is bound in the configuration file of each test case, the test case is bound to the corresponding processing node.
5. The processor performance evaluation method according to claim 4, characterized in that, Each of the processing nodes includes N processing cores, where N is an integer greater than 1; The step of binding a test case to a corresponding processing node by specifying the core number of the processing core to which the test case is bound in the configuration file of each test case includes: N core binding instructions are inserted into the configuration file of each test case to bind the N test case copies to the N processing cores in a one-to-one correspondence. The process of running each test case based on its configuration file to obtain a test score for each test case includes: Based on the configuration file of each test case, each copy of the test case is run in parallel on the N processing cores corresponding to the test case to obtain the test score corresponding to the test case.
6. The processor performance evaluation method according to claim 3, characterized in that, The process of running each test case based on its configuration file to obtain a test score for each test case includes: Each processing node runs each test case in a preset order based on its corresponding test case configuration file.
7. The processor performance evaluation method according to claim 6, characterized in that, In each of the test cases that are run sequentially, the next test case will run after the previous test case has finished running and a preset time has elapsed.
8. The processor performance evaluation method according to claim 3, characterized in that, Running each test case based on its configuration file includes: Based on the configuration file of each test case, each test case is run using the processing node corresponding to each test case and the local memory of the processing node.
9. The processor performance evaluation method according to claim 1, characterized in that, Determining the performance score of the processor to be evaluated based on the test scores of each of the test cases includes: The performance score of the processor to be evaluated is obtained by taking the geometric mean of the test scores of each test case.
10. The processor performance evaluation method according to claim 1, characterized in that, The test case set includes the CPU test case set from a standard performance evaluation company.
11. A processor performance evaluation device, characterized in that, include: An acquisition unit is used to acquire the architecture information of the processor to be evaluated. The architecture information includes the processing nodes contained in the processor to be evaluated and the processing cores contained in each processing node. The processor to be evaluated is a processor based on a non-uniform memory access architecture, and the memory corresponding to each processing node is different from each other. A partitioning unit is used to divide each test case in the test case set into at least two test case groups according to the architecture information, and each processing node corresponds one-to-one with each test case group; The running unit is used to run the test case set based on the correspondence between each processing node and each test case group, and to obtain the test score corresponding to each test case, wherein each processing node is used to run the test case group corresponding to itself. A determining unit is configured to determine the performance score of the processor to be evaluated based on the test scores of each of the test cases.
12. An electronic device, characterized in that, include: A processor and a memory, wherein the processor and the memory are electrically connected; The memory is used to store executable program code; The processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, so as to implement the processor performance evaluation method of any one of claims 1 to 10.
13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more programs, which can be executed by one or more processors to implement the performance evaluation method of the processor according to any one of claims 1 to 10.