Operator test method, device and equipment, medium and program product

Abstract

The invention relates to the technical field of computers, and discloses an operator testing method, device and equipment, a medium and a program product. The method comprises the steps that a test process corresponding to a to-be-measured sub-piece is acquired, a test calculation graph corresponding to the to-be-measured sub-piece is acquired according to the test process, and nodes of the test calculation graph are in one-to-one correspondence with test tasks; task scheduling is carried out according to the test calculation graph, execution units and execution types corresponding to all the test tasks are obtained, and all the test tasks are added to task queues matched with the corresponding execution units; and according to the execution type corresponding to each test task, executing each test task in the matched task queue through each execution unit, and obtaining a test result corresponding to the to-be-measured sub-task. According to the scheme of the embodiment, the execution of the test task can be optimized, the operator test efficiency can be improved, and heterogeneous hardware resources are fully utilized.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to an operator testing method, device, equipment, medium and program product. Background Art

[0002] Operator testing refers to the process of verifying and testing the performance of basic operations (operators) within an algorithm. It is widely used in deep learning frameworks, data processing systems, hardware acceleration, and other fields. With the development of AI hardware, the complexity of operator testing is also increasing. Especially in heterogeneous hardware environments, how to efficiently manage and schedule these heterogeneous hardware resources to achieve operator testing goals has become a critical issue.

[0003] Currently, existing operator testing methods typically execute tests serially, with significant dependencies between test phases. This results in underutilized hardware resources. For example, some existing testing frameworks only support testing on a single piece of hardware, or require each phase to complete before the next one can begin. This limits test parallelism and efficiency. Summary of the Invention

[0004] The present invention provides an operator testing method, apparatus, equipment, medium and program product, which can improve operator testing efficiency and fully utilize heterogeneous hardware resources.

[0005] According to one aspect of the present invention, there is provided an operator testing method, comprising:

[0006] Obtain the test process corresponding to the operator to be tested, and based on the test process, obtain the test calculation graph corresponding to the operator to be tested, where the nodes of the test calculation graph correspond one-to-one to the test tasks;

[0007] Perform task scheduling according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to the task queue matched with the corresponding execution unit;

[0008] According to the execution type corresponding to each test task, each test task in the matching task queue is executed by each execution unit to obtain the test result corresponding to the operator to be tested.

[0009] According to another aspect of the present invention, there is provided an operator testing device, comprising:

[0010] A calculation graph acquisition module is used to obtain the test process corresponding to the operator to be tested, and obtain the test calculation graph corresponding to the operator to be tested based on the test process, where the nodes of the test calculation graph correspond one-to-one to the test tasks;

[0011] A task scheduling module is used to schedule tasks according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to the task queue matched by the corresponding execution unit;

[0012] The task execution module is used to execute each test task in the matching task queue through each execution unit according to the execution type corresponding to each test task, and obtain the test result corresponding to the operator to be tested.

[0013] According to another aspect of the present invention, an electronic device is provided, comprising:

[0014] at least one processor; and

[0015] a memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the operator testing method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and the computer program is configured to enable a processor to implement the operator testing method according to any embodiment of the present invention when executed.

[0018] According to another aspect of the present invention, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the operator testing method according to any embodiment of the present invention.

[0019] The technical solution of the embodiment of the present invention obtains the test process corresponding to the operator to be tested, and obtains the test calculation graph corresponding to the operator to be tested based on the test process, and the nodes of the test calculation graph correspond to the test tasks one by one; performs task scheduling according to the test calculation graph, obtains the execution unit and execution type corresponding to each test task, and adds each test task to the task queue matched by the corresponding execution unit; according to the execution type corresponding to each test task, executes each test task in the matched task queue by each execution unit to obtain the test result corresponding to the operator to be tested; by converting the test process of the operator test into the form of a calculation graph, and adopting the scheduling method of the calculation graph to realize asynchronous parallel execution of test tasks and scheduling of heterogeneous hardware resources, the execution of test tasks can be optimized, the efficiency of operator testing can be improved, and heterogeneous hardware resources can be fully utilized.

[0020] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0022] Figure 1 This is a flowchart of an operator testing method provided according to the first embodiment of the present invention;

[0023] Figure 2 This is a flow chart of a traditional operator testing method provided according to the first embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a test calculation graph provided according to embodiment 1 of the present invention;

[0025] Figure 4 This is a flow chart of a traditional operator testing method provided according to the first embodiment of the present invention;

[0026] Figure 5 This is a flowchart of another operator testing method provided in Example 1 of the present invention;

[0027] Figure 6 is a schematic diagram of a task queue provided according to embodiment 1 of the present invention;

[0028] Figure 7 1 is a schematic diagram of pipeline execution during testing of multiple operator units according to the first embodiment of the present invention;

[0029] Figure 8 This is a flowchart of an operator testing method provided according to the second embodiment of the present invention;

[0030] Figure 9 This is a structural diagram of an operator testing device provided according to a third embodiment of the present invention;

[0031] Figure 10 It is a structural diagram of an electronic device for implementing the operator testing method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0033] It should be noted that the terms "first," "second," "target," etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or that are inherent to these processes, methods, products, or apparatus.

[0034] Example 1

[0035] Figure 1 A flowchart of an operator testing method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of testing an operator unit. The method can be executed by an operator testing device. The operator testing device can be implemented in the form of hardware and / or software. Typically, the operator testing device can be configured in an electronic device, such as a computer device or a server. Figure 1 As shown, the method includes:

[0036] S110 , obtaining a test process corresponding to the operator to be tested, and obtaining a test calculation graph corresponding to the operator to be tested based on the test process, wherein the nodes of the test calculation graph correspond to the test tasks in a one-to-one manner.

[0037] The test process can include test phases such as data generation, operator execution, result comparison, and evaluation. Each test phase can consist of multiple test tasks. The test process can be pre-configured by the tester. In this embodiment, a test configuration file can be obtained and parsed to obtain the attribute information of the operator to be tested and the test process. For example, the attribute information may include the operator name, operator type, etc.

[0038] It should be noted that the process of the traditional operator testing method can be as follows Figure 2As shown, each test task is executed serially, which has the defects of not being able to fully utilize heterogeneous hardware resources and being inefficient. In this embodiment, an efficient execution management mechanism is formed by converting the test process of the operator test into a calculation graph. A calculation graph is a data structure used to describe the calculation process of a mathematical expression. It is a directed acyclic graph composed of nodes and edges. Nodes represent variables or operations (such as matrix multiplication, addition, etc.), and edges represent data flows or computational dependencies. For example, the test calculation graph can be as follows Figure 3 As shown, each level corresponds to a test phase, and each node corresponds to a test task.

[0039] Optionally, according to the test process, obtaining a test calculation graph corresponding to the operator to be tested may include:

[0040] According to the test process, multiple test tasks corresponding to the operator to be tested and the dependency relationship between the test tasks are obtained;

[0041] Each node is generated according to each of the test tasks, and each of the nodes is connected according to the dependency relationship between the test tasks to obtain a test calculation graph corresponding to the operator to be tested.

[0042] Specifically, when constructing a test calculation graph, first, according to the test process, obtain multiple test phases and the test tasks contained in each test phase, and obtain the dependency relationship between the test tasks by analyzing whether the execution of each test task depends on the execution results of other test tasks. For example, the test phase is input data generation, and the test tasks it contains include data preprocessing, data set loading, etc.; or, the test phase is operator execution, and the test tasks it contains include baseline operator execution and operator execution to be tested. Baseline operator execution refers to the execution of standard operators based on the Central Processing Unit (CPU) or Graphics Processing Unit (GPU), and operator execution to be tested refers to the execution of the operator to be tested on an artificial intelligence (AI) chip, GPU or other dedicated hardware; or, the test phase is result comparison and evaluation, and the test tasks it contains include comparing the execution results of the baseline operator and the operator to be tested, and outputting a test report.

[0043] Each test phase is then converted into a layer, and each test task is converted into a node at the corresponding layer to obtain the initial computation graph. Furthermore, based on the dependencies between test tasks, if the current node is detected to depend on multiple nodes in the previous layer, the current node is connected to each node until all nodes are checked, resulting in a directed acyclic graph as the test computation graph corresponding to the operator to be tested.

[0044] The node attributes of each node can include computing resource type, input / output data, execution time, and task priority. Computing resource type refers to the type of hardware resources corresponding to each test task (such as CPU, GPU, AI chip, disk I / O, etc.); input / output data refers to the input and output data of each node; execution time refers to the estimated execution time of each node; task priority means that if there are multiple parallel tasks, the priority can be set to flexibly control the execution order.

[0045] In this embodiment, by converting the test process of the operator test into a computational graph, parallel scheduling and execution of the test tasks can be achieved.

[0046] S120: Perform task scheduling according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to a task queue matched with the corresponding execution unit.

[0047] It should be noted that the process of the traditional operator testing method is as follows Figure 4 As shown, it does not focus on the various heterogeneous hardware and systems involved in the test process or environment. For example, in the operator unit test, the hardware and systems involved include mechanical hard disks, solid-state hard disks, distributed storage systems, AI-specific chips, CPUs, GPUs, Ethernet, etc. Each test task is usually executed sequentially, that is, after one task is completed, the next task can be executed. Each test task generally relies on a single hardware resource to complete, resulting in unused hardware resources being idle when each test task is executed, and these resources are not fully utilized. To solve this problem, this embodiment uses the characteristics of the computational graph to support asynchronous parallel execution. The process of the operator testing method of this embodiment can be as follows Figure 5 As shown in the figure, compared with the traditional serial execution method, the execution time of a single operator test is significantly reduced.

[0048] Among them, the execution unit can be a hardware device that executes the test task; the execution type can include serial execution and parallel execution. In this embodiment, the execution unit corresponding to the test task matched by each node can be obtained based on the node attributes of each node in the test calculation graph; at the same time, the execution type corresponding to the test task matched by each node can be obtained by judging whether there is an edge between the nodes for connection based on the structure of the test calculation graph. Typically, if there is an edge connection between the current node and at least one other node in the previous level, the execution type corresponding to the test task matched by the current node is determined to be serial execution; and if there is no edge connection between the current node and all other nodes in the previous level, the execution type corresponding to the test task matched by the current node is determined to be parallel execution.

[0049] It is understandable that different nodes rely on different types of hardware resources. In this embodiment, the use of various heterogeneous hardware resources can be queued, that is, the use of each execution unit corresponds to a first-in-first-out task queue. In this way, multiple tasks can be issued to a certain execution unit at the same time, and multiple tasks are executed serially in a first-in-first-out manner. Different execution units correspond to different task queues, and each task queue is executed asynchronously, such as Figure 6 Therefore, after analyzing the test calculation graph, when the dependencies between multiple task nodes are relatively loose (i.e., multiple nodes need to run on different heterogeneous hardware), the batch test tasks can be disassembled and the disassembled tasks can be distributed to the task queues of the corresponding heterogeneous hardware resources for execution. Based on this, for large-scale operator testing, the following can be formed: Figure 7 The pipeline approach shown can fully utilize all hardware resources while achieving large-scale and efficient parallel processing.

[0050] In this embodiment, by asynchronously and parallelly executing tasks at different stages, the bottlenecks in traditional methods are avoided, the overall time for testing large batches of operator units can be significantly shortened, and the testing efficiency can be significantly improved.

[0051] Optionally, performing task scheduling according to the test computation graph and obtaining an execution unit and an execution type corresponding to each test task may include:

[0052] According to the test calculation graph, the hardware resource requirements corresponding to each of the nodes are obtained, and according to the hardware resource requirements corresponding to each of the nodes, the execution units matching the test tasks corresponding to each of the nodes are obtained;

[0053] According to the test calculation graph, the dependency relationship between the nodes is obtained, and according to the dependency relationship between the nodes, the execution type matching the test task corresponding to each of the nodes is obtained.

[0054] In this embodiment, the hardware resource type corresponding to each node can be obtained based on the node attributes of the test computation graph as the hardware resource requirement. Then, based on this hardware resource requirement and the preset mapping relationship between hardware resource types and execution units, the test task can be assigned to the most appropriate execution unit. For example, baseline operator execution can be assigned to the GPU to accelerate large-scale matrix operations; or, the execution of the operator to be tested can be assigned to the AI ​​chip to utilize hardware acceleration features; or, data loading can be assigned to disk I / O to improve efficiency by reading data blocks in parallel.

[0055] Optionally, when there are multiple execution units of a hardware resource type at the same time, for example, when there are multiple CPUs and GPUs, the test task is assigned to the most suitable execution unit based on the hardware resource requirements, which may include: obtaining multiple candidate execution units corresponding to the current node based on the hardware resource requirements corresponding to the current node and the hardware resource type corresponding to each execution unit; performing load balancing according to the load conditions of each candidate execution unit and the execution time corresponding to the current node, so as to screen the most suitable execution unit among the candidate execution units.

[0056] Secondly, this embodiment also supports dynamic adjustment of task execution order and resource allocation. For example, if a test task is detected to be unable to execute due to hardware failure or insufficient resources during execution, the test task is scheduled to another execution unit with the same hardware resource type as the current execution unit to ensure that the overall test process is not affected.

[0057] In this embodiment, by performing resource allocation and load balancing, overloading of certain hardware units or waste of resources can be avoided, and automatic scheduling and parallel execution of multiple heterogeneous hardware can be achieved.

[0058] Secondly, the dependencies between nodes can be determined by analyzing the structure of the test computation graph (edges between nodes). Alternatively, the dependencies between nodes can be determined by analyzing the input and output data of each node. Dependencies determine which tasks can be executed in parallel and which tasks must wait for other tasks to complete before executing. Then, based on the pre-set execution type classification rules and the dependencies between nodes, the execution type of the test task corresponding to each node can be determined.

[0059] Optionally, obtaining the execution type that matches the test task corresponding to each node according to the dependency relationship between the nodes may include:

[0060] If it is detected that there is a dependency relationship between the current node and at least one other node, then the execution type matching the test task corresponding to the current node is determined to be serial execution;

[0061] If it is detected that there is no dependency relationship between the current node and any other nodes, it is determined that the execution type matched by the test task corresponding to the current node is parallel execution.

[0062] If the execution of the current node depends on another node (e.g., the calculation of the current node requires the calculation result of another node as input), then it can be considered that there is a dependency relationship between the current node and the other node. In this embodiment, as long as it is detected that the execution of the current node depends on any other node, the corresponding execution type can be determined to be serial execution; if the execution of the current node does not depend on any other node, the corresponding execution type can be determined to be parallel execution.

[0063] In this embodiment, by analyzing the dependency relationship between nodes, the test task is divided into serial execution and parallel execution, which can improve the execution efficiency of the test task.

[0064] S130 , according to the execution type corresponding to each test task, execute each test task in the matching task queue through each execution unit to obtain the test result corresponding to the operator to be tested.

[0065] Specifically, each execution unit can read test tasks from the matching task queue in sequence according to the first-in, first-out rule. It can then determine whether the currently read test task can be executed immediately based on its corresponding execution type. For example, if the execution type is determined to be parallel execution, it can be executed immediately. If the execution type is determined to be serial execution, it must wait until all dependent test tasks have completed execution before the currently read test task can be executed. Thus, after all test tasks corresponding to the operator to be tested have completed execution, the final test results and test report can be obtained.

[0066] In this embodiment, through the scheduling mechanism based on the computational graph, the parallel computing capabilities of heterogeneous hardware are fully utilized, the idle time of hardware resources is reduced, and the hardware utilization rate during the test process is improved.

[0067] The technical solution of the embodiment of the present invention obtains the test process corresponding to the operator to be tested, and obtains the test calculation graph corresponding to the operator to be tested based on the test process, and the nodes of the test calculation graph correspond to the test tasks one by one; performs task scheduling according to the test calculation graph, obtains the execution unit and execution type corresponding to each test task, and adds each test task to the task queue matched by the corresponding execution unit; according to the execution type corresponding to each test task, executes each test task in the matched task queue by each execution unit to obtain the test result corresponding to the operator to be tested; by converting the test process of the operator test into the form of a calculation graph, and adopting the scheduling method of the calculation graph to realize asynchronous parallel execution of test tasks and scheduling of heterogeneous hardware resources, the execution of test tasks can be optimized, the efficiency of operator testing can be improved, and heterogeneous hardware resources can be fully utilized.

[0068] Example 2

[0069] Figure 8 This is a flowchart of an operator testing method provided by Example 2 of the present invention. This embodiment is a further refinement of the above technical solution. The technical solution in this embodiment can be combined with one or more of the above implementation methods. Figure 8 As shown, the method includes:

[0070] S210 , obtaining a test process corresponding to the operator to be tested, and according to the test process, obtaining multiple test tasks corresponding to the operator to be tested, as well as dependency relationships between the test tasks.

[0071] S220 , generating nodes according to the test tasks, and connecting the nodes according to the dependency relationships between the test tasks, to obtain a test calculation graph corresponding to the operator to be tested.

[0072] S230. According to the test calculation graph, obtain the hardware resource requirements corresponding to each of the nodes, and according to the hardware resource requirements corresponding to each of the nodes, obtain the execution unit that matches the test task corresponding to each of the nodes.

[0073] S240: Obtain the dependency relationship between the nodes according to the test calculation graph, and obtain the execution type matched by the test task corresponding to each node according to the dependency relationship between the nodes.

[0074] S250: Add each of the test tasks to a task queue matched by a corresponding execution unit, and obtain a priority corresponding to each test task in a current task queue matched by a current execution unit.

[0075] In this embodiment, priority control can be performed when executing test tasks in the task queue, that is, the task queue is a priority queue. Typically, for test tasks with different priorities, their execution order can be adjusted according to the priority of the task. For example, some critical test tasks may need to be executed first, while other low-priority test tasks can be deferred.

[0076] Specifically, when a new test task enters the current task queue, the priority level of each test task in the current task queue is obtained based on the node attributes. Priorities can include high, medium, and low, and different priorities can be represented by different numerical values.

[0077] S260: Adjust the execution order of each test task in the current task queue according to the priority corresponding to each test task, and obtain the adjusted execution order corresponding to each test task in the current task queue.

[0078] Specifically, a preset sorting algorithm can be used to sort the test tasks in the current task queue in order of priority from high to low, and an adjusted execution order can be assigned to each test task based on the sorting result. In the task queue, a test task with a higher priority has a higher execution order. This embodiment does not specifically limit the type of sorting algorithm. Optionally, for test tasks with the same priority, a corresponding execution order can be assigned to them according to the first-in-first-out rule.

[0079] S270. According to the execution type and the adjusted execution order corresponding to each test task in the current task queue, each test task in the current task queue is executed by the current execution unit to obtain the task execution result corresponding to each test task, and according to the task execution result corresponding to each test task, the test result corresponding to the operator to be tested is obtained.

[0080] Specifically, based on the adjusted execution order, the test task with the highest execution order in the current task queue is read. Based on the execution type corresponding to the test task, a determination is made as to whether immediate execution is possible. If so, the test task is executed by the current execution unit to obtain the task execution result. Once all test tasks corresponding to the operator to be tested have been completed, the task execution result of the output test report task is obtained to obtain the final test report as the test result corresponding to the operator to be tested.

[0081] The technical solution of the embodiment of the present invention is to obtain the priority corresponding to each test task in the current task queue matched by the current execution unit after adding each test task to the task queue matched by the corresponding execution unit, adjust the execution order of each test task in the current task queue according to the priority corresponding to each test task, and obtain the adjusted execution order corresponding to each test task in the current task queue; execute each test task in the current task queue through the current execution unit according to the execution type and adjusted execution order corresponding to each test task in the current task queue, obtain the task execution result corresponding to each test task, and obtain the test result corresponding to the operator to be tested according to the task execution result corresponding to each test task; by performing priority control on the execution of test tasks, the flexibility and efficiency of test task execution can be improved, and different operator testing scenarios can be applied.

[0082] Optionally, executing each test task in the current task queue by the current execution unit according to the execution type and the adjusted execution order corresponding to each test task in the current task queue and obtaining the task execution result corresponding to each test task may include:

[0083] Obtain the current test task according to the adjusted execution order corresponding to each test task in the current task queue; if it is detected that the execution type corresponding to the current test task is parallel execution, directly execute the current test task through the current execution unit to obtain the task execution result corresponding to the current test task;

[0084] If it is detected that the execution type corresponding to the current test task is serial execution, it is determined whether all other test tasks on which the current test task depends have been completed. If so, the current test task is executed by the current execution unit to obtain the task execution result corresponding to the current test task.

[0085] Among them, the current test task can be the test task with the highest execution order in the current task queue. Specifically, if the execution type corresponding to the current test task is parallel execution, it means that its execution does not need to rely on any other test tasks, and it can be executed immediately through the current execution unit, and the task execution result can be obtained. In another case, if the execution type is serial execution, first determine whether the other test tasks it depends on have been completed. If so, it can be executed immediately; and if there are one or more other test tasks it depends on that have not yet been completed, then put the current test task back into the current task queue, and readjust the execution order of each test task in the current task queue, and execute each test task according to the execution type corresponding to each test task and the readjusted execution order. Alternatively, the current test task can be marked, and after a set interval, it can be re-determined whether the other test tasks it depends on have been completed. If not, the above process is repeated until it is detected that the other test tasks it depends on have been completed, and the current test task is executed by the current execution unit.

[0086] In this embodiment, efficient assistance and operation of various heterogeneous hardware resources in the test process are achieved based on the computation graph, thereby improving the efficiency and throughput of large-scale test case operation; by converting each step in the operator unit test into a node in the computation graph, and using the scheduling algorithm of the computation graph to achieve asynchronous parallel execution and scheduling of heterogeneous resources, the execution of tasks at each stage of the test process is optimized, significantly improving the test efficiency.

[0087] It should be noted that the technical solution of this embodiment can also be widely applied to other types of unit testing and performance testing scenarios, especially in scenarios that require parallel computing of multiple hardware resources. For example, it can be applied to performance testing of hardware accelerators, operator testing in big data processing, and functional verification of operating systems and middleware. During the development of hardware accelerators (such as AI chips, field programmable logic gate arrays, etc.), the solution of this embodiment can be used to test the execution performance of the basic operation units (operators) in the accelerator. Through computational graph scheduling, comparative tests can be performed on different hardware platforms (such as CPU, GPU, AI chips, etc.) to evaluate the hardware acceleration effect and optimize the computing efficiency of the accelerator.

[0088] Secondly, in big data processing frameworks (such as Hadoop, Spark, etc.), the processing of large-scale data sets is often involved, and different computing tasks may be involved at different stages. By using the computational graph scheduling method of this embodiment, the execution effect of each computing task on different hardware platforms (such as distributed computing nodes, memory, hard disk, etc.) can be efficiently tested, thereby helping to optimize the entire data processing pipeline. Finally, in the development process of operating systems or middleware, the scheduling and management of multiple hardware resources may be involved. By abstracting the various functional modules in the operating system (such as process scheduling, memory management, I / O processing, etc.) as nodes in the computational graph, the collaborative work and performance of the various modules of the system can be tested. Through the scheduling of the computational graph, the performance of the system in various hardware environments can be efficiently verified.

[0089] Example 3

[0090] Figure 9 This is a structural diagram of an operator testing device provided by the third embodiment of the present invention. Figure 9 As shown, the device includes: a calculation graph acquisition module 310, a task scheduling module 320 and a task execution module 330; wherein,

[0091] A computation graph acquisition module 310 is configured to acquire a test process corresponding to an operator to be tested, and based on the test process, acquire a test computation graph corresponding to the operator to be tested, wherein the nodes of the test computation graph correspond one-to-one to the test tasks.

[0092] The task scheduling module 320 is used to schedule tasks according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to the task queue matched with the corresponding execution unit;

[0093] The task execution module 330 is configured to execute each test task in the matching task queue through each execution unit according to the execution type corresponding to each test task, and obtain the test result corresponding to the operator to be tested.

[0094] The technical solution of the embodiment of the present invention obtains the test process corresponding to the operator to be tested, and obtains the test calculation graph corresponding to the operator to be tested based on the test process, and the nodes of the test calculation graph correspond to the test tasks one by one; performs task scheduling according to the test calculation graph, obtains the execution unit and execution type corresponding to each test task, and adds each test task to the task queue matched by the corresponding execution unit; according to the execution type corresponding to each test task, executes each test task in the matched task queue by each execution unit to obtain the test result corresponding to the operator to be tested; by converting the test process of the operator test into the form of a calculation graph, and adopting the scheduling method of the calculation graph to realize asynchronous parallel execution of test tasks and scheduling of heterogeneous hardware resources, the execution of test tasks can be optimized, the efficiency of operator testing can be improved, and heterogeneous hardware resources can be fully utilized.

[0095] Optionally, a computation graph acquisition module 310 is specifically configured to acquire, according to the test process, multiple test tasks corresponding to the operator to be tested, and dependencies between the test tasks;

[0096] Each node is generated according to each of the test tasks, and each of the nodes is connected according to the dependency relationship between the test tasks to obtain a test calculation graph corresponding to the operator to be tested.

[0097] Optionally, the task scheduling module 320 is specifically configured to obtain the hardware resource requirements corresponding to each of the nodes according to the test computation graph, and obtain the execution unit matching the test task corresponding to each of the nodes according to the hardware resource requirements corresponding to each of the nodes;

[0098] According to the test calculation graph, the dependency relationship between the nodes is obtained, and according to the dependency relationship between the nodes, the execution type matching the test task corresponding to each of the nodes is obtained.

[0099] Optionally, the task scheduling module 320 is specifically configured to determine that the execution type matched by the test task corresponding to the current node is serial execution if a dependency relationship is detected between the current node and at least one other node;

[0100] If it is detected that there is no dependency relationship between the current node and any other nodes, it is determined that the execution type matched by the test task corresponding to the current node is parallel execution.

[0101] Optionally, the task execution module 330 is specifically configured to obtain the priority corresponding to each test task in the current task queue matched by the current execution unit;

[0102] Adjust the execution order of each test task in the current task queue according to the priority corresponding to each test task, and obtain the adjusted execution order corresponding to each test task in the current task queue;

[0103] According to the execution type and the adjusted execution order corresponding to each test task in the current task queue, each test task in the current task queue is executed by the current execution unit to obtain the task execution result corresponding to each test task.

[0104] Optionally, the task execution module 330 is specifically configured to obtain the current test task according to the adjusted execution order corresponding to each test task in the current task queue, and if it is detected that the execution type corresponding to the current test task is parallel execution, directly execute the current test task through the current execution unit to obtain the task execution result corresponding to the current test task;

[0105] If it is detected that the execution type corresponding to the current test task is serial execution, it is determined whether all other test tasks on which the current test task depends have been completed. If so, the current test task is executed by the current execution unit to obtain the task execution result corresponding to the current test task.

[0106] The operator testing device provided in the embodiment of the present invention can execute the operator testing method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0107] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0108] Example 4

[0109] Figure 10 A schematic diagram of the structure of an electronic device 40 that can be used to implement an embodiment of the present invention is shown. The electronic device 40 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 40 can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0110] like Figure 10As shown, the electronic device 40 includes at least one processor 41, and a memory connected to the at least one processor 41, such as a read-only memory (ROM) 42, a random access memory (RAM) 43, etc., wherein the memory stores a computer program that can be executed by the at least one processor, and the processor 41 can perform various appropriate actions and processes according to the computer program stored in the read-only memory 42 or the computer program loaded from the storage unit 48 to the random access memory 43. Various programs and data required for the operation of the electronic device 40 can also be stored in the RAM 43. The processor 41, ROM 42 and RAM 43 are connected to each other via a bus 44. An input / output (I / O) interface 45 is also connected to the bus 44.

[0111] Multiple components in the electronic device 40 are connected to the I / O interface 45, including an input unit 46, such as a keyboard, a mouse, etc.; an output unit 47, such as various types of displays, speakers, etc.; a storage unit 48, such as a magnetic disk, an optical disk, etc.; and a communication unit 49, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0112] Processor 41 can be any general-purpose and / or specialized processing component with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a central processing unit, a graphics processing unit, various specialized artificial intelligence computing chips, various processors running machine learning model algorithms, a digital signal processor, and any suitable processor, controller, microcontroller, etc. Processor 41 executes the various methods and processes described above, such as the operator testing method.

[0113] In some embodiments, the operator testing method can be implemented as a computer program that is tangibly contained in a computer-readable storage medium, such as a storage unit 48. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 40 via the ROM 42 and / or the communication unit 49. When the computer program is loaded into the RAM 43 and executed by the processor 41, one or more steps of the operator testing method described above can be performed. Alternatively, in other embodiments, the processor 41 can be configured to perform the operator testing method in any other appropriate manner (e.g., by means of firmware).

[0114] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays, application specific integrated circuits, application specific standard products, system-on-a-chip systems, on-load programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0116] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory, a read-only memory, an erasable programmable read-only memory, an optical fiber, a portable compact disk read-only memory, an optical storage device, a magnetic storage device or any suitable combination of the foregoing.

[0117] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device 40 having: a display device (e.g., a cathode ray tube or a liquid crystal display) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or a trackball) through which the user can provide input to the electronic device 40. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0118] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks, wide area networks, blockchain networks, and the Internet.

[0119] A computing system may include clients and servers. The client and server are generally remote from each other and typically interact through a communication network. The client and server relationship arises through computer programs running on the respective computers and having a client-server relationship to each other. The server may be a cloud server.

[0120] This embodiment may also include a computer program product, which includes a computer program. When the computer program is executed by a processor, it implements the operator testing method provided by any embodiment of the present invention.

[0121] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0122] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. An operator testing method, characterized in that: include: Obtain the test process corresponding to the operator to be tested, and based on the test process, obtain the test calculation graph corresponding to the operator to be tested, where the nodes of the test calculation graph correspond one-to-one to the test tasks; Perform task scheduling according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to the task queue matched with the corresponding execution unit; According to the execution type corresponding to each test task, each test task in the matching task queue is executed by each execution unit to obtain the test result corresponding to the operator to be tested.

2. The method according to claim 1, characterized in that According to the test process, obtaining a test calculation graph corresponding to the operator to be tested includes: According to the test process, multiple test tasks corresponding to the operator to be tested and the dependency relationship between the test tasks are obtained; Each node is generated according to each of the test tasks, and each of the nodes is connected according to the dependency relationship between the test tasks to obtain a test calculation graph corresponding to the operator to be tested.

3. The method according to claim 1, characterized in that Performing task scheduling according to the test computation graph to obtain the execution unit and execution type corresponding to each test task includes: According to the test calculation graph, the hardware resource requirements corresponding to each of the nodes are obtained, and according to the hardware resource requirements corresponding to each of the nodes, the execution units matching the test tasks corresponding to each of the nodes are obtained; According to the test calculation graph, the dependency relationship between the nodes is obtained, and according to the dependency relationship between the nodes, the execution type matching the test task corresponding to each of the nodes is obtained.

4. The method according to claim 3, characterized in that According to the dependency relationship between the nodes, the execution type matching the test task corresponding to each node is obtained, including: If it is detected that there is a dependency relationship between the current node and at least one other node, then the execution type matching the test task corresponding to the current node is determined to be serial execution; If it is detected that there is no dependency relationship between the current node and any other nodes, it is determined that the execution type matched by the test task corresponding to the current node is parallel execution.

5. The method according to claim 1, wherein According to the execution type corresponding to each test task, each execution unit executes each test task in the matching task queue to obtain a test result corresponding to the operator to be tested, including: Get the priority of each test task in the current task queue matched by the current execution unit; Adjust the execution order of each test task in the current task queue according to the priority corresponding to each test task, and obtain the adjusted execution order corresponding to each test task in the current task queue; According to the execution type and the adjusted execution order corresponding to each test task in the current task queue, each test task in the current task queue is executed by the current execution unit to obtain the task execution result corresponding to each test task.

6. The method according to claim 5, characterized in that According to the execution type and the adjusted execution order corresponding to each test task in the current task queue, each test task in the current task queue is executed by the current execution unit, and the task execution result corresponding to each test task is obtained, including: Obtain the current test task according to the adjusted execution order corresponding to each test task in the current task queue; if it is detected that the execution type corresponding to the current test task is parallel execution, directly execute the current test task through the current execution unit to obtain the task execution result corresponding to the current test task; If it is detected that the execution type corresponding to the current test task is serial execution, it is determined whether all other test tasks on which the current test task depends have been completed. If so, the current test task is executed by the current execution unit to obtain the task execution result corresponding to the current test task.

7. An operator testing device, characterized in that: include: A calculation graph acquisition module is used to obtain the test process corresponding to the operator to be tested, and obtain the test calculation graph corresponding to the operator to be tested based on the test process, where the nodes of the test calculation graph correspond one-to-one to the test tasks; A task scheduling module is used to schedule tasks according to the test calculation graph, obtain the execution unit and execution type corresponding to each test task, and add each test task to the task queue matched by the corresponding execution unit; The task execution module is used to execute each test task in the matching task queue through each execution unit according to the execution type corresponding to each test task, and obtain the test result corresponding to the operator to be tested.

8. An electronic device, characterized in that: The electronic device comprises: at least one processor, and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor. The computer program is executed by the at least one processor to enable the at least one processor to perform the operator testing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and the computer program is configured to enable a processor to implement the operator testing method according to any one of claims 1 to 6 when the computer program is executed.

10. A computer program product, characterized in that The invention comprises a computer program, which implements the operator testing method according to any one of claims 1 to 6 when executed by a processor.

Images