Hardware testing method, system and equipment and storage medium

Abstract

The invention discloses a hardware testing method, system and device and a storage medium. The method comprises the steps that a sub-graph sequence of a corresponding calculation graph is generated according to a to-be-tested model or algorithm; the calculation graph is used for representing a data structure of a model or an algorithm; the calculation graph is composed of a plurality of nodes and edges, the nodes represent the calculation steps of the model or algorithm, and the edges represent the logic relation between the calculation steps; determining a sub-graph execution sequence and an execution position based on a calling sequence among the sub-graph sequences; according to the sub-graph execution sequence, the sub-graphs are input into execution positions corresponding to hardware for calculation, and hardware performance indexes are obtained. According to the method, the sub-graph sequence of the calculation graph is automatically generated from the model or the algorithm, and the execution sequence and the position are determined, so that the efficiency and the accuracy of the hardware performance test are improved, the test result is more reliable, and the comprehensive evaluation and optimization of the use of hardware resources are facilitated.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular to a hardware testing method, system, device and storage medium. Background Art

[0002] With the development of hardware technology, new generations of GPUs are emerging, and they are widely used in various artificial intelligence software. These intelligent hardware, especially GPUs, are designed for artificial intelligence algorithms, enabling AI software to achieve high-speed computing relative to traditional CPUs. GPUs support the calculation of deep neural networks through a variety of hardware units, but their performance and compatibility vary depending on different designs and compilers.

[0003] When evaluating the GPU's support for AI software, we must consider not only its computing speed and accuracy, but also its synergy with the compiler. Different GPU designs lead to differences in computing power and compatibility, which directly affect their adaptability to upper-level AI software. Although GPUs perform well in performing AI computing tasks, their performance does not exist in isolation, but rather works together with the CPU to allocate computing tasks based on the compiler's decisions.

[0004] Therefore, to fully evaluate the computing power of the GPU, we cannot rely solely on the calculation results, but need to go deep into the hardware level and consider the collaborative working mechanism of the GPU and CPU and the compiler's task allocation strategy. Summary of the invention

[0005] The main purpose of the present invention is to provide a hardware testing method, system, device and storage medium, which not only improves the efficiency and accuracy of hardware performance testing, but also makes the test results more reliable, and helps to comprehensively evaluate and optimize the use of hardware resources by automatically generating a subgraph sequence of a computational graph from a model or algorithm and determining the execution sequence and location.

[0006] In order to achieve the above objectives, the embodiments of the present application provide the following technical solutions:

[0007] According to a first aspect of an embodiment of the present application, a hardware testing method is provided, the method comprising:

[0008] Receiving a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested;

[0009] Generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps;

[0010] Determine the subgraph execution sequence and execution position based on the calling order between the subgraph sequences;

[0011] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index.

[0012] Optionally, generating a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested includes:

[0013] Generate an initial computational graph based on the computational graph compiler for the model or algorithm to be tested;

[0014] The initial computation graph is divided into corresponding subgraph sequences according to a segmentation rule, wherein each subgraph in the subgraph sequence represents a set of computation steps that can be computed in parallel.

[0015] Optionally, generating an initial computational graph based on a computational graph compiler for the model or algorithm to be tested includes:

[0016] Constructing a deep neural network according to the model or algorithm to be tested, determining each network layer of the deep neural network as a node in a computational graph; determining weights and biases between network layers as edges between nodes in the computational graph;

[0017] Through the computational graph compiler, all computational steps in the model or algorithm are integrated into an initial computational graph, and the initial computational graph represents all computational steps in the model or algorithm.

[0018] Optionally, determining the subgraph execution sequence and execution position based on the calling order between the subgraph sequences includes:

[0019] Determining data dependencies between subgraphs based on a calling order between the subgraph sequences;

[0020] The subgraph execution sequence and execution location are determined according to subgraph attributes and data dependencies between the subgraphs; the attributes include subgraphs without data dependencies and subgraphs executed only on the CPU, and the execution location includes the CPU and / or GPU.

[0021] Optionally, the hardware performance indicator includes a calculation graph coverage;

[0022] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including:

[0023] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the number of GPU subgraphs and / or the number of CPU subgraphs;

[0024] The computation graph coverage is obtained according to the ratio of the number of GPU subgraphs and / or the number of CPU subgraphs in the subgraph sequence of the initial computation graph, where the computation graph coverage includes GPU coverage and CPU coverage.

[0025] Optionally, the hardware performance indicator includes a proportion of computing time;

[0026] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including:

[0027] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; the calculation time includes GPU calculation time and / or CPU calculation time;

[0028] The calculation time proportion is obtained according to the ratio of the GPU calculation time and / or the CPU calculation time of all subgraphs in the total calculation time. The calculation time proportion includes the GPU calculation time proportion and the CPU calculation time proportion. The total calculation time is the total calculation time consumed by all subgraphs at the execution location of the hardware.

[0029] Optionally, the hardware performance indicator includes computing speed;

[0030] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including:

[0031] According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; the calculation time includes GPU calculation time and / or CPU calculation time;

[0032] The calculation speed is obtained according to the calculation workload of all subgraphs and the GPU calculation time and / or the CPU calculation time of all subgraphs, and the calculation speed includes the GPU calculation speed and the CPU calculation speed.

[0033] According to a second aspect of an embodiment of the present application, a hardware testing system is provided, the system comprising:

[0034] A message receiving module, used for receiving a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested;

[0035] A subgraph generation module, used to generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps;

[0036] An execution sequence module is used to determine the subgraph execution sequence and execution position based on the calling sequence between the subgraph sequences;

[0037] The indicator calculation module is used to input the subgraph into the execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance indicator.

[0038] According to a third aspect of an embodiment of the present application, an electronic device is provided, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method described in the first aspect above.

[0039] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, on which computer-readable instructions are stored. The computer-readable instructions can be executed by a processor to implement the method described in the first aspect above.

[0040] In summary, the embodiments of the present application provide a hardware testing method, system, device and storage medium, which receives a hardware testing instruction, and the hardware testing instruction carries a model or algorithm to be tested; generates a corresponding subgraph sequence of a calculation graph according to the model or algorithm to be tested; the calculation graph is used to characterize the data structure of the model or algorithm; the calculation graph is composed of a number of nodes and edges, the nodes represent the calculation steps of the model or algorithm, and the edges represent the logical relationship between the calculation steps; the subgraph execution sequence and execution position are determined based on the call sequence between the subgraph sequences; according to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain hardware performance indicators. By automatically generating a subgraph sequence of a calculation graph from a model or algorithm and determining the execution sequence and position, not only the efficiency and accuracy of hardware performance testing are improved, but also the test results are made more reliable, which helps to comprehensively evaluate and optimize the use of hardware resources. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0042] The structures, proportions, sizes, etc. illustrated in this specification are only used to match the contents disclosed in the specification so as to facilitate understanding and reading by persons familiar with the technology. They are not used to limit the conditions under which the present invention can be implemented, and therefore have no substantial technical significance. Any structural modification, change in proportion or adjustment of size shall still fall within the scope of the technical contents disclosed in the present invention without affecting the effects and purposes that can be achieved by the present invention.

[0043] Figure 1 A schematic diagram of a hardware testing method flow chart provided in an embodiment of the present application;

[0044] Figure 2 A hardware test logic diagram provided for an embodiment of the present application;

[0045] Figure 3 A block diagram of a hardware testing system provided in an embodiment of the present application;

[0046] Figure 4 A structural diagram of an electronic device provided in an embodiment of the present application is shown;

[0047] Figure 5 A diagram showing a computer-readable storage medium provided by an embodiment of the present application.

[0048] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0049] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0050] It should be noted that all directional indications in the embodiments of the present invention (such as up, down, left, right, front, back, etc.) are only used to explain the relative position relationship, movement status, etc. between the components under a certain specific posture (as shown in the accompanying drawings). If the specific posture changes, the directional indication will also change accordingly.

[0051] In addition, in the present invention, descriptions such as "first", "second", etc. are only used for descriptive purposes and cannot be understood as indicating or implying their relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0052] In the present invention, unless otherwise clearly specified and limited, the terms "connection", "fixation", etc. should be understood in a broad sense. For example, "fixation" can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0053] In addition, the technical solutions between the various embodiments of the present invention can be combined with each other, but it must be based on the fact that ordinary technicians in the field can implement it. When the combination of technical solutions is contradictory or cannot be implemented, it should be deemed that such combination of technical solutions does not exist and is not within the scope of protection required by the present invention.

[0054] Figure 1 A hardware testing method provided in an embodiment of the present application is shown, the method comprising:

[0055] Step 101: receiving a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested;

[0056] Step 102: Generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps;

[0057] Step 103: Determine the subgraph execution sequence and execution position based on the calling order between the subgraph sequences;

[0058] Step 104: According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index.

[0059] In a possible implementation, in step 102, generating a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested includes:

[0060] Generate an initial computational graph based on a computational graph compiler for the model or algorithm to be tested; generate a corresponding subgraph sequence from the initial computational graph according to a segmentation rule, wherein each subgraph in the subgraph sequence represents a set of computational steps that can be calculated in parallel.

[0061] In a possible implementation, generating an initial computation graph based on a computation graph compiler for the model or algorithm to be tested includes:

[0062] A deep neural network is constructed according to the model or algorithm to be tested, and each network layer of the deep neural network is determined as a node in a computational graph; weights and biases between network layers are determined as edges between nodes in the computational graph; and all computational steps in the model or algorithm are integrated into an initial computational graph through the computational graph compiler, wherein the initial computational graph represents all computational steps in the model or algorithm.

[0063] The embodiment of the present application aims to propose a method for evaluating the compatibility of intelligent hardware based on the underlying representation, so as to accurately measure the support performance of intelligent hardware for upper-level application software. The underlying representation of the core refers to the calculation graph generated by the hardware compilation process. As a data structure, the calculation graph can effectively describe complex calculation processes or algorithms. In the calculation graph, nodes (or "operations" or "operators") represent specific calculation steps, while edges represent data flow or control flow, which connect different operations and reveal the dependencies between operations. From the perspective of the programming framework, any computing task can be represented by a calculation graph.

[0064] In a possible implementation, in step 103, determining the subgraph execution sequence and execution position based on the calling order between the subgraph sequences includes:

[0065] The data dependency relationship between the subgraphs is determined based on the calling order between the subgraph sequences; the subgraph execution sequence and execution location are determined according to the subgraph attributes and the data dependency relationship between the subgraphs; the attributes include subgraphs without data dependencies and subgraphs that are executed only on the CPU, and the execution location includes the CPU and / or GPU.

[0066] In a possible implementation, in step 104, the hardware performance indicator includes a calculation graph coverage; the step of inputting the subgraph into the execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance indicator includes: according to the subgraph execution sequence, inputting the subgraph into the execution position corresponding to the hardware for calculation to obtain the number of GPU subgraphs and / or the number of CPU subgraphs; obtaining the calculation graph coverage according to the ratio of the number of GPU subgraphs and / or the number of CPU subgraphs in the subgraph sequence of the initial calculation graph, and the calculation graph coverage includes GPU coverage and CPU coverage.

[0067] In a possible implementation, in step 104, the hardware performance indicator includes a proportion of computing time; the subgraph is input into the execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance indicator, including: according to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; the calculation time includes GPU calculation time and / or CPU calculation time; the calculation time proportion is obtained according to the ratio of the GPU calculation time and / or CPU calculation time of all subgraphs in the total calculation time, the calculation time proportion includes the GPU calculation time proportion and the CPU calculation time proportion, and the total calculation time is the total calculation time of all subgraphs at the execution position of the hardware.

[0068] In a possible implementation, in step 104, the hardware performance indicator includes a calculation speed; the subgraph is input into an execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance indicator, including: according to the subgraph execution sequence, the subgraph is input into an execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; the calculation time includes GPU calculation time and / or CPU calculation time; the calculation speed is obtained according to the calculation workload of all subgraphs and the GPU calculation time of all subgraphs and / or CPU calculation time, and the calculation speed includes GPU calculation speed and CPU calculation speed.

[0069] The embodiment of the present application evaluates the compatibility of intelligent hardware with upper-level software by testing the execution efficiency of the calculation graph. The calculation graph can be decomposed into multiple subgraphs based on the calculation relationship, and assigning these subgraphs to the hardware for execution can achieve a high degree of parallel processing. In order to quantify the execution capability of the hardware, the embodiment of the present application introduces two key indicators: coverage ratio and time ratio. The coverage ratio refers to the percentage of subgraphs executed on the hardware to the entire calculation graph. The higher this indicator is, the stronger the overall execution capability of the hardware for the calculation graph. The time ratio refers to the ratio of GPU calculation time to all calculation time. The lower the time ratio, the stronger the computing power.

[0070] In summary, the embodiments of the present application provide a more accurate evaluation method to determine the actual support capability of hardware for upper-layer application software by analyzing the execution of intelligent hardware on the computational graph, including the coverage ratio and time ratio of the subgraph. This method helps developers and users select the intelligent hardware that best suits their needs and ensures that the software can run efficiently on the hardware.

[0071] Figure 2The overall flow diagram provided by the embodiment of the present application is shown, and an initial calculation graph is generated by the model. Based on the calculation graph, a subgraph set corresponding to the initial calculation graph is formed through a subgraph segmentation algorithm. According to the calling sequence of the subgraphs, the subgraphs are sent to the intelligent hardware from small to large for calculation to obtain the return results. The coverage ratio is calculated based on the coverage ratio of the subgraphs that can be calculated on the intelligent hardware to the initial calculation graph. The calculation speed of the intelligent hardware is calculated based on the time it takes for all subgraphs to be calculated. Specifically, it includes the following stages:

[0072] Phase 1: The model first generates an initial computational graph, which represents the structure of the entire computational task.

[0073] The initial computational graph is a complete computational graph generated by the computational graph compiler. This computational graph is generated by the compiler and contains all the computational processes. The computational graph is a core concept of deep neural networks. When building a neural network, we abstract each layer in the network as a node in the computational graph, and the weights and biases between layers represent the edges between nodes. In this way, the learning process of the neural network can be viewed as a series of computational steps on the computational graph. The computational graph is an intermediate representation of hardware calculations, and the computational graph can be executed on the hardware after being further interpreted by the hardware compiler. In fact, on intelligent hardware, the computational graph is a unified representation. The computational graph can be formed from an algorithm or generated from a model. Through the computational graph compiler, we generate a complete computational graph for all calculations.

[0074] Step 1-1: Build a neural network: When building a deep neural network, each layer of the network is considered as a node in the computational graph.

[0075] Step 1-2: Define nodes and edges: Abstract the weights and biases between network layers as edges between nodes in the computational graph, representing the flow of data and parameters.

[0076] Step 1-3: Generate the initial computational graph: Through the computational graph compiler, the entire neural network construction process is converted into a complete computational graph, which contains all the computational steps of the network.

[0077] Steps 1-4: Intermediate representation of the computation graph: The computation graph serves as an intermediate representation of hardware computation, mapping the learning process of the neural network into a series of operations on the computation graph.

[0078] Step 1-5: Computational Graph Compilation: The hardware compiler further interprets the computational graph and converts it into a form that can be executed on smart hardware.

[0079] Phase 2: Process the initial computational graph according to the subgraph segmentation algorithm to generate a subgraph set sequence corresponding to the initial computational graph. Each subgraph contains a set of operations that can be calculated in parallel.

[0080] In order to make the computation graph parallel, a complete computation graph is often divided into several subgraphs, and the subgraphs are put into batches on intelligent hardware for parallel computation. There are many bases for subgraph division, including:

[0081] (1) Maximum parallel partitioning: This algorithm maximizes the parallelism of the partitioned subgraphs based on the dependencies between the data and the estimated time for subgraph computation.

[0082] (2) Operator segmentation: Since intelligent hardware does not support all operators, operators that are not supported by intelligent hardware will be executed on the CPU, so this part of the graph is segmented.

[0083] Apply the maximum parallel splitting algorithm to analyze data dependencies and subgraph computation time estimates to ensure that the split subgraphs can achieve the maximum parallel computing potential. For operators not supported by intelligent hardware, split them into separate subgraphs, which will be executed on the CPU, while other subgraphs are executed on intelligent hardware. The initial computation graph is processed by the splitting algorithm to generate a sequence of subgraph sets, each of which contains a set of operations that can be computed in parallel.

[0084] The third stage: Determine the calling order and execution sequence between subgraph sequences through the subgraph dependency analysis algorithm.

[0085] According to the relationship between the subgraph input and output, the computational subgraphs are sorted in the order of execution, and the subgraphs that can be computed in parallel are put together and marked. The subgraphs on the CPU are also distinguished. According to the order of execution and the execution location, the subgraphs executed on the intelligent hardware and CPU are dropped into the corresponding GPU or CPU for calculation.

[0086] Step 3-1: Analyze the input and output of each subgraph and determine the data dependencies between them.

[0087] Step 3-2: Determine the execution order of the subgraphs based on the dependencies between the subgraphs.

[0088] Step 3-3: Identify and mark those subgraphs that can be computed in parallel without data dependencies.

[0089] Step 3-4: Identify those subgraphs that can only be executed on the CPU, usually because the smart hardware does not support certain specific operators.

[0090] Phase 4: Based on the results of subgraph dependency analysis, a subgraph execution sequence is determined to ensure the correctness and order of the calculation.

[0091] The subgraphs are assigned to the corresponding GPU or CPU according to their execution order and execution location.

[0092] Perform subgraph dependency analysis to determine which subgraphs can be calculated on the GPU and which need to be executed on the CPU.

[0093] Based on the results of subgraph dependency analysis, we can determine which subgraphs can be calculated on the GPU. These subgraphs are compared with the initial calculation graph according to the coverage relationship to obtain the calculation graph coverage index.

[0094] Phase 5: According to the determined subgraph execution sequence, the subgraphs are sent to the intelligent hardware one by one for calculation. The calculation of each subgraph can be performed in parallel to improve the overall calculation efficiency.

[0095] Phase 6: Calculate the coverage ratio of the subgraphs that can be executed on intelligent hardware to the initial computational graph.

[0096]

[0097] Compare the subgraphs that can be calculated on the GPU with the initial calculation graph, and calculate the coverage index, that is, the percentage of subgraphs tested on the intelligent hardware to the initial calculation graph. The same is true for the CPU. The higher the coverage ratio, the stronger the hardware's ability to execute the calculation graph.

[0098] Stage 7: Record the total time required to complete the calculation of all subgraphs, and calculate the calculation speed of the intelligent hardware based on this time.

[0099]

[0100] The percentage of the total computing time of all subgraphs scheduled to the intelligent hardware to the total computing time is calculated to evaluate the activity of the GPU in the entire computing process. The same is true for the CPU.

[0101]

[0102] The average computing time required to complete all subgraphs scheduled to the intelligent hardware is calculated to evaluate the computing speed of the GPU. The same is true for the CPU.

[0103] During the calculation process, the calculation time of the subgraph on the CPU and GPU is recorded respectively, and the CPU calculation speed and GPU calculation speed can be obtained respectively. By executing the calculation graph multiple times, recording the calculation time each time, and then calculating the average calculation time, a more accurate performance indicator can be obtained. The faster the calculation speed, the stronger the computing power of the hardware. Based on the coverage ratio and calculation speed, the compatibility and computing power of the intelligent hardware are comprehensively evaluated.

[0104] Phase 8: Collect the calculation results of all subgraphs, integrate them according to the predetermined order or dependency, and output the final calculation results.

[0105] Based on the calculated indicators, evaluate the intelligent hardware's ability to execute the computational graph, including its computational speed and participation in the entire computational process. If the coverage or computational speed does not meet expectations, analyze the possible reasons and make optimization suggestions, such as improving the subgraph segmentation algorithm, adjusting hardware resource allocation, or optimizing software scheduling strategies.

[0106] The method provided in the embodiment of the present application uses a computational graph to evaluate the hardware, which can fully reflect the hardware's ability to handle complex computing tasks. This method not only focuses on the computing speed, but also takes into account the compatibility and stability of the hardware when executing the computational graph, thereby providing a more accurate and comprehensive performance evaluation. By testing the entire graph and subgraphs of the computational graph separately, the embodiment of the present application can make full use of the parallel processing characteristics of the hardware. This method allows multiple subgraphs to be tested simultaneously, simulating the scenario of hardware processing multiple tasks in the real world, thereby more accurately evaluating the parallel computing capabilities and efficiency of the hardware.

[0107] Traditional hardware performance evaluation often relies on the test results of upper-layer application software, while the embodiments of the present application provide a method for directly evaluating hardware performance from the bottom layer, avoiding the deviation and influence that may be caused by upper-layer application software. This bottom-level evaluation method can reveal the true performance of the hardware when executing the calculation graph, providing a more direct reference for hardware selection and optimization.

[0108] In summary, the embodiment of the present application provides a hardware testing method, by receiving a hardware testing instruction, the hardware testing instruction carries a model or algorithm to be tested; generating a corresponding subgraph sequence of a computational graph according to the model or algorithm to be tested; the computational graph is used to characterize the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps; determining the subgraph execution sequence and execution position based on the calling sequence between the subgraph sequences; according to the subgraph execution sequence, inputting the subgraph into the execution position corresponding to the hardware for calculation to obtain hardware performance indicators. By automatically generating a subgraph sequence of a computational graph from a model or algorithm and determining the execution sequence and position, not only the efficiency and accuracy of hardware performance testing are improved, but also the test results are made more reliable, which helps to comprehensively evaluate and optimize the use of hardware resources.

[0109] Based on the same technical concept, the embodiment of the present application also provides a hardware testing system, such as Figure 3 As shown, the system comprises:

[0110] The message receiving module 301 is used to receive a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested;

[0111] A subgraph generation module 302 is used to generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps;

[0112] An execution sequence module 303, used to determine a subgraph execution sequence and execution position based on a calling sequence between subgraph sequences;

[0113] The index calculation module 304 is used to input the subgraph into the execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance index.

[0114] The present application also provides an electronic device corresponding to the method provided in the above embodiment. Figure 4 , which shows an electronic device diagram provided by some embodiments of the present application. The electronic device 20 may include: a processor 200, a memory 201, a bus 202 and a communication interface 203, wherein the processor 200, the communication interface 203 and the memory 201 are connected via the bus 202; the memory 201 stores a computer program that can be run on the processor 200, and the processor 200 executes the method provided by any of the aforementioned embodiments of the present application when running the computer program.

[0115] The memory 201 may include a high-speed random access memory (RAM), and may also include a non-volatile memory, such as at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one physical port (which may be wired or wireless), and the Internet, wide area network, local area network, metropolitan area network, etc. may be used.

[0116] The bus 202 may be an ISA bus, a PCI bus, or an EISA bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. The memory 201 is used to store programs, and the processor 200 executes the programs after receiving execution instructions. The method disclosed in any implementation of the aforementioned embodiment of the present application may be applied to the processor 200, or implemented by the processor 200.

[0117] The processor 200 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 200. The above processor 200 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 201, and the processor 200 reads the information in the memory 201 and completes the steps of the above method in combination with its hardware.

[0118] The electronic device provided in the embodiment of the present application and the method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, operated or implemented by them.

[0119] The present application also provides a computer-readable storage medium corresponding to the method provided in the above embodiment. Figure 5The computer-readable storage medium shown is a CD 30 on which a computer program (ie, a program product) is stored. When the computer program is run by a processor, the method provided in any of the aforementioned embodiments is executed.

[0120] It should be noted that examples of the computer-readable storage medium may also include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0121] The computer-readable storage medium provided in the above-mentioned embodiments of the present application and the method provided in the embodiments of the present application are based on the same inventive concept and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0122] It should be noted that the above embodiments illustrate the present application rather than limit the present application, and that those skilled in the art may design alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets should not be constructed as a limitation to the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of multiple such elements. The present application may be implemented by means of hardware including several different elements and by means of a suitably programmed computer. In a unit claim that lists several devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names.

[0123] The above is only a preferred specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

[0124] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. All equivalent structural changes made by using the contents of the present invention specification and drawings under the concept of the present invention, or directly / indirectly applied in other related technical fields are included in the patent protection scope of the present invention.

Claims

1. A hardware testing method, characterized in that: The method comprises: Receiving a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested; Generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps; Determine the subgraph execution sequence and execution position based on the calling order between the subgraph sequences; According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index.

2. The method according to claim 1, characterized in that Generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested, including: Generate an initial computational graph based on the computational graph compiler for the model or algorithm to be tested; The initial computation graph is divided into corresponding subgraph sequences according to a segmentation rule, wherein each subgraph in the subgraph sequence represents a set of computation steps that can be computed in parallel.

3. The method according to claim 2, characterized in that The step of generating an initial computational graph based on a computational graph compiler for the model or algorithm to be tested includes: Constructing a deep neural network according to the model or algorithm to be tested, determining each network layer of the deep neural network as a node in a computational graph; determining weights and biases between network layers as edges between nodes in the computational graph; Through the computational graph compiler, all computational steps in the model or algorithm are integrated into an initial computational graph, and the initial computational graph represents all computational steps in the model or algorithm.

4. The method according to claim 1, characterized in that The subgraph execution sequence and execution position are determined based on the calling order between subgraph sequences, including: Determining data dependencies between subgraphs based on a calling order between the subgraph sequences; The subgraph execution sequence and execution location are determined according to subgraph attributes and data dependencies between the subgraphs; the attributes include subgraphs without data dependencies and subgraphs executed only on the CPU, and the execution location includes the CPU and / or GPU.

5. The method according to any one of claims 1 to 4, characterized in that: The hardware performance indicators include computational graph coverage; According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including: According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the number of GPU subgraphs and / or the number of CPU subgraphs; The computation graph coverage is obtained according to the ratio of the number of GPU subgraphs and / or the number of CPU subgraphs in the subgraph sequence of the initial computation graph, where the computation graph coverage includes GPU coverage and CPU coverage.

6. The method according to any one of claims 1 to 4, characterized in that: The hardware performance indicators include the proportion of computing time; According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including: According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; The computing time includes GPU computing time and / or CPU computing time; The calculation time proportion is obtained according to the ratio of the GPU calculation time and / or the CPU calculation time of all subgraphs in the total calculation time. The calculation time proportion includes the GPU calculation time proportion and the CPU calculation time proportion. The total calculation time is the total calculation time consumed by all subgraphs at the execution location of the hardware.

7. The method according to any one of claims 1 to 4, characterized in that: The hardware performance indicators include computing speed; According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation to obtain the hardware performance index, including: According to the subgraph execution sequence, the subgraph is input into the execution position corresponding to the hardware for calculation, and the calculation time of all subgraphs is recorded; the calculation time includes GPU calculation time and / or CPU calculation time; The calculation speed is obtained according to the calculation workload of all subgraphs and the GPU calculation time and / or the CPU calculation time of all subgraphs, and the calculation speed includes the GPU calculation speed and the CPU calculation speed.

8. A hardware testing system, characterized in that: The system comprises: A message receiving module, used for receiving a hardware test instruction, wherein the hardware test instruction carries a model or algorithm to be tested; A subgraph generation module, used to generate a subgraph sequence of a corresponding computational graph according to the model or algorithm to be tested; the computational graph is used to represent the data structure of the model or algorithm; the computational graph is composed of a number of nodes and edges, the nodes represent the computational steps of the model or algorithm, and the edges represent the logical relationship between the computational steps; An execution sequence module is used to determine the subgraph execution sequence and execution position based on the calling sequence between the subgraph sequences; The indicator calculation module is used to input the subgraph into the execution position corresponding to the hardware for calculation according to the subgraph execution sequence to obtain the hardware performance indicator.

9. An electronic device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and the computer-readable instructions can be executed by a processor to implement the method according to any one of claims 1-7.

Images