Executable instruction generation apparatus, unified operator library construction apparatus, method, and medium

Abstract

A kind of executable instruction generation device, uniform operator library construction device, method and medium are disclosed. Among them, the executable instruction generation device includes at least one first processor, the first processor is configured to: determine the neural network model to be compiled;From the heterogeneous computing platform, determine the target computing unit corresponding to the first operator in the neural network model;In the uniform operator library, determine the target implementation code corresponding to the first operator and the target computing unit;Wherein, the uniform operator library includes the multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent;The target implementation code is compiled to generate executable instructions for deployment on the heterogeneous computing platform. In the embodiment of the present disclosure, based on the uniform operator library, the consistency of operator across hardware execution can be ensured.

Description

Technical Field

[0001] This disclosure relates to neural network model technology, and in particular to an executable instruction generation device, a unified operator library construction device, method, and medium. Background Technology

[0002] When applying neural network models to heterogeneous computing, operators within these models may exhibit inconsistencies in execution across different hardware. For example, the same operator executed on computing unit A and computing unit B in a heterogeneous computing platform may produce completely different results. How to resolve this issue of inconsistent operator execution across hardware is a pressing technical problem for those skilled in the art. Summary of the Invention

[0003] To address the aforementioned technical problems, this disclosure provides an executable instruction generation apparatus, a unified operator library construction apparatus, a method, and a medium.

[0004] According to one aspect of the present disclosure, an executable instruction generation apparatus is provided, including at least one first processor, the first processor being configured to: Determine the neural network model to be compiled; From the heterogeneous computing platform, determine the target computing unit corresponding to the first operator in the neural network model; In a unified operator library, target implementation code corresponding to the first operator and the target computing unit is determined; wherein, the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent; The target implementation code is compiled to generate executable instructions for deployment on the heterogeneous computing platform.

[0005] According to another aspect of the present disclosure, a unified operator library construction apparatus is provided, including at least one second processor, the second processor being configured to: Identify at least one class of operators included in a deep learning framework; For each type of operator, identify the types of computational units that support the execution of the operator; Based on the operator and the various types of computing units, determine the implementation code of the operator in each type of computing unit; In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are consistent, and an operator library corresponding to the operator is constructed based on the implementation code of the operator in each type of computing unit. A unified operator library is constructed based on the operator libraries corresponding to various operators.

[0006] According to another aspect of the present disclosure, an executable instruction generation method is provided, comprising: Determine the neural network model to be compiled; From the heterogeneous computing platform, determine the target computing unit corresponding to the first operator in the neural network model; In a unified operator library, target implementation code corresponding to the first operator and the target computing unit is determined; wherein, the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent; The target implementation code is compiled to generate executable instructions for deployment on the heterogeneous computing platform.

[0007] According to another aspect of the present disclosure, a method for constructing a unified operator library is provided, comprising: Identify at least one class of operators included in a deep learning framework; For each type of operator, identify the types of computational units that support the execution of the operator; Based on the operator and the various types of computing units, determine the implementation code of the operator in each type of computing unit; In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are consistent, and an operator library corresponding to the operator is constructed based on the implementation code of the operator in each type of computing unit. A unified operator library is constructed based on the operator libraries corresponding to various operators.

[0008] According to another aspect of the present disclosure, a computer-readable storage medium is provided, the storage medium storing a computer program that is executed by a processor to implement the above-described executable instruction generation method or to implement the above-described unified operator library construction method.

[0009] According to another aspect of the present disclosure, a computer program product is provided, which, when the instructions in the computer program product are executed by a processor, executes the above-described executable instruction generation method or the above-described unified operator library construction method.

[0010] Based on the executable instruction generation apparatus, unified operator library construction apparatus, method, medium, and program product provided in the above embodiments of this disclosure, the first processor, for a neural network model to be compiled, determines the target computing unit corresponding to the first operator in the neural network model from a heterogeneous computing platform, and determines the target implementation code corresponding to the first operator and the target computing unit in the unified operator library, and compiles the target implementation code to generate executable instructions for deployment on the heterogeneous computing platform. It should be noted that the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes in the corresponding computing units are consistent. Therefore, for the first operator, executing the corresponding executable instructions through different types of computing units in the same heterogeneous computing platform to calculate the same data will also yield consistent calculation results. Similarly, executing the corresponding executable instructions through computing units in different heterogeneous computing platforms to calculate the same data will also yield consistent calculation results. As can be seen, in the embodiments of this disclosure, based on a unified operator library, the execution results of the first operator are consistent on different computing units in the same heterogeneous computing platform, and the execution results of the first operator are also consistent on computing units in different heterogeneous computing platforms. Therefore, the consistency of operator execution across hardware can be ensured. Attached Figure Description

[0011] Figure 1 This is a schematic diagram of an exemplary application scenario of this disclosure.

[0012] Figure 2-1 This is one of the structural schematic diagrams of an executable instruction generation apparatus provided in some exemplary embodiments of this disclosure.

[0013] Figure 2-2 This is a second schematic diagram of the structure of an executable instruction generation apparatus provided in some exemplary embodiments of this disclosure.

[0014] Figure 3-1 This is one of the structural schematic diagrams of a unified operator library construction apparatus provided by some exemplary embodiments of this disclosure.

[0015] Figure 3-2 This is a schematic diagram illustrating the principle of verifying the consistency of implementation code results in some exemplary embodiments of this disclosure.

[0016] Figure 3-3 This is the second schematic diagram of the structure of the unified operator library construction apparatus provided by some exemplary embodiments of this disclosure.

[0017] Figure 4 This is a schematic diagram illustrating the application of the unified operator library to various stages in some exemplary embodiments of this disclosure.

[0018] Figure 5This is a schematic diagram of the architecture of a heterogeneous computing platform provided by some exemplary embodiments of this disclosure.

[0019] Figure 6 This is a flowchart illustrating an executable instruction generation method provided by some exemplary embodiments of this disclosure.

[0020] Figure 7 This is one of the flowcharts illustrating a unified operator library construction method provided by some exemplary embodiments of this disclosure.

[0021] Figure 8 This is the second flowchart illustrating a unified operator library construction method provided by some exemplary embodiments of this disclosure.

[0022] Figure 9 This is the third flowchart illustrating the unified operator library construction method provided by some exemplary embodiments of this disclosure.

[0023] Figure 10 This is the fourth flowchart illustrating the unified operator library construction method provided by some exemplary embodiments of this disclosure. Detailed Implementation

[0024] To explain this disclosure, exemplary embodiments of the disclosure will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the disclosure, and not all of them. It should be understood that the disclosure is not limited to exemplary embodiments.

[0025] It should be noted that, unless otherwise specifically stated, the relative arrangement, numerical expressions, and values ​​of the components and steps set forth in these embodiments do not limit the scope of this disclosure.

[0026] Application Overview When neural network models are applied to heterogeneous computing, operators in the neural network model may exhibit inconsistencies in execution across hardware.

[0027] For example, the heterogeneous computing platform used for developing a neural network model is heterogeneous computing platform P1, and the heterogeneous computing platform used for deploying the neural network model is heterogeneous computing platform P2. The neural network model includes an operator X. During the development phase, operator X is executed on computing unit A in heterogeneous computing platform P1, and during the deployment phase, operator X is executed on computing unit B in heterogeneous computing platform P2. Furthermore, the execution result of operator X on computing unit B in heterogeneous computing platform P2 is inconsistent with the execution result of operator X on computing unit A in heterogeneous computing platform P1. In this case, computing unit A and computing unit B are located in different heterogeneous computing platforms, and computing unit A and computing unit B are obviously different hardware. Therefore, operator X suffers from cross-hardware execution inconsistency.

[0028] For example, if the heterogeneous computing platform used for deploying a neural network model is P2, and the neural network model includes an operator X, during deployment, the execution of operator X is required to complete a corresponding computational task. Since this computational task is very large, operator X can be executed simultaneously on computing units A and B within the heterogeneous computing platform P2. This allows computing units A and B to complete a portion of the computational task, respectively. Furthermore, the execution result of operator X on computing unit B within the heterogeneous computing platform P2 will be inconsistent with the execution result of operator X on computing unit A within the heterogeneous computing platform P2. In this case, computing units A and B are located within the same heterogeneous computing platform, but they are still different hardware components. Therefore, operator X suffers from inconsistencies in execution across hardware.

[0029] It should be noted that inconsistencies in operator execution across hardware can easily affect the computational correctness and reliability of neural network models during the deployment phase. How to resolve these inconsistencies is a pressing technical problem for those skilled in the art.

[0030] Exemplary System Figure 1 An exemplary application scenario of this disclosure is shown, including an executable instruction generation device 110, a unified operator library construction device 120, and a heterogeneous computing platform 130.

[0031] Optionally, the heterogeneous computing platform 130 can integrate various types of computing units to achieve complex computing tasks through the collaborative work of these different types of computing units. These various types of computing units include, but are not limited to, a Central Processing Unit (CPU), a Neural Processing Unit (NPU), a Graphics Processing Unit (GPU), and a Digital Signal Processor (DSP).

[0032] Optionally, the unified operator library construction device 120 is hardware in an electronic device that has the function of building a unified operator library, such as a processor. The electronic device here includes, but is not limited to, terminal devices, computer systems, servers, etc. Terminal devices include, but are not limited to, personal computers, tablets, etc. Examples of well-known terminal devices, computing systems, environments, and / or configurations suitable for use with terminal devices, computer systems, servers, etc., include, but are not limited to: personal computer systems, server computer systems, thin clients, thick clients, handheld or laptop devices, microprocessor-based systems, set-top boxes, programmable consumer electronics, network personal computers, minicomputer systems, mainframe computer systems, and distributed cloud computing environments including any of the above systems. Terminal devices may also include, in vehicle scenarios, in-vehicle intelligent terminals, in-vehicle infotainment terminals, in-vehicle sensing terminals, in-vehicle computing terminals; or in robotic scenarios, robot body control terminals, robot sensing terminals, robot interaction terminals, robot execution terminals, etc. The unified operator library construction device 120 can execute software programs through hardware (such as a processor) to realize the construction of the unified operator library. Specifically, the unified operator library construction device 120 can be configured to: determine at least one type of operator included in the deep learning framework; for each type of operator, determine various types of computational units that support the execution of the operator; based on the operator and each type of computational unit, determine the implementation code of the operator in each type of computational unit; in response to determining that the execution results of the implementation code of the operator in each type of computational unit are consistent based on the test data corresponding to the operator, construct the operator library corresponding to the operator based on the implementation code of the operator in each type of computational unit; and construct a unified operator library based on the operator libraries corresponding to each type of operator.

[0033] Optionally, the executable instruction generation device 110 is hardware with instruction generation capabilities in an electronic device. The type of electronic device in which the executable instruction generation device 110 is located is described in the previous section and will not be repeated here. Furthermore, the electronic device in which the executable instruction generation device 110 is located and the electronic device in which the unified operator library construction device 120 is located can be the same electronic device or different electronic devices. The executable instruction generation device 110 can execute software programs through hardware to generate executable instructions for deployment on the heterogeneous computing platform 130. Specifically, the executable instruction generation device 110 can be configured to: determine the neural network model to be compiled; determine the target computation unit corresponding to the first operator in the neural network model from the heterogeneous computing platform 130; determine the target implementation code corresponding to the first operator and the target computation unit in the unified operator library; wherein the unified operator library includes multiple implementation codes of the first operator in different types of computation units, and the execution results of the multiple implementation codes of the first operator in different types of computation units are consistent; compile the target implementation code to generate executable instructions for deployment on the heterogeneous computing platform 130.

[0034] Exemplary device Figure 2-1 This is a schematic diagram of the structure of an executable instruction generation apparatus 110 provided in some exemplary embodiments of this disclosure. The executable instruction generation apparatus 110 can be applied during the compilation stage. Figure 2-1 As shown, the executable instruction generation device 110 includes a first processor 210, which is configured to: determine a neural network model to be compiled; determine a target computing unit corresponding to a first operator in the neural network model from the heterogeneous computing platform 130; determine target implementation code corresponding to the first operator and the target computing unit in a unified operator library; wherein the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent; and compile the target implementation code to generate executable instructions for deployment on the heterogeneous computing platform 130.

[0035] Optionally, the first processor 210 is processing hardware in the executable instruction generation apparatus 110, configured to perform at least one of the following functions: executing a computer program to generate instructions through compilation; executing instructions; processing data; and coordinating or controlling the operation of other components. The first processor 210 may be, for example, but not limited to, a CPU, a microcontroller unit (MCU), etc. The number of first processors 210 in the executable instruction generation apparatus 110 may be one or more. For ease of understanding, the following description will use the case where the number of first processors 210 in the executable instruction generation apparatus 110 is one as an example.

[0036] Optionally, the neural network model to be compiled can be any neural network model that requires compilation, such as, but not limited to, semantic segmentation models, instance segmentation models, object detection models, environment perception models, end-to-end models, and Transformer models (which are neural network models based on a self-attention mechanism). The neural network model to be compiled can include multiple types of operators, such as, but not limited to, convolution operators, pooling operators, quantization operators, dequantization operators, and remap operators. The first operator in this disclosure can be any operator in the neural network model to be compiled. The neural network model to be compiled can include multiple operators, for example, the number of operators is M1, where M1 is an integer greater than or equal to 2, and the number of the first operator is also M1.

[0037] Optionally, the first processor 210 may determine the types of computing units that support the execution of the first operator operation based on the types of computing units included in the heterogeneous computing platform 130, and designate at least a portion of the computing units that support the execution of the first operator operation as the target computing units corresponding to the first operator. For example, if the number of computing units in the heterogeneous computing platform 130 that support the execution of the first operator operation is M2, where M2 is an integer greater than or equal to 2, then the number of target computing units corresponding to the first operator is greater than or equal to 1 and less than or equal to M2.

[0038] Optionally, the unified operator library can be constructed by the unified operator library construction device 120 executing a software program. The constructed unified operator library can be deployed at any location accessible by the first processor 210. The unified operator library can include operator libraries corresponding to various types of operators. That is, the unified operator library includes multiple operator libraries, and each operator library can include multiple implementation codes of a type of operator in different types of computing units, and the execution results of the multiple implementation codes in the corresponding computing units are consistent. The consistent execution results of the multiple implementation codes in the corresponding computing units can be understood as follows: if the same data to be calculated is processed according to the multiple implementation codes in the corresponding computing units, the calculation results are consistent. It should be noted that the consistent calculation results in this disclosure can be understood as: the calculation results are the same or the calculation result error is within a preset range.

[0039] As an example, the unified operator library may include a first operator library corresponding to the convolution operator, a second operator library corresponding to the quantization operator, a third operator library corresponding to the dequantization operator, and a fourth operator library corresponding to the remapping operator. The first operator library may include the CPU implementation code D1, the GPU implementation code D2, and the DSP implementation code D3 for the convolution operator. Furthermore, the execution results of implementation code D1 on the CPU, D2 on the GPU, and D3 on the DSP are consistent. Implementation code D1 can be understood as program code that conforms to the CPU's operating mechanism and implements the convolution operation logic. Implementation code D2 can be understood as program code that conforms to the GPU's operating mechanism and implements the convolution operation logic. Implementation code D3 can be understood as program code that conforms to the DSP's operating mechanism and implements the convolution operation logic. Generally, different types of computing units have different operating mechanisms. An operating mechanism can be understood as the unique execution mode and computing paradigm of a computing unit. An execution mode can be understood as the specific implementation method of the computing unit in terms of the timing of instruction and data flow interaction and hardware resource utilization strategies, such as whether the computing unit can perform computation in parallel, whether it can perform computation in a pipelined manner, how it performs computation in parallel, and how it performs computation in a pipelined manner. A computing paradigm is used to define how to divide a task into several concurrently executable subtasks, as well as the interaction and dependencies between these subtasks. Typical computing paradigms include data parallelism and streaming parallelism. As an example, when processing data, a CPU may process only one number at a time, while a DSP can process 100 numbers concurrently. Therefore, in implementation code D1, the number of loop executions needs to be defined for this data processing, while in implementation code D3, this number of loop executions does not need to be defined. The consistency of the execution results of code D1 on the CPU, D2 on the GPU, and D3 on the DSP can be understood as follows: if the same data to be computed is used, performing convolution calculations according to code D1 on the CPU, D2 on the GPU, and D3 on the DSP will yield consistent convolution results (e.g., the results are the same or nearly the same). The contents of the second, third, and fourth operator libraries are similar to those described in the introduction to the first operator library, and will not be repeated here.

[0040] Optionally, the first processor 210 can determine the operator library corresponding to the first operator based on a unified operator library, and then determine the implementation code of the first operator in the target computing unit within the operator library corresponding to the first operator, and use this implementation code as the target implementation code corresponding to the first operator and the target computing unit. Optionally, the target computing unit corresponding to the first operator can be one or more. If the target computing unit corresponding to the first operator is one, then there is one set of target implementation code; if the target computing units corresponding to the first operator are multiple, then there are multiple sets of target implementation code.

[0041] Optionally, the first processor 210 can utilize a compiler adapted to the heterogeneous computing platform 130 to compile the target implementation code, generating binary machine instructions. These machine instructions serve as the executable instructions corresponding to the target implementation code, and the executable instructions can be deployed on the heterogeneous computing platform 130. If there are multiple target implementation codes, the compiler can generate executable instructions corresponding to each target implementation code, and each executable instruction can be deployed on the heterogeneous computing platform 130. Here, each executable instruction can constitute an instruction library, and deploying the instruction library on the heterogeneous computing platform 130 is equivalent to deploying all executable instructions on the heterogeneous computing platform 130.

[0042] As an example, the first operator library includes convolution operator implementation code D1 on the CPU, convolution operator implementation code D2 on the GPU, and convolution operator implementation code D3 on the DSP. Since the first operator in the neural network model is the convolution operator, and the heterogeneous computing platform 130 includes a CPU, GPU, and DSP, three target computing units can be identified from the heterogeneous computing platform 130: CPU, GPU, and DSP. There are three sets of target implementation code corresponding to the first operator and the target computing units: implementation code D1, implementation code D2, and implementation code D3. Compiling implementation code D1 yields executable instruction Z1, which can be used to implement convolution computation based on CPU resources. Compiling implementation code D2 yields executable instruction Z2, which can be used to implement convolution computation based on GPU resources. Compiling implementation code D3 yields executable instruction Z3, which can be used to implement convolution computation based on DSP resources. Executable instructions Z1, Z2, and Z3 can all be deployed on the heterogeneous computing platform 130.

[0043] It should be noted that if the convolution operator yields consistent results when implementing code D1, code D2, and code D3 on the same input data, then the executable instructions Z1, Z2, and Z3 generated based on code D1, generated based on code D2, and generated based on code D3 will also yield consistent results when performing convolution calculations on the same input data. That is, for the same data to be calculated, the convolution calculation results will be consistent whether the CPU in the heterogeneous computing platform 130 executes executable instruction Z1, the GPU in the heterogeneous computing platform 130 executes executable instruction Z2, or the DSP in the heterogeneous computing platform 130 executes executable instruction Z3.

[0044] In the above example, if the first operator is a convolution operator, the number of target computing units determined from the heterogeneous computing platform 130 is three. In a specific implementation, the number of target computing units determined from the heterogeneous computing platform 130 can also be two, for example, the two target computing units could be a CPU and a GPU, or a CPU and a DSP. Of course, the number of target computing units determined from the heterogeneous computing platform 130 can also be only one, for example, this target computing unit could be a CPU, a GPU, or a DSP.

[0045] In the embodiments of this disclosure, the first processor 210, for the neural network model to be compiled, determines the target computing unit corresponding to the first operator in the neural network model from the heterogeneous computing platform 130, and determines the target implementation code corresponding to the first operator and the target computing unit in the unified operator library, and compiles the target implementation code to generate executable instructions for deployment on the heterogeneous computing platform 130. It should be noted that the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes in the corresponding computing units are consistent. Therefore, for the first operator, executing the corresponding executable instructions through different types of computing units in the same heterogeneous computing platform 130 to calculate the same data will also yield consistent calculation results. Similarly, executing the corresponding executable instructions through computing units in different heterogeneous computing platforms 130 to calculate the same data will also yield consistent calculation results. As can be seen, in the embodiments of this disclosure, based on a unified operator library, the execution results of the first operator are consistent across different computing units on the same heterogeneous computing platform 130, and also consistent across computing units on different heterogeneous computing platforms 130. Therefore, the consistency of operator execution across hardware can be ensured. It should be noted that since the execution results of the first operator are also consistent across computing units on different heterogeneous computing platforms 130, the consistency of operator execution across platforms can also be ensured.

[0046] In some optional examples, the first operator includes multiple implementations of code for different types of computational units: The first implementation code for the first operator to perform calculations through the first computation unit; and The second implementation code for the first operator to perform calculations through the second computation unit, wherein the type of the second computation unit is different from that of the first computation unit.

[0047] As an example, the first computing unit can be one of the four: CPU, NPU, GPU, and DSP, and the second computing unit can be the other one of the four: CPU, NPU, GPU, and DSP.

[0048] Optionally, the first implementation code for the first operator to be calculated by the first computing unit can satisfy: (1) the programming language of the first implementation code is a programming language supported by the first computing unit; (2) the first implementation code conforms to the operating mechanism of the first computing unit; and (3) the first implementation code implements the operation logic of the first operator.

[0049] Optionally, the second implementation code for the first operator to be computed by the second computing unit can satisfy the following: (1) the programming language of the second implementation code is a programming language supported by the second computing unit; (2) the second implementation code conforms to the operating mechanism of the second computing unit; and (3) the second implementation code implements the operation logic of the first operator. Here, the operation logic implemented by the second implementation code and the first implementation code can be the same.

[0050] It should be noted that, since the first implementation code and the second implementation code satisfy the above conditions, for the same input data, the first calculation result obtained by executing the executable instructions corresponding to the first implementation code on the first computing unit in the heterogeneous computing platform 130 to perform the calculation corresponding to the first operator is consistent with the second calculation result obtained by executing the executable instructions corresponding to the second implementation code on the second computing unit in the heterogeneous computing platform 130 to perform the calculation corresponding to the first operator. Therefore, based on the first implementation code and the second implementation code, it is possible to achieve the same calculation result as that obtained by performing calculations on the first operator through the first computing unit or the second computing unit in the heterogeneous computing platform 130.

[0051] In some optional examples, such as Figure 2-2As shown, the executable instruction generation apparatus 110 may further include a first memory 220. The first memory 220 is a circuit in the executable instruction generation apparatus 110 that stores instructions and data. The first memory 220 includes one or more computer program products, which include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. Volatile memory includes, for example, random access memory (RAM) and / or cache memory. Non-volatile memory includes, for example, read-only memory (ROM), hard disk, flash memory, etc. The first memory 220 may store at least one of the following: instructions generated by the first processor 210, instructions to be executed by the first processor 210, data to be processed by the first processor 210, and processing results obtained by the first processor 210 after data processing.

[0052] In some optional examples, such as Figure 2-2 As shown, the executable instruction generation apparatus 110 may further include a first input device 230 and a first output device 240, which are interconnected via a bus system and / or other forms of connection mechanisms (not shown). The first input device 230 includes, for example, a keyboard, a mouse, etc. The first output device 240 includes, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc., and the first output device 240 can output various information.

[0053] Of course, for the sake of simplicity, Figure 2-2 Only some of the components of the executable instruction generation apparatus 110 relevant to this disclosure are shown, omitting components such as buses and input / output interfaces. In addition, the executable instruction generation apparatus 110 may include any other suitable components depending on the specific application.

[0054] Figure 3-1 This is a schematic diagram of the structure of a unified operator library construction apparatus 120 provided in some exemplary embodiments of this disclosure. For example... Figure 3-1 As shown, the unified operator library construction device 120 includes a second processor 310, which is configured to: determine at least one type of operator included in the deep learning framework; for each type of operator, determine various types of computational units that support the execution of the operator; based on the operator and the various types of computational units, determine the implementation code of the operator in each type of computational unit; in response to determining that the execution results of the implementation code of the operator in each type of computational unit are consistent based on the test data corresponding to the operator, construct the operator library corresponding to the operator based on the implementation code of the operator in each type of computational unit; and construct a unified operator library based on the operator libraries corresponding to each type of operator.

[0055] Optionally, the second processor 310 is processing hardware in the unified operator library construction apparatus 120, configured to perform at least one of the following functions: executing computer programs to construct a unified operator library; executing instructions; processing data; and coordinating or controlling the operation of other components. The second processor 310 may be, for example, but not limited to, a CPU, an MCU, etc. The number of second processors 310 in the unified operator library construction apparatus 120 may be one or more. For ease of understanding, the following description will use the case where the number of second processors 310 in the unified operator library construction apparatus 120 is one as an example.

[0056] Optionally, a deep learning framework is a collection of software libraries, toolsets, and runtime environments that simplify and accelerate the entire process of designing, training, validating, deploying, and managing neural network models by providing modular, automated, and high-performance building blocks. Deep learning frameworks include, but are not limited to, PyTorch and TensorFlow. Deep learning frameworks include at least one class of operators, such as, but not limited to, convolution operators, pooling operators, quantization operators, dequantization operators, remapping operators, element-wise operators, and Rectified Linear Unit (ReLU) operators.

[0057] Optionally, for each type of operator in at least one class of operators included in the deep learning framework, the second processor 310 can perform the following operations (hereinafter referred to as target operations for ease of description): determine the types of computational units that support the execution of the operator; determine the implementation code of the operator in each type of computational unit based on the operator and each type of computational unit; in response to the test data corresponding to the operator, determine that the execution results of the implementation code of the operator in each type of computational unit are consistent; and construct the operator library corresponding to the operator based on the implementation code of the operator in each type of computational unit. Since the target operations for each type of operator are similar, for the sake of brevity, the following text mainly focuses on the target operations of one type of operator.

[0058] Optionally, the second processor 310 can, for each type of operator, determine from the set of computing units that support the execution of that type of operator. The set of computing units can include various types of computing units that can be integrated into the heterogeneous computing platform 130. There can be multiple types of computing units corresponding to any type of operator.

[0059] Optionally, the second processor 310 can determine the operating mechanism of each type of computing unit, and based on the operator and the operating mechanism of each type of computing unit, determine the implementation code of the operator in each type of computing unit. The second processor 310 can also determine whether the execution results of the implementation code of the operator in each type of computing unit are consistent based on the test data corresponding to the operator. The test data can be tensor data, which serves as the input data for the operator. Based on the implementation code and the computing unit, the operator can perform calculations on the test data to obtain calculation results, thereby testing the consistency of the execution results of different implementation codes. If the operator is a convolution operator, the test data may include feature tensors (e.g., images or feature maps) and weight tensors; if the operator is a modified linear unit operator, the test data may only include feature tensors. If the execution results of the implementation code of the operator in each type of computing unit are consistent, the second processor 310 can construct an operator library corresponding to the operator based on the implementation code of the operator in each type of computing unit. The operator library corresponding to the operator may include the implementation code of the operator in each type of computing unit.

[0060] It should be noted that for each type of operator within at least one class of operators included in a deep learning framework, corresponding operator libraries can be constructed according to the relevant introductions above. The collection of these operator libraries can serve as a unified operator library. Based on this unified operator library, the execution results of the same operator are consistent across different computing units on the same heterogeneous computing platform 130, and also consistent across computing units on different heterogeneous computing platforms 130. Therefore, it ensures consistency in operator execution across hardware and across platforms, thereby ensuring the computational correctness and reliability of the neural network model during the deployment phase.

[0061] In some optional examples, the implementation code for the operator in each type of computational unit includes: The third implementation code for the operator to be calculated through the third computation unit; The fourth implementation code for the operator to be calculated through the fourth computation unit.

[0062] Optionally, the third implementation code of the operator calculated by the third computing unit can satisfy: (1) the programming language of the third implementation code is a programming language supported by the third computing unit; (2) the third implementation code conforms to the operating mechanism of the third computing unit; and (3) the third implementation code implements the operation logic of the operator.

[0063] Optionally, the fourth implementation code of the operator being calculated by the fourth computation unit can satisfy: (1) the programming language of the fourth implementation code is a programming language supported by the fourth computation unit; (2) the fourth implementation code conforms to the operation mechanism of the fourth computation unit; and (3) the fourth implementation code implements the operation logic of the operator.

[0064] In the embodiments of this disclosure, when the second processor 310 determines that the execution results of the implementation code of the operator in each type of computing unit are consistent, it is specifically configured to: generate first operation result data corresponding to the test data based on the third implementation code and using the third computing unit in the test heterogeneous computing platform; generate second operation result data corresponding to the test data based on the fourth implementation code and using the fourth computing unit in the test heterogeneous computing platform; determine a first matching degree between the first operation result data and the second operation result data; and determine that the execution results of the third implementation code and the fourth implementation code are consistent in response to the first matching degree satisfying a first preset condition.

[0065] It should be noted that testing heterogeneous computing platforms can be used to assist in testing the consistency of execution results across different implementations of code. For example, a heterogeneous computing platform 130 used during neural network model development can be used. The testing heterogeneous computing platform may include a third computing unit and a fourth computing unit, the type of which may differ from that of the third computing unit. As an example, the third computing unit may be either a CPU or a GPU, and the fourth computing unit may be either a CPU or a GPU.

[0066] Optionally, the second processor 310 can utilize a compiler adapted to the test heterogeneous computing platform to compile the third implementation code, generate corresponding executable instructions, and call the third computing unit in the test heterogeneous computing platform to execute the executable instructions to perform calculations on the test data, thereby generating first operation result data corresponding to the test data. The first operation result data may be, for example, tensor data. Similarly, the second processor 310 can utilize a compiler adapted to the test heterogeneous computing platform to compile the fourth implementation code, generate corresponding executable instructions, and call the fourth computing unit in the test heterogeneous computing platform to execute the executable instructions to perform calculations on the test data, thereby generating second operation result data corresponding to the test data. The second operation result data may be, for example, tensor data. The second processor 310 can also determine a first matching degree between the first operation result data and the second operation result data. For example, the second processor 310 can use a cosine similarity algorithm or other similarity algorithms to calculate the first matching degree between the first operation result data and the second operation result data.

[0067] After determining the first matching degree, the second processor 310 can determine whether the first matching degree meets the first preset condition. For example, the first preset condition could be: the first matching degree is greater than a first preset matching degree. The first preset matching degree could be, for example, 90%, 92%, 95%, etc., and will not be listed here. If the first matching degree is greater than the first preset matching degree, the second processor 310 can determine that the first matching degree meets the first preset condition, the first calculation result data and the second calculation result data are consistent, and the second processor 310 can further determine that the execution results of the third implementation code and the fourth implementation code are consistent. If the first matching degree is less than or equal to the first preset matching degree, the second processor 310 can determine that the first matching degree does not meet the first preset condition, the first calculation result data and the second calculation result data are inconsistent, and the second processor 310 can further determine that the execution results of the third implementation code and the fourth implementation code are inconsistent.

[0068] In the embodiments of this disclosure, by calling the third and fourth computing units in the test heterogeneous computing platform, the test data can be obtained as the operation result data corresponding to the third and fourth implementation codes, respectively. Then, by calculating the matching degree between the operation result data and judging whether the calculated first matching degree meets the first preset condition, it is possible to efficiently and quickly determine whether the execution results of the third and fourth implementation codes are consistent.

[0069] In some optional examples, the second processor 310 is further configured to: select one implementation code from the implementation code of the operator in each type of computing unit as the baseline implementation code based on the different types of computing units included in the test heterogeneous computing platform; in response to determining that the execution results of the implementation code of the operator in each type of computing unit are inconsistent based on the test data corresponding to the operator, optimize the implementation code of the operator in each type of computing unit other than the baseline implementation code to obtain the optimized implementation code; and construct the operator library corresponding to the operator based on the baseline implementation code and the optimized implementation code.

[0070] Optionally, when the second processor 310 selects an implementation code as the baseline implementation code from the implementation codes of the operator in each type of computing unit, it can select according to the following rule: the computing unit corresponding to the selected implementation code is the computing unit included in the test heterogeneous computing platform. The second processor 310 can also determine the implementation codes of the operator in each type of computing unit other than the baseline implementation code; each determined implementation code can be used as a non-baseline implementation code. The number of non-baseline implementation codes can be one or more. It should be noted that the baseline implementation code can be understood as the implementation code that serves as the benchmark. The execution result generated based on the baseline implementation code is used as a reference benchmark during consistency comparison, and the execution result generated based on the non-baseline implementation code needs to be compared with this reference benchmark.

[0071] If the execution results of the operator's implementation code differ across different types of computation units, the second processor 310 can optimize the non-baseline implementation code to obtain optimized implementation code. For example, it can output prompts to developers, suggesting that they optimize the non-baseline implementation code. Alternatively, the second processor 310 can automatically optimize the non-baseline implementation code according to pre-configured optimization rules. It should be noted that optimized implementation code can be understood as implementation code whose execution result, after optimization, aligns with the baseline implementation code, based on the non-baseline implementation code.

[0072] After obtaining the optimized implementation code, the second processor 310 can construct an operator library corresponding to the operator based on the baseline implementation code and the optimized implementation code. For example, the second processor 310 can verify whether the execution results of the optimized implementation code and the baseline implementation code are consistent based on test data. If the execution results of the optimized implementation code and the baseline implementation code are consistent, an operator library including the baseline implementation code and the optimized implementation code can be constructed. If the execution results of the optimized implementation code and the baseline implementation code are inconsistent, the optimized implementation code can be further optimized to obtain the latest optimized implementation code. If the latest optimized implementation code and the baseline implementation code are consistent, an operator library including the baseline implementation code and the latest optimized implementation code can be constructed. If the latest optimized implementation code and the baseline implementation code are inconsistent, the optimized implementation code can be optimized again to obtain an updated optimized implementation code, and so on. This process will not be elaborated further here.

[0073] In the embodiments of this disclosure, the implementation code of the operator in each type of computing unit can be divided into baseline implementation code and non-baseline implementation code. The execution result of the baseline implementation code is used as a reference benchmark to optimize the non-baseline implementation code, so that the execution result of the optimized non-baseline implementation code is aligned with the execution result of the baseline implementation code. This facilitates the rapid acquisition of different implementation codes with consistent execution results. Based on different implementation codes with consistent execution results, the operator library corresponding to the operator can be constructed efficiently and reliably, and a unified operator library can be further constructed. Based on the unified operator library, the consistency of operator execution across hardware and the consistency of cross-platform execution can be ensured.

[0074] In some optional examples, the implementation code of the operator in each type of computational unit, in addition to the baseline implementation code, includes: The fifth implementation code, which performs calculations through the fifth computational unit, has a type different from any computational unit in the heterogeneous computing platform being tested. The fifth implementation code conforms to the operating mechanism of the fifth computational unit; and... The sixth implementation code is used to simulate the operation mechanism of the fifth computing unit. The sixth computing unit is located in the test heterogeneous computing platform.

[0075] Here, the fifth and sixth implementation codes can each be considered as non-baseline implementation codes. The type of the fifth computing unit differs from the type of any computing unit in the test heterogeneous computing platform. The sixth computing unit is located in the test heterogeneous computing platform; that is, the heterogeneous computing platform 130 in the development phase includes the sixth computing unit but not the fifth computing unit. Furthermore, the heterogeneous computing platform 130 in the deployment phase may include both the sixth and fifth computing units.

[0076] Optionally, the sixth implementation code used to simulate the operating mechanism of the fifth computing unit can be understood as follows: the sixth implementation code is used to support the reproduction of the execution mode, computing paradigm, etc. unique to the fifth computing unit on the sixth computing unit.

[0077] In some optional embodiments of this disclosure, the operators are implemented in the implementation code of each type of computing unit, and the seventh implementation code in which the operators are calculated in the seventh computing unit is used as the baseline implementation code. Here, the seventh computing unit may be a computing unit included in the heterogeneous computing platform 130 (i.e., the test heterogeneous computing platform) during the deployment phase.

[0078] When the execution results of the implementation code of the operators in each type of computing unit are inconsistent, the second processor 310 is specifically configured to: generate the third operation result data corresponding to the test data based on the seventh implementation code and using the seventh computing unit in the test heterogeneous computing platform; generate the fourth operation result data corresponding to the test data based on the sixth implementation code and using the sixth computing unit in the test heterogeneous computing platform; determine the second matching degree between the third operation result data and the fourth operation result data; and, in response to the second matching degree satisfying the second preset condition, determine that the execution results of the sixth implementation code and the seventh implementation code are inconsistent, and that the execution results of the fifth implementation code and the seventh implementation code are inconsistent.

[0079] Optionally, the generation methods of the third and fourth operation result data can refer to the relevant introductions above regarding the generation methods of the first and second operation result data, and the determination method of the second matching degree can refer to the relevant introductions above regarding the determination method of the first matching degree, and will not be repeated here.

[0080] After determining the second matching degree, the second processor 310 can determine whether the second matching degree meets the second preset condition. For example, the second preset condition could be: the second matching degree is less than or equal to the second preset matching degree. If the second matching degree is less than or equal to the second preset matching degree, the second processor 310 can determine that the second matching degree meets the second preset condition, the third operation result data and the fourth operation result data are inconsistent, and the second processor 310 can further determine that the execution results of the sixth implementation code and the seventh implementation code are inconsistent, and the execution results of the fifth implementation code and the seventh implementation code are inconsistent. If the second matching degree is greater than the second preset matching degree, the second processor 310 can determine that the second matching degree does not meet the second preset condition, the third operation result data and the fourth operation result data are consistent, and the second processor 310 can further determine that the execution results of the sixth implementation code and the seventh implementation code are consistent, and the execution results of the fifth implementation code and the seventh implementation code are consistent.

[0081] In an optional example, such as Figure 3-2 As shown, the heterogeneous computing platform 130 in the development phase (i.e., the test heterogeneous computing platform) includes a CPU and a GPU, but not a DSP. The heterogeneous computing platform 130 in the deployment phase (i.e., the embedded heterogeneous computing platform) includes a CPU, a GPU, and a DSP. Therefore, the fifth computing unit can be a DSP, and the sixth computing unit can be a CPU. The operators are implemented in the code of each type of computing unit. The sixth implementation code where the operator performs calculations through the sixth computing unit can be a DSP RefC kernel, and the fifth implementation code where the operator performs calculations through the fifth computing unit can be a DSP kernel. The seventh implementation code, serving as the baseline implementation code, can be a GPU kernel or a CPU kernel. For ease of understanding, it is assumed below that the seventh implementation code is a GPU kernel.

[0082] In practice, based on the GPU kernel, the GPU in the heterogeneous computing platform can be used to generate the third operation result data corresponding to the test data; based on the DSP RefC kernel, the CPU in the heterogeneous computing platform can be used to generate the fourth operation result data corresponding to the test data. If the second matching degree between the third and fourth operation result data is less than or equal to the second preset matching degree, it can be determined that the execution results of the DSP RefC kernel and the GPU kernel are inconsistent. Furthermore, since the DSP RefC kernel is used to simulate the DSP's operating mechanism, theoretically, the execution results of the DSP kernel and the DSP RefC kernel should be consistent. Therefore, it can be inferred that the execution results of the DSP kernel and the GPU kernel are also inconsistent. That is, although the heterogeneous computing platform does not include a DSP, causing the heterogeneous computing platform to be unable to execute the executable instructions corresponding to the DSP kernel, by introducing the DSP RefC kernel, it is still possible to infer whether the execution results of the DSP kernel and the GPU kernel are consistent.

[0083] As can be seen, by calling the seventh and sixth computing units in the heterogeneous computing test platform, the test data can be obtained corresponding to the computation results of the seventh and sixth implementation codes, respectively. Then, by calculating the matching degree between the computation results and determining whether the calculated second matching degree meets the second preset condition, it is possible to efficiently and quickly determine whether the execution results of the sixth and seventh implementation codes are consistent, and whether the execution results of the fifth and seventh implementation codes are consistent. Even if the heterogeneous computing test platform does not include the fifth computing unit, by introducing the sixth implementation code to simulate the operating mechanism of the fifth computing unit, it is possible to effectively verify whether the fifth implementation code used for computation through the fifth computing unit meets the requirement of consistent execution results.

[0084] In some optional examples, when the second processor 310 determines the implementation code of the operator in each type of computing unit based on the operator and each type of computing unit, it is specifically configured to: determine the operation implementation logic corresponding to the operator; and determine the implementation code of the operator in each type of computing unit according to the operation implementation logic based on each type of computing unit.

[0085] Optionally, the second processor 310 can determine the operation implementation logic corresponding to the operator based on the type of the operator, using any implementable method. The operation implementation logic corresponding to the operator can, for example, be used to indicate the multiple ordered steps required to obtain the output data of the operator from the input data.

[0086] In the embodiments of this disclosure, the second processor 310 can uniformly determine the implementation code of the operator in each type of computing unit according to the operation implementation logic corresponding to the operator and in combination with the operating mechanism of each type of computing unit. This helps ensure the logical consistency of the implementation code of the operator in each type of computing unit. For example, although the functions used in the implementation code corresponding to different types of computing units may differ, these implementation codes are all used to instruct the execution of multiple ordered steps in the previous section. Based on this, it is beneficial to ensure that the execution results of the implementation code of the operator in each type of computing unit are consistent. Alternatively, by optimizing some implementation code (e.g., non-baseline implementation code), the consistency of the execution results of the implementation code of the operator in each type of computing unit can be quickly achieved.

[0087] In some optional examples, such as Figure 3-3 As shown, the unified operator library construction apparatus 120 may further include a second memory 320. The second memory 320 is the circuitry within the unified operator library construction apparatus 120 that stores instructions and data. The second memory 320 includes one or more computer program products, which include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The second memory 320 may store at least one of the following: the unified operator library constructed by the second processor 310, instructions to be executed by the second processor 310, data to be processed by the second processor 310, and processing results obtained by the second processor 310 from data processing.

[0088] In some optional examples, such as Figure 3-3 As shown, the unified operator library construction apparatus 120 may further include a second input device 330 and a second output device 340, these components being interconnected via a bus system and / or other forms of connection mechanisms (not shown). The second input device 330 includes, for example, a keyboard, a mouse, etc. The second output device 340 includes, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc., and the second output device 340 can output various types of information.

[0089] Of course, for the sake of simplicity, Figure 3-3 Only some of the components in the unified operator library construction apparatus 120 relevant to this disclosure are shown, omitting components such as buses and input / output interfaces. In addition, the unified operator library construction apparatus 120 may include any other suitable components depending on the specific application.

[0090] In some optional examples, such as Figure 4As shown, in the development process of neural network models, the most primitive neural network model is the Torch model. The Torch model is converted into an ONNX model through optimization (such as pruning and fusion), and the ONNX model is converted into an AI model through compilation. The AI ​​model is used for deployment on embedded heterogeneous computing platforms. Regardless of whether it is a Torch model, an ONNX model, or an AI model, its backend execution can rely on a unified operator library to ensure that the execution results of the neural network model are consistent at different stages (such as consistent execution results in the development and deployment phases), without the need for manual adjustments to ensure the alignment of execution results, which helps to reduce development and maintenance costs.

[0091] Exemplary heterogeneous computing platform Figure 5 This is a schematic diagram of the architecture of a heterogeneous computing platform 130 provided in some exemplary embodiments of this disclosure. For example... Figure 5 As shown, the heterogeneous computing platform 130 integrates various types of computing units, such as a first computing unit, a second computing unit, a third computing unit, and a fourth computing unit. Furthermore, executable instructions are deployed on the heterogeneous computing platform 130. These executable instructions may include, for example, a first executable instruction corresponding to the first computing unit (Dequantize operator), a second executable instruction corresponding to the second computing unit (Dequantize operator), a third executable instruction corresponding to the third computing unit (Dequantize operator), a fourth executable instruction corresponding to the fourth computing unit (Dequantize operator), a fifth executable instruction corresponding to the first computing unit (Remap operator), a sixth executable instruction corresponding to the second computing unit (Remap operator), a seventh executable instruction corresponding to the third computing unit (Remap operator), and an eighth executable instruction corresponding to the fourth computing unit (Remap operator). The first to eighth executable instructions can all be generated by compiling the corresponding implementation code from a unified operator library.

[0092] In some optional examples, a front-end interface and an interface framework layer can also be configured. The front-end interface may include, for example, a front-end C++ interface, a front-end Python interface, etc. The interface framework layer may include multiple Application Programming Interfaces (APIs), such as the Dequantize API and the Remap API.

[0093] In practical implementation, if the front-end C++ interface or the front-end Python interface is called, the application programming interface in the interface framework layer can be further called. Assuming the called application programming interface in the interface framework layer is the Dequantize API, one executable instruction can be selected as the target executable instruction from the first, second, third, and fourth executable instructions. This selection can be made by the upper-layer scheduling system based on the load of the computing units corresponding to the first, second, third, and fourth executable instructions, or it can be based on configuration information. If the target executable instruction is the first executable instruction, it can be executed on the first computing unit in the heterogeneous computing platform 130 to achieve Dequantize computation. If the target executable instruction is the second executable instruction, it can be executed on the second computing unit in the heterogeneous computing platform 130 to achieve Dequantize computation. Since both the first and second executable instructions are generated based on a unified operator library, the execution results are consistent whether the Dequantize calculation is performed by executing the first executable instruction or by executing the second executable instruction. In other words, the embodiments of this disclosure can ensure the consistency of operator execution across hardware.

[0094] Exemplary methods Figure 6 This is a flowchart illustrating an executable instruction generation method provided by some exemplary embodiments of this disclosure. The executable instruction generation method can be applied to the executable instruction generation apparatus 110 in the above embodiments. Figure 6 As shown, the executable instruction generation method includes: Step 610: Determine the neural network model to be compiled; Step 620: Determine the target computing unit corresponding to the first operator in the neural network model from the heterogeneous computing platform; Step 630: In the unified operator library, determine the target implementation code corresponding to the first operator and the target computing unit; wherein, the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent; Step 640: Compile the target implementation code to generate executable instructions for deployment on heterogeneous computing platforms.

[0095] In some optional examples, the first operator includes multiple implementations of code for different types of computational units: The first implementation code for the first operator to perform calculations through the first computation unit; and The second implementation code for the first operator to perform calculations through the second computation unit, wherein the type of the second computation unit is different from that of the first computation unit.

[0096] Figure 7 This is a flowchart illustrating a unified operator library construction method provided by some exemplary embodiments of this disclosure. The unified operator library construction method can be applied to the unified operator library construction apparatus 120. For example... Figure 7 As shown, the unified operator library construction method includes: Step 710: Determine at least one class of operators included in the deep learning framework; Step 720: For each type of operator, determine the types of computational units that support the execution of the operators; Step 730: Based on the operators and various types of computational units, determine the implementation code of the operators in each type of computational unit; Step 740: In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are consistent, and based on the implementation code of the operator in each type of computing unit, the operator library corresponding to the operator is constructed. Step 750: Construct a unified operator library based on the operator libraries corresponding to various operators.

[0097] In some optional examples, the implementation code of the operator in each type of computation unit includes third implementation code where the operator is computed through a third computation unit and fourth implementation code where the operator is computed through a fourth computation unit; like Figure 8 As shown, the execution results of the determination operator in step 740 are consistent in the implementation code of each type of computation unit, including: Step 810: Based on the third implementation code, use the third computing unit in the test heterogeneous computing platform to generate the first operation result data corresponding to the test data; Step 820: Based on the fourth implementation code, the fourth computing unit in the test heterogeneous computing platform is used to generate the second operation result data corresponding to the test data; Step 830: Determine the first matching degree between the first operation result data and the second operation result data; Step 840: In response to the first matching degree satisfying the first preset condition, determine that the execution results of the third implementation code and the fourth implementation code are consistent.

[0098] In some optional examples, such as Figure 9 As shown, the unified operator library construction method also includes: Step 910: Based on the different types of computing units included in the test heterogeneous computing platform, select one implementation code from the implementation code of each type of computing unit as the benchmark implementation code; Step 920: In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are inconsistent. The implementation code of the operator in each type of computing unit, excluding the baseline implementation code, is optimized to obtain the optimized implementation code. Step 930: Based on the baseline implementation code and the optimized implementation code, construct the operator library corresponding to the operator.

[0099] In some optional examples, the implementation code of the operator in each type of computational unit, in addition to the baseline implementation code, includes: The fifth implementation code, which performs calculations through the fifth computational unit, has a type different from any computational unit in the heterogeneous computing platform being tested. The fifth implementation code conforms to the operating mechanism of the fifth computational unit; and... The sixth implementation code is used to simulate the operation mechanism of the fifth computing unit. The sixth computing unit is located in the test heterogeneous computing platform.

[0100] In some optional examples, the operators are implemented in the implementation code of each type of computation unit, with the seventh implementation code in which the operators are computed in the seventh computation unit serving as the baseline implementation code; like Figure 10 As shown, the execution results of the determination operator in step 920 are inconsistent in the implementation code of each type of computation unit, including: Step 1010: Based on the seventh implementation code, use the seventh computing unit in the test heterogeneous computing platform to generate the third operation result data corresponding to the test data; Step 1020: Based on the sixth implementation code, use the sixth computing unit in the test heterogeneous computing platform to generate the fourth operation result data corresponding to the test data; Step 1030: Determine the second matching degree between the third operation result data and the fourth operation result data; Step 1040: In response to the second matching degree satisfying the second preset condition, it is determined that the execution results of the sixth implementation code and the seventh implementation code are inconsistent, and the execution results of the fifth implementation code and the seventh implementation code are also inconsistent.

[0101] In some optional examples, step 730 includes: Determine the operational implementation logic corresponding to the operator; Based on the various types of computing units, and according to the operational implementation logic, the implementation code of the operators in each type of computing unit is determined.

[0102] In the methods disclosed herein, the various optional embodiments, optional implementation methods and optional examples disclosed in the above exemplary device section can be flexibly selected and combined as needed to achieve the corresponding functions and effects, and this disclosure does not list them all.

[0103] The beneficial technical effects corresponding to the exemplary embodiments of this method can be found in the corresponding beneficial technical effects of the exemplary device section above, and will not be repeated here.

[0104] Exemplary computer program products and computer-readable storage media In addition to the methods and apparatus described above, embodiments of this disclosure may also be computer program products comprising computer program instructions that, when executed by a processor, cause the processor to perform the steps in the executable instruction generation method or unified operator library construction method according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0105] Computer program products can be written in any combination of one or more programming languages ​​to perform the operations of embodiments of this disclosure. These programming languages ​​include object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on a user's computing device, partially on a user's computing device, as a standalone software package, partially on a user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0106] Furthermore, embodiments of this disclosure may also be computer-readable storage media storing computer program instructions that, when executed by a processor, cause the processor to perform the steps in the executable instruction generation method or unified operator library construction method according to various embodiments of this disclosure as described in the "Exemplary Methods" section of this specification.

[0107] The computer-readable storage medium may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0108] The basic principles of this disclosure have been described above with reference to specific embodiments. However, the advantages, benefits, and effects mentioned in this disclosure are merely examples and not limitations, and should not be considered as essential features of each embodiment of this disclosure. The specific details disclosed above are for illustrative and facilitative purposes only, and are not limitations. These details do not limit the scope of this disclosure to the necessity of employing the specific details described above.

[0109] Various modifications and variations can be made to this disclosure without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this disclosure and their equivalents, this disclosure is also intended to include such modifications and variations.

Claims

1. An executable instruction generation apparatus, comprising at least one first processor, the first processor being configured to: Determine the neural network model to be compiled; From the heterogeneous computing platform, determine the target computing unit corresponding to the first operator in the neural network model; Within the unified operator library, target implementation code corresponding to the first operator and the target computation unit is determined; wherein... The unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent. The target implementation code is compiled to generate executable instructions for deployment on the heterogeneous computing platform.

2. The executable instruction generation apparatus according to claim 1, wherein, The first operator includes various implementation codes in different types of computational units, including: The first implementation code for the first operator to perform calculations through the first computing unit; and The second implementation code for the first operator to be calculated by the second computing unit, wherein the type of the second computing unit is different from the type of the first computing unit.

3. A unified operator library construction apparatus, comprising at least one second processor, the second processor being configured to: Identify at least one class of operators included in a deep learning framework; For each type of operator, identify the types of computational units that support the execution of the operator; Based on the operator and the various types of computing units, determine the implementation code of the operator in each type of computing unit; In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are consistent, and an operator library corresponding to the operator is constructed based on the implementation code of the operator in each type of computing unit. A unified operator library is constructed based on the operator libraries corresponding to various operators.

4. The unified operator library construction apparatus according to claim 3, wherein, The implementation code of the operator in each type of computing unit includes third implementation code in which the operator performs calculations through a third computing unit and fourth implementation code in which the operator performs calculations through a fourth computing unit; When the second processor determines that the execution results of the operator are consistent across the implementation code of each type of computing unit, it is specifically configured as follows: Based on the third implementation code, the third computing unit in the test heterogeneous computing platform is used to generate the first operation result data corresponding to the test data; Based on the fourth implementation code, the fourth computing unit in the test heterogeneous computing platform is used to generate the second operation result data corresponding to the test data; Determine the first matching degree between the first operation result data and the second operation result data; In response to the first matching degree satisfying the first preset condition, it is determined that the execution results of the third implementation code and the fourth implementation code are consistent.

5. The unified operator library construction apparatus according to claim 3, wherein, The second processor is also configured to: Based on the different types of computing units included in the test heterogeneous computing platform, one implementation code is selected from the implementation code of each type of computing unit as the benchmark implementation code; In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are inconsistent, and the implementation code of the operator in each type of computing unit other than the baseline implementation code is optimized to obtain the optimized implementation code; Based on the baseline implementation code and the optimized implementation code, construct the operator library corresponding to the operator.

6. The unified operator library construction apparatus according to claim 5, wherein, The implementation code of the operators in each type of computing unit, excluding the baseline implementation code, includes: The operator is computed through a fifth computational unit, the type of which differs from any computational unit in the heterogeneous computing platform being tested. This fifth implementation code conforms to the operating mechanism of the fifth computational unit. The operator is calculated by a sixth computing unit located in the test heterogeneous computing platform. The sixth implementation code is used to simulate the operating mechanism of the fifth computing unit.

7. The unified operator library construction apparatus according to claim 6, wherein, The operators are respectively in the implementation code of each type of computing unit, and the seventh implementation code in which the operators are calculated in the seventh computing unit is used as the base implementation code; The second processor determines that when the execution results of the implementation code of the various types of computing units are inconsistent, the operator is specifically configured as follows: Based on the seventh implementation code, the third operation result data corresponding to the test data is generated using the seventh computing unit in the test heterogeneous computing platform; Based on the sixth implementation code, the sixth computing unit in the heterogeneous computing platform is used to generate the fourth operation result data corresponding to the test data; Determine the second matching degree between the third operation result data and the fourth operation result data; In response to the second matching degree satisfying the second preset condition, it is determined that the execution results of the sixth implementation code and the seventh implementation code are inconsistent, and the execution results of the fifth implementation code and the seventh implementation code are also inconsistent.

8. The unified operator library construction apparatus according to claim 3, wherein, The second processor, based on the operator and the various types of computing units, determines that the operator is specifically configured as follows when implementing the code of each type of computing unit: Determine the operation implementation logic corresponding to the operator; Based on the aforementioned types of computing units, and in accordance with the aforementioned operational implementation logic, the implementation code of the operator in each type of computing unit is determined.

9. A method for generating executable instructions, comprising: Determine the neural network model to be compiled; From the heterogeneous computing platform, determine the target computing unit corresponding to the first operator in the neural network model; In a unified operator library, target implementation code corresponding to the first operator and the target computing unit is determined; wherein, the unified operator library includes multiple implementation codes of the first operator in different types of computing units, and the execution results of the multiple implementation codes of the first operator in different types of computing units are consistent; The target implementation code is compiled to generate executable instructions for deployment on the heterogeneous computing platform.

10. A method for constructing a unified operator library, comprising: Identify at least one class of operators included in a deep learning framework; For each type of operator, identify the types of computational units that support the execution of the operator; Based on the operator and the various types of computing units, determine the implementation code of the operator in each type of computing unit; In response to the test data corresponding to the operator, it is determined that the execution results of the implementation code of the operator in each type of computing unit are consistent, and an operator library corresponding to the operator is constructed based on the implementation code of the operator in each type of computing unit. A unified operator library is constructed based on the operator libraries corresponding to various operators.

11. A computer-readable storage medium storing a computer program that is executed by a processor to perform the executable instruction generation method of claim 9 or the unified operator library construction method of claim 10.

Images