Operator hybrid compilation method and apparatus, electronic device, and medium

Abstract

The application provides an operator mixed compilation method and device, electronic equipment and medium, and relates to the technical field of compilers. The method comprises the following steps: determining at least one hardware adaptation characteristic of each to-be-compiled operator according to operator attribute information of each to-be-compiled operator in a calculation graph; determining a hardware adaptation score of each to-be-compiled operator based on the at least one hardware adaptation characteristic, so as to classify each to-be-compiled operator into a first category suitable for first hardware execution, a second category suitable for second hardware execution, or a third category suitable for cooperative execution of the first hardware and the second hardware according to the hardware adaptation score, and obtain a classification result; and compiling each to-be-compiled operator based on the classification result, so as to obtain hardware configuration information corresponding to each to-be-compiled operator, so that the first hardware and / or the second hardware executes each to-be-compiled operator according to the hardware configuration information. The optimal operator allocation decision can be automatically completed without human intervention.

Description

Technical Field

[0001] This application relates to the field of compiler technology, and in particular to an operator hybrid compilation method, apparatus, electronic device and medium. Background Technology

[0002] With the rapid development of Artificial Intelligence (AI) technology, the scale and complexity of deep neural network models are constantly increasing. To balance the processing efficiency of regular computations (such as matrix multiplication and convolution) and irregular computations (such as activation functions and normalization), AI acceleration chips typically adopt a heterogeneous computing architecture, integrating two types of computing units on the same chip: AME (RISC-V Attached Matrix Extension) and CGRA (Coarse-Grained Reconfigurable Array). AME is suitable for processing regular operators, while CGRA is suitable for processing irregular operators.

[0003] In related technologies, the operator task allocation problem in the above-mentioned heterogeneous computing architecture is usually addressed by manual annotation. That is, senior engineers pre-analyze each operator, manually specify whether each operator should be assigned to AME or CGRA for execution, and write the annotation results into the compiler.

[0004] However, the relevant technologies have the following drawbacks: First, manual annotation is costly, requiring analysis of each operator, which is time-consuming and labor-intensive; second, manual annotation is difficult to achieve global optimization; and third, manual annotation is required again when the model structure changes (such as the appearance of new operators), lacking adaptive capabilities. Summary of the Invention

[0005] This application provides an operator hybrid compilation method, apparatus, electronic device, and medium to solve the technical problems of high manual annotation cost, difficulty in achieving global optimum in operator task allocation, and lack of adaptive capability in related technologies.

[0006] A first aspect of this application proposes an operator hybrid compilation method, comprising: determining at least one hardware adaptation feature for each operator to be compiled based on operator attribute information of at least one operator to be compiled in a computation graph, wherein the at least one hardware adaptation feature is used to characterize the degree of adaptation of each operator to be compiled to be executed by different hardware; determining a hardware adaptation score for each operator to be compiled based on the at least one hardware adaptation feature, so as to classify each operator to be compiled into a first category suitable for execution by a first hardware, or a second category suitable for execution by a second hardware, or a third category suitable for joint execution by the first hardware and the second hardware, thereby obtaining a classification result; and compiling each operator to be compiled based on the classification result to obtain hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware execute each operator to be compiled according to the hardware configuration information.

[0007] In some embodiments, at least one hardware adaptation feature includes at least one of computational alignment, operator type encoding value, and data reuse distance. Determining at least one hardware adaptation feature of each operator to be compiled based on operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph includes: determining the computational alignment of each operator to be compiled based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware, wherein the computational alignment characterizes the degree of matching between the computational load of each operator to be compiled and the first computational load; determining the operator type encoding value of each operator to be compiled based on the operator type of each operator to be compiled, wherein the operator type includes regular operators and irregular operators; and determining the data reuse distance of each operator to be compiled based on the data flow information of each operator to be compiled, wherein data reuse characterizes the degree to which the input data of each operator to be compiled is reused.

[0008] In some embodiments, determining the computational alignment of each operator to be compiled based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware includes: splitting each operator to be compiled into a task based on the computational load of each operator to be compiled and the first computational load of the first hardware to obtain at least one subtask corresponding to each operator to be compiled; determining the number of matching subtasks in the at least one subtask whose computational load matches the first computational load; and determining the computational alignment based on the ratio of the number of matching subtasks to the total number of at least one subtask.

[0009] In some embodiments, based on at least one hardware adaptation feature, a hardware adaptation score is determined for each operator to be compiled, so as to classify each operator to be compiled into a first category suitable for execution on a first hardware, a second category suitable for execution on a second hardware, or a third category suitable for joint execution on the first and second hardware, according to the hardware adaptation score. The classification result includes: determining at least one weight corresponding to at least one hardware adaptation feature; performing a weighted operation on at least one hardware adaptation feature based on at least one weight to obtain a hardware adaptation score for each operator to be compiled; and classifying each operator to be compiled into a first category, a second category, or a third category based on the hardware adaptation score, a preset score threshold, and a preset boundary to obtain a classification result.

[0010] In some embodiments, based on hardware adaptation scores, preset score thresholds, and preset boundaries, each operator to be compiled is classified into a first category, a second category, or a third category to obtain the classification result, including: determining a first score threshold based on the preset score threshold and preset boundaries, wherein the preset score threshold is greater than the first score threshold; classifying the first operator to be compiled into the first category if the first hardware adaptation score of the first operator to be compiled is greater than the preset score threshold, wherein the first operator to be compiled is any one of at least one operator to be compiled; classifying the first operator to be compiled into the second category if the first hardware adaptation score is less than the first score threshold; and classifying the first operator to be compiled into the third category if the first hardware adaptation score is greater than or equal to the first score threshold and the first hardware adaptation score is less than or equal to the preset score threshold.

[0011] In some embodiments, compiling each operator to be compiled according to the classification result to obtain hardware configuration information for each operator to be compiled includes: when the second operator to be compiled is a first category, compiling the second operator to be compiled to generate first hardware configuration information matching the first hardware, the first hardware configuration information being the configuration data required by the first hardware to execute the second operator to be compiled, and the second operator to be compiled being any one of at least one operator to be compiled; when the second operator to be compiled is a second category, compiling the second operator to be compiled to generate second hardware configuration information matching the second hardware, the second hardware configuration information being the configuration data required by the second hardware to execute the second operator to be compiled; when the second operator to be compiled is a third category, compiling the second operator to be compiled to generate third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware, the third hardware configuration information being the configuration data required by the first hardware to execute the first part of the computational task of the second operator to be compiled, and the fourth hardware configuration information being the configuration data required by the second hardware to execute the second part of the computational task of the second operator to be compiled; wherein, the first part of the computational task and the second part of the computational task are obtained by task splitting the second operator to be compiled.

[0012] In some embodiments, the second operator to be compiled is compiled to generate third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware. Before this, the method includes: splitting the second operator to be compiled into a task based on the computational load of the second operator to be compiled and the first computational load of the first hardware to obtain a first part of the computational task suitable for execution on the first hardware and a second part of the computational task suitable for execution on the second hardware; wherein the computational load of the first part of the computational task matches the first computational load of the first hardware, and the computational load of the second part of the computational task does not match the first computational load of the first hardware.

[0013] A second aspect of this application provides an operator hybrid compilation apparatus, the apparatus comprising: The determining unit is used to determine at least one hardware adaptation feature of each operator to be compiled based on the operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph. The at least one hardware adaptation feature is used to characterize the degree of adaptation of each operator to be compiled to be executed by different hardware. The classification unit is used to determine the hardware adaptation score of each operator to be compiled based on at least one hardware adaptation feature, so as to classify each operator to be compiled into a first category suitable for execution on the first hardware, or a second category suitable for execution on the second hardware, or a third category suitable for joint execution on the first hardware and the second hardware, and obtain the classification result. The compilation unit is used to compile each operator to be compiled according to the classification results, and obtain the hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware can execute each operator to be compiled according to the hardware configuration information.

[0014] A third aspect of this application provides an electronic device comprising: a processor and a memory for storing a computer program capable of running on the processor, wherein the processor, when running the computer program, performs the method described in any embodiment of the first aspect.

[0015] A fourth aspect of this application provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to perform the methods described in any embodiment of the first aspect of this application.

[0016] In summary, the operator hybrid compilation method proposed in this application determines at least one hardware adaptation feature for each operator to be compiled based on the operator attribute information of at least one operator to be compiled in the computation graph. This hardware adaptation feature characterizes the degree of adaptation of each operator to be compiled to different hardware execution methods. Based on this feature, a hardware adaptation score is determined for each operator to be compiled, and the operators are classified into three categories: a first category suitable for execution on a first hardware, a second category suitable for execution on a second hardware, or a third category suitable for collaborative execution by both the first and second hardware. Based on the classification results, each operator to be compiled is processed to obtain the corresponding hardware configuration information, enabling the first and / or second hardware to execute each operator according to the hardware configuration information. Compared to manual annotation schemes, this application achieves fully automated operator scheduling, adaptively making optimal decisions based on operator hardware characteristics without manual intervention. This avoids the subjectivity and non-optimal nature of manual annotation. Furthermore, by introducing collaborative execution categories, it fully leverages the complementary advantages of different hardware, effectively improving the utilization efficiency of heterogeneous computing resources and operator execution performance.

[0017] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0018] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0019] Figure 1 A flowchart illustrating the first operator hybrid compilation method provided in this application embodiment; Figure 2 A flowchart illustrating the second operator hybrid compilation method provided in this application embodiment; Figure 3 A general flowchart of a specific operator hybrid compilation method provided for embodiments of this application; Figure 4 This is a schematic diagram of the structure of an operator hybrid compiler device provided in an embodiment of this application; Figure 5 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0020] The embodiments of this application are described in detail below. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this application, and should not be construed as limiting this application.

[0021] With the rapid development of artificial intelligence technology, the scale and complexity of deep neural network models are constantly increasing, leading to a sharp rise in the demand for computing resources. To efficiently handle the numerous regular computations such as matrix multiplication and convolution, as well as irregular computations such as activation functions and normalization, that exist in neural networks, modern AI accelerator chips generally adopt heterogeneous computing architectures. This involves integrating multiple different types of computing units on the same chip to balance computational efficiency and flexibility. Currently, mainstream AI accelerator chips typically integrate the following two types of computing units: (1) AME: AME is a matrix operation extension instruction in the RISC-V instruction set, specifically designed for efficient matrix multiplication. The chip has a dedicated matrix operation unit that can recognize and execute AME instructions. AME is implemented using dedicated hardware circuitry and is deeply optimized for regular operators such as matrix multiplication and convolution, resulting in extremely high execution efficiency. A single AME instruction can process 16384 bits of data simultaneously. However, because the hardware circuitry of AME is designed specifically for regular matrix calculations, it has poor flexibility and cannot efficiently handle irregular operators such as activation functions and normalization.

[0022] (2) CGRA: CGRA is a hardware array consisting of multiple processing elements (PEs). Each processing element can be dynamically configured with different operational functions (such as addition, multiplication, activation functions, etc.) through configuration words. CGRA has extremely high flexibility and can adapt to various irregular operators. Function switching can be achieved simply by modifying the configuration words. Because the processing units of CGRA are designed for general purposes and lack dedicated optimizations for regular operators, the efficiency of performing regular matrix calculations is significantly lower than that of AME.

[0023] This demonstrates that AME exhibits a significant performance advantage in handling regular matrix computations, while CGRA offers flexible adaptability when dealing with irregular computations. How to rationally allocate computational tasks across these two types of hardware units is a key challenge in compiler design for heterogeneous computing architectures.

[0024] In related technologies, the operator task allocation problem for the aforementioned heterogeneous computing architecture is mainly addressed through manual annotation. Specifically, senior engineers with hardware architecture knowledge pre-analyze each operator in the computation graph before model deployment. Based on their engineering experience and hardware characteristics, they manually determine whether the operator is suitable to be executed by AME or CGRA, and write the judgment results into the compiler configuration in the form of annotations to complete the compilation process.

[0025] However, the relevant technologies have significant drawbacks: First, manual annotation is costly, requiring analysis of each operator individually, which is time-consuming and labor-intensive; second, manual annotation struggles to achieve global optimization, for example, an operator might be partially suitable for AME and partially suitable for CGRA, but manual annotation only allows for one of the two; third, manual annotation is required again when the model structure changes (e.g., when new operators appear), lacking adaptive capabilities. Therefore, there is an urgent need for a compilation method that can automatically and optimally classify operators for execution on different hardware.

[0026] To address the aforementioned issues, this application proposes a hybrid operator compilation method. This method automatically extracts hardware adaptation features (including computational alignment, operator type encoding values, and data reuse distance) from each operator to be compiled. Based on these features, a hardware adaptation score is calculated. Operators are then automatically classified according to their hardware adaptation scores into those suitable for execution on the first hardware (e.g., AME), the second hardware (e.g., CGRA), or a combination of both. The method then automatically compiles and generates the corresponding hardware configuration information based on the classification results to complete execution. Compared to manual annotation schemes, this application achieves fully automated operator scheduling. It adaptively makes optimal decisions based on hardware features and model changes without human intervention, avoiding the subjectivity and non-optimal aspects of manual annotation. Furthermore, by introducing a collaborative execution category, it fully leverages the complementary advantages of different hardware, effectively improving the overall performance and deployment efficiency of heterogeneous computing systems. This method exhibits strong adaptability, versatility, and practicality.

[0027] The operator hybrid compilation method provided in this application will be described in detail below with reference to the accompanying drawings.

[0028] Figure 1 A flowchart illustrating the first operator hybrid compilation method provided in this application embodiment. Figure 1 As shown, the operator hybrid compilation method includes steps 101-103.

[0029] Step 101: Determine at least one hardware adaptation feature of each operator to be compiled based on the operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph.

[0030] In the embodiments of this application, the computation graph, i.e., the AI ​​model, is represented as a directed acyclic graph in a framework (such as PyTorch, TensorFlow). Each node in the graph represents a computational operation (i.e., an operator), and the edges represent the flow direction of data (tensors) between operators. The computation graph describes the complete computational process of the entire neural network from input to output.

[0031] Operator attribute information is the inherent information of an operator, including the operator's computational load (such as the dimension M×N×K of matrix multiplication), operator type (such as regular or irregular operator), and data flow relationship (such as which computations use the input data).

[0032] Hardware compatibility features refer to feature values ​​extracted and calculated from operator attribute information, which are used to quantify the degree of compatibility between the operator and different hardware.

[0033] In some embodiments, at least one hardware adaptation feature includes at least one of computational alignment, operator type encoding value, and data reuse distance.

[0034] The computational alignment refers to the degree of matching between the computational load of the operator to be compiled and the single-processing capability of the hardware. For example, when AME performs matrix operations, its hardware circuit has a fixed single-processing capability (i.e., the maximum amount of data that an AME instruction can process, represented by MLEN (Matrix Lane Enable Number), for example, MLEN = 16384 bits). When the computational load of the operator to be compiled is exactly an integer multiple of MLEN, the AME can work at full capacity without wasting computational power; when the computational load of the operator to be compiled is not an integer multiple of MLEN, the AME will generate idle cycles during execution, resulting in wasted computational power. The computational alignment is used to quantify this degree of matching, with a value range of [0,1]. The closer the value is to 1, the higher the degree of matching and the higher the utilization rate of the AME.

[0035] The operator type encoding value is a numerical representation of the computational pattern type of the operator to be compiled. Based on the different computational patterns, operators to be compiled can be divided into two categories: regular operators and irregular operators. Regular operators refer to operators with regular computational patterns, continuous data access, and suitable for dedicated hardware acceleration, such as matrix multiplication (MatMul) and convolution (Conv). Irregular operators refer to operators with irregular computational patterns, scattered data access, and requiring flexible hardware processing, such as activation functions (ReLU) and normalization (LayerNorm). To facilitate participation in subsequent weighted calculations, regular operators are encoded as the value 1.0, and irregular operators are encoded as the value 0.0.

[0036] Data reuse distance refers to the degree to which input data of an operator is reused during computation, and is used to measure the temporal locality of data. In this application, the data reuse distance is obtained by performing data flow analysis on the operator, representing the total number of times the same input data is reused during the operator's computation. The value is normalized to the range [0,1], and the larger the value, the higher the degree of data reuse. A large data reuse distance (close to 1) means that the input data is repeatedly used during computation, and the data needs to be repeatedly loaded from main memory. The dedicated loading path of AME is more efficient, so it is more suitable for AME execution. A small data reuse distance (close to 0) means that the input data is only used once or very rarely, and the data can reside in the PE local memory of CGRA for a long time, so it is suitable for CGRA execution.

[0037] Step 102: Based on at least one hardware adaptation feature, determine the hardware adaptation score of each operator to be compiled, so as to classify each operator to be compiled into a first category suitable for execution on the first hardware, or a second category suitable for execution on the second hardware, or a third category suitable for joint execution on the first hardware and the second hardware, and obtain the classification result.

[0038] In the embodiments of this application, the first hardware and the second hardware are different types of computing units in a heterogeneous computing architecture. The first hardware can be AME or CGRA, and the second hardware can be CGRA or AME. It is understood that when the first hardware is AME, the second hardware is CGRA; and when the first hardware is CGRA, the second hardware is AME. This application uses an example where the first hardware is AME and the second hardware is CGRA for illustration.

[0039] This application employs a classification decision function to determine the hardware adaptation score of each operator to be compiled based on at least one hardware adaptation feature. The hardware adaptation score is a comprehensive score obtained by weighted summation of one or more hardware adaptation features according to their respective weights. A higher hardware adaptation score indicates that the operator is more suitable for execution by AME, while a lower score indicates that it is more suitable for execution by CGRA. A score in the intermediate range indicates that both AME and CGRA are required for execution.

[0040] Step 103: Based on the classification results, compile each operator to be compiled to obtain the hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware can execute each operator to be compiled according to the hardware configuration information.

[0041] In the embodiments of this application, compilation refers to the process of using a compiler to convert a high-level abstract description of the operator to be compiled (such as matrix multiplication) into low-level configuration data that can be recognized and executed by heterogeneous hardware.

[0042] Specifically, this application employs a Just-In-Time (JIT) compiler to compile each operator. The JIT compiler dynamically acquires the actual input size and computational load information of the operator during program runtime, and accordingly calculates hardware adaptation features in real time, completing classification decisions and compilation configuration, thereby enabling adaptation to changes in model structure and dynamic input shapes.

[0043] Hardware configuration information is control data sent to the hardware execution unit, including a description of the computation task, data address, hardware control parameters, etc. The configuration information of the first hardware (AME) is represented by dedicated instruction codes, while the configuration information of the second hardware (CGRA) is represented by a configuration bitstream. In specific implementations, the hardware configuration information is the "configuration package" required for the hardware to execute the operator.

[0044] Based on the classification results of each operator to be compiled, this application executes different compilation logic for different categories: If the operator to be compiled is of the first category, then hardware configuration information (such as AME instruction encoding) matching the first hardware is generated, so that the first hardware can independently complete all calculations; If the operator to be compiled is of the second category, then hardware configuration information (such as CGRA configuration bitstream) matching the second hardware is generated, so that the second hardware can independently complete all calculations; If the operator to be compiled is of the third category, the computation task of the operator is first split, and then the configuration information of the first hardware and the second hardware are generated separately, and the two work together to complete the computation.

[0045] After compilation, the generated hardware configuration information is sent to the corresponding hardware execution unit to drive the hardware to complete the operator's computation task.

[0046] In summary, the operator hybrid compilation method proposed in this application determines at least one hardware adaptation feature for each operator to be compiled based on the operator attribute information of at least one operator to be compiled in the computation graph. Based on this feature, a hardware adaptation score is determined for each operator to be compiled, and each operator is classified into a first category suitable for execution on first hardware, a second category suitable for execution on second hardware, or a third category suitable for collaborative execution by first and second hardware, resulting in a classification result. Based on the classification result, each operator to be compiled is processed to obtain the corresponding hardware configuration information, enabling the first and / or second hardware to execute each operator according to the hardware configuration information. This achieves fully automated operator scheduling, enabling adaptive optimal decision-making based on operator hardware features without manual intervention, avoiding the subjectivity and non-optimal nature of manual annotation. Furthermore, by introducing collaborative execution categories, it fully leverages the complementary advantages of different hardware, effectively improving the utilization efficiency of heterogeneous computing resources and operator execution performance.

[0047] As one possible implementation method, Figure 2 A flowchart of the second operator hybrid compilation method provided in an embodiment of this application is shown. Based on... Figure 1 The illustrated embodiment further defines steps 101, 102, and 103, wherein step 101 includes steps 201, 202, and 203; step 102 includes steps 204, 205, and 206; and step 103 includes steps 207, 208, and 209, as shown. Figure 2 As shown, the method includes the following steps: Step 201: Based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware, determine the computational load alignment of each operator to be compiled.

[0048] In the embodiments of this application, the computational load of the operator to be compiled refers to the total amount of computation that the operator to be compiled needs to complete. The first computational load refers to the maximum amount of computation that the first hardware (AME) can process in a single instruction. This application takes the maximum amount of computation as MLEN = 16384 bits as an example, that is, a single AME instruction can process a maximum of 16384 bits of matrix data. It is understood that the specific value of MLEN is only for illustrative purposes, and MLEN can adopt different values ​​depending on the hardware implementation. This application does not limit this value.

[0049] The computational alignment represents the degree of matching between the computational load of each operator to be compiled and the first computational load, with a value range of [0,1]. The closer the value is to 1, the more fully the operator's computational load can utilize the processing power of AME, and the higher the execution efficiency of AME; the closer the value is to 0, the worse the matching degree, and the more computational power will be wasted when executed by AME.

[0050] In some embodiments, determining the computational alignment of each operator to be compiled based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware includes: splitting each operator to be compiled into a task based on the computational load of each operator to be compiled and the first computational load of the first hardware to obtain at least one subtask corresponding to each operator to be compiled; determining the number of matching subtasks in the at least one subtask whose computational load matches the first computational load; and determining the computational alignment based on the ratio of the number of matching subtasks to the total number of at least one subtask.

[0051] Specifically, the computational alignment (denoted as alignment) needs to be calculated through task splitting and proportional calculation, as follows: alignment = alignment MLEN. / Among them, aligning MLEN =The computational load is exactly equal to the number of MLEN subtasks (i.e., the number of matching subtasks); Total number of subtasks = the total number of subtasks after splitting.

[0052] In one example, taking a 128×128×128 (i.e., M×N×K) matrix multiplication as an example, the AME can process a 64×64×64 matrix block at a time (corresponding to MLEN=16,384 bits, FP16 precision). The matrix multiplication task is split according to the first computational load (MLEN) of the first hardware. Since 128 can be split into 2×64 (i.e., 128=2×64, with a remainder of 0), after splitting along the three dimensions of M, N, and K, a total of 2×2×2=8 subtasks are obtained, each subtask being 64×64×64 in size. The computational load of each subtask is 64×64×64×16bit=4,194,304 bits, which is equal to 4 times the MLEN (4,194,304 / 16,384=256). Since the computational load of each subtask is an integer multiple of MLEN (i.e., aligned with MLEN), and the number of matched subtasks is 8, with a total of 8 subtasks, the computational alignment is 8 / 8 = 1. This result indicates that the computational load of the operator to be compiled matches the processing capacity of AME, and it is preliminarily determined that it is suitable for execution by AME.

[0053] Step 202: Determine the operator type encoding value for each operator to be compiled based on its operator type.

[0054] In the embodiments of this application, the operator types include regular operators and irregular operators.

[0055] In some embodiments, for regular operators (such as MatMul, Conv), the operator type encoding value is 1, which is suitable for AME execution; for irregular operators (such as ReLU, LayerNorm, Softmax), the operator type encoding value is 0, which is suitable for CGRA execution.

[0056] Step 203: Determine the data reuse distance of each operator to be compiled based on the data flow information of each operator to be compiled.

[0057] In the embodiments of this application, the data flow information describes the reading and usage pattern of the operator input data during the calculation process, including information such as the number of times the data is read, the access order, and the data dependencies, which is obtained by the compiler performing data flow analysis on the operator.

[0058] Data reuse distance characterizes the degree to which the input data of each operator to be compiled is reused. In this application, data reuse distance represents the total number of times the same input data is reused during the operator computation process. The value is normalized to the range [0,1]. The larger the value, the higher the degree of data reuse, and the more suitable it is for AME to execute.

[0059] Specifically, the compiler performs data flow analysis on the operators to be compiled, counting the total number of times each input data is used in the computation process. For the matrix multiplication operator Y = X × W, the weight matrix W is repeatedly read when calculating each row of the output matrix. If the output matrix has 128 rows, W is reused 128 times, which is close to 1 after normalization, making it suitable for execution by AME. For the ReLU operator, each input element participates in the calculation only once and is not reused, which is close to 0 after normalization, making it suitable for execution by CGRA.

[0060] Step 204: Determine at least one weight corresponding to at least one hardware adaptation feature.

[0061] In the embodiments of this application, weight refers to the coefficient assigned to each hardware adaptation feature, used to represent the importance of that feature in the classification decision. The larger the weight value, the greater the influence of the corresponding hardware adaptation feature on the final classification result. The specific values ​​of each weight can be determined by searching for the optimal combination on the validation set through offline experiments (such as grid search).

[0062] In some embodiments, the hardware adaptation features include three items: computational alignment, operator type encoding value, and data reuse distance, with corresponding weights w1, w2, and w3, respectively. In one example, the weights in this application are w1=0.5, w2=0.3, and w3=0.2, meaning that computational alignment has the greatest impact on the classification result, followed by operator type, while the impact of data reuse distance is relatively small.

[0063] It is understood that the above weight values ​​are merely illustrative and do not constitute a limitation on the scope of protection of this application. In practical applications, the weight coefficients can be determined and adjusted according to the performance characteristics of the specific hardware platform, the needs of the application scenario, or through automated search methods, and this application does not impose any restrictions on this.

[0064] Step 205: Based on at least one weight, perform weighted operations on at least one hardware adaptation feature to obtain the hardware adaptation score of each operator to be compiled.

[0065] In the embodiments of this application, a hardware adaptation score is obtained by summing multiple hardware adaptation features after multiplying them by their corresponding weights using a classification decision function. The calculation formula is as follows: score = w1× alignment + w2× op_type + w3× reuse_distance; Here, `alignment` represents the computational alignment; `op_type` represents the operator type encoding value (regular operator `op_type=1`, irregular operator `op_type=0`); and `reuse_distance` represents the data reuse distance. In essence, the hardware compatibility score quantifies the overall compatibility between the compiled operator and different hardware. A higher score indicates that the operator is more suitable for execution by the first hardware (AME); a lower score indicates that the operator is more suitable for execution by the second hardware (CGRA).

[0066] Step 206: Based on the hardware adaptation score, the preset score threshold, and the preset boundary, classify each operator to be compiled into a first category, a second category, or a third category to obtain the classification result.

[0067] In the embodiments of this application, the preset score threshold is a pre-set benchmark value used for classification and comparison, denoted as threshold in this application. This value is used to distinguish the score boundary between "suitable for the first hardware" and "requires collaboration or is suitable for the second hardware".

[0068] The preset boundary is used to determine the tolerance range of the third category (collaborative execution), and is represented by margin in this application. This value, together with the preset score threshold, defines the lower limit of the score interval for the third category.

[0069] It is understood that the specific values ​​of the preset score threshold and preset boundary can be set according to actual needs, and are not limited in the embodiments of this application.

[0070] In some embodiments, based on hardware adaptation scores, preset score thresholds, and preset boundaries, each operator to be compiled is classified into a first category, a second category, or a third category to obtain the classification result, including: determining a first score threshold based on the preset score threshold and preset boundaries, wherein the preset score threshold is greater than the first score threshold; classifying the first operator to be compiled into the first category if the first hardware adaptation score of the first operator to be compiled is greater than the preset score threshold, wherein the first operator to be compiled is any one of at least one operator to be compiled; classifying the first operator to be compiled into the second category if the first hardware adaptation score is less than the first score threshold; and classifying the first operator to be compiled into the third category if the first hardware adaptation score is greater than or equal to the first score threshold and the first hardware adaptation score is less than or equal to the preset score threshold.

[0071] Wherein, the first score threshold = threshold - margin. If the hardware adaptation score of the operator to be compiled meets the following conditions, it is classified into the corresponding category: If score > threshold, then the operator to be compiled is classified as the first category (i.e., all are executed by AME). If score < threshold-margin, the operator to be compiled is classified into the second category (that is, executed entirely by the CGRA); If threshold-margin ≤ score ≤ threshold, the operator to be compiled is classified into the third category (requiring collaborative execution by the AME and the CGRA).

[0072] In step 207, when the second operator to be compiled is of the first category, the second operator to be compiled is compiled to generate first hardware configuration information matching the first hardware.

[0073] In the embodiments of the present application, the second operator to be compiled is any one of the at least one operator to be compiled. The first hardware configuration information refers to underlying configuration data matching the first hardware (AME), which is all the control information required for the first hardware to execute the second operator to be compiled. In a specific implementation, the manifestation of the first hardware configuration information may be the encoding of AME extended instructions, which includes calculation task descriptions (such as matrix multiplication), data address information (input data reading address, output result writing address), hardware control parameters (such as matrix dimension, MLEN alignment mode), etc., for the first hardware to directly parse and execute.

[0074] When the second operator to be compiled is classified into the first category (suitable for execution by the first hardware), the compiler compiles the operator to generate first hardware configuration information corresponding to the operator and matching the first hardware. The generated first hardware configuration information is then delivered to the first hardware, and the first hardware independently completes all calculation tasks of the operator according to the configuration information.

[0075] In step 208, when the second operator to be compiled is of the second category, the second operator to be compiled is compiled to generate second hardware configuration information matching the second hardware.

[0076] In the embodiments of the present application, the second hardware configuration information refers to underlying configuration data matching the second hardware (CGRA), which is all the control information required for the second hardware to execute the second operator to be compiled. In a specific implementation, the manifestation of the second hardware configuration information may be a configuration bitstream of the CGRA, which includes information such as the function configuration of each processing element (PE) (for example, configuring a specific PE as an adder, multiplier or comparator), the data interconnection relationship between processing elements, and data load and store control, for the second hardware to load and execute directly.

[0077] When the second operator to be compiled is classified as the second category (applicable to execution on the second hardware), the compiler compiles the operator and generates second hardware configuration information that corresponds to the operator and matches the second hardware. The generated second hardware configuration information is then sent to the second hardware, which independently completes all the computational tasks of the operator based on this configuration information.

[0078] Step 209: If the second operator to be compiled is of the third category, compile the second operator to be compiled to generate third hardware configuration information that matches the first hardware and fourth hardware configuration information that matches the second hardware.

[0079] In the embodiments of this application, the third hardware configuration information is the configuration data required for the first hardware to execute the first part of the computational task of the second operator to be compiled, and the fourth hardware configuration information is the configuration data required for the second hardware to execute the second part of the computational task of the second operator to be compiled; wherein, the first part of the computational task and the second part of the computational task are obtained by splitting the second operator to be compiled into tasks. When the operator to be compiled is classified into the third category (requiring collaborative execution), it indicates that the computational workload of the operator is partially aligned with MLEN and partially misaligned. In this case, splitting it into two parts for separate execution results in optimal overall performance.

[0080] In some embodiments, before generating hardware configuration information for the third category of operators to be compiled, the compiler first performs task splitting on the second operator to be compiled. Specifically, based on the computational load (AME) of the second operator to be compiled and the first computational load (MLEN) of the first hardware, the second operator to be compiled is split into tasks to obtain a first part of the computational tasks suitable for execution on the first hardware and a second part of the computational tasks suitable for execution on the second hardware; wherein, the computational load of the first part of the computational tasks matches the first computational load of the first hardware (i.e., the computational amount is an integer multiple of MLEN), and the computational load of the second part of the computational tasks does not match the first computational load of the first hardware (i.e., the computational amount is not an integer multiple of MLEN, and there is a remainder).

[0081] Specifically, the first part of the computational task is a set of subtasks whose computational load matches MLEN, extracted from the total computational task of the second operator to be compiled, and executed by AME. The computational load of the first part of the computational task corresponds to the computational load of the AME main block, and the calculation formula is as follows: AME main block computation cost = floor(total computation cost / MLEN) × MLEN; Where floor is the floor function; total computation is the total computational load of the operator to be compiled.

[0082] The second part of the computational task is a set of remaining subtasks that are split from the total computational task of the second operator to be compiled, and whose computational load does not match that of MLEN. This set is executed by the second hardware. The computational load of the second part of the computational task corresponds to the computational load of the CGRA tail block, which is the remaining portion of the total computational load minus the computational load of the AME main block. The calculation formula is as follows: CGRA tail block computation amount = total computation amount - AME main block computation amount; After splitting the operators to be compiled that need to be executed collaboratively, hardware configuration information matching AME is generated for the part of the computational tasks that need to be executed by AME, and hardware configuration information matching CGRA is generated for the part of the computational tasks that need to be executed by CGRA. The above hardware configuration information is then injected into AME and CGRA respectively, so that AME and CGRA can collaboratively execute the operator to be compiled.

[0083] In summary, this application calculates three hardware adaptation features—computational alignment, operator type encoding value, and data reuse distance—based on the computational load, operator type, and data flow information of each operator to be compiled. A hardware adaptation score is obtained by weighted summation of these features. Based on a comparison of the score with preset score thresholds and boundaries, each operator to be compiled is automatically classified into three categories: a first category suitable for AME execution, a second category suitable for CGRA execution, or a third category suitable for collaborative execution of both. Finally, corresponding hardware configuration information is generated for each category. Under the third category, the operator to be compiled is task-split to generate two sets of hardware configuration information, enabling collaborative execution of the first and second hardware. Compared to manual annotation schemes, this application automatically calculates based on operator features, eliminating the need for manual annotation by engineers and achieving full automation of operator scheduling. It adaptively makes optimal decisions based on operator hardware features without human intervention. Online calculation of classification features allows for adaptation to changes in model structure. Furthermore, by introducing collaborative execution categories, it fully leverages the complementary advantages of different hardware, effectively improving the utilization efficiency of heterogeneous computing resources and operator execution performance. In addition, the classification decision function is a rule engine, which is simple to implement and easy to integrate into the JIT compiler.

[0084] Furthermore, to help with better understanding, such as Figure 3 As shown, Figure 3 This is a general flowchart of a specific operator hybrid compilation method provided in an embodiment of this application.

[0085] Reference Figure 3The overall procedure of the operator hybrid compilation method provided by the embodiments of the present application is as follows: a JIT compiler first receives a computation graph, which contains a plurality of operators to be compiled and their data dependency relationships. The compiler traverses each operator in the computation graph, and extracts three hardware adaptation features for the currently processed operator: MLE N alignment (i.e., computation amount alignment), operator type (i.e., the encoded value of the operator type), and data reuse distance. After feature extraction is completed, the compiler calculates the hardware adaptation score of the operator according to a hardware adaptation score formula. After obtaining the hardware adaptation score, the compiler compares the score with a preset score threshold and a preset margin: if the score is greater than the preset score threshold (score>threshold), the operator is classified as "AME-suitable", the compiler generates a corresponding AME configuration package (i.e., hardware configuration information), and the AME executes all computation tasks of the operator; if the score is less than the difference between the preset score threshold and the preset margin (score<threshold - margin), the operator is classified as "CGRA-suitable", the compiler generates a corresponding CGRA configuration package, and the CGRA executes all computation tasks of the operator; if the score falls between the two (threshold - margin ≤score ≤ threshold), the operator is classified as "cooperation-required", the compiler splits the computation task of the operator into two parts: an AME main block and a CGRA tail block, generates an AME configuration package and a CGRA configuration package respectively, and the AME and the CGRA execute in cooperation. Finally, the compiler injects the generated configuration packages into the corresponding hardware execution units to complete the compilation and execution of the operator. The compiler traverses all operators in the computation graph until all operators are processed.

[0086] Figure 4 is a schematic structural diagram of an operator hybrid compilation apparatus provided by the embodiments of the present application. As shown in Figure 4 , the operator hybrid compilation apparatus 400 includes: a determining unit 410, configured to determine at least one hardware adaptation feature of each operator to be compiled according to the operator attribute information of each of the at least one operator to be compiled in the computation graph, where the at least one hardware adaptation feature is used to characterize the adaptation degree of each operator to be compiled when executed by different hardware; a classification unit 420, configured to determine the hardware adaptation score of each operator to be compiled based on the at least one hardware adaptation feature, so as to classify each operator to be compiled into a first category suitable for execution by a first hardware, a second category suitable for execution by a second hardware, or a third category suitable for cooperative execution by the first hardware and the second hardware according to the hardware adaptation score, to obtain a classification result; The compilation unit 430 is used to compile each operator to be compiled based on the classification results, and obtain the hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware can execute each operator to be compiled according to the hardware configuration information.

[0087] In some embodiments, at least one hardware adaptation feature includes at least one of computational alignment, operator type encoding value, and data reuse distance. The determining unit 410 is configured to: determine the computational alignment of each operator to be compiled based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware, wherein the computational alignment characterizes the degree of matching between the computational load of each operator to be compiled and the first computational load; determine the operator type encoding value of each operator to be compiled according to the operator type of each operator to be compiled, wherein the operator type includes regular operators and irregular operators; and determine the data reuse distance of each operator to be compiled according to the data flow information of each operator to be compiled, wherein data reuse characterizes the degree to which the input data of each operator to be compiled is reused.

[0088] In some embodiments, the determining unit 410 is configured to: split each operator to be compiled into a task based on the computational load of each operator to be compiled and the first computational load of the first hardware, to obtain at least one subtask corresponding to each operator to be compiled; determine the number of matching subtasks in the at least one subtask whose computational load matches the first computational load; and determine the computational load alignment degree according to the ratio of the number of matching subtasks to the total number corresponding to the at least one subtask.

[0089] In some embodiments, the classification unit 420 is configured to: determine at least one weight corresponding to at least one hardware adaptation feature; perform weighted operation on at least one hardware adaptation feature based on at least one weight to obtain hardware adaptation score for each operator to be compiled; and classify each operator to be compiled into a first category, a second category, or a third category based on the hardware adaptation score, a preset score threshold, and a preset boundary to obtain a classification result.

[0090] In some embodiments, the classification unit 420 is configured to: determine a first score threshold based on a preset score threshold and a preset boundary, wherein the preset score threshold is greater than the first score threshold; classify the first operator to be compiled into a first category if the first hardware adaptation score of the first operator to be compiled is greater than the preset score threshold, wherein the first operator to be compiled is any one of at least one operator to be compiled; classify the first operator to be compiled into a second category if the first hardware adaptation score is less than the first score threshold; and classify the first operator to be compiled into a third category if the first hardware adaptation score is greater than or equal to the first score threshold and the first hardware adaptation score is less than or equal to the preset score threshold.

[0091] In some embodiments, the compilation unit 430 is configured to: compile the second operator to be compiled into a first category to generate first hardware configuration information matching the first hardware, wherein the first hardware configuration information is configuration data required by the first hardware to execute the second operator to be compiled, and the second operator to be compiled is any one of at least one operator to be compiled; compile the second operator to be compiled into a second category to generate second hardware configuration information matching the second hardware, wherein the second hardware configuration information is configuration data required by the second hardware to execute the second operator to be compiled; and compile the second operator to be compiled into a third category to generate third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware, wherein the third hardware configuration information is configuration data required by the first hardware to execute a first part of the computational task of the second operator to be compiled, and the fourth hardware configuration information is configuration data required by the second hardware to execute a second part of the computational task of the second operator to be compiled; wherein the first part of the computational task and the second part of the computational task are obtained by task splitting the second operator to be compiled.

[0092] In some embodiments, the apparatus includes: a splitting unit, configured to compile a second operator to be compiled, generating third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware; prior to this, based on the computational load of the second operator to be compiled and the first computational load of the first hardware, the second operator to be compiled is split into tasks to obtain a first part of the computational task suitable for execution on the first hardware and a second part of the computational task suitable for execution on the second hardware; wherein the computational load of the first part of the computational task matches the first computational load of the first hardware, and the computational load of the second part of the computational task does not match the first computational load of the first hardware.

[0093] Since the apparatus provided in this application corresponds to the methods provided in the above-mentioned embodiments, the implementation of the methods is also applicable to the apparatus provided in this embodiment, and will not be described in detail in this embodiment.

[0094] The methods and apparatus provided in the embodiments of this application have been described above. To implement the functions of the methods provided in the embodiments of this application, the electronic device may include a hardware structure and software modules, and may implement the above functions in the form of a hardware structure, software modules, or a hardware structure plus software modules. One of the above functions may be executed in the form of a hardware structure, software modules, or a hardware structure plus software modules.

[0095] Figure 5 This is a block diagram illustrating an electronic device 500 for implementing the above-described operator hybrid compilation method according to an exemplary embodiment. (Refer to...) Figure 5The electronic device 500 may include a communication interface 501, capable of interacting with other devices; a processor 502, connected to the communication interface 501 to enable interaction with other devices, used to execute the methods provided by one or more of the above-mentioned technical solutions when running a computer program; and a memory 503, on which the computer program is stored. Specifically, the specific processing procedure of the processor 502 can refer to the operator hybrid compilation method described in the above embodiments of this application.

[0096] Of course, in practical applications, the various components in electronic device 500 are coupled together through bus system 504. It can be understood that bus system 504 is used to realize the connection and communication between these components. In addition to a data bus, bus system 504 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 5 The general designated all buses as Bus System 504.

[0097] The memory 503 in this embodiment is used to store various types of data to support the operation of the electronic device 500. Examples of such data include any computer program used to operate on the electronic device 500.

[0098] The methods disclosed in the embodiments of this application can be applied to processor 502, or implemented by processor 502. Processor 502 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the integrated logic circuit of the hardware in processor 502 or by instructions in the form of software. The processor 502 may be a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Processor 502 can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as being executed by a hardware decoding processor, or being executed by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory 503. Processor 502 reads the information in memory 503 and combines its hardware to complete the steps of the aforementioned method.

[0099] In an exemplary embodiment, the electronic device 500 may be implemented by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), general-purpose processors, controllers, microcontrollers (MCUs), microprocessors, or other electronic components to perform the aforementioned method.

[0100] Embodiments of this application also propose a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause a computer to execute the operator hybrid compilation method described in the above embodiments of this application.

[0101] Embodiments of this application also propose a computer program product, including a computer program, which is executed by a processor using the operator hybrid compilation method described in the above embodiments of this application.

[0102] Embodiments of this application also propose a chip including one or more interface circuits and one or more processors; the interface circuits are used to receive signals from the memory of an electronic device and send signals to the processors, the signals including computer instructions stored in the memory, and when the processor executes the computer instructions, it causes the electronic device to execute the operator hybrid compilation method described in the above embodiments of this application.

[0103] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0104] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with an embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0105] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0106] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (a non-exhaustive list) of computer-readable media include: an electrical connection having one or more wires (control method), a portable computer disk drive (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic device, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which programs can be printed, because programs can be obtained electronically, for example, by optically scanning the paper or other media, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0107] It should be understood that various parts of the embodiments of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0108] Those skilled in the art will understand that all or part of the steps of the methods described in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0109] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium mentioned above can be a read-only memory, a disk, or an optical disk, etc.

[0110] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A hybrid operator compilation method, characterized in that, The method includes: Based on the operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph, at least one hardware adaptation feature of each operator to be compiled is determined, and the at least one hardware adaptation feature is used to characterize the degree of adaptation of each operator to be compiled to be executed by different hardware. Based on the at least one hardware adaptation feature, determine the hardware adaptation score of each operator to be compiled, and classify each operator to be compiled into a first category suitable for execution on the first hardware, or a second category suitable for execution on the second hardware, or a third category suitable for execution on both the first hardware and the second hardware, based on the hardware adaptation score, to obtain a classification result; Based on the classification results, each operator to be compiled is compiled to obtain the hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware execute each operator to be compiled according to the hardware configuration information.

2. The method according to claim 1, characterized in that, The at least one hardware adaptation feature includes at least one of computational alignment, operator type encoding value, and data reuse distance. The step of determining at least one hardware adaptation feature of each operator to be compiled based on the operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph includes: Based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware, the computational load alignment of each operator to be compiled is determined, wherein the computational load alignment represents the degree of matching between the computational load of each operator to be compiled and the first computational load; Based on the operator type of each operator to be compiled, determine the operator type encoding value of each operator to be compiled, wherein the operator type includes regular operators and irregular operators; Based on the data flow information of each operator to be compiled, the data reuse distance of each operator to be compiled is determined, whereby data reuse characterizes the degree to which the input data of each operator to be compiled is reused.

3. The method according to claim 2, characterized in that, The step of determining the computational load alignment of each operator to be compiled based on the computational load of each operator to be compiled and the first computational load corresponding to the first hardware includes: Based on the computational load of each operator to be compiled and the first computational load of the first hardware, the operators to be compiled are split into tasks to obtain at least one subtask corresponding to each operator to be compiled. Determine the number of matching subtasks in which the computational load of each subtask in the at least one subtask matches the first computational load; The computational alignment is determined based on the ratio of the number of matching subtasks to the total number of the at least one subtask.

4. The method according to claim 1, characterized in that, Based on the at least one hardware adaptation feature, a hardware adaptation score is determined for each operator to be compiled. The operators are then classified according to the hardware adaptation score into a first category suitable for execution on the first hardware, a second category suitable for execution on the second hardware, or a third category suitable for joint execution on the first and second hardware. The classification results include: Determine at least one weight corresponding to the at least one hardware adaptation feature; Based on the at least one weight, the at least one hardware adaptation feature is weighted and processed to obtain the hardware adaptation score of each operator to be compiled. Based on the hardware adaptation score, the preset score threshold, and the preset boundary, each operator to be compiled is classified into the first category, the second category, or the third category to obtain the classification result.

5. The method according to claim 4, characterized in that, The classification of each operator to be compiled into the first category, the second category, or the third category based on the hardware adaptation score, a preset score threshold, and a preset boundary, yields the following classification results: Based on the preset score threshold and the preset boundary, a first score threshold is determined, wherein the preset score threshold is greater than the first score threshold; If the first hardware adaptation score of the first operator to be compiled is greater than the preset score threshold, the first operator to be compiled is classified into the first category, and the first operator to be compiled is any one of the at least one operators to be compiled. If the first hardware adaptation score is less than the first score threshold, the first operator to be compiled will be classified into the second category. If the first hardware adaptation score is greater than or equal to the first score threshold and the first hardware adaptation score is less than or equal to the preset score threshold, the first operator to be compiled is classified into the third category.

6. The method according to claim 1, characterized in that, The step of compiling each operator to be compiled based on the classification result to obtain the hardware configuration information of each operator to be compiled includes: When the second operator to be compiled is the first category, the second operator to be compiled is compiled to generate first hardware configuration information that matches the first hardware. The first hardware configuration information is the configuration data required by the first hardware to execute the second operator to be compiled. The second operator to be compiled is any one of the at least one operator to be compiled. When the second operator to be compiled is of the second category, the second operator to be compiled is compiled to generate second hardware configuration information that matches the second hardware. The second hardware configuration information is the configuration data for the second hardware to execute the second operator to be compiled. When the second operator to be compiled is the third category, the second operator to be compiled is compiled to generate third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware. The third hardware configuration information is the configuration data required by the first hardware to execute the first part of the computation task of the second operator to be compiled, and the fourth hardware configuration information is the configuration data required by the second hardware to execute the second part of the computation task of the second operator to be compiled. The first part of the computation task and the second part of the computation task are obtained by splitting the second operator to be compiled into tasks.

7. The method according to claim 6, characterized in that, Before compiling the second operator to be compiled to generate third hardware configuration information matching the first hardware and fourth hardware configuration information matching the second hardware, the method includes: Based on the computational load of the second operator to be compiled and the first computational load of the first hardware, the second operator to be compiled is split into tasks to obtain a first part of the computational task suitable for execution by the first hardware and a second part of the computational task suitable for execution by the second hardware. The computational load of the first part of the computing tasks matches the first computing load of the first hardware, while the computational load of the second part of the computing tasks does not match the first computing load of the first hardware.

8. An operator hybrid compiler apparatus, characterized in that, The device includes: The determining unit is used to determine at least one hardware adaptation feature of each operator to be compiled based on the operator attribute information of each operator to be compiled in at least one operator to be compiled in the computation graph. The at least one hardware adaptation feature is used to characterize the degree of adaptation of each operator to be compiled to be executed by different hardware. A classification unit is configured to determine the hardware adaptation score of each operator to be compiled based on the at least one hardware adaptation feature, so as to classify each operator to be compiled into a first category suitable for execution on the first hardware, or a second category suitable for execution on the second hardware, or a third category suitable for execution on the first hardware and the second hardware in cooperation, and obtain a classification result. The compilation unit is used to compile each of the operators to be compiled according to the classification results, and obtain the hardware configuration information corresponding to each operator to be compiled, so that the first hardware and / or the second hardware execute each operator to be compiled according to the hardware configuration information.

9. An electronic device, characterized in that, include: The processor and the memory used to store computer programs that can run on the processor. When the processor is used to run the computer program, it performs the method according to any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1 to 7.

Images