Instruction processing method and apparatus, electronic device, computer-readable storage medium

Abstract

The present disclosure provides an instruction processing method and device, electronic equipment and computer readable storage medium. The method comprises: determining an operator execution sequence to be executed by a target processing core in a many-core system; determining a second operator instruction to be executed in alternation with a first operator instruction from the operator execution sequence according to a first communication time length of a routing instruction and a first calculation time length of a calculation instruction in the first operator instruction; a time length relationship between a second communication time length of a routing instruction and a second calculation time length of a calculation instruction in the second operator instruction is different from a time length relationship between the first communication time length and the first calculation time length; splitting the first operator instruction and the second operator instruction to obtain a plurality of first execution segments and a plurality of second execution segments; and mapping the plurality of first execution segments and the plurality of second execution segments to the target processing core to enable the target processing core to execute the plurality of first execution segments and the plurality of second execution segments in alternation. According to embodiments of the present disclosure, the resource utilization rate of the processing core can be improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of computer, in particular to an instruction processing method and device, electronic equipment and computer readable storage medium. BACKGROUND

[0002] With the popularization of artificial intelligence applications, the demand for neural network computing is increasing, and neural network computing is becoming more and more important. However, the large amount of computation and storage required by neural networks makes it difficult for general-purpose processors to effectively perform these tasks. Therefore, people have developed many specialized neural network acceleration architectures, among which many-core systems (including many-core chips or boards, etc.) are an important technical path. In many-core systems, different operators of a network are usually distributed on different processing cores when executing a neural network, a process called mapping, and the resulting scheme is called a mapping scheme. Data transmission between processing cores is achieved through the on-chip network (or routing system) of the many-core system, and there is usually a large amount of data transmission within the neural network. Because of the diversity of neural networks, different operators have different demands for computation and communication, which may cause different performance bottlenecks. SUMMARY

[0003] The present disclosure provides an instruction processing method and device, electronic equipment and computer readable storage medium.

[0004] In a first aspect, the present disclosure provides an instruction processing method, comprising:

[0005] determining an operator execution sequence to be executed by a target processing core in a many-core system, the operator execution sequence comprising a plurality of operator instructions, the operator instructions comprising routing instructions and computing instructions, the routing instructions being used to perform data transmission operations of the operator instructions, and the computing instructions being used to perform computing operations of the operator instructions; the many-core system comprising a plurality of processing cores, and the target processing core being any one of the plurality of processing cores;

[0006] determining, according to a first communication duration of a routing instruction and a first computing duration of a computing instruction in a first operator instruction of the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction from the operator execution sequence; wherein a duration relationship between a second communication duration of a routing instruction and a second computing duration of a computing instruction in the second operator instruction is different from a duration relationship between the first communication duration and the first computing duration;

[0007] splitting the first operator instruction and the second operator instruction according to the first communication duration, the first computing duration, the second communication duration and the second computing duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0008] map the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction to the target processing core, so as to cause the target processing core to interleave execution of the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction.

[0009] In a second aspect, the present disclosure provides an instruction processing apparatus, comprising:

[0010] an operator sequence determining module configured to determine an operator execution sequence to be executed by a target processing core in a many-core system, the operator execution sequence comprising a plurality of operator instructions, the operator instructions comprising routing instructions and computing instructions, the routing instructions being configured to perform data transmission operations of the operator instructions, and the computing instructions being configured to perform computing operations of the operator instructions, the many-core system comprising a plurality of processing cores, and the target processing core being any one of the plurality of processing cores;

[0011] an instruction determining module configured to determine, according to a first communication duration of a routing instruction and a first computing duration of a computing instruction in a first operator instruction of the operator execution sequence, a second operator instruction interleaved with the first operator instruction from the operator execution sequence, wherein a time duration relationship between a second communication duration of a routing instruction and a second computing duration of a computing instruction in the second operator instruction is different from a time duration relationship between the first communication duration and the first computing duration;

[0012] an instruction splitting module configured to split the first operator instruction and the second operator instruction according to the first communication duration, the first computing duration, the second communication duration, and the second computing duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0013] an instruction mapping module configured to map the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction to the target processing core, so as to cause the target processing core to interleave execution of the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction.

[0014] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the above-mentioned instruction processing method.

[0015] In a fourth aspect, the present disclosure provides a computer readable storage medium having stored thereon a computer program, wherein the computer program, when executed by a processor, implements the instruction processing method described above.

[0016] The embodiments provided by the present disclosure can determine the operator instructions of communication bottlenecks and calculation bottlenecks in the operator execution sequence of the processing core, split the corresponding operator instructions of the bottlenecks into multiple execution segments, and map the multiple execution segments of the operator instructions to the processing core, so that the processing core executes the execution segments of the operator instructions in an interleaved manner, thereby improving the utilization of the calculation resources and the communication resources of the processing core, and improving the performance and efficiency of the neural network execution.

[0017] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0018] The accompanying drawings are included to provide a further understanding of the present disclosure and constitute a part of the specification, which together with the embodiments of the present disclosure serve to explain the present disclosure, and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent from the detailed description of the specific example embodiments described below, with reference to the accompanying drawings, in which:

[0019] Figure 1 is a schematic diagram of a roofline model;

[0020] Figure 2 is a flowchart of an instruction processing method provided by an embodiment of the present disclosure;

[0021] Figure 3 is a schematic diagram of an execution process of the operator instruction of the communication bottleneck provided by an embodiment of the present disclosure;

[0022] Figure 4 is a schematic diagram of interleaved execution of the operator instruction provided by an embodiment of the present disclosure;

[0023] Figure 5 is a schematic diagram of interleaved execution of the operator instruction provided by an embodiment of the present disclosure;

[0024] Figure 6 is a schematic diagram of parallel execution of the operator instruction provided by an embodiment of the present disclosure;

[0025] Figure 7 is a block diagram of an instruction processing apparatus provided by an embodiment of the present disclosure;

[0026] Figure 8 is a block diagram of an electronic device provided by an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments of this disclosure to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0028] Where there is no conflict, the various embodiments of this disclosure and the features thereof in the embodiments may be combined with each other.

[0029] As used herein, the term “and / or” includes any and all combinations of one or more related enumerated entries.

[0030] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. As used herein, the singular forms “a” and “the” are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will also be understood that when the terms “comprising” and / or “made of” are used in this specification, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Words such as “connected” or “linked” are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0031] Unless otherwise specified, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this disclosure, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined herein.

[0032] In the field of neural network acceleration computing, the roofline model is often used to assess the performance of the current operator. A processing core has two important metrics: computing power and bandwidth. Computing power refers to the upper limit of a processing core's performance, such as the number of floating-point or integer operations it can perform per second. Bandwidth, on the other hand, refers to the upper limit of the computing platform's bandwidth, indicating the maximum amount of data exchange a single processing core can complete per second. Dividing these two metrics yields the upper limit of the processing core's computational intensity, describing the maximum number of computations a processing core can perform per unit of data exchange.

[0033] Figure 1 This is a schematic diagram of the roofline model. (For example...) Figure 1As shown, the abscissa of the roofline model is the computational intensity I, and the ordinate is the computational performance P. The roofline model divides the performance space into two regions: a bandwidth-limited region and a computation-limited region. At the beginning of processing, data is transmitted at the maximum memory bandwidth β (bytes / s); after reaching the maximum computational performance π, the maximum computational intensity Imax = π / β is reached. In this roofline model, performance is limited depending on the balance between computational resources and memory access speed.

[0034] In the mapping strategy of the many-core system, a processing core is often assigned an operator execution sequence, and different operators in the operator execution sequence are likely to be in different regions of the roofline model and have different resource bottlenecks. Sequential execution of instructions in the operator execution sequence causes low utilization of overall computing and communication resources of the processing core, resulting in a decrease in the overall performance of the neural network model execution.

[0035] According to the instruction processing method provided in the embodiments of the present disclosure, the operator instructions of the communication bottleneck and the computation bottleneck in the operator execution sequence of the processing core can be determined, the corresponding operator instructions can be divided into multiple execution segments, and the multiple execution segments of the operator instructions can be mapped to the processing core, so that the processing core can execute the execution segments of the operator instructions in an interleaved manner, thereby improving the utilization of the computing resources and the communication resources of the processing core and improving the performance and efficiency of the neural network execution.

[0036] The instruction processing method according to the embodiments of the present disclosure can be executed by an electronic device such as a terminal device or a server. The terminal device can be a vehicle-mounted device, a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor invoking computer-readable program instructions stored in a memory. Alternatively, the method can be executed by a server.

[0037] In some possible implementation manners, the instruction processing method according to the embodiments of the present disclosure can be executed by a compiler, or by a processor in a many-core system, a processing core for control, or a pre-set processing core, and the present disclosure does not limit this.

[0038] Figure 2 A flowchart of an instruction processing method according to an embodiment of the present disclosure is provided. Referring to Figure 2 The method includes:

[0039] In step S21, an operator execution sequence to be executed by a target processing core in a many-core system is determined, the operator execution sequence including a plurality of operator instructions, the operator instructions including routing instructions and computing instructions, the routing instructions being used for performing data transmission operations of the operator instructions, and the computing instructions being used for performing computing operations of the operator instructions; the many-core system including a plurality of processing cores, and the target processing core being any one of the plurality of processing cores.

[0040] In step S22, a second operator instruction staggered with a first operator instruction in the operator execution sequence is determined from the operator execution sequence according to a first communication duration of a routing instruction in the first operator instruction and a first computing duration of a computing instruction in the first operator instruction; wherein a duration relationship between a second communication duration of a routing instruction in the second operator instruction and a second computing duration of a computing instruction in the second operator instruction is different from a duration relationship between the first communication duration and the first computing duration.

[0041] In step S23, the first operator instruction and the second operator instruction are split according to the first communication duration, the first computing duration, the second communication duration, and the second computing duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction.

[0042] In step S24, the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core staggeredly executes the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction.

[0043] For example, the many-core system can be a many-core chip or a board card, or any electronic device including a many-core chip or a board card, the many-core system including a plurality of processing cores, data transmission between the processing cores and data transmission between the processing cores and external devices being achieved through an on-chip network (or a routing system) of the many-core system.

[0044] In some possible implementation manners, in a process of accelerating a neural network calculation by using a many-core system, each processing core of the many-core system corresponds to an operator execution sequence to be executed, the operator execution sequence including a plurality of operator instructions, the operator instructions including operation instructions of a neural network operator, and being used for performing coarse-grained operations of the neural network operator.

[0045] The neural network operator includes coarse-grained operations that are processed in parallel, and the coarse-grained operations are used for operations with a communication duration greater than a first threshold value and / or a computing duration greater than a second threshold value.

[0046] That is, the neural network operator can include one or more coarse-grained operations that require parallel computation, and the coarse-grained operation represents an operation that has a large amount of computation, requires a long data communication time or a long computation time. For example, the neural network operator can be a convolution operator, and the convolution operator includes coarse-grained operations such as a matrix multiplication operation and a matrix addition operation. The matrix multiplication operation, which is a coarse-grained operation, includes a plurality of element multiplication operations, which can be implemented by parallel computation. In this example, the element multiplication operation is a fine-grained operation.

[0047] In some possible implementation manners, the first threshold of the communication duration and the second threshold of the computation duration are respectively used to indicate the length of the data communication time and the computation time required for one processing of the coarse-grained operation. If the communication duration required for one processing is greater than the first threshold, it is considered that the data communication time is long. Conversely, it is considered that the data communication time is short. Similarly, if the computation duration required for one processing is greater than the second threshold, it is considered that the computation time is long. Conversely, it is considered that the computation time is short.

[0048] It should be understood that a person skilled in the art can set the specific values of the first threshold and the second threshold according to actual conditions, and the present disclosure does not limit this.

[0049] In some possible implementation manners, the operator instruction includes a routing instruction and a computation instruction, the routing instruction is used to perform a data transmission operation of the operator instruction, and the computation instruction is used to perform a computation operation of the operator instruction.

[0050] In some possible implementation manners, for any one of the plurality of processing cores (hereinafter referred to as a target processing core), the operator execution sequence to be executed by the target processing core can be determined in step S21.

[0051] For any one of the operator instructions in the operator execution sequence, the communication duration of the routing instruction and the computation duration of the computation instruction in the operator instruction can be determined according to the computation resource and the communication resource of the target processing core, and the data amount (for example, the weight data and the convolution data to be processed in the convolution operator, the generated convolution result data, etc.) and the computation amount (for example, the multiplication and accumulation computation amount in the convolution) required by the operator instruction, so as to determine whether the operator instruction is a communication bottleneck operator instruction or a computation bottleneck operator instruction.

[0052] In some possible implementation manners, if the communication duration of the routing instruction in the operator instruction is greater than the computation duration of the computation instruction, it is determined that the operator instruction is a communication bottleneck operator instruction. Conversely, if the communication duration of the routing instruction in the operator instruction is less than the computation duration of the computation instruction, it is determined that the operator instruction is a computation bottleneck operator instruction.

[0053] Figure 3A schematic diagram of an execution process of an operator instruction of a communication bottleneck is provided for embodiments of the present disclosure. Referring to Figure 3 The operator instruction includes a routing instruction R1 and a calculation instruction C1, and the communication duration of the routing instruction R1 is greater than the calculation duration of the calculation instruction C1. In the execution process, the pipeline between the routing instruction R1 and the calculation instruction C1 needs to be maintained. That is, the routing instruction R1 receives / transmits data at the same time, and the calculation instruction C1 performs calculation using the data or synchronously generates the data at the same time, so as to reduce the data that needs to be buffered. In addition, there is a data dependency relationship between the routing instruction R1 and the calculation instruction C1, and the processing core needs to receive a certain amount of data transmitted by the routing instruction R1 before starting the calculation operation of the calculation instruction C1. The utilization rate of the calculation resource in the execution process is low.

[0054] In some possible implementation manners, for any one operator instruction (hereinafter referred to as a first operator instruction) in the operator execution sequence of the target processing core, a second operator instruction that is staggered with the first operator instruction can be determined from the operator execution sequence according to a first communication duration of a routing instruction and a first calculation duration of a calculation instruction in the first operator instruction in step S22.

[0055] The time duration relationship between a second communication duration of a routing instruction and a second calculation duration of a calculation instruction in the second operator instruction is different from the time duration relationship between the first communication duration and the first calculation duration. That is, if the time duration relationship between the first communication duration and the first calculation duration is that the first communication duration is greater than the first calculation duration, the time duration relationship between the second communication duration and the second calculation duration is that the second communication duration is less than the second calculation duration; otherwise, if the time duration relationship between the first communication duration and the first calculation duration is that the first communication duration is less than the first calculation duration, the time duration relationship between the second communication duration and the second calculation duration is that the second communication duration is greater than the second calculation duration. The processing process can be understood as follows: a corresponding calculation bottleneck operator instruction is screened for the communication bottleneck operator instruction; or a corresponding communication bottleneck operator instruction is screened for the calculation bottleneck operator instruction, so as to be processed subsequently.

[0056] In some possible implementation manners, if there is one operator instruction that meets the condition in the operator execution sequence, the operator instruction is directly selected as the second operator instruction; if there are multiple operator instructions that meet the condition in the operator execution sequence, one or more of the multiple operator instructions are selected as the second operator instruction; and if there is no second operator instruction that meets the condition in the operator execution sequence, the search fails, and the next first operator instruction is searched for a corresponding second operator instruction. The present disclosure does not limit the specific selection process of the second operator instruction.

[0057] In some possible implementation manners, after the second operator instruction is determined, the first operator instruction and the second operator instruction can be split according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction in step S23.

[0058] The operator instruction can be split based on the communication duration and the calculation duration of the first operator instruction and the second operator instruction according to parameters such as a data flow granularity and an instruction context switching speed. The data flow granularity can represent a minimum data amount transmitted each time or a minimum data amount required for each calculation operation, for example, 64B. The instruction context switching speed can represent a minimum time duration required for each switching during segmented execution of the operator instruction by the processing core, for example, 1ms. The present disclosure does not limit the types and specific numbers of parameters required for splitting the operator instruction.

[0059] In some possible implementation manners, if the first operator instruction is a communication bottleneck operator instruction, the calculation instructions of the first operator instruction and the second operator instruction can be split to obtain the first execution segments and the second execution segments as calculation execution segments. If the first operator instruction is a calculation bottleneck operator instruction, the routing instructions of the first operator instruction and the second operator instruction can be split to obtain the first execution segments and the second execution segments as routing execution segments.

[0060] In some possible implementation manners, the routing instructions and the calculation instructions of the first operator instruction and the second operator instruction can also be split at the same time to obtain the first execution segments and the second execution segments including the routing execution segments and the calculation execution segments. The present disclosure does not limit the specific splitting manners.

[0061] In some possible implementation manners, the order of the operator instructions in the operator execution sequence can be adjusted so that the first operator instruction and the second operator instruction are adjacent, and information of the execution segments and an indication of interleaved execution are added. Alternatively, the order of the operator instructions in the operator execution sequence can not be adjusted, and the information of the execution segments and the indication of interleaved execution are directly added to the first operator instruction and the second operator instruction.

[0062] In some possible implementation manners, the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction can be mapped to the target processing core in step S24. After receiving the execution segments of the first operator instruction and the second operator instruction and the indication of interleaved execution, the target processing core can interleave the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction during execution of the first operator instruction and the second operator instruction, to achieve complementation of the communication resources and the calculation resources.

[0063] According to an embodiment of the present disclosure, the communication bottleneck and the calculation bottleneck of the operator instruction in the operator execution sequence of the processing core can be determined, the corresponding operator instruction can be divided into multiple execution segments, and the multiple execution segments of the operator instruction can be mapped to the processing core, so that the processing core executes the execution segments of the operator instruction in an interleaved manner, thereby improving the utilization of the calculation resource and the communication resource of the processing core and improving the performance and efficiency of the neural network execution.

[0064] The instruction processing method according to an embodiment of the present disclosure will be described below.

[0065] As described above, the operator execution sequence to be executed by the target processing core of the many-core system can be determined in step S21. The operator execution sequence includes multiple operator instructions, and the operator instructions correspond to coarse-grained operations of the neural network operators.

[0066] In the related art, the compiler sequentially maps the multiple operator instructions in the operator execution sequence to the target processing core according to the data flow, so that the target processing core sequentially executes the multiple operator instructions. However, the communication time length of the routing instruction and the calculation time length of the calculation instruction in the operator instruction can be different, and a communication bottleneck or a calculation bottleneck occurs, resulting in low resource utilization.

[0067] In some possible implementation manners, for a first operator instruction in the operator execution sequence, a second operator instruction interleaved with the first operator instruction can be determined from the operator execution sequence according to a first communication time length of a routing instruction and a first calculation time length of a calculation instruction in the first operator instruction in step S22.

[0068] The time length relationship between a second communication time length of a routing instruction and a second calculation time length of a calculation instruction in the second operator instruction is different from the time length relationship between the first communication time length and the first calculation time length. That is, if the time length relationship between the first communication time length and the first calculation time length is that the first communication time length is greater than the first calculation time length, the time length relationship between the second communication time length and the second calculation time length is that the second communication time length is less than the second calculation time length; otherwise, if the time length relationship between the first communication time length and the first calculation time length is that the first communication time length is less than the first calculation time length, the time length relationship between the second communication time length and the second calculation time length is that the second communication time length is greater than the second calculation time length.

[0069] In some possible implementation manners, step S22 can include:

[0070] In the case where the first communication time length is greater than the first calculation time length, the second operator instruction is determined according to the operator instruction in the operator execution sequence, which has a communication time length less than a calculation time length and has no data dependency relationship with the first operator instruction;

[0071] In a case where the first communication duration is less than the first computation duration, a second operator instruction is determined according to an operator instruction in the operator execution sequence that has a communication duration greater than a computation duration and has no data dependency relationship with the first operator instruction.

[0072] For example, if the first communication duration is greater than the first computation duration, the first operator instruction is an operator instruction of a communication bottleneck, and an operator instruction of a computation bottleneck needs to be selected as an operator instruction of a bottleneck complement to be executed interleaved with the first operator instruction to improve resource utilization.

[0073] That is, an operator instruction in the operator execution sequence that has a communication duration less than a computation duration is selected from the operator execution sequence. Also, for the selected operator instruction, if the operator instruction has a data dependency relationship with the first operator instruction, the first operator instruction needs to be executed before the operator instruction is executed, or the operator instruction needs to be executed before the first operator instruction is executed, and the operator instruction cannot be executed interleaved with the first operator instruction and needs to be excluded. In this way, a second operator instruction can be determined from among operator instructions in the operator execution sequence that have a communication duration less than a computation duration and have no data dependency relationship with the first operator instruction.

[0074] In some possible implementation manners, in a case where the first communication duration is greater than the first computation duration, the step of determining a second operator instruction according to an operator instruction in the operator execution sequence that has a communication duration less than a computation duration and has no data dependency relationship with the first operator instruction can include:

[0075] In a case where the first communication duration is greater than the first computation duration, it is determined whether there is a target operator instruction in the operator execution sequence that has a communication duration of zero and has no data dependency relationship with the first operator instruction.

[0076] In a case where the target operator instruction exists in the operator execution sequence, the target operator instruction is determined as the second operator instruction.

[0077] For example, a communication duration of zero indicates that data required by the operator instruction is already in a target processing core, and no communication is needed, and no additional routing instruction between processing cores is needed, so this type of operator instruction can be preferentially selected for interleaved execution.

[0078] In this way, for the first operator instruction of a communication bottleneck that has a first communication duration greater than a first computation duration, it can be determined whether there is a target operator instruction in the operator execution sequence that has a communication duration of zero and has no data dependency relationship with the first operator instruction. If the target operator instruction exists, the target operator instruction can be directly determined as the second operator instruction for interleaved execution.

[0079] In some possible implementation manners, in the processing of the attention mechanism in the neural network, it is necessary to calculate query features (Query, Q), key features (Key, K), and value features (Value, V) based on input information and weights; and then perform linear transformation based on the Q, K, and V to determine attention features and the like. The input information and weights required by the operator instruction for calculating the Q, K, and V need to be externally input, and the operator instruction may be a communication bottleneck. The data required by the operator instruction for performing linear transformation based on the Q, K, and V are generated by the processing core calculation, and do not need external input. In this case, the operator instruction for performing linear transformation based on the Q, K, and V is a calculation bottleneck, and there is no routing instruction or the communication duration of the routing instruction is zero.

[0080] In this way, the selection efficiency of the second operator instruction can be improved, and the efficiency of subsequent instruction splitting and the efficiency of interleaved execution in the processing core can be improved.

[0081] In some possible implementation manners, if there is no target operator instruction with a communication duration of zero and no data dependency relationship with the first operator instruction in the operator execution sequence, an operator instruction (hereinafter referred to as a fourth operator instruction) with a communication duration less than a calculation duration and no data dependency relationship with the first operator instruction can be selected from the operator execution sequence. If there is only one fourth operator instruction, the fourth operator instruction can be directly selected as the second operator instruction; if there are multiple fourth operator instructions, further selection is needed.

[0082] In some possible implementation manners, in the case where there are multiple fourth operator instructions, one of the multiple fourth operator instructions can be selected as the second operator instruction at random, or the fourth operator instruction with the minimum interval from the first operator instruction in the operator execution sequence can be selected as the second operator instruction; the fourth operator instruction with the shortest communication duration can also be selected as the second operator instruction for interleaved execution; and the fourth operator instruction that best matches the first operator instruction can also be selected as the second operator instruction for interleaved execution. The disclosure does not limit the specific selection manner in this case.

[0083] In some possible implementation manners, the best matching with the first operator instruction can mean that the bottleneck complementarity is best, for example, the sum of the first communication duration and the second communication duration is equal to or close to the sum of the first calculation duration and the second calculation duration.

[0084] In some possible implementation manners, if the first communication duration is less than the first calculation duration, the first operator instruction is a calculation bottleneck operator instruction, and a communication bottleneck operator instruction needs to be selected as the bottleneck complementary operator instruction, and interleaved execution is performed with the first operator instruction to improve resource utilization.

[0085] That is, an operator instruction in the operator execution sequence is selected, which has a communication duration longer than a calculation duration in the operator execution sequence. Moreover, for the selected operator instruction, if the operator instruction has a data dependency relationship with the first operator instruction, the first operator instruction needs to be executed first, or the first operator instruction needs to be executed after the operator instruction is executed, and the operator instruction cannot be interleaved with the first operator instruction and needs to be excluded. In this way, the second operator instruction can be determined from the operator instruction in the operator execution sequence, which has a communication duration longer than a calculation duration and has no data dependency relationship with the first operator instruction.

[0086] In some possible implementation manners, if the operator instruction (hereinafter referred to as a fifth operator instruction) having a communication duration longer than a calculation duration and having no data dependency relationship with the first operator instruction has only one, the fifth operator instruction can be directly selected as the second operator instruction; if the fifth operator instruction has multiple, the fifth operator instruction needs to be further selected. In this case, one of the multiple fifth operator instructions can be selected as the second operator instruction, or the fifth operator instruction having the smallest interval with the first operator instruction in the operator execution sequence can be selected as the second operator instruction. The disclosure does not limit the specific selection manner in this case.

[0087] In this way, the operator instruction corresponding to the communication bottleneck can be screened out for the operator instruction corresponding to the calculation bottleneck, or the operator instruction corresponding to the calculation bottleneck can be screened out for the operator instruction corresponding to the communication bottleneck, so as to be interleaved in subsequent processing, thereby improving the utilization of the calculation resource and the communication resource of the processing core.

[0088] In some possible implementation manners, after the second operator instruction is determined, the first operator instruction and the second operator instruction can be divided into multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration in step S23.

[0089] In some possible implementation manners, if the first operator instruction is the operator instruction corresponding to the communication bottleneck, that is, the first communication duration is longer than the first calculation duration, and the second communication duration is shorter than the second calculation duration, the calculation instructions of the first operator instruction and the second operator instruction can be divided. In this case, step S23 can include:

[0090] The calculation instructions of the first operator instruction are divided into multiple first calculation execution segments, and the calculation instructions of the second operator instruction are divided into multiple second calculation execution segments; the first execution segment includes the first calculation execution segment, and the second execution segment includes the second calculation execution segment.

[0091] That is, for the communication bottleneck operator instruction, the computing instructions of the first operator instruction and the second operator instruction can be divided based on the switching parameters such as the data flow water granularity and the instruction context switching speed, and the computing time length of the first operator instruction and the second operator instruction, to obtain the first computing execution segment and the second computing execution segment, so that the first execution segment includes the first computing execution segment, and the second execution segment includes the second computing execution segment.

[0092] In some possible implementation manners, the step of dividing the computing instructions of the first operator instruction to obtain the plurality of first computing execution segments comprises:

[0093] The computing instructions of the first operator instruction are divided based on the switching parameters and the first computing time length of the first operator instruction to obtain a plurality of first computing execution segments, and the routing instructions of the first operator instruction maintain a data dependency relationship with the plurality of first computing execution segments.

[0094] The switching parameters include a data flow water granularity and an instruction context switching speed, the data flow water granularity is used to represent a minimum data amount of each communication transmission and / or a minimum data amount required for each computing operation, and the instruction context switching speed is used to represent a minimum time length required for each switching during the segmented execution of the operator instruction by the processing core.

[0095] For the communication bottleneck first operator instruction, the computing instructions of the first operator instruction can be divided based on the switching parameters and the first computing time length of the first operator instruction. The execution time length corresponding to the first computing execution segment can be greater than or equal to the instruction context switching speed, so as to ensure that the computing execution segment can be successfully switched during the interleaved execution. In addition, the computing data amount corresponding to the first computing execution segment can be greater than or equal to the minimum data amount required for each computing operation in the data flow water granularity, so as to ensure that the computing execution segment can normally perform the computing operation. The disclosure does not limit the specific division manner.

[0096] After the division, the routing instructions of the first operator instruction maintain a data dependency relationship with the plurality of first computing execution segments, that is, the data flow water between the routing instructions and the plurality of first computing execution segments is maintained. Before a first computing execution segment is executed, it is ensured that the computing data corresponding to the first computing execution segment has been transmitted to the target processing core through the routing instruction, so as to guarantee the execution of the first computing execution segment, realize the synchronous performance of the data transmission and the data computing, and thus improve the computing efficiency.

[0097] The division manner of the computing instructions of the second operator instruction is similar to that of the computing instructions of the first operator instruction, which will not be described herein again.

[0098] Further, in step S24, the plurality of first computing execution segments of the first operator instruction and the plurality of second computing execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core executes the plurality of first computing execution segments of the first operator instruction and the plurality of second computing execution segments of the second operator instruction in an interleaved manner.

[0099] The process in which the target processing core executes the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction in an interleaved manner comprises: the target processing core executes the plurality of first computing execution segments of the first operator instruction and the plurality of second computing execution segments of the second operator instruction in an interleaved manner.

[0100] Figure 4 A schematic diagram of the interleaved execution of the operator instructions provided by the embodiments of the present disclosure is shown in FIG. 2. Referring to FIG. 2, Figure 4 The first operator instruction comprises a routing instruction R1 and a computing instruction C1, and the communication duration of the routing instruction R1 is greater than the computing duration of the computing instruction C1. Correspondingly, the second operator instruction can comprise a routing instruction R2 (not shown) and a computing instruction C2, or can only comprise the computing instruction C2 (i.e., the communication duration of the routing instruction R2 is zero).

[0101] Referring to FIG. 2, Figure 4 The computing instructions C1 and C2 can be respectively split to obtain the plurality of first computing execution segments and the plurality of second computing execution segments. In the execution process, the target processing core executes the first computing execution segments and the second computing execution segments in an interleaved manner. Moreover, the execution process maintains the pipeline between the routing instruction R1 and the computing instruction C1. That is, the routing instruction R1 receives / transmits data at the same time, and the computing instruction C1 performs computing using the data or synchronously generates the data at the same time.

[0102] In this case, if the communication duration of the routing instruction of the second operator instruction is zero, the target processing core directly executes the routing instruction R1 of the first operator instruction; if the communication duration of the routing instruction of the second operator instruction is not zero, the target processing core can first execute the routing instruction of the second operator instruction, transmit the computing data required by the second operator instruction, and then execute the routing instruction R1 of the first operator instruction. The present disclosure does not limit the specific processing manner.

[0103] In this way, the interleaved execution of the computing instructions under the communication bottleneck in time can be realized, thereby improving the utilization rate of the computing resources.

[0104] In some possible implementation manners, if the first operator instruction is a computing bottleneck operator instruction, i.e., the first communication duration is less than the first computing duration, and the second communication duration is greater than the second computing duration, the routing instructions of the first operator instruction and the second operator instruction can be split. In this case, step S23 can comprise:

[0105] The routing instruction of the first operator instruction is split to obtain a plurality of first routing execution segments, and the routing instruction of the second operator instruction is split to obtain a plurality of second routing execution segments; the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment.

[0106] That is, for the operator instruction of the calculation bottleneck, the routing instruction of the first operator instruction and the routing instruction of the second operator instruction are respectively split based on the communication duration of the first operator instruction and the second operator instruction according to parameters such as data stream water granularity and instruction context switching speed, to obtain the first routing execution segment and the second routing execution segment, so that the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment.

[0107] Further, in step S24, the plurality of first routing execution segments of the first operator instruction and the plurality of second routing execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core staggeredly executes the plurality of first routing execution segments of the first operator instruction and the plurality of second routing execution segments of the second operator instruction.

[0108] The process that the target processing core staggeredly executes the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction includes that the target processing core staggeredly executes the plurality of first routing execution segments of the first operator instruction and the plurality of second routing execution segments of the second operator instruction.

[0109] Figure 5 The schematic diagram of the staggered execution of the operator instruction provided by the embodiment of the present disclosure. Referring to Figure 5 , the first operator instruction includes routing instruction R1 and calculation instruction C1, and the communication duration of the routing instruction R1 is less than the calculation duration of the calculation instruction C1; accordingly, the second operator instruction can include routing instruction R2 and calculation instruction C2 (not shown).

[0110] Referring to Figure 5 , the routing instructions R1 and R2 can be respectively split to obtain a plurality of first routing execution segments and a plurality of second routing execution segments. In the execution process, the target processing core staggeredly executes the first routing execution segment and the second routing execution segment. Moreover, the execution process maintains the flow of the routing instruction R1 and the calculation instruction C1. That is, the routing instruction R1 receives / transmits data at the same time, and the calculation instruction C1 simultaneously performs calculation using the data or synchronously generates the data.

[0111] In this case, the target processing core can execute the calculation instruction of the first operator instruction first, and then execute the calculation instruction of the second operator instruction, and the present disclosure does not limit the specific processing mode.

[0112] In this way, the routing instructions under the calculation bottleneck can be staggered in time, thereby improving the utilization of communication resources.

[0113] In some possible implementation manners, for the operator instructions of the communication bottleneck and the operator instructions of the calculation bottleneck, the routing instructions and the calculation instructions can also be split at the same time. In this case, for the first communication duration being greater than the first calculation duration and the second communication duration being less than the second calculation duration, or the first communication duration being less than the first calculation duration and the second communication duration being greater than the second calculation duration, the following processing can be performed.

[0114] The step S23 comprises:

[0115] The calculation instructions of the first operator instructions are split to obtain a plurality of first calculation execution segments, and the routing instructions of the first operator instructions are split to obtain a plurality of first routing execution segments; the first execution segment comprises the first calculation execution segment and the first routing execution segment.

[0116] The calculation instructions of the second operator instructions are split to obtain a plurality of second calculation execution segments, and the routing instructions of the second operator instructions are split to obtain a plurality of second routing execution segments; the second execution segment comprises the second calculation execution segment and the second routing execution segment.

[0117] That is, the routing instructions of the first operator instructions and the second operator instructions can be split based on the data flow water granularity, the instruction context switching speed and other parameters, the communication duration and the calculation duration of the first operator instructions and the second operator instructions, to obtain the first routing execution segment and the second routing execution segment; and the calculation instructions of the first operator instructions and the second operator instructions are split to obtain the first calculation execution segment and the second calculation execution segment, so that the first execution segment comprises the first routing execution segment and the first calculation execution segment, and the second execution segment comprises the second routing execution segment and the second calculation execution segment.

[0118] Further, in step S24, the plurality of first execution segments of the first operator instructions and the plurality of second execution segments of the second operator instructions are mapped to the target processing core, so that the target processing core staggered executes the plurality of first execution segments of the first operator instructions and the plurality of second execution segments of the second operator instructions.

[0119] The process that the target processing core staggered executes the plurality of first execution segments of the first operator instructions and the plurality of second execution segments of the second operator instructions comprises: the target processing core staggered executes the plurality of first routing execution segments of the first operator instructions and the plurality of second routing execution segments of the second operator instructions, and staggered executes the plurality of first calculation execution segments of the first operator instructions and the plurality of second calculation execution segments of the second operator instructions.

[0120] In some embodiments, the execution process can be represented as Figure 4 the combination of the routing instructions R1 and R2 in the Figure 5 In the execution process, the pipeline between the routing instruction R1 and the computing instruction C1 is maintained, and the pipeline between the routing instruction R2 and the computing instruction C2 is maintained.

[0121] In this way, the routing instructions and the computing instructions can be jointly staggered in time, thereby improving the utilization of communication resources and computing resources at the same time, and further improving the performance of the neural network execution.

[0122] In some possible implementation manners, the processing core of the many-core system usually includes a computing unit such as a MAC (Multiply-Accumulate) array, for executing the computing process of the neural network operator. The computing unit can be spatially split to perform the computation of the operator instructions in parallel. In this case, for the operator instruction of the communication bottleneck, the utilization of the computing resource can also be improved through spatial parallel execution.

[0123] In some possible implementation manners, the instruction processing method according to the embodiments of the present disclosure further includes:

[0124] In the case where the first communication duration of the first operator instruction is greater than the first computing duration, a third operator instruction executed in parallel with the first operator instruction is determined;

[0125] According to the computing instruction of the first operator instruction and the computing instruction of the third operator instruction, a first execution region and a second execution region of the computing unit in the target processing core are determined;

[0126] The first operator instruction and the third operator instruction, and the first execution region and the second execution region are mapped to the target processing core, so that the target processing core executes the computing instruction of the first operator instruction through the first execution region, and executes the computing instruction of the third operator instruction in parallel through the second execution region.

[0127] For example, if the first operator instruction is an operator instruction of a communication bottleneck, that is, the first communication duration is greater than the first computing duration, a third operator instruction executed in parallel with the first operator instruction can be determined from the operator execution sequence. The determination manner of the third operator instruction is similar to that of the second operator instruction.

[0128] In some possible implementation manners, an operator instruction in the operator execution sequence that has a calculation bottleneck (i.e., a communication duration is less than a calculation duration) and has no data dependency relationship with the first operator instruction can be determined first; if there is only one determined operator instruction, the determined operator instruction can be directly used as the third operator instruction; if there are multiple determined operator instructions, an operator instruction with a zero communication duration is preferred; if there is no operator instruction with a zero communication duration, an operator instruction is selected randomly or an operator instruction that has a minimum interval with the first operator instruction is selected.

[0129] In some possible implementation manners, a first execution region and a second execution region of a calculation unit in a target processing core are determined according to the calculation instruction of the first operator instruction and the calculation instruction of the third operator instruction. The first execution region and the second execution region can be uniformly divided, or be divided based on calculation amounts of the calculation instruction of the first operator instruction and the calculation instruction of the third operator instruction, and the specific division manner is not limited in the present disclosure.

[0130] In some possible implementation manners, the order of the operator instructions in the operator execution sequence can be adjusted, so that the first operator instruction and the second operator instruction are adjacent, and information of the execution region and an identifier of parallel execution are added; or the order of the operator instructions in the operator execution sequence can not be adjusted, and the information of the execution region and the identifier of parallel execution are directly added to the first operator instruction and the second operator instruction, and the present disclosure does not limit this.

[0131] In some possible implementation manners, after the first execution region and the second execution region are determined, the first operator instruction and the third operator instruction and the first execution region and the second execution region can be mapped to the target processing core, so that the target processing core executes the calculation instruction of the first operator instruction through the first execution region, and executes the calculation instruction of the third operator instruction through the second execution region in parallel.

[0132] Figure 6 An illustrative diagram of parallel execution of operator instructions provided by an embodiment of the present disclosure is provided. Referring to Figure 6 , the first operator instruction includes a routing instruction R1 and a calculation instruction C1, and a communication duration of the routing instruction R1 is greater than a calculation duration of the calculation instruction C1; accordingly, the second operator instruction can include a routing instruction R2 (not shown) and a calculation instruction C2, or can only include the calculation instruction C2 (i.e., a communication duration of the routing instruction R2 is zero).

[0133] Referring to Figure 6The MAC array of the target processing core can be divided into two execution regions, one of which executes the calculation instruction C1, and the other of which executes the calculation instruction C2 in parallel. In addition, the pipeline between the routing instruction R1 and the calculation instruction C1 is maintained during execution. That is, the routing instruction R1 is executed while receiving / transmitting data, and the calculation instruction C1 is executed while performing calculation or generating data using the data.

[0134] In this case, if the communication duration of the routing instruction of the second operator instruction is zero, the target processing core directly executes the routing instruction R1 of the first operator instruction; if the communication duration of the routing instruction of the second operator instruction is not zero, the target processing core can first execute the routing instruction of the second operator instruction, transmit the calculation data required by the second operator instruction, and then execute the routing instruction R1 of the first operator instruction. The present disclosure does not limit the specific processing manner.

[0135] In this way, the calculation instruction under the communication bottleneck can be executed in parallel in space, thereby improving the utilization of the calculation resource.

[0136] According to the instruction processing method of the embodiment of the present disclosure, the operator instructions of the communication bottleneck and the calculation bottleneck in the operator execution sequence of the processing core can be determined, the utilization of the calculation resource and the communication resource of the processing core is improved by controlling the time-interleaved execution or the space-parallel execution of the plurality of operator instructions, and the performance and the efficiency of the neural network execution are improved.

[0137] In the case of time-interleaved execution, the corresponding operator instruction can be divided into a plurality of execution segments according to the embodiment of the present disclosure, and the plurality of execution segments of the operator instruction are mapped to the processing core, so that the processing core executes the execution segments of different operator instructions in an interleaved manner, thereby improving the utilization of the calculation resource and the communication resource of the processing core; in the case of space-parallel execution, the calculation unit of the processing core can be divided into a plurality of execution regions according to the embodiment of the present disclosure, and the calculation instructions of different operator instructions are executed in parallel in space, thereby improving the utilization of the calculation resource of the processing core.

[0138] The instruction processing method according to the embodiment of the present disclosure can be applied to the application scenario of accelerating the calculation of the neural network by the many-core system. The neural network can be any type of neural network such as an artificial neural network ANN, a spiking neural network SNN, etc., the neural network is used to execute any one of an image processing task, a speech processing task, a text processing task, and a video processing task, and the data processed by the neural network includes any one of an image, a speech, a text, and a video, and the present disclosure does not limit this.

[0139] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form combined embodiments without deviating from the principle logic. Limited by the length of the present disclosure, the present disclosure will not be described again. Those skilled in the art can understand that in the above-mentioned method of the specific embodiment, the specific execution order of each step should be determined according to its function and possible internal logic.

[0140] In addition, the present disclosure also provides an instruction processing apparatus, an electronic device, and a computer readable storage medium, which can be used to implement any one of the instruction processing methods provided by the present disclosure. The corresponding technical solutions and descriptions are described in the method part and will not be described again.

[0141] Figure 7 A block diagram of an instruction processing apparatus provided by an embodiment of the present disclosure.

[0142] Referring to Figure 7 The instruction processing apparatus provided by the embodiment of the present disclosure comprises:

[0143] The operator sequence determination module 71 is configured to determine an operator execution sequence to be executed by a target processing core in a many-core system, the operator execution sequence comprising a plurality of operator instructions, the operator instructions comprising routing instructions and calculation instructions, the routing instructions being configured to perform data transmission operations of the operator instructions, and the calculation instructions being configured to perform calculation operations of the operator instructions; the many-core system comprising a plurality of processing cores, and the target processing core being any one of the plurality of processing cores;

[0144] The instruction determination module 72 is configured to determine, according to a first communication time length of a routing instruction and a first calculation time length of a calculation instruction in a first operator instruction of the operator execution sequence, a second operator instruction to be executed in an interleaved manner with the first operator instruction from the operator execution sequence; wherein a time length relationship between a second communication time length of a routing instruction and a second calculation time length of a calculation instruction in the second operator instruction is different from a time length relationship between the first communication time length and the first calculation time length.

[0145] The instruction splitting module 73 is configured to split the first operator instruction and the second operator instruction according to the first communication time length, the first calculation time length, the second communication time length, and the second calculation time length, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction.

[0146] The instruction mapping module 74 is configured to map the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction to the target processing core, so that the target processing core executes the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction in an interleaved manner.

[0147] In some possible implementation manners, the instruction determining module 72 is configured to:

[0148] In a case where the first communication duration is greater than the first calculation duration, determining a second operator instruction according to an operator instruction in the operator execution sequence, which has a communication duration smaller than a calculation duration and has no data dependency relationship with the first operator instruction.

[0149] In a case where the first communication duration is smaller than the first calculation duration, determining a second operator instruction according to an operator instruction in the operator execution sequence, which has a communication duration greater than a calculation duration and has no data dependency relationship with the first operator instruction.

[0150] In some possible implementation manners, the first communication duration is greater than the first calculation duration and the second communication duration is smaller than the second calculation duration,

[0151] The instruction splitting module 73 is configured to split the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments, and split the calculation instruction of the second operator instruction to obtain a plurality of second calculation execution segments; the first execution segment includes a first calculation execution segment, and the second execution segment includes a second calculation execution segment.

[0152] The instruction mapping module 74 is configured to: the target processing core interleaves execution of the plurality of first calculation execution segments of the first operator instruction and the plurality of second calculation execution segments of the second operator instruction.

[0153] In some possible implementation manners, the instruction splitting module 73 is configured to:

[0154] The calculation instruction of the first operator instruction is split according to a switching parameter and a first calculation duration of the first operator instruction to obtain a plurality of first calculation execution segments, and a routing instruction of the first operator instruction maintains a data dependency relationship with the plurality of first calculation execution segments; the switching parameter includes a data flow water granularity and an instruction context switching speed, the data flow water granularity is used to represent a minimum data amount of each communication transmission and / or a minimum data amount required for each calculation operation, and the instruction context switching speed is used to represent a minimum time duration required for each switching during segmented execution of the operator instruction by the processing core.

[0155] In some possible implementation manners, the first communication duration is smaller than the first calculation duration and the second communication duration is greater than the second calculation duration,

[0156] The instruction splitting module 73 is configured to split the routing instruction of the first operator instruction to obtain a plurality of first routing execution segments, and split the routing instruction of the second operator instruction to obtain a plurality of second routing execution segments; the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment.

[0157] The instruction mapping module 74 is configured to stagger execution of the plurality of first routing execution segments of the first operator instruction and the plurality of second routing execution segments of the second operator instruction by the target processing core.

[0158] In some possible implementation manners, the instruction determining module 72 is configured to, in a case where the first communication duration is greater than the first calculation duration, determine whether there is a target operator instruction with a zero communication duration and no data dependency relationship with the first operator instruction in the operator execution sequence; and in a case where there is the target operator instruction in the operator execution sequence, determine the target operator instruction as the second operator instruction.

[0159] In some possible implementation manners, the first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration, or the first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration,

[0160] The instruction splitting module 73 is configured to split the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments, and split the routing instruction of the first operator instruction to obtain a plurality of first routing execution segments; the first execution segment includes the first calculation execution segment and the first routing execution segment; split the calculation instruction of the second operator instruction to obtain a plurality of second calculation execution segments, and split the routing instruction of the second operator instruction to obtain a plurality of second routing execution segments; the second execution segment includes the second calculation execution segment and the second routing execution segment.

[0161] The instruction mapping module 74 is configured to stagger execution of the plurality of first routing execution segments of the first operator instruction and the plurality of second routing execution segments of the second operator instruction by the target processing core, and stagger execution of the plurality of first calculation execution segments of the first operator instruction and the plurality of second calculation execution segments of the second operator instruction.

[0162] In some possible implementation manners, the apparatus further includes:

[0163] The parallel instruction determining module is configured to, in a case where a first communication duration of the first operator instruction is greater than a first calculation duration, determine a third operator instruction executed in parallel with the first operator instruction.

[0164] The region determining module is configured to determine a first execution region and a second execution region of a computing unit in the target processing core according to the calculation instruction of the first operator instruction and the calculation instruction of the third operator instruction.

[0165] The region mapping module is configured to map the first operator instruction and the third operator instruction, and the first execution region and the second execution region to the target processing core, so that the target processing core executes the calculation instruction of the first operator instruction through the first execution region, and executes the calculation instruction of the third operator instruction through the second execution region in parallel.

[0166] In some possible implementation manners, the operator instruction includes operation instructions of a neural network operator, and the neural network operator includes a coarse-grained operation, which is used to represent an operation with a communication duration longer than a first threshold and / or a calculation duration longer than a second threshold.

[0167] Figure 8 A block diagram of an electronic device is provided according to an embodiment of the present disclosure.

[0168] With reference to Figure 8 According to an embodiment of the present disclosure, an electronic device is provided, which includes at least one processor 501, at least one memory 502, and one or more I / O interfaces 503 connected between the processor 501 and the memory 502. The memory 502 stores one or more computer programs executable by the at least one processor 501. The one or more computer programs are executed by the at least one processor 501, so that the at least one processor 501 can perform the instruction processing method described above.

[0169] According to an embodiment of the present disclosure, a computer readable storage medium is provided, which stores a computer program. When the computer program is executed by a processor, the instruction processing method described above is implemented. The computer readable storage medium can be a volatile or non-volatile computer readable storage medium.

[0170] According to an embodiment of the present disclosure, a computer program product is provided, which includes a computer readable code or a non-volatile computer readable storage medium carrying the computer readable code. When the computer readable code is run in a processor of an electronic device, the processor in the electronic device executes the instruction processing method described above.

[0171] Those of ordinary skill in the art will realize and understand that all or some of the steps in the methods disclosed above and the functional modules / units in the systems and devices can be implemented as software, firmware, hardware, and appropriate combinations thereof. In hardware implementation, the division between the functional modules / units referred to in the above description does not necessarily correspond to the division of physical components; for example, one physical component can have multiple functions, or one function or step can be performed by several physical components in cooperation. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on computer readable storage media, which can include computer storage media (or non-transitory media) and communication media (or transitory media).

[0172] As is well known to those of ordinary skill in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer readable program instructions, data structures, program modules or other data. Computer storage media include, but are not limited to, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, compact disc read only memory (CD-ROM), digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. Further, it should be understood by those of ordinary skill in the art that communication media typically embodies computer readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transport mechanism and includes any information delivery media.

[0173] Computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and / or a wireless network. The network can comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing / processing device.

[0174] Computer readable program instructions for carrying out operations of the present disclosure can be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state setting data, or any combination of one or more of the above in any combination of one or more programming languages including an object oriented programming language such as Smalltalk, C++ or the like and conventional procedural programming languages such as the "C" programming language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection can be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) can execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present disclosure.

[0175] The computer program product described herein can be embodied in a specific manner by hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium. In another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), and the like.

[0176] The various aspects of the present disclosure are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It should be understood that each block of the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, can be implemented by computer readable program instructions.

[0177] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can include a non-transitory computer readable storage medium that can be a computer- readable storage medium having no data storage cycles that change state. The instructions can be executed by one or more processors of a computer, to cause a series of operational elements or steps to be performed on the computer to produce a computer implemented process; such that the instructions, which execute via one or more computer program product, implement a computer implemented process for performing the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0178] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational elements or steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process such that the instructions that execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0179] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational elements or steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer implemented process such that the instructions that execute on the computer, other programmable data processing apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0180] Example embodiments have been disclosed and although specific terms are employed, they are used in a generic and descriptive sense only and not for purposes of limitation. In some instances, it will be apparent to those skilled in the art that features, characteristics or elements described with respect to a particular embodiment can be used, alone or in combination, with other embodiments unless specifically recited otherwise. Accordingly, various modifications, alterations, and improvements will readily occur to those skilled in the art with the foregoing description. Accordingly, the present disclosure is not intended to be limited by the method, system, and apparatus disclosed herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. An instruction processing method, characterized in that, include: The system determines the operator execution sequence to be executed by the target processing core in the many-core system. The operator execution sequence includes multiple operator instructions, each of which includes routing instructions and computation instructions. The routing instructions are used to perform data transmission operations of the operator instructions, and the computation instructions are used to perform computation operations of the operator instructions. The many-core system includes multiple processing cores, and the target processing core is any one of the multiple processing cores. Based on the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence, a second operator instruction that is interleaved with the first operator instruction is determined from the operator execution sequence; wherein, the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration; Based on the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, the first operator instruction and the second operator instruction are segmented to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction; The first execution segments of the first operator instruction and the second execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core executes the first execution segments of the first operator instruction and the second execution segments of the second operator instruction in an interleaved manner.

2. The method according to claim 1, characterized in that, The step of determining a second operator instruction that is interleaved with the first operator instruction from the operator execution sequence based on the first communication duration of the routing instruction and the first computation duration of the computation instruction in the first operator instruction of the operator execution sequence includes: If the first communication duration is longer than the first calculation duration, a second operator instruction is determined based on the operator instruction in the operator execution sequence whose communication duration is shorter than the calculation duration and which has no data dependency relationship with the first operator instruction. If the first communication duration is less than the first computation duration, a second operator instruction is determined based on the operator instruction in the operator execution sequence whose communication duration is greater than the computation duration and which has no data dependency on the first operator instruction.

3. The method according to claim 1, characterized in that, The first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration. The step of segmenting the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: The calculation instructions of the first operator instruction are divided into multiple first calculation execution segments, and the calculation instructions of the second operator instruction are divided into multiple second calculation execution segments; the first execution segment includes a first calculation execution segment, and the second execution segment includes a second calculation execution segment; The target processing core interleaved execution of multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The target processing core executes multiple first computation execution segments of the first operator instruction and multiple second computation execution segments of the second operator instruction in an interleaved manner.

4. The method according to claim 3, characterized in that, The calculation instructions of the first operator instruction are segmented to obtain multiple first calculation execution segments, including: Based on the switching parameters and the first calculation duration of the first operator instruction, the calculation instructions of the first operator instruction are divided into multiple first calculation execution segments, and the routing instructions of the first operator instruction maintain a data dependency relationship with the multiple first calculation execution segments. The switching parameters include data pipeline granularity and instruction context switching speed. The data pipeline granularity is used to represent the minimum amount of data transmitted in each communication and / or the minimum amount of data required for each computation operation. The instruction context switching speed is used to represent the minimum duration required for each switch during the segmented execution of operator instructions in the processing kernel.

5. The method according to claim 1, characterized in that, The first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration. The step of segmenting the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: The routing instructions of the first operator instruction are segmented to obtain multiple first routing execution segments, and the routing instructions of the second operator instruction are segmented to obtain multiple second routing execution segments; the first execution segment includes a first routing execution segment, and the second execution segment includes a second routing execution segment; The target processing core interleaved execution of multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The target processing core executes multiple first route execution segments of the first operator instruction and multiple second route execution segments of the second operator instruction in an interleaved manner.

6. The method according to claim 2, characterized in that, When the first communication duration is longer than the first computation duration, determining the second operator instruction based on operator instructions in the operator execution sequence whose communication duration is shorter than the computation duration and which have no data dependency on the first operator instruction includes: If the first communication duration is longer than the first computation duration, determine whether there is a target operator instruction in the operator execution sequence that has a communication duration of zero and no data dependency with the first operator instruction; If a target operator instruction exists in the operator execution sequence, the target operator instruction is determined as the second operator instruction.

7. The method according to claim 1, characterized in that, The first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration, or the first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration. The step of segmenting the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: The computation instructions of the first operator instruction are segmented to obtain multiple first computation execution segments, and the routing instructions of the first operator instruction are segmented to obtain multiple first routing execution segments; the first execution segment includes a first computation execution segment and a first routing execution segment; The calculation instructions of the second operator instruction are divided into multiple second calculation execution segments, and the routing instructions of the second operator instruction are divided into multiple second routing execution segments; the second execution segment includes a second calculation execution segment and a second routing execution segment; The target processing core interleaved execution of multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The target processing core interleaved execution of multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction, and interleaved execution of multiple first calculation execution segments of the first operator instruction and multiple second calculation execution segments of the second operator instruction.

8. The method according to claim 1, characterized in that, The method further includes: If the first communication duration of the first operator instruction is greater than the first computation duration, a third operator instruction is determined to be executed in parallel with the first operator instruction. Based on the calculation instructions of the first operator instruction and the calculation instructions of the third operator instruction, the first execution region and the second execution region of the computing unit in the target processing core are determined; The first operator instruction and the third operator instruction, as well as the first execution region and the second execution region, are mapped to the target processing core, so that the target processing core executes the calculation instructions of the first operator instruction through the first execution region and executes the calculation instructions of the third operator instruction in parallel through the second execution region.

9. The method according to claim 1, characterized in that, The operator instructions include operation instructions for neural network operators, which include coarse-grained operations. The coarse-grained operations are used to represent operations where the communication time required to complete one processing step is greater than a first threshold and / or the computation time is greater than a second threshold.

10. An instruction processing apparatus, characterized in that, include: An operator sequence determination module is used to determine the operator execution sequence to be executed by the target processing core in a many-core system. The operator execution sequence includes multiple operator instructions, each of which includes a routing instruction and a calculation instruction. The routing instruction is used to perform data transmission operations of the operator instructions, and the calculation instruction is used to perform calculation operations of the operator instructions. The many-core system includes multiple processing cores, and the target processing core is any one of the multiple processing cores. The instruction determination module is used to determine, based on the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence, a second operator instruction that is executed interleaved with the first operator instruction; wherein, the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration; The instruction segmentation module is used to segment the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction; An instruction mapping module is used to map multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction to the target processing core, so that the target processing core executes the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction in an interleaved manner.

11. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to perform the instruction processing method as described in any one of claims 1-9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the instruction processing method as described in any one of claims 1-9.

Images