Instruction processing method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an instruction processing method and device, electronic equipment and a computer readable storage medium. The method comprises the steps that a to-be-executed operator execution sequence of a target processing core in a many-core system is determined; according to a first communication duration of a routing instruction in a first operator instruction of the operator execution sequence and a first calculation duration of a calculation instruction, determining a second operator instruction executed in a staggered manner with the first operator instruction from the operator execution sequence; the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration; segmenting the first operator instruction and the second operator instruction to obtain a plurality of first execution segments and a plurality of second execution segments; and mapping the plurality of first execution segments and the second execution segments to the target processing core, so that the target processing core executes the plurality of first execution segments and the second execution segments in a staggered manner. According to the embodiment of the invention, the resource utilization rate of the processing core can be improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to an instruction processing method and device, an electronic device, and a computer-readable storage medium. Background Art

[0002] With the popularity of artificial intelligence applications, the demand for neural network computing is growing, and neural network computing is becoming more and more important. However, the large amount of computing and storage required by neural networks makes it difficult for general-purpose processors to effectively perform these tasks. Therefore, many specialized neural network acceleration architectures have been developed, among which many-core systems (including many-core chips or boards, etc.) are an important technical path. In many-core systems, when executing neural networks, different operators of the network are usually distributed on different processing cores. This process is called mapping, and the resulting scheme is called a mapping scheme. Data transmission between processing cores is achieved through the on-chip network (or routing system) of the many-core system, and there is usually a large amount of data transmission inside the neural network. Due to the diversity of neural networks, different operators have different requirements for computing and communication, which may cause different performance bottlenecks. Summary of the invention

[0003] The present disclosure provides an instruction processing method and device, an electronic device, and a computer-readable storage medium.

[0004] In a first aspect, the present disclosure provides an instruction processing method, the method comprising:

[0005] Determining an operator execution sequence to be executed by a target processing core in a many-core system, wherein the operator execution sequence includes a plurality of operator instructions, wherein the operator instructions include routing instructions and calculation instructions, wherein the routing instructions are used to execute a data transmission operation of the operator instructions, and the calculation instructions are used to execute a calculation operation of the operator instructions; the many-core system includes a plurality of processing cores, and the target processing core is any one of the plurality of processing cores;

[0006] Determine, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence; wherein the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration;

[0007] According to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, the first operator instruction and the second operator instruction are divided to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0008] The plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction are mapped to the target processing core so that the target processing core interleaves the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction.

[0009] In a second aspect, the present disclosure provides an instruction processing device, the device comprising:

[0010] an operator sequence determination module, configured to determine an operator execution sequence to be executed by a target processing core in a many-core system, wherein the operator execution sequence includes a plurality of operator instructions, wherein the operator instructions include routing instructions and calculation instructions, wherein the routing instructions are used to execute a data transmission operation of the operator instructions, and the calculation instructions are used to execute a calculation operation of the operator instructions; the many-core system includes a plurality of processing cores, and the target processing core is any one of the plurality of processing cores;

[0011] An instruction determination module, configured to determine, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to a first communication duration of a routing instruction and a first calculation duration of a calculation instruction in a first operator instruction of the operator execution sequence; wherein a duration relationship between a second communication duration of the routing instruction and a second calculation duration of the calculation instruction in the second operator instruction is different from a duration relationship between the first communication duration and the first calculation duration;

[0012] an instruction segmentation module, configured to segment the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0013] An instruction mapping module is used to map multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction to the target processing core, so that the target processing core interleaves the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction.

[0014] In a third aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned instruction processing method.

[0015] In a fourth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned instruction processing method when executed by a processor.

[0016] The embodiments provided by the present disclosure can determine the operator instructions of the communication bottleneck and the computing bottleneck in the operator execution sequence of the processing core, divide the corresponding operator instructions with complementary bottlenecks into multiple execution segments, and map the multiple execution segments of the operator instructions to the processing core, so that the processing core interleaves the execution segments of the operator instructions, thereby improving the utilization rate of the computing resources and communication resources of the processing core, and improving the performance and efficiency of the neural network execution.

[0017] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0019] Figure 1 This is a schematic diagram of the roofline model;

[0020] Figure 2 A flowchart of an instruction processing method provided by an embodiment of the present disclosure;

[0021] Figure 3 A schematic diagram of an operator instruction execution process of a communication bottleneck provided by an embodiment of the present disclosure;

[0022] Figure 4 A schematic diagram of interleaved execution of operator instructions provided in an embodiment of the present disclosure;

[0023] Figure 5 A schematic diagram of interleaved execution of operator instructions provided in an embodiment of the present disclosure;

[0024] Figure 6 A schematic diagram of parallel execution of operator instructions provided in an embodiment of the present disclosure;

[0025] Figure 7 A block diagram of an instruction processing device provided by an embodiment of the present disclosure;

[0026] Figure 8 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0028] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0029] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0030] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0031] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0032] In the field of neural network acceleration computing, the roofline model is often used in related technologies to judge the performance of the current operator execution. There are two important indicators for processing cores, computing power and bandwidth. Computing power refers to the performance upper limit of the processing core, such as the number of floating-point or integer operations that can be completed per second. Bandwidth is also the bandwidth upper limit of the computing platform, which refers to the upper limit of the amount of data exchange that a processing core can complete per second. Dividing the two indicators can get the upper limit of the computing intensity of the processing core, which describes how many times the processing core can perform calculations per unit data exchange.

[0033] Figure 1 is a schematic diagram of the roofline model. Figure 1As shown in the figure, the horizontal axis of the roofline model is the computing density I, and the vertical axis is the computing performance P. The roofline model divides the performance space into two areas: the bandwidth-limited area and the computing-limited area. At the beginning of processing, data is transmitted with the maximum memory bandwidth β (bytes / second); after reaching the maximum computing performance π, the maximum computing density Imax=π / β is reached. In this roofline model, performance limitation depends on the balance between computing resources and memory access speed.

[0034] In the mapping strategy of many-core systems, a processing core is often assigned an operator execution sequence. Different operators in the operator execution sequence are likely to be in different areas of the roofline model and have different resource bottlenecks. The sequential execution of instructions in the operator execution sequence will result in low utilization of the overall computing and communication resources of the processing core, resulting in a decrease in the overall performance of the neural network model execution.

[0035] According to the instruction processing method of the embodiment of the present disclosure, it is possible to determine the operator instructions of the communication bottleneck and the computing bottleneck in the operator execution sequence of the processing core, divide the corresponding operator instructions into multiple execution segments, and map the multiple execution segments of the operator instructions to the processing core, so that the processing core interleaves the execution segments of the operator instructions, thereby improving the utilization rate of the computing resources and communication resources of the processing core, and improving the performance and efficiency of the neural network execution.

[0036] The instruction processing method according to the embodiment of the present disclosure can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a vehicle-mounted device, a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory. Alternatively, the method can be executed by a server.

[0037] In some possible implementations, the instruction processing method according to the embodiments of the present disclosure may be executed by a compiler, or by a processor in a multi-core system, a processing core for control, or a pre-set processing core, and the present disclosure does not limit this.

[0038] Figure 2 A flowchart of an instruction processing method provided by an embodiment of the present disclosure. Figure 2 , the method comprising:

[0039] In step S21, an operator execution sequence to be executed by a target processing core in a many-core system is determined, wherein the operator execution sequence includes a plurality of operator instructions, wherein the operator instructions include routing instructions and calculation instructions, wherein the routing instructions are used to execute a data transmission operation of the operator instructions, and the calculation instructions are used to execute a calculation operation of the operator instructions; the many-core system includes a plurality of processing cores, and the target processing core is any one of the plurality of processing cores;

[0040] In step S22, according to the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction is determined from the operator execution sequence; wherein the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration;

[0041] In step S23, the first operator instruction and the second operator instruction are segmented according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0042] In step S24, multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core interleaves the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction.

[0043] For example, the many-core system can be a many-core chip or board, or any electronic device including a many-core chip or board. The many-core system includes multiple processing cores, and data transmission between processing cores and data transmission between processing cores and external devices are realized through the on-chip network (or routing system) of the many-core system.

[0044] In some possible implementations, in the process of accelerating the calculation of a neural network through a many-core system, each processing core of the many-core system corresponds to an operator execution sequence to be executed, and the operator execution sequence includes multiple operator instructions. The operator instructions include operation instructions of the neural network operator, which are used to perform coarse-grained operations of the neural network operator.

[0045] The neural network operator includes coarse-grained operations for parallel processing, and the coarse-grained operations are used to indicate operations in which a communication time greater than a first threshold and / or a calculation time greater than a second threshold is required to complete one processing.

[0046] That is, a neural network operator may include one or more coarse-grained operations that require parallel computing, and a coarse-grained operation represents an operation with a large amount of computation and a long data communication time or computing time. For example, a neural network operator may be a convolution operator, which includes coarse-grained operations such as matrix multiplication operations and matrix addition operations; and a matrix multiplication operation that is a coarse-grained operation includes multiple element multiplication operations, which can be implemented through parallel computing. In this example, the element multiplication operation is a fine-grained operation.

[0047] In some possible implementations, the first threshold of the communication time and the second threshold of the computation time are used to indicate the length of the data communication time and the computation time required for the coarse-grained operation to complete one processing, respectively; if the communication time required to complete one processing is greater than the first threshold, the data communication time is considered to be long; otherwise, the data communication time is considered to be short. Similarly, if the computation time required to complete one processing is greater than the second threshold, the computation time is considered to be long; otherwise, the computation time is considered to be short.

[0048] It should be understood that those skilled in the art may set specific values ​​of the first threshold and the second threshold according to actual conditions, and the present disclosure does not limit this.

[0049] In some possible implementations, the operator instruction includes a routing instruction and a calculation instruction, the routing instruction is used to execute the data transmission operation of the operator instruction, and the calculation instruction is used to execute the calculation operation of the operator instruction.

[0050] In some possible implementations, for any one of the multiple processing cores (hereinafter referred to as a target processing core), an operator execution sequence to be executed by the target processing core may be determined in step S21.

[0051] Among them, for any operator instruction in the operator execution sequence, the communication duration of the routing instruction and the calculation duration of the calculation instruction in the operator instruction can be determined according to the computing resources and communication resources of the target processing core, as well as the amount of data that the operator instruction needs to transmit (such as weight data and convolution data to be processed in the convolution operator, generated convolution result data, etc.) and the amount of calculation (such as the amount of multiplication and accumulation in convolution). The calculation duration can determine whether the operator instruction is an operator instruction of a communication bottleneck or an operator instruction of a calculation bottleneck.

[0052] In some possible implementations, if the communication duration of the routing instruction in the operator instruction is greater than the calculation duration of the calculation instruction, the operator instruction is determined to be an operator instruction of a communication bottleneck; conversely, if the communication duration of the routing instruction in the operator instruction is less than the calculation duration of the calculation instruction, the operator instruction is determined to be an operator instruction of a calculation bottleneck.

[0053] Figure 3A schematic diagram of the operator instruction execution process of the communication bottleneck provided by the embodiment of the present disclosure. Figure 3 , the operator instruction includes a routing instruction R1 and a calculation instruction C1, and the communication duration of the routing instruction R1 is longer than the calculation duration of the calculation instruction C1. During the execution process, it is necessary to maintain the pipeline between the routing instruction R1 and the calculation instruction C1. That is, while the routing instruction R1 is receiving / sending data, the calculation instruction C1 is simultaneously using the data for calculation or synchronously generating the data, thereby reducing the data that needs to be cached. In addition, there is a data dependency relationship between the routing instruction R1 and the calculation instruction C1. The processing core needs to receive a certain amount of data transmitted by the routing instruction R1 before it can start the calculation operation of the calculation instruction C1. The utilization rate of computing resources during this execution process is low.

[0054] In some possible implementations, for any operator instruction in the operator execution sequence of the target processing core (hereinafter referred to as the first operator instruction), in step S22, a second operator instruction to be executed interleaved with the first operator instruction can be determined from the operator execution sequence based on the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction.

[0055] Among them, the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration. That is, if the duration relationship between the first communication duration and the first calculation duration is that the first communication duration is greater than the first calculation duration, then the duration relationship between the second communication duration and the second calculation duration is that the second communication duration is less than the second calculation duration; conversely, if the duration relationship between the first communication duration and the first calculation duration is that the first communication duration is less than the first calculation duration, then the duration relationship between the second communication duration and the second calculation duration is that the second communication duration is greater than the second calculation duration. This processing process can be understood as: filtering out the corresponding calculation bottleneck operator instructions for the communication bottleneck operator instructions; or filtering out the corresponding communication bottleneck operator instructions for the calculation bottleneck operator instructions for subsequent processing.

[0056] In some possible implementations, if there is an operator instruction that meets the condition in the operator execution sequence, the operator instruction is directly used as the second operator instruction; if there are multiple operator instructions that meet the condition in the operator execution sequence, one or more are selected from the multiple operator instructions as the second operator instruction; if there is no second operator instruction that meets the condition in the operator execution sequence, the search fails, and the next first operator instruction is searched for the corresponding second operator instruction. The present disclosure does not limit the specific selection process of the second operator instruction.

[0057] In some possible implementations, after determining the second operator instruction, in step S23, the first operator instruction and the second operator instruction can be divided according to the first communication duration, the first calculation duration, the second communication duration and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction.

[0058] Among them, the operator instructions can be segmented based on the communication duration and calculation duration of the first operator instruction and the second operator instruction according to parameters such as data flow granularity and instruction context switching speed. The data flow granularity can represent the minimum amount of data transmitted each time or the minimum amount of data required for each calculation operation, such as 64B; the instruction context switching speed can represent the minimum duration required for each switch during the segmented execution of the operator instruction by the processing core, such as 1ms. The present disclosure does not limit the type and specific number of parameters required for the segmentation of operator instructions.

[0059] In some possible implementations, if the first operator instruction is an operator instruction of a communication bottleneck, the computing instructions of the first operator instruction and the second operator instruction may be split, and the obtained first execution segment and the second execution segment are computing execution segments; if the first operator instruction is an operator instruction of a computing bottleneck, the routing instructions of the first operator instruction and the second operator instruction may be split, and the obtained first execution segment and the second execution segment are routing execution segments.

[0060] In some possible implementations, the routing instructions and calculation instructions of the first operator instruction and the second operator instruction can also be split separately, and the obtained first execution segment and second execution segment include a routing execution segment and a calculation execution segment. The present disclosure does not limit the specific splitting method.

[0061] In some possible implementations, the order of operator instructions in the operator execution sequence can be adjusted so that the first operator instruction and the second operator instruction are adjacent, and the execution segment information and the interleaved execution identifier are added; or the order of operator instructions in the operator execution sequence may not be adjusted, and the execution segment information and the interleaved execution identifier are directly added to the first operator instruction and the second operator instruction.

[0062] In some possible implementations, in step S24, multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction may be mapped to the target processing core. After receiving the execution segments of the first operator instruction and the second operator instruction and the identification of interleaved execution, the target processing core may interleave the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction during the execution of the first operator instruction and the second operator instruction, so as to achieve complementarity of communication resources and computing resources.

[0063] According to the embodiments of the present disclosure, it is possible to determine the operator instructions of the communication bottleneck and the computing bottleneck in the operator execution sequence of the processing core, divide the corresponding operator instructions into multiple execution segments, and map the multiple execution segments of the operator instructions to the processing core, so that the processing core interleaves the execution segments of the operator instructions, thereby improving the utilization rate of the computing resources and communication resources of the processing core, and improving the performance and efficiency of the neural network execution.

[0064] The instruction processing method according to the embodiment of the present disclosure is described below.

[0065] As mentioned above, the operator execution sequence to be executed by the target processing core of the many-core system may be determined in step S21. The operator execution sequence includes a plurality of operator instructions, and the operator instructions correspond to the coarse-grained operations of the neural network operators.

[0066] In related technologies, the compiler maps multiple operator instructions in the operator execution sequence to the target processing core in sequence according to the data pipeline, so that the target processing core executes multiple operator instructions in sequence. However, the communication duration of the routing instruction and the calculation duration of the calculation instruction in the operator instruction may be different, resulting in communication bottlenecks or calculation bottlenecks, resulting in low resource utilization.

[0067] In some possible implementations, for the first operator instruction in the operator execution sequence, in step S22, a second operator instruction to be executed interleaved with the first operator instruction can be determined from the operator execution sequence based on the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction.

[0068] Among them, the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration. That is, if the duration relationship between the first communication duration and the first calculation duration is that the first communication duration is greater than the first calculation duration, then the duration relationship between the second communication duration and the second calculation duration is that the second communication duration is less than the second calculation duration; conversely, if the duration relationship between the first communication duration and the first calculation duration is that the first communication duration is less than the first calculation duration, then the duration relationship between the second communication duration and the second calculation duration is that the second communication duration is greater than the second calculation duration.

[0069] In some possible implementations, step S22 may include:

[0070] When the first communication duration is greater than the first calculation duration, determining a second operator instruction according to an operator instruction in the operator execution sequence whose communication duration is less than the calculation duration and has no data dependency relationship with the first operator instruction;

[0071] When the first communication duration is less than the first calculation duration, a second operator instruction is determined according to an operator instruction in the operator execution sequence whose communication duration is greater than the calculation duration and has no data dependency with the first operator instruction.

[0072] For example, if the first communication duration is greater than the first computing duration, the first operator instruction is an operator instruction of the communication bottleneck, and an operator instruction of the computing bottleneck needs to be selected as an operator instruction complementary to the bottleneck and executed interleaved with the first operator instruction to improve resource utilization.

[0073] That is, select an operator instruction whose communication time is less than the calculation time in the operator execution sequence from the operator execution sequence. And, for the selected operator instruction, if the operator instruction has a data dependency relationship with the first operator instruction, it is necessary to execute the operator instruction after the first operator instruction is executed, or the first operator instruction is executed after the operator instruction is executed. The operator instruction cannot be interleaved with the first operator instruction and needs to be excluded. In this way, the second operator instruction can be determined from the operator instructions in the operator execution sequence whose communication time is less than the calculation time and have no data dependency relationship with the first operator instruction.

[0074] In some possible implementations, when the first communication duration is greater than the first calculation duration, the step of determining the second operator instruction according to an operator instruction in the operator execution sequence whose communication duration is less than the calculation duration and has no data dependency relationship with the first operator instruction may include:

[0075] When the first communication duration is greater than the first calculation duration, determining whether there is a target operator instruction in the operator execution sequence whose communication duration is zero and has no data dependency relationship with the first operator instruction;

[0076] In a case where a target operator instruction exists in the operator execution sequence, the target operator instruction is determined as a second operator instruction.

[0077] For example, if the communication duration is zero, it means that the data required by the operator instruction is already in the target processing core, and no communication is required. No additional routing instructions between processing cores are required, and this type of operator instruction can be preferentially selected for interleaved execution.

[0078] In this way, for the first operator instruction with a communication bottleneck and a first communication duration greater than the first computing duration, it is possible to first determine whether there is a target operator instruction in the operator execution sequence with a communication duration of zero and no data dependency with the first operator instruction; if there is a target operator instruction, the target operator instruction can be directly determined as the second operator instruction for interleaved execution.

[0079] In some possible implementations, in the processing of the attention mechanism in the neural network, it is necessary to calculate the query feature (Query, Q), key feature (Key, K) and value (Value, V) features based on the input information and weights; then perform linear transformation based on Q, K, V to determine information such as attention features. The input information and weights required for the operator instructions for calculating Q, K, V require external input, which may be the operator instructions of the communication bottleneck; while the data required for the operator instructions for linear transformation based on Q, K, V are all generated by the processing core calculation and do not require external input. In this case, the operator instructions for linear transformation based on Q, K, V are the operator instructions of the computing bottleneck, and there is no routing instruction or the communication duration of the routing instruction is zero.

[0080] In this way, the efficiency of selecting the second operator instruction can be improved, and the efficiency of subsequent instruction splitting and the efficiency of interleaved execution in the processing core can be improved.

[0081] In some possible implementations, if there is no target operator instruction in the operator execution sequence whose communication duration is zero and has no data dependency with the first operator instruction, an operator instruction (hereinafter referred to as the fourth operator instruction) whose communication duration is less than the calculation duration and has no data dependency with the first operator instruction can be selected from the operator execution sequence. If there is only one fourth operator instruction, the fourth operator instruction can be directly used as the second operator instruction; if there are multiple fourth operator instructions, further selection is required.

[0082] In some possible implementations, when there are multiple fourth operator instructions, one of the multiple fourth operator instructions may be selected as the second operator instruction, or the fourth operator instruction with the smallest interval with the first operator instruction in the operator execution sequence may be selected as the second operator instruction; the fourth operator instruction with the shortest communication time may also be used as the second operator instruction for interleaved execution; the fourth operator instruction that best matches the first operator instruction may also be used as the second operator instruction for interleaved execution. The present disclosure does not limit the specific selection method in this case.

[0083] Among them, the best match with the first operator instruction can indicate the best bottleneck complementarity, for example, the sum of the first communication duration and the second communication duration is equal to or close to the sum of the first calculation duration and the second calculation duration.

[0084] In some possible implementations, if the first communication duration is less than the first computing duration, the first operator instruction is an operator instruction of the computing bottleneck, and it is necessary to select the operator instruction of the communication bottleneck as an operator instruction that complements the bottleneck and execute it interleaved with the first operator instruction to improve resource utilization.

[0085] That is, select an operator instruction whose communication time is longer than the calculation time in the operator execution sequence from the operator execution sequence. And, for the selected operator instruction, if the operator instruction has a data dependency relationship with the first operator instruction, it is necessary to execute the operator instruction after the first operator instruction is executed, or the first operator instruction is executed after the operator instruction is executed. The operator instruction cannot be interleaved with the first operator instruction and needs to be excluded. In this way, the second operator instruction can be determined from the operator instructions in the operator execution sequence whose communication time is longer than the calculation time and have no data dependency relationship with the first operator instruction.

[0086] In some possible implementations, if the communication duration is greater than the calculation duration, and there is only one operator instruction (hereinafter referred to as the fifth operator instruction) that has no data dependency relationship with the first operator instruction, the fifth operator instruction can be directly used as the second operator instruction; if there are multiple fifth operator instructions, further selection is required. Among them, one can be arbitrarily selected from multiple fifth operator instructions as the second operator instruction, or the fifth operator instruction with the smallest interval with the first operator instruction in the operator execution sequence can be selected as the second operator instruction. The present disclosure does not limit the specific selection method in this case.

[0087] In this way, the operator instructions of the communication bottleneck can be filtered out for the operator instructions of the computing bottleneck, or the operator instructions of the computing bottleneck can be filtered out for the operator instructions of the communication bottleneck, so as to be interleaved in subsequent processing, thereby improving the utilization rate of core computing resources and communication resources.

[0088] In some possible implementations, after determining the second operator instruction, in step S23, the first operator instruction and the second operator instruction can be divided according to the first communication duration, the first calculation duration, the second communication duration and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction.

[0089] In some possible implementations, if the first operator instruction is an operator instruction of a communication bottleneck, that is, the first communication duration is greater than the first calculation duration, and the second communication duration is less than the second calculation duration, the calculation instructions of the first operator instruction and the second operator instruction may be split. In this case, step S23 may include:

[0090] The calculation instructions of the first operator instruction are divided into multiple first calculation execution segments, and the calculation instructions of the second operator instruction are divided into multiple second calculation execution segments; the first execution segment includes a first calculation execution segment, and the second execution segment includes a second calculation execution segment.

[0091] That is to say, for the operator instructions of the communication bottleneck, the calculation instructions of the first operator instruction and the second operator instruction can be divided respectively based on the calculation time of the first operator instruction and the second operator instruction according to the switching parameters such as the data flow granularity and the instruction context switching speed, so as to obtain the first calculation execution segment and the second calculation execution segment, so that the first execution segment includes the first calculation execution segment, and the second execution segment includes the second calculation execution segment.

[0092] In some possible implementations, the step of dividing the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments includes:

[0093] According to the switching parameter and the first calculation duration of the first operator instruction, the calculation instruction of the first operator instruction is divided to obtain a plurality of first calculation execution segments, wherein the routing instruction of the first operator instruction maintains a data dependency relationship with the plurality of first calculation execution segments;

[0094] Among them, the switching parameters include data flow granularity and instruction context switching speed. The data flow granularity is used to indicate the minimum amount of data transmitted for each communication and / or the minimum amount of data required for each computing operation, and the instruction context switching speed is used to indicate the minimum duration required for each switch during the segmented execution of operator instructions by the processing core.

[0095] For the first operator instruction of the communication bottleneck, the calculation instruction of the first operator instruction can be segmented according to the switching parameter and the first calculation duration of the first operator instruction. Among them, the execution duration corresponding to the first calculation execution segment can be greater than or equal to the instruction context switching speed to ensure that the calculation execution segment can be successfully switched during the interleaved execution process. In addition, the amount of calculation data corresponding to the first calculation execution segment can be greater than or equal to the minimum amount of data required for each calculation operation in the data flow granularity to ensure that the calculation execution segment can perform the calculation operation normally. The present disclosure does not limit the specific segmentation method.

[0096] After segmentation, the routing instructions of the first operator instructions maintain a data dependency relationship with multiple first computing execution segments, that is, the data flow between the routing instructions and multiple first computing execution segments is maintained. Before a first computing execution segment is executed, it is ensured that the computing data corresponding to the first computing execution segment has been transmitted to the target processing core through the routing instructions, thereby ensuring the execution of the first computing execution segment and realizing the synchronization of data transmission and data calculation, thereby improving computing efficiency.

[0097] Among them, the segmentation method of the calculation instruction of the second operator instruction is similar to the segmentation method of the calculation instruction of the first operator instruction, and will not be repeated here.

[0098] Furthermore, in step S24, multiple first computing execution segments of the first operator instruction and multiple second computing execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core interleaves the multiple first computing execution segments of the first operator instruction and the multiple second computing execution segments of the second operator instruction.

[0099] The process of the target processing core interleavingly executing multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: the target processing core interleavingly executing multiple first calculation execution segments of the first operator instruction and multiple second calculation execution segments of the second operator instruction.

[0100] Figure 4 A schematic diagram of interleaved execution of operator instructions provided in an embodiment of the present disclosure. Figure 4 The first operator instruction includes a routing instruction R1 and a calculation instruction C1, and the communication duration of the routing instruction R1 is greater than the calculation duration of the calculation instruction C1; accordingly, the second operator instruction may include a routing instruction R2 (not shown) and a calculation instruction C2, or may only include the calculation instruction C2 (that is, the communication duration of the routing instruction R2 is zero).

[0101] Reference Figure 4 , the computing instructions C1 and C2 can be divided respectively to obtain multiple first computing execution segments and multiple second computing execution segments. During the execution process, the target processing core interleaves the first computing execution segment and the second computing execution segment. In addition, the execution process maintains the pipeline between the routing instruction R1 and the computing instruction C1. That is, while the routing instruction R1 receives / sends data, the computing instruction C1 simultaneously uses the data to perform calculations or synchronously generates the data.

[0102] In this case, if the communication duration of the routing instruction of the second operator instruction is zero, the target processing core directly executes the routing instruction R1 of the first operator instruction; if the communication duration of the routing instruction of the second operator instruction is not zero, the target processing core may first execute the routing instruction of the second operator instruction, transmit the calculation data required by the second operator instruction, and then execute the routing instruction R1 of the first operator instruction. The present disclosure does not limit the specific processing method.

[0103] In this way, computing instructions under the communication bottleneck can be executed in time staggered, thereby improving the utilization of computing resources.

[0104] In some possible implementations, if the first operator instruction is an operator instruction of a computing bottleneck, that is, the first communication duration is less than the first computing duration, and the second communication duration is greater than the second computing duration, the routing instructions of the first operator instruction and the second operator instruction may be segmented. In this case, step S23 may include:

[0105] The routing instructions of the first operator instructions are divided to obtain multiple first routing execution segments, and the routing instructions of the second operator instructions are divided to obtain multiple second routing execution segments; the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment.

[0106] That is to say, for operator instructions that are computing bottlenecks, the routing instructions of the first operator instruction and the second operator instruction can be divided respectively based on the communication duration between the first operator instruction and the second operator instruction according to parameters such as data flow granularity and instruction context switching speed, so as to obtain the first routing execution segment and the second routing execution segment, so that the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment.

[0107] Furthermore, in step S24, multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction are mapped to the target processing core so that the target processing core interleaves the multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction.

[0108] The process of the target processing core interleavingly executing multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: the target processing core interleavingly executing multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction.

[0109] Figure 5 A schematic diagram of interleaved execution of operator instructions provided in an embodiment of the present disclosure. Figure 5 The first operator instruction includes a routing instruction R1 and a calculation instruction C1, and the communication duration of the routing instruction R1 is shorter than the calculation duration of the calculation instruction C1; accordingly, the second operator instruction may include a routing instruction R2 and a calculation instruction C2 (not shown).

[0110] Reference Figure 5 , the routing instructions R1 and R2 can be divided respectively to obtain multiple first routing execution segments and multiple second routing execution segments. During the execution process, the target processing core interleaves the first routing execution segment and the second routing execution segment. In addition, the execution process maintains the pipeline between the routing instruction R1 and the calculation instruction C1. That is, while the routing instruction R1 receives / sends data, the calculation instruction C1 simultaneously uses the data to perform calculations or synchronously generates the data.

[0111] In this case, the target processing core may first execute the calculation instructions of the first operator instruction, and then execute the calculation instructions of the second operator instruction. The present disclosure does not limit the specific processing method.

[0112] In this way, the routing instructions under the computing bottleneck can be staggered in time, thereby improving the utilization of communication resources.

[0113] In some possible implementations, for operator instructions of communication bottlenecks and operator instructions of computing bottlenecks, routing instructions and computing instructions may also be split at the same time. In this case, if the first communication duration is greater than the first computing duration and the second communication duration is less than the second computing duration, or if the first communication duration is less than the first computing duration and the second communication duration is greater than the second computing duration, the following processing may be performed.

[0114] Wherein, step S23 comprises:

[0115] The calculation instructions of the first operator instruction are divided into a plurality of first calculation execution segments, and the routing instructions of the first operator instruction are divided into a plurality of first routing execution segments; the first execution segments include a first calculation execution segment and a first routing execution segment;

[0116] The calculation instructions of the second operator instructions are divided to obtain multiple second calculation execution segments, and the routing instructions of the second operator instructions are divided to obtain multiple second routing execution segments; the second execution segments include a second calculation execution segment and a second routing execution segment.

[0117] That is to say, according to parameters such as data flow granularity and instruction context switching speed, based on the communication duration and calculation duration of the first operator instruction and the second operator instruction, the routing instructions of the first operator instruction and the second operator instruction can be divided respectively to obtain the first routing execution segment and the second routing execution segment; and, the calculation instructions of the first operator instruction and the second operator instruction can be divided respectively to obtain the first calculation execution segment and the second calculation execution segment, so that the first execution segment includes the first routing execution segment and the first calculation execution segment, and the second execution segment includes the second routing execution segment and the second calculation execution segment.

[0118] Furthermore, in step S24, multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction are mapped to the target processing core, so that the target processing core interleaves the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction.

[0119] Among them, the process of the target processing core interleaving the execution of multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction includes: the target processing core interleaving the execution of multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction, and interleaving the execution of multiple first calculation execution segments of the first operator instruction and multiple second calculation execution segments of the second operator instruction.

[0120] In some embodiments, the execution process can be expressed as Figure 4 In the calculation instructions C1 and C2, Figure 5 The combination of routing instructions R1 and R2. During execution, the pipeline between routing instruction R1 and calculation instruction C1 is maintained, and the pipeline between routing instruction R2 and calculation instruction C2 is maintained.

[0121] In this way, routing instructions and computing instructions can be executed in a temporally interleaved manner, thereby improving the utilization of communication resources and computing resources at the same time, and further improving the performance of neural network execution.

[0122] In some possible implementations, the processing cores of the many-core system usually include computing units such as MAC (Multiply-Accumulate) arrays, which are used to perform the computation process of the neural network operator. The computing unit can be split in space to perform the computation of the operator instructions in parallel. In this case, for the operator instructions of the communication bottleneck, the utilization of computing resources can also be improved by spatial parallel execution.

[0123] In some possible implementations, the instruction processing method according to the embodiment of the present disclosure further includes:

[0124] When the first communication duration of the first operator instruction is greater than the first calculation duration, determining a third operator instruction to be executed in parallel with the first operator instruction;

[0125] Determining a first execution region and a second execution region of a computing unit in the target processing core according to a computing instruction of the first operator instruction and a computing instruction of the third operator instruction;

[0126] The first operator instruction and the third operator instruction, as well as the first execution region and the second execution region are mapped to the target processing core, so that the target processing core executes the calculation instructions of the first operator instruction through the first execution region, and executes the calculation instructions of the third operator instruction in parallel through the second execution region.

[0127] For example, if the first operator instruction is an operator instruction of a communication bottleneck, that is, the first communication duration is greater than the first calculation duration, a third operator instruction executed in parallel with the first operator instruction can be determined from the operator execution sequence. The third operator instruction is determined in a similar manner to the second operator instruction.

[0128] In some possible implementations, the operator instruction that is the computational bottleneck (i.e., the communication duration is less than the computational duration) in the operator execution sequence and has no data dependency with the first operator instruction can be first determined; if there is only one operator instruction determined, it can be directly used as the third operator instruction; if there are multiple operator instructions determined, the operator instruction with a zero communication duration is preferably selected; if there is no operator instruction with a zero communication duration, then any operator instruction selected from the determined operator instructions or the operator instruction with the smallest interval with the first operator instruction is selected.

[0129] In some possible implementations, a first execution region and a second execution region of a computing unit in a target processing core are determined according to a computing instruction of the first operator instruction and a computing instruction of the third operator instruction. The first execution region and the second execution region may be divided evenly, or may be divided based on the computing amount of the computing instruction of the first operator instruction and the computing instruction of the third operator instruction, and the present disclosure does not limit the specific division method.

[0130] In some possible implementations, the order of operator instructions in the operator execution sequence can be adjusted so that the first operator instruction and the second operator instruction are adjacent, and the execution area information and the parallel execution identifier are added; it is also possible not to adjust the order of operator instructions in the operator execution sequence, and directly add the execution area information and the parallel execution identifier for the first operator instruction and the second operator instruction, and the present disclosure does not impose any restrictions on this.

[0131] In some possible implementations, after determining the first execution area and the second execution area, the first operator instruction and the third operator instruction as well as the first execution area and the second execution area can be mapped to the target processing core so that the target processing core executes the calculation instructions of the first operator instruction through the first execution area, and executes the calculation instructions of the third operator instruction in parallel through the second execution area.

[0132] Figure 6 A schematic diagram of the parallel execution of operator instructions provided in the embodiment of the present disclosure. Figure 6 The first operator instruction includes a routing instruction R1 and a calculation instruction C1, and the communication duration of the routing instruction R1 is greater than the calculation duration of the calculation instruction C1; accordingly, the second operator instruction may include a routing instruction R2 (not shown) and a calculation instruction C2, or may only include the calculation instruction C2 (that is, the communication duration of the routing instruction R2 is zero).

[0133] Reference Figure 6, the computing unit MAC array of the target processing core can be divided into two execution areas, one execution area executes the computing instruction C1, and the other execution area executes the computing instruction C2 in parallel. In addition, the pipeline between the routing instruction R1 and the computing instruction C1 is maintained during the execution process. That is, while the routing instruction R1 receives / sends data, the computing instruction C1 simultaneously uses the data for calculation or synchronously generates the data.

[0134] In this case, if the communication duration of the routing instruction of the second operator instruction is zero, the target processing core directly executes the routing instruction R1 of the first operator instruction; if the communication duration of the routing instruction of the second operator instruction is not zero, the target processing core may first execute the routing instruction of the second operator instruction, transmit the calculation data required by the second operator instruction, and then execute the routing instruction R1 of the first operator instruction. The present disclosure does not limit the specific processing method.

[0135] In this way, computing instructions under communication bottlenecks can be executed in parallel in space, thereby improving the utilization of computing resources.

[0136] According to the instruction processing method of the embodiment of the present disclosure, it is possible to determine the operator instructions of the communication bottleneck and the computing bottleneck in the operator execution sequence of the processing core, and by controlling the staggered execution of multiple operator instructions in time or parallel execution in space, the utilization rate of the computing resources and communication resources of the processing core is improved, thereby improving the performance and efficiency of the neural network execution.

[0137] In the case of temporal interleaved execution, according to the embodiments of the present disclosure, the corresponding operator instructions can be divided into multiple execution segments, and the multiple execution segments of the operator instructions can be mapped to the processing core, so that the processing core interleaves the execution segments of different operator instructions, thereby improving the utilization rate of the computing resources and communication resources of the processing core; in the case of spatial parallel execution, according to the embodiments of the present disclosure, the computing unit of the processing core can be divided into multiple execution areas, and the computing instructions of different operator instructions can be executed in spatial parallel, thereby improving the utilization rate of the computing resources of the processing core.

[0138] The instruction processing method according to the embodiment of the present disclosure can be applied to the application scenario of accelerating the calculation of neural networks through a many-core system. The neural network can be any type of neural network such as an artificial neural network ANN, a spiking neural network SNN, etc. The neural network is used to perform any one of image processing tasks, speech processing tasks, text processing tasks, and video processing tasks. The data processed by the neural network includes any one of images, speech, text, and video, and the present disclosure does not limit this.

[0139] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0140] In addition, the present disclosure also provides an instruction processing device, an electronic device, and a computer-readable storage medium, all of which can be used to implement any instruction processing method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0141] Figure 7 A block diagram of an instruction processing device provided in an embodiment of the present disclosure.

[0142] Reference Figure 7 , an embodiment of the present disclosure provides an instruction processing device, the device comprising:

[0143] An operator sequence determination module 71 is used to determine an operator execution sequence to be executed by a target processing core in a many-core system, wherein the operator execution sequence includes multiple operator instructions, and the operator instructions include routing instructions and calculation instructions. The routing instructions are used to perform data transmission operations of the operator instructions, and the calculation instructions are used to perform calculation operations of the operator instructions. The many-core system includes multiple processing cores, and the target processing core is any one of the multiple processing cores.

[0144] An instruction determination module 72 is used to determine, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence; wherein the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration;

[0145] An instruction segmentation module 73, configured to segment the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction;

[0146] The instruction mapping module 74 is used to map the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction to the target processing core, so that the target processing core interleaves the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction.

[0147] In some possible implementations, the instruction determination module 72 is configured to:

[0148] When the first communication duration is greater than the first calculation duration, determining a second operator instruction according to an operator instruction in the operator execution sequence whose communication duration is less than the calculation duration and has no data dependency relationship with the first operator instruction;

[0149] When the first communication duration is less than the first calculation duration, a second operator instruction is determined according to an operator instruction in the operator execution sequence whose communication duration is greater than the calculation duration and has no data dependency with the first operator instruction.

[0150] In some possible implementations, the first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration.

[0151] The instruction segmentation module 73 is used to: segment the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments, and segment the calculation instruction of the second operator instruction to obtain a plurality of second calculation execution segments; the first execution segment includes a first calculation execution segment, and the second execution segment includes a second calculation execution segment;

[0152] The instruction mapping module 74 is used to cause the target processing core to interleavely execute multiple first computing execution segments of the first operator instruction and multiple second computing execution segments of the second operator instruction.

[0153] In some possible implementations, the instruction segmentation module 73 is used to:

[0154] According to the switching parameter and the first computing duration of the first operator instruction, the computing instruction of the first operator instruction is segmented to obtain a plurality of first computing execution segments, and a data dependency relationship is maintained between the routing instruction of the first operator instruction and the plurality of first computing execution segments; wherein the switching parameter includes a data flow granularity and an instruction context switching speed, the data flow granularity is used to indicate the minimum amount of data transmitted for each communication and / or the minimum amount of data required for each computing operation, and the instruction context switching speed is used to indicate the minimum duration required for each switching during the segmented execution of the operator instruction by the processing core.

[0155] In some possible implementations, the first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration.

[0156] The instruction segmentation module 73 is used to: segment the routing instruction of the first operator instruction to obtain a plurality of first routing execution segments, and segment the routing instruction of the second operator instruction to obtain a plurality of second routing execution segments; the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment;

[0157] The instruction mapping module 74 is used to: cause the target processing core to interleavely execute multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction.

[0158] In some possible implementations, the instruction determination module 72 is used to: when the first communication duration is greater than the first calculation duration, determine whether there is a target operator instruction in the operator execution sequence whose communication duration is zero and has no data dependency with the first operator instruction; when there is a target operator instruction in the operator execution sequence, determine the target operator instruction as the second operator instruction.

[0159] In some possible implementations, the first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration, or the first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration.

[0160] The instruction segmentation module 73 is used to: segment the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments, and segment the routing instruction of the first operator instruction to obtain a plurality of first routing execution segments; the first execution segment includes a first calculation execution segment and a first routing execution segment; segment the calculation instruction of the second operator instruction to obtain a plurality of second calculation execution segments, and segment the routing instruction of the second operator instruction to obtain a plurality of second routing execution segments; the second execution segment includes a second calculation execution segment and a second routing execution segment;

[0161] Among them, the instruction mapping module 74 is used to: the target processing core interleaves the execution of multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction, and interleaves the execution of multiple first calculation execution segments of the first operator instruction and multiple second calculation execution segments of the second operator instruction.

[0162] In some possible implementations, the device further includes:

[0163] a parallel instruction determination module, configured to determine a third operator instruction to be executed in parallel with the first operator instruction when the first communication duration of the first operator instruction is greater than the first calculation duration;

[0164] a region determination module, configured to determine a first execution region and a second execution region of a computing unit in the target processing core according to a calculation instruction of the first operator instruction and a calculation instruction of the third operator instruction;

[0165] A region mapping module is used to map the first operator instruction and the third operator instruction, as well as the first execution region and the second execution region to the target processing core, so that the target processing core executes the calculation instructions of the first operator instruction through the first execution region, and executes the calculation instructions of the third operator instruction in parallel through the second execution region.

[0166] In some possible implementations, the operator instructions include operation instructions of a neural network operator, and the neural network operator includes coarse-grained operations, and the coarse-grained operations are used to represent operations in which a communication time greater than a first threshold and / or a calculation time greater than a second threshold is required to complete a processing.

[0167] Figure 8 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0168] Reference Figure 8 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 501; at least one memory 502, and one or more I / O interfaces 503, connected between the processor 501 and the memory 502; wherein the memory 502 stores one or more computer programs that can be executed by the at least one processor 501, and the one or more computer programs are executed by the at least one processor 501 so that the at least one processor 501 can execute the above-mentioned instruction processing method.

[0169] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above instruction processing method when executed by a processor. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0170] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned instruction processing method.

[0171] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0172] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0173] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0174] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0175] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0176] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0177] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0178] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0179] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0180] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A command processing method, characterized in that: include: Determining an operator execution sequence to be executed by a target processing core in a many-core system, wherein the operator execution sequence includes a plurality of operator instructions, wherein the operator instructions include routing instructions and calculation instructions, wherein the routing instructions are used to execute a data transmission operation of the operator instructions, and the calculation instructions are used to execute a calculation operation of the operator instructions; the many-core system includes a plurality of processing cores, and the target processing core is any one of the plurality of processing cores; Determine, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to the first communication duration of the routing instruction and the first calculation duration of the calculation instruction in the first operator instruction of the operator execution sequence; wherein the duration relationship between the second communication duration of the routing instruction and the second calculation duration of the calculation instruction in the second operator instruction is different from the duration relationship between the first communication duration and the first calculation duration; According to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, the first operator instruction and the second operator instruction are divided to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction; The plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction are mapped to the target processing core so that the target processing core interleaves the plurality of first execution segments of the first operator instruction and the plurality of second execution segments of the second operator instruction.

2. The method according to claim 1, characterized in that: The determining, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to a first communication duration of a routing instruction and a first calculation duration of a calculation instruction in a first operator instruction of the operator execution sequence, comprises: When the first communication duration is greater than the first calculation duration, determining a second operator instruction according to an operator instruction in the operator execution sequence whose communication duration is less than the calculation duration and has no data dependency relationship with the first operator instruction; When the first communication duration is less than the first calculation duration, a second operator instruction is determined according to an operator instruction in the operator execution sequence whose communication duration is greater than the calculation duration and has no data dependency with the first operator instruction.

3. The method according to claim 1, characterized in that The first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration, The first operator instruction and the second operator instruction are segmented according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The calculation instructions of the first operator instruction are divided into a plurality of first calculation execution segments, and the calculation instructions of the second operator instruction are divided into a plurality of second calculation execution segments; the first execution segment includes a first calculation execution segment, and the second execution segment includes a second calculation execution segment; The target processing core interleavingly executes a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction, including: The target processing core interleaves and executes a plurality of first computing execution stages of the first operator instruction and a plurality of second computing execution stages of the second operator instruction.

4. The method according to claim 3, characterized in that: The step of dividing the calculation instruction of the first operator instruction to obtain a plurality of first calculation execution segments includes: According to the switching parameter and the first calculation duration of the first operator instruction, the calculation instruction of the first operator instruction is divided to obtain a plurality of first calculation execution segments, wherein the routing instruction of the first operator instruction maintains a data dependency relationship with the plurality of first calculation execution segments; Among them, the switching parameters include data flow granularity and instruction context switching speed. The data flow granularity is used to indicate the minimum amount of data transmitted for each communication and / or the minimum amount of data required for each computing operation, and the instruction context switching speed is used to indicate the minimum duration required for each switch during the segmented execution of operator instructions by the processing core.

5. The method according to claim 1, characterized in that: The first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration, The first operator instruction and the second operator instruction are segmented according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The routing instructions of the first operator instruction are divided to obtain a plurality of first routing execution segments, and the routing instructions of the second operator instruction are divided to obtain a plurality of second routing execution segments; the first execution segment includes the first routing execution segment, and the second execution segment includes the second routing execution segment; The target processing core interleavingly executes a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction, including: The target processing core interleaves execution of a plurality of first routing execution stages of the first operator instruction and a plurality of second routing execution stages of the second operator instruction.

6. The method according to claim 2, characterized in that The determining, when the first communication duration is greater than the first calculation duration, of a second operator instruction according to an operator instruction in the operator execution sequence whose communication duration is less than the calculation duration and has no data dependency relationship with the first operator instruction, includes: When the first communication duration is greater than the first calculation duration, determining whether there is a target operator instruction in the operator execution sequence whose communication duration is zero and has no data dependency relationship with the first operator instruction; In a case where a target operator instruction exists in the operator execution sequence, the target operator instruction is determined as a second operator instruction.

7. The method according to claim 1, characterized in that The first communication duration is greater than the first calculation duration and the second communication duration is less than the second calculation duration, or the first communication duration is less than the first calculation duration and the second communication duration is greater than the second calculation duration, The first operator instruction and the second operator instruction are segmented according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration to obtain multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction, including: The calculation instructions of the first operator instruction are divided into a plurality of first calculation execution segments, and the routing instructions of the first operator instruction are divided into a plurality of first routing execution segments; the first execution segments include a first calculation execution segment and a first routing execution segment; The calculation instruction of the second operator instruction is divided into a plurality of second calculation execution segments, and the routing instruction of the second operator instruction is divided into a plurality of second routing execution segments; the second execution segment includes a second calculation execution segment and a second routing execution segment; The target processing core interleavingly executes a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction, including: The target processing core interleaves the execution of multiple first routing execution segments of the first operator instruction and multiple second routing execution segments of the second operator instruction, and interleaves the execution of multiple first calculation execution segments of the first operator instruction and multiple second calculation execution segments of the second operator instruction.

8. The method according to claim 1, characterized in that: The method further comprises: When the first communication duration of the first operator instruction is greater than the first calculation duration, determining a third operator instruction to be executed in parallel with the first operator instruction; Determining a first execution region and a second execution region of a computing unit in the target processing core according to a computing instruction of the first operator instruction and a computing instruction of the third operator instruction; The first operator instruction and the third operator instruction, as well as the first execution region and the second execution region are mapped to the target processing core, so that the target processing core executes the calculation instructions of the first operator instruction through the first execution region, and executes the calculation instructions of the third operator instruction in parallel through the second execution region.

9. The method according to claim 1, characterized in that: The operator instruction includes an operation instruction of a neural network operator, and the neural network operator includes a coarse-grained operation, and the coarse-grained operation is used to represent an operation in which a communication time required to complete a processing is greater than a first threshold and / or a calculation time is greater than a second threshold.

10. An instruction processing device, characterized in that: include: an operator sequence determination module, configured to determine an operator execution sequence to be executed by a target processing core in a many-core system, wherein the operator execution sequence includes a plurality of operator instructions, wherein the operator instructions include routing instructions and calculation instructions, wherein the routing instructions are used to execute a data transmission operation of the operator instructions, and the calculation instructions are used to execute a calculation operation of the operator instructions; the many-core system includes a plurality of processing cores, and the target processing core is any one of the plurality of processing cores; An instruction determination module, configured to determine, from the operator execution sequence, a second operator instruction to be executed interleaved with the first operator instruction according to a first communication duration of a routing instruction and a first calculation duration of a calculation instruction in a first operator instruction of the operator execution sequence; wherein a duration relationship between a second communication duration of the routing instruction and a second calculation duration of the calculation instruction in the second operator instruction is different from a duration relationship between the first communication duration and the first calculation duration; an instruction segmentation module, configured to segment the first operator instruction and the second operator instruction according to the first communication duration, the first calculation duration, the second communication duration, and the second calculation duration, to obtain a plurality of first execution segments of the first operator instruction and a plurality of second execution segments of the second operator instruction; An instruction mapping module is used to map multiple first execution segments of the first operator instruction and multiple second execution segments of the second operator instruction to the target processing core, so that the target processing core interleaves the multiple first execution segments of the first operator instruction and the multiple second execution segments of the second operator instruction.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs executable by the at least one processor, and the one or more computer programs are executed by the at least one processor to enable the at least one processor to execute the instruction processing method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the computer program implements the instruction processing method according to any one of claims 1 to 9.

Images