Compiling method, chip, electronic device and computer readable storage medium for executing program

Abstract

A compiling method of an execution program, a chip, an electronic device and a computer readable storage medium are disclosed. The compiling method of the execution program comprises: determining the number of processing cores required for executing an original program according to the attribute of the data of the original program (S201); determining the position of a synchronization point in the original program according to the parameter of the original program (S202); and compiling the original program into an execution program according to the number of processing cores and the position of the synchronization point (S203). The above method generates the execution program according to the attribute of the input and output data of the original program and the parameter, and solves the technical problem that the execution program in the prior art needs to frequently access an external memory when executing.

Description

Technical Field

[0001] The present disclosure relates to the field of program compilation and processors, and in particular to a compilation method, chip, electronic device, and computer-readable storage medium for executing a program. Background Art

[0002] With the advancement of science and technology, human society is rapidly entering the intelligent era. A key characteristic of this era is the increasing variety and volume of data available, coupled with the increasing demand for faster data processing. Chips are the cornerstone of task scheduling and fundamentally determine our ability to process data. From an application perspective, there are two main approaches to chip development: general-purpose chips, such as CPUs (Central Processing Units), offer significant flexibility but exhibit relatively low effective computing power when processing domain-specific algorithms. Specialized chips, such as Tensor Processing Units (TPUs), offer high effective computing power in specific domains, but their processing capabilities are limited or even incapable of handling the more flexible and diverse needs of general-purpose applications. Given the diverse and massive amounts of data in the intelligent era, chips must possess both extreme flexibility to handle diverse and rapidly evolving algorithms and robust processing capabilities to rapidly process vast and rapidly growing volumes of data.

[0003] When multi-core CPUs or GPUs are processing neural network tasks, there are generally two approaches:

[0004] The first is that each processing core processes its own task independently, and the cores do not affect each other. Figure 1a The second is that some or all of the processing cores work together in parallel to process a task, each completing a part of the task, as shown in the attached figure. Figure 1b shown.

[0005] In the two methods described above, the compiler compiles a suitable program to perform neural network calculations based on the structure of the neural network and the characteristics of traditional multi-core CPUs or GPUs. The cache in each processing core is transparent to the program and cannot be directly and independently accessed. For the processing core, all data reading and writing during the calculation are based on the access address of the DDR (Double Data Rate) memory. In neural network calculations, a large amount of intermediate data is generated, and most of the intermediate data in each layer is not related. Due to the spatial and temporal locality of the cache, it can easily reduce the hit rate in neural network calculations, resulting in frequent access to the DDR memory, thereby reducing the calculation speed of the neural network and reducing power consumption. Summary of the Invention

[0006] This summary is provided to briefly introduce concepts that will be described in detail in the detailed description below. This summary is not intended to identify key features or essential features of the claimed technical solution, nor is it intended to limit the scope of the claimed technical solution.

[0007] In order to solve the technical problems of inflexible task scheduling and complex control of processing cores in the prior art, the embodiments of the present disclosure propose the following technical solutions:

[0008] In a first aspect, an embodiment of the present disclosure provides a method for compiling an execution program, comprising:

[0009] Determining the number of processing cores required to execute the original program according to attributes of data of the original program;

[0010] determining a position of a synchronization point in the original program according to parameters of the original program;

[0011] The original program is compiled into an execution program according to the number of the processing cores and the positions of the synchronization points.

[0012] Furthermore, determining the number of processing cores required to execute the original program according to the attributes of the data of the original program includes:

[0013] Obtaining the total amount of data of each subroutine in the original program; wherein the total amount of data of the subroutine includes the sum of the size of the input data and the size of the output data of the subroutine;

[0014] The number of processing cores required to execute the original program is determined based on the total amount of data for each subroutine and the size of the data storage area of ​​the processing core.

[0015] Furthermore, the calculating the number of processing cores required to execute the original program according to the total amount of data of each subroutine and the size of the data storage area of ​​the processing core includes:

[0016] calculating a quotient of a total amount of data of each subroutine and a size of a data storage area of ​​the processing core;

[0017] The value obtained by rounding up the maximum quotient among the multiple quotients is used as the number of processing cores required to execute the original program.

[0018] Furthermore, determining the position of the synchronization point in the original program according to the parameters of the original program includes:

[0019] The positions of the synchronization points of the plurality of subroutines are determined according to the size of the parameter storage area of ​​the processing core and the sizes of the parameters of the plurality of subroutines.

[0020] Furthermore, determining the position of the synchronization point of the subroutine according to the size of the parameter storage area of ​​the processing core and the size of the parameter of the subroutine includes:

[0021] determining the number of parameters of the subroutine that can be stored in the parameter storage area of ​​the processing core according to the size of the parameter storage area of ​​the processing core and the size of the parameters of the subroutine;

[0022] The position of the synchronization point of the subroutine is determined according to the number of parameters of the subroutine that can be stored in the parameter storage area of ​​the processing core.

[0023] Furthermore, after determining the position of the synchronization point, the method further includes:

[0024] A synchronization instruction is added at the synchronization point; wherein the synchronization instruction is used to enable the system including the multiple processing cores to generate a synchronization signal.

[0025] Furthermore, the execution program includes a plurality of program segments, and each program segment includes instructions in the original program and control instructions required for the processing core to execute the instructions in the original program.

[0026] Furthermore, the original program is a neural network, and the subroutine is a layer of sub-network in the neural network.

[0027] Furthermore, obtaining the total amount of data of each subroutine in the original program includes:

[0028] obtaining the neural network;

[0029] Analyze the size of input data and output data of each layer of sub-network in the neural network;

[0030] Generate the total amount of data for each layer of the neural network.

[0031] In a second aspect, an embodiment of the present disclosure provides a chip, including:

[0032] Multiple processing cores and synchronization signal generators; wherein each processing core includes a data storage area and a parameter storage area;

[0033] The plurality of processing cores are used to be grouped according to an execution program, wherein the processing cores in each group are used to execute a plurality of program segments in the execution program; the data storage area is used to store input data and output data of the plurality of program segments, and the parameter storage area is used to store parameters of the plurality of program segments;

[0034] The synchronization signal generator is used to send a synchronization signal to all processing cores when all processing cores executing the program segment have completed execution.

[0035] In a third aspect, an embodiment of the present disclosure provides an electronic device, comprising: a memory for storing computer-readable instructions; and one or more processors for executing the computer-readable instructions, so that when the processor is running, it implements any of the compilation methods for executing the program in the first aspect.

[0036] In a fourth aspect, an embodiment of the present disclosure provides a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions, which are used to enable a computer to execute any of the compilation methods of the execution program described in the first aspect.

[0037] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, characterized in that it includes computer instructions. When the computer instructions are executed by a computing device, the computing device can execute any of the compilation methods of the execution program in the first aspect.

[0038] In a sixth aspect, an embodiment of the present disclosure provides a computing device, characterized in that it includes any chip described in the second aspect.

[0039] The disclosed embodiments disclose a method for compiling an execution program, a chip, an electronic device, and a computer-readable storage medium. The method for compiling an execution program includes: determining the number of processing cores required to execute the original program based on the attributes of the original program's data; determining the location of synchronization points in the original program based on the parameters of the original program; and compiling the original program into an execution program based on the number of processing cores and the location of the synchronization points. The above method compiles and generates the execution program based on the attributes and parameters of the original program's input and output data, resolving the technical problem in the prior art of requiring frequent access to external memory during execution of the execution program.

[0040] The above description is only an overview of the technical solution of the present disclosure. In order to more clearly understand the technical means of the present disclosure, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present disclosure more obvious and easy to understand, the following specifically cites preferred embodiments and describes them in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The above and other features, advantages, and aspects of the various embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that the originals and elements are not necessarily drawn to scale.

[0042] Figure 1a and 1b It is a schematic diagram of the prior art;

[0043] Figure 2 A flowchart of a method for compiling an execution program provided in an embodiment of the present disclosure;

[0044] Figure 3 A further flowchart of the method for compiling an execution program provided in an embodiment of the present disclosure;

[0045] Figure 4 A further flowchart of the method for compiling an execution program provided in an embodiment of the present disclosure;

[0046] Figure 5 A further flowchart of the method for compiling an execution program provided in an embodiment of the present disclosure;

[0047] Figure 6a An example diagram of a chip provided in accordance with an embodiment of the present disclosure;

[0048] Figure 6b A schematic diagram of grouping processing cores when the chip provided by an embodiment of the present disclosure executes the execution program;

[0049] Figure 7 is a schematic diagram of a neural network to be compiled in an embodiment of the present disclosure;

[0050] Figure 8 This is a schematic diagram of data partitioning between two processing cores in the first group in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0051] The following describes embodiments of the present disclosure in more detail with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments described herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are for illustrative purposes only and are not intended to limit the scope of protection of the present disclosure.

[0052] It should be understood that the various steps described in the method embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this respect.

[0053] As used herein, the term "including" and its variations are open-ended, i.e., "including but not limited to." The term "based on" means "based, at least in part, on." The term "one embodiment" means "at least one embodiment," the term "another embodiment" means "at least one additional embodiment," and the term "some embodiments" means "at least some embodiments." Other terms are defined in the following description.

[0054] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0055] It should be noted that the modifications of "one" and "multiple" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, they should be understood as "one or more".

[0056] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0057] Figure 2 A flowchart of a method for compiling an execution program provided in an embodiment of the present disclosure. The method for compiling an execution program is used in a system including multiple processing cores, wherein the processing core includes a storage area for storing relevant data of the execution program. The relevant data of the execution program includes input data, output data, program instruction data, and parameter data of the execution program, and accordingly, the storage area includes a data storage area for storing the input data and output data of the execution program, a program storage area for storing the program instruction data, and a parameter storage area for storing the parameter data.

[0058] The method comprises:

[0059] Step S201, determining the number of processing cores required to execute the original program according to the attributes of the data of the original program;

[0060] In this step, the attributes of the original program data include the total amount of input and output data of the original program. In this step, the original program data is analyzed, and a processing core allocation plan is planned based on the original program data attributes to determine how many processing cores are required to parallelize the original program.

[0061] Optionally, step S201 includes:

[0062] Step S301, obtaining the total amount of data of each subroutine in the original program; wherein the total amount of data of the subroutine includes the sum of the size of the input data and the size of the output data of the subroutine;

[0063] Step S302: Determine the number of processing cores required to execute the original program based on the total amount of data of each subroutine and the size of the data storage area of ​​the processing core.

[0064] Exemplarily, the original program is a neural network, and the subroutine is a sub-network layer within the neural network. In this optional embodiment, the original program includes multiple subroutines, and the original program is executed sequentially according to the order of the subroutines. The output data of a subroutine is used as the input data of the next subroutine or the output data of the original program.

[0065] When the original program is a neural network, step S301 includes:

[0066] Obtain the neural network; analyze the size of input data and the size of output data of each layer in the neural network; and generate the total amount of data for each layer in the neural network. A neural network can typically be represented in the form of a graph, with each layer including the size of input data and the size of output data, such as the dimensions of the input and output data. Thus, by analyzing the graph representing the neural network, the total amount of data for each layer of the subnetwork can be obtained.

[0067] It can be understood that the original program can be other types of programs, and the subroutine of the original program can be a program module in the original program divided according to functional modules or a program module that obtains output data from output data according to the program execution order.

[0068] Since there may be intermediate data when the original program is executed, such as the output data of the above-mentioned subroutine, in order to avoid accessing the storage area outside the processing core for each intermediate data, the total amount of data of each subroutine of the original program is first calculated in this step.

[0069] After obtaining the total amount of data of each subroutine of the original program, in step S302, the size of the data storage area of ​​the processing core is obtained, and the number of processing cores required to execute the original program is calculated based on the total amount of data of the subroutine and the size of the data storage area.

[0070] Since the total amount of data for each subroutine may be different, in order to be able to execute the complete original program, the maximum number of processing cores required to execute each subroutine can be selected as the number of processing cores required to execute the original program. Therefore, optionally, step S302 includes:

[0071] Step S401, calculating the quotient of the total amount of data of each subroutine and the size of the data storage area of ​​the processing core;

[0072] Step S402: rounding up the maximum quotient among the multiple quotients as the number of processing cores required to execute the original program.

[0073] In step S401, assuming that the size of the data storage area of ​​each processing core is the same, multiple quotients are obtained by dividing the total amount of data of each subroutine by the size of the data storage area, wherein the quotient may be an integer or not. Therefore, in step S402, the rounded-up value of the maximum quotient among the multiple quotients is taken as the number of processing cores required to execute the original program. When the size of the data storage area of ​​each processing core is different, the minimum value of the data storage area in the processing core can be used as the size of the data storage area used when calculating the quotient to ensure that each processing core can store the input data and output data of each subroutine without using an external memory. It can be understood that in the above steps S401 and S402, the minimum number of processing cores required to execute each subroutine is calculated, and then the maximum value of these numbers of processing cores is taken as the minimum number of processing cores required to execute the original program.

[0074] In step S201, the number of processing cores required to execute the original program is obtained, and the multiple processing cores are grouped according to the number of processing cores required by the original program, and the number of processing cores in each group is the required number of processing cores; multiple original programs can be executed in parallel between each group.

[0075] Assume that the number of processing cores required to execute the original program is N min After the multiple processing cores are grouped, the number of processing cores in each group is N g , then N needs to be satisfied g ≥N min , the number of processing cores in the system is N, then N=A*N g, where A is a positive integer, i.e. the number of groups. For example, N min =2,N=9, then N g =3, A=3.

[0076] After calculating the number of processing cores required to execute the original program in this step, the amount of input data for each processing core can be further calculated, that is, the input data of each subroutine is evenly distributed to each processing core in a group of processing cores. Specifically, the input data can be evenly divided into N g The input data is divided into parts; however, in some cases, the input data between the processing cores may overlap, in which case it is necessary to calculate the number of processing cores N in each group. g Adjust when N g Satisfies: M0 ≥ D m +D in / N g ; Where M0 is the size of the data storage area of ​​the processing core, D m It is the incremental data generated by the overlapping part of the data when each processing core is added. Therefore, the input data that each processing core needs to allocate is D m +D in / N g .

[0077] In this way, through the above step S201 , the grouping planning of multiple processing cores in the system executing the original program can be completed during the compilation phase of the original program.

[0078] Return to Attachment Figure 2 , the compilation method of the execution program further includes:

[0079] Step S202: determining the position of the synchronization point in the original program according to the parameters of the original program.

[0080] The instructions of the original program between the two synchronization points are program instructions that the processing core needs to execute in one synchronization cycle.

[0081] Optionally, the step S202 includes: determining the positions of the synchronization points of the multiple subroutines according to the size of the parameter storage area of ​​the processing core and the sizes of the parameters of the multiple subroutines.

[0082] The parameter storage area of ​​the processing core is used to store the parameters of the original program, such as the size of the convolution kernel, weight value, step size and other parameters used by each layer of sub-network in the convolutional neural network. The size of the parameter storage area in the processing core determines the amount of program instructions that can be executed in a synchronization cycle. If the parameter storage area in the processing core can store all the parameters of the next layer of sub-network, then in a synchronization cycle, the processing core can complete the calculation of a layer of sub-network without reading parameters from the storage area outside the processing core. The synchronization point can be set at the end of the program instructions of each sub-network. In some cases, the parameter storage area of ​​the processing core is relatively small and cannot store all the parameters of a layer of sub-network. In this case, it is necessary to insert a synchronization point in the sub-network to determine the position that can be executed using the parameters stored in the parameter storage area.

[0083] Furthermore, the above step S202 includes:

[0084] Step S501, determining the number of the subroutine parameters that can be stored in the parameter storage area of ​​the processing core according to the size of the parameter storage area of ​​the processing core and the size of the subroutine parameters;

[0085] Step S502 : determining the position of the synchronization point of the subroutine according to the number of parameters of the subroutine that can be stored in the parameter storage area of ​​the processing core.

[0086] In this embodiment, the number of parameters that can be stored in the parameter storage area is determined by the size of the subroutine's parameters and the size of the parameter storage area. The positions of the synchronization points of the multiple subroutines are determined based on the number of parameters. For example, if the size of the subroutine's parameters is 50 KB and the size of the parameter storage area is 25 KB, a synchronization point needs to be inserted at the midpoint of the subroutine. The above synchronization point position determination is performed for each subroutine to obtain the positions of all synchronization points in the original program.

[0087] Furthermore, after obtaining the position of the synchronization point, step S202 further includes: adding a synchronization instruction at the position of the synchronization point; wherein the synchronization instruction is used to enable the system including multiple processing cores to generate a synchronization signal. That is, after the processing core executes the program instructions between the synchronization points, it continues to execute the synchronization instruction. Optionally, after the processing core executes the synchronization instruction, it generates a synchronization request signal to request the system including multiple processing cores to generate a synchronization signal, and the system includes a synchronization signal generator. After receiving the synchronization request signal issued by each processing core participating in the program execution in the system, the synchronization signal generator generates a synchronization signal to enable the multiple processing cores to enter the next different cycle to execute subsequent program instructions.

[0088] Return to Attachment Figure 2 , the compilation method of the execution program further includes:

[0089] Step S203 : compile the original program into an execution program according to the number of the processing cores and the positions of the synchronization points.

[0090] The execution program includes multiple program segments, each of which is generated with the synchronization point as the demarcation point. Each program segment includes instructions from the original program and control instructions required for the processing core to execute the instructions in the original program. As described above, synchronization points are inserted into subroutines, and multiple program segments are demarcated with the synchronization points as the demarcation points. Therefore, the instructions contained in a program segment may be part or all of the instructions in a subroutine, or instructions from multiple subroutines.

[0091] The control instructions are used by the processing core to read the parameters and / or the next program segment required for the next synchronization cycle during each synchronization cycle. The program segment also includes the synchronization instructions at the end, which are used to generate a synchronization request signal. The number of processing cores is used to generate allocation information and / or grouping information in the execution program, which is used to group the multiple processing cores and allocate program segments, parameters, input data, etc. when the system including multiple processing cores executes the execution program.

[0092] Through the above-mentioned execution program compilation method, the original program is compiled into an execution program suitable for execution by a multi-processing core system, wherein the basis for compiling and generating the execution program is the attributes and parameters of the input and output data of the original program, thereby enhancing the compatibility of the execution program with the multi-processing core system and improving the effective computing power of the multi-processing core system; in addition, the intermediate data generated by the original program are all moved within the multi-processing core system and do not need to be exchanged with the external memory, thereby reducing latency, reducing the pressure on the bandwidth of the external memory, and reducing the power consumption of the entire multi-processing core system.

[0093] Figure 6a This is an example of a structural diagram of a system including multiple processing cores provided by an embodiment of the present disclosure. Figure 6a As shown, in this example, the system including multiple processing cores is a chip, and the chip 600 includes:

[0094] Multiple processing cores 601 and synchronization signal generators 602; each processing core includes a data storage area and a parameter storage area;

[0095] The multiple processing cores 601 are used to be grouped according to the execution program, wherein the processing cores in each group are used to execute multiple program segments in the execution program; the data storage area is used to store input data and output data of the multiple program segments, and the parameter storage area is used to store parameters of the multiple program segments;

[0096] The synchronization signal generator 602 is used to send a synchronization signal to all processing cores when all processing cores executing the program segment have completed execution.

[0097] Below through Figure 6a The structure of the chip shown in FIG. 1 is used as an example to illustrate the compilation process of the above-mentioned execution program. Figure 6a As shown, the chip 600 includes four processing cores, namely C1, C2, C3, and C4. Each processing core includes a 1MB data storage area. Each processing core also includes a parameter storage area and a program storage area (not shown). An external memory DDR is connected to the chip for storing the input data, parameters, and final output data of the original program.

[0098] The original program takes a 2-layer neural network as an example. Its structure and the total amount of input and output data of each layer are as follows: Figure 7 As shown in the figure, the input data of the first layer L1 of the neural network is 400KB, and the output data is 800KB; the input data size of the second layer L2 of the neural network is the same as the output data size of the first layer, which is 800KB, and the output data is 10KB.

[0099] The above-mentioned execution program compilation method is executed by a neural network compiler. According to the embodiment of the above-mentioned execution program compilation method, the neural network compiler first performs step S301 to analyze the neural network, obtain the sum of the input data and output data size of each layer, and generate the total amount of data for each layer in Table 1:

[0100] Layer InData(KB) OutData(KB) Total Data(KB) L1 400 800 1200 L2 800 10 810

[0101] Table 1

[0102] After that, the steps in step S302 are executed to calculate the number of processing cores required to execute the original program based on the total amount of data in each layer and the size of the data storage area of ​​the processing core. If a decimal is generated during the calculation, it is rounded up. The calculation results are shown in Table 2 below:

[0103] Layer Calculate Core Number Core Number L1 1200 / 1000 = 1.2 2 L2 810 / 1000 = 0.8 1

[0104] Table 2

[0105] The calculation result indicates that 2 processing cores are required to execute the subnetwork program of the first layer and 1 processing core is required to execute the subnetwork program of the second layer. Since each layer of the subnetwork in the neural network needs to be executed in sequence, and the subnetworks cannot be executed in parallel, at least 2 processing cores are required to execute the neural network. That is, the number N of processing cores required to execute the neural network min = 2.

[0106] Then, the number Ng of processing cores in each group in the processing core group of the chip is determined, and Ng needs to meet the following two conditions to maximize the utilization rate of the chip computing power:

[0107] a. N g >= N min ;

[0108] b. N = A * N g , A is a positive integer.

[0109] In the above example, the chip includes 4 processing cores, N = 4, Nmin= 2, and A = 2 can be obtained: N g = N min = 2, that is, the 4 processing cores in the chip are divided into two groups, each group including 2 cores, and the cores in each group calculate the same task in parallel, and the 2 groups can process 2 tasks in parallel. As shown in Figure 6b , the grouping of the processing cores in the chip shown in Figure 6a , wherein C1 and C2 are the first group Group1, and C3 and C4 are the second group Group2.

[0110] For each group, according to the number N g of processing cores in the group, the input and output data of each layer of the neural network are split so that they can be calculated in parallel on two processing cores, and the calculation amount of the two processing cores is balanced. The parameters used by the two processing cores in the same synchronization period are the same.

[0111] Figure 8 A schematic diagram of splitting data for the two processing cores in the first group. As shown in Figure 8As shown in , the input data of L1 is first split. The input data of L1 is 400KB, which is divided into two processing cores for calculation. It is divided into two 200KB sub-input data (not necessarily completely equal. In some cases, the two processing cores will use part of the same input data. At this time, this part of the input data needs to be given to the two processing cores at the same time, so the two parts of the sub-input data will be larger than 200KB). They are respectively assigned to C1 and C2 in Group 1 as input data; when they complete the calculation of the first layer L1 of the neural network, each will generate half of the 800KB output data of L1, 400KB sub-output data; C1 and C2 will automatically The sub-output data of L1 of each of them is used as the sub-input data of their respective L2 (not necessarily the output of each L1 can be used as the input of its L2. It is possible that part of the sub-output data of L1 of C1 needs to be allocated to C2, and used together with the sub-output data of L1 of C2 as the sub-input data of L2. Similarly, it is possible that part of the sub-output data of L1 of C2 also needs to be allocated to C1 as the input of its L2. However, the data exchange between the two is carried out in the storage area within the chip, and there is no need to read and write the DDR storage outside the chip). When the two complete the calculation of L2, each will generate 5KB of sub-output data, which is half of the 10KB output data of L2, and the two sub-output data will be combined into a total of 10KB of output data. In this way, the chip completes the operation of a complete execution program. The situation of Group2 is similar to that of Group1, except that the input data is different. Therefore, Figure 6a The chip in the chip can perform two neural network computing tasks in parallel.

[0112] After obtaining the input data partitioning scheme, the embodiment of step S202 is continued. The location of the synchronization point is determined based on the parameters of each layer and the size of the parameter storage area within the processing core. This allows each processing core to read parameters from the parameter storage area within the processing core to perform neural network calculations during each synchronization cycle. Furthermore, during the synchronization cycle, the parameters to be used in the next synchronization cycle can be read from the off-chip storage area, i.e., the DDR, to the parameter storage area within the processing core. Synchronization instructions are inserted at the location of the synchronization point based on the location of the synchronization point.

[0113] Then continue to execute the embodiment in step S203 to generate the N g The program segments to be executed in the processing core can be generated by calling a traditional compiler. Each program segment is generated by the neural network program between two synchronization points.

[0114] When the chip executes the execution program, the Figure 8The input data is divided into two parts in the manner of FIG. 1 , and then the processing core executes the program segment in the execution program according to the input data and parameters. Each time a synchronization instruction is executed, a synchronization request is generated and sent to the synchronization signal generator. After the synchronization signal generator receives the synchronization request issued by each processing core in the chip, a synchronization signal is generated and sent to each processing core, so that each processing core enters the next synchronization cycle and continues to execute the program segment of the execution program using new parameters until the execution of the execution program is completed and the output result is obtained.

[0115] The compilation method of the above-mentioned execution program disclosed in the present invention can be executed when the number of processing cores is known in advance. In this case, the execution program obtained after compilation can carry allocation information, including grouping information of the processing cores, the division method of input data (including intermediate data used as input data of other layers), etc.; the compilation method of the execution program can also be executed when the number of processing cores is not known in advance. In this case, the execution program obtained after compilation can carry the number of processing cores required to execute the execution program and the grouping strategy of the processing cores. When executing the execution program, the number of processing cores of the current chip is obtained first, and the optimal grouping method for the current chip to execute the execution program is calculated based on the grouping strategy and the number of processing cores required to execute the execution program, and then tasks are assigned to the processing cores of each group according to the grouping method.

[0116] It can be seen from the above examples that the original program is compiled using the execution program compilation method in the embodiment of the present disclosure, and the execution program is executed using the chip in the embodiment of the present disclosure. When executing the execution program, for each processing core, since the parameters used are the same, the processing core has the same amount of calculation in each synchronization cycle, so that in each synchronization cycle, the calculation time of all processing cores is consistent, avoiding the difference in calculation time between different processing cores, and the loss of computing power caused by the processing core that completes the calculation first having to wait for the processing core that completes the calculation later, thereby greatly improving the effective computing power of the chip; in addition, all processing cores use the same parameters, so the parameters only need to be read from the DDR once, and can be shared by all processing cores, which greatly improves the parameter reuse rate, reduces the demand for DDR bandwidth, and reduces power consumption.

[0117] An embodiment of the present disclosure further provides an electronic device, comprising: a memory for storing computer-readable instructions; and one or more processors for executing the computer-readable instructions, so that when the processors are executed, any of the compilation methods for executing the program in the embodiments is implemented.

[0118] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium, characterized in that the non-transitory computer-readable storage medium stores computer instructions, which are used to enable a computer to execute any of the compilation methods of the execution program described in the aforementioned embodiments.

[0119] An embodiment of the present disclosure further provides a computer program product, which is characterized in that it includes computer instructions. When the computer instructions are executed by a computing device, the computing device can execute any of the program compilation methods described in the aforementioned embodiments.

[0120] An embodiment of the present disclosure further provides a computing device, characterized in that it includes any chip described in the embodiments.

[0121] The flowcharts and block diagrams in the accompanying drawings of the present disclosure illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of the boxes in the block diagram and / or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0122] The units involved in the embodiments described in this disclosure may be implemented in software or hardware, wherein the name of a unit does not necessarily limit the unit itself.

[0123] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0124] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include an electrical connection based on one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

Claims

1. A method for compiling an execution program, for use in a system comprising multiple processing cores, characterized in that: include: Determining the number of processing cores required to execute the original program according to attributes of data of the original program; determining a position of a synchronization point in the original program according to parameters of the original program; Compiling the original program into an execution program suitable for execution by the system of the multiple processing cores based on the number of the processing cores and the positions of the synchronization points, wherein the execution program includes multiple program segments, the program segments are generated with the positions of the synchronization points as demarcation points, and each program segment includes instructions in the original program and control instructions required for the processing cores to execute the instructions in the original program; The plurality of processing cores are grouped according to the number of processing cores required by the original program, the number of processing cores in each group being the required number of processing cores, and the plurality of original programs are executed in parallel between the groups; The determining, based on the attributes of the data of the original program, the number of processing cores required to execute the original program includes: Obtaining the total amount of data of each subroutine in the original program; wherein the total amount of data of the subroutine includes the sum of the size of the input data and the size of the output data of the subroutine; The number of processing cores required to execute the original program is determined according to the total amount of data of each subprogram and the size of the data storage area of ​​the processing core.

2. The method for compiling an execution program according to claim 1, wherein: The calculating the number of processing cores required to execute the original program according to the total amount of data of each subroutine and the size of the data storage area of ​​the processing core includes: calculating a quotient of a total amount of data of each subroutine and a size of a data storage area of ​​the processing core; The value obtained by rounding up the maximum quotient among the multiple quotients is used as the number of processing cores required to execute the original program.

3. The method for compiling an execution program according to any one of claims 1 to 2, wherein: The determining the position of the synchronization point in the original program according to the parameters of the original program includes: The positions of the synchronization points of the plurality of subroutines are determined according to the size of the parameter storage area of ​​the processing core and the sizes of the parameters of the plurality of subroutines.

4. The method for compiling an execution program according to claim 3, wherein: The determining the position of the synchronization point of the subroutine according to the size of the parameter storage area of ​​the processing core and the size of the parameter of the subroutine includes: determining the number of the subroutine parameters that can be stored in the parameter storage area of ​​the processing core according to the size of the parameter storage area of ​​the processing core and the size of the subroutine parameters; The position of the synchronization point of the subroutine is determined according to the number of parameters of the subroutine that can be stored in the parameter storage area of ​​the processing core.

5. The method for compiling an execution program according to any one of claims 1 to 2 and 4, wherein: After determining the position of the synchronization point, the method further includes: A synchronization instruction is added at the synchronization point; wherein the synchronization instruction is used to enable the system including the multiple processing cores to generate a synchronization signal.

6. The method for compiling an execution program according to any one of claims 1 to 2 and 4, wherein: The original program is a neural network, and the subprogram is a sub-network in the neural network.

7. The method for compiling an execution program according to claim 1, wherein: The obtaining of the total amount of data of each subroutine in the original program includes: Get the neural network; Analyze the size of input data and output data of each layer of sub-network in the neural network; Generate the total amount of data for each layer of the sub-network in the neural network.

8. A chip, characterized in that: include: Multiple processing cores and synchronization signal generators; Each processing core includes a data storage area and a parameter storage area; A neural network compiler, configured to execute the method for compiling an execution program according to any one of claims 1 to 7; The plurality of processing cores are used to be grouped according to execution programs suitable for execution by the system of the plurality of processing cores, wherein the processing cores in each group are used to execute a plurality of program segments in the execution program; wherein the data storage area is used to store input data and output data of the plurality of program segments, and the parameter storage area is used to store parameters of the plurality of program segments; The synchronization signal generator is used to send a synchronization signal to all processing cores when all processing cores executing the program segment have completed execution.

Images