Multi-task program compiling method and device, and multi-core chip
By reallocating and inserting synchronization instructions in the BSP computing model, the problem that the same group of cores can only complete one task is solved, realizing multi-task parallel computing, optimizing the computing resources of the core group, improving system efficiency and reducing costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- STREAM COMPUTING INC
- Filing Date
- 2020-11-27
- Publication Date
- 2026-07-03
AI Technical Summary
In the existing BSP computing model, the same set of cores can only complete one task, and the insertion of synchronous instructions is based on data correlation, which leads to an imbalance in computing time between cores and reduces system efficiency.
By determining the initial synchronization instruction insertion point for each task, an initial program segment is generated, its computational cost is calculated, and the program segments are reallocated in the computing core group so that each core performs the same amount of computation. Finally, the synchronization instruction is inserted, and the program segments of the core group are optimized.
It enables multiple tasks to run in the same computing core group, saving the number of chips, optimizing computing resources, increasing the total computing power of the chip, reducing power consumption, and reducing the programming difficulty for users.
Smart Images
Figure CN114546392B_ABST
Abstract
Description
Technical Field
[0001] This disclosure belongs to the field of program compilation technology, specifically relating to a multi-task program compilation method and apparatus, and a multi-core chip. Background Technology
[0002] The Bulk Synchronous Parallel Computing Model (BSP) is a parallel computing model that can be used in system-level applications, such as parallel computing in a cluster of computers composed of multiple servers, or in chip-level applications, such as parallel computing in multi-core chips.
[0003] When performing neural network calculations, multi-core chips using the BSP (Block Spectrum Processing) mechanism have unique application advantages. These multi-core chips mainly implement BSP calculations through the following three circuit components:
[0004] 1. A set of cores with local memory that can run independently;
[0005] 2. Inter-core communication bus or on-chip network (NoC);
[0006] 3. Circuits for achieving inter-core synchronization.
[0007] In the entire BSP task, a total of N cores operate. Each core has its own memory circuitry, computational and control circuitry, and can run independently to complete specific tasks. All cores begin working under the synchronization of synchronization signals. The time span between two successive synchronization signals is usually called a superstep. The entire BSP computation proceeds in a pipeline, using supersteps as units, and the workflow is as follows: Figure 1 As shown. The workflow is as follows:
[0008] 1. When the synchronization signal arrives, it signifies the start of a superstep, and all cores begin calculations simultaneously;
[0009] 2. Once each core has completed its calculations, it enters the communication phase, sending out the data that needs to be transmitted and simultaneously receiving the data that needs to be received.
[0010] 3. After each core completes its own calculations and communication, it enters a waiting state;
[0011] Once all cores have completed computation and communication during the task, it signifies the end of this superstep and the start of the next superstep, i.e., the arrival of a new synchronization signal, which initiates the next computation and transmission.
[0012] Within the entire chip, all cores are divided into M groups according to BSP tasks. Each group has a synchronization signal generator, Core_ctrl, which manages the generation of synchronization signals for all cores within the group. At this point, all cores within each group share the same synchronization signal, although synchronization signals from different groups may differ. Cores send synchronization request messages to Core_ctrl via a network (or bus). Core_ctrl generates the corresponding synchronization signal based on the received message and sends it to each core as a message. Core_ctrl is typically a core or a dedicated circuit. The network here refers to the data transmission network between the cores. A schematic diagram of the entire BSP computing system is shown below. Figure 2 As shown.
[0013] The complete workflow of a BSP task in a Group is as follows:
[0014] 1. Core_ctrl_G1 sends a synchronization signal message to all cores in Group 1, initiating the first superstep, causing all cores in this group to start running their own program segments;
[0015] 2. All cores begin executing the program segments within this time period until the program segments have finished executing;
[0016] 3. If a Core_1_i program segment finishes running, this core will start sending the calculation result data according to the task until all data has been sent;
[0017] 4. After the data transmission is complete, this core sends a message to Core_ctrl_G1, informing Core_ctrl_G1 that the core has completed the operation and communication in this superstep and can proceed to the next superstep. Then, this core enters the state of waiting for the next synchronization signal.
[0018] 5. After receiving the message, Core_ctrl_G1 checks whether all other cores have also completed their computation and communication. If Core_ctrl_G1 finds that some cores have not sent messages to notify it that their computation and communication are complete, it waits; if Core_ctrl_G1 finds that all cores have sent messages to notify it that their computation and communication are complete, it sends the next synchronization signal sync_x and enters the next hyperstep.
[0019] In this mechanism, each core in the group only begins to execute the special synchronization instructions inserted in its own program segment after it has finished running its own program segment and sent all the calculation results data. Through the on-chip network (NoC) or bus, it notifies the synchronization signal generator Core_ctrl_G1, which then generates a synchronization signal shared by all cores.
[0020] However, the existing solutions mentioned above have the following drawbacks:
[0021] 1. A core in the same group can only complete one task;
[0022] 2. Within the same task program, synchronization instructions are inserted based on data relevance. That is, within the current synchronization time period, a core needs to complete a full task program segment and obtain all the data that needs to be sent to other cores within this time period before inserting a synchronization instruction. However, because the execution time required for each core to obtain all the data can vary greatly and is very unbalanced, this reduces the efficiency of the entire chip or system. Summary of the Invention
[0023] This disclosure aims to address at least one of the problems existing in the prior art by providing a multi-tasking program compilation method and apparatus, and a multi-core chip.
[0024] One aspect of this disclosure provides a method for compiling a multitasking program, the method comprising:
[0025] Determine the initial synchronization instruction insertion point for each task;
[0026] For each initial synchronization instruction insertion point, an initial program segment is generated within each synchronization segment.
[0027] Calculate the computational cost required for each of the initial program segments;
[0028] Based on the computational amount required for each initial program segment, all initial program segments are redistributed among the computational cores of the computational core group, such that each computational core performs the same amount of computation. Based on the result of the redistribution, a new program segment and a final synchronization instruction insertion point are generated for each computational core.
[0029] Based on each of the final synchronization instruction insertion points, a synchronization instruction is inserted into the corresponding new program segment to generate the final program segment of the computing core group.
[0030] In some alternative implementations, the step of reallocating all the initial program segments in the computing core group according to the computational cost required for each initial program segment includes:
[0031] Based on the computing power of each computing core in the computing core group and the amount of computation required for each initial program segment, all the initial program segments are redistributed to each computing core.
[0032] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0033] Based on the computing power of the computing core, the initial program segments belonging to the different tasks are allocated to the same computing core.
[0034] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0035] Based on the computing power of the computing core, multiple initial program segments of one of the tasks are assigned to the same computing core.
[0036] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0037] Based on the computing power of the computing core, at least one initial program segment of at least one of the tasks is split into multiple sub-initial program segments, and the split sub-initial program segments are assigned to different computing cores.
[0038] In some alternative implementations, the task is a neural network, and determining the initial synchronization instruction insertion point for each task includes:
[0039] Based on the data flow in each of the neural networks, the initial synchronization instruction insertion point for each neural network is determined.
[0040] Another aspect of this disclosure provides a multitasking program compilation apparatus, the apparatus comprising:
[0041] The determination module is used to determine the initial synchronization instruction insertion point for each task.
[0042] The generation module is used to generate the initial program segment in each synchronization segment for each initial synchronization instruction insertion point.
[0043] The calculation module is used to calculate the computational amount required for each of the initial program segments;
[0044] The allocation module is used to reallocate all the initial program segments among the computing cores of the computing core group according to the amount of computation required for each initial program segment, so that each computing core performs the same amount of computation, and generate a new program segment and a final synchronization instruction insertion point for each computing core based on the result of the reallocation.
[0045] An insertion module is used to insert synchronization instructions into the corresponding new program segment according to each of the final synchronization instruction insertion points, thereby generating the final program segment of the computing core group.
[0046] In some optional implementations, the allocation module is configured to reallocate all the initial program segments in the computing core group according to the computational amount required by each initial program segment, including:
[0047] The allocation module is specifically used for:
[0048] Based on the computing power of each computing core in the computing core group and the amount of computation required for each initial program segment, all the initial program segments are redistributed to each computing core.
[0049] In some optional implementations, the allocation module is configured to reallocate all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computational amount required by each initial program segment, including:
[0050] The allocation module is specifically used for:
[0051] Based on the computing power of the computing core, the initial program segments belonging to the different tasks are allocated to the same computing core.
[0052] In some optional implementations, the allocation module is configured to reallocate all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computational amount required by each initial program segment, including:
[0053] The allocation module is specifically used for:
[0054] Based on the computing power of the computing core, multiple initial program segments of one of the tasks are assigned to the same computing core.
[0055] In some optional implementations, the allocation module is configured to reallocate all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computational amount required by each initial program segment, including:
[0056] The allocation module is specifically used for:
[0057] Based on the computing power of the computing core, at least one initial program segment of at least one of the tasks is split into multiple sub-initial program segments, and the split sub-initial program segments are assigned to different computing cores.
[0058] In some alternative implementations, the task is a neural network, and the determining module is configured to determine the initial synchronization instruction insertion point for each task, including:
[0059] The determining module is specifically used to determine the initial synchronization instruction insertion point of each neural network based on the data flow in each neural network.
[0060] Another aspect of this disclosure provides a multi-core chip including at least one computing core group, wherein the at least one computing core group is configured to perform the methods described above; or,
[0061] At least one of the computing cores includes the apparatus described above.
[0062] Another aspect of this disclosure provides an electronic device comprising:
[0063] One or more processors;
[0064] A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.
[0065] Another aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the implementation of the method described above.
[0066] This disclosed multi-tasking program compilation method and apparatus, along with a multi-core chip, allows multiple tasks to run within the same computing core group, thus reducing the number of chips required in a multi-tasking system and lowering costs. Furthermore, it optimizes the computing resources of the cores, maximizing the effective computing power of each core and thereby maximizing the chip's overall computing power, ultimately accelerating task completion and saving power. Finally, the entire synchronous insertion process can be automated by the compiler, significantly reducing the programming difficulty for users. Attached Figure Description
[0067] Figure 1 This is a schematic diagram of the BSP calculation workflow in related technologies;
[0068] Figure 2 This is a schematic diagram of the entire BSP computing system in the relevant technology;
[0069] Figure 3 This is a schematic block diagram illustrating the composition of an electronic device according to an embodiment of the present disclosure;
[0070] Figure 4 This is a flowchart of a multi-tasking program compilation method according to another embodiment of the present disclosure;
[0071] Figure 5 This is a flowchart of a multi-tasking program compilation method according to another embodiment of the present disclosure;
[0072] Figure 6This is a chip architecture diagram of another embodiment of the present disclosure;
[0073] Figure 7 This is a schematic diagram illustrating the structure of a neural network and the computational requirements of each layer according to another embodiment of the present disclosure.
[0074] Figure 8a This is a schematic diagram showing the distribution of initial synchronization insertion points according to another embodiment of this disclosure;
[0075] Figure 8b This is a schematic diagram showing the distribution of the initial program segments according to another embodiment of this disclosure;
[0076] Figure 9a This is a schematic diagram illustrating the allocation of initial program segments according to another embodiment of this disclosure;
[0077] Figure 9b This is a schematic diagram showing the final distribution of each initial program segment and the final synchronization instruction insertion point according to another embodiment of this disclosure;
[0078] Figure 10 for Figure 9b The diagram shows two CNN computation pipelines in the time domain.
[0079] Figure 11 This is a schematic diagram of the structure of a multitasking program compilation apparatus according to another embodiment of the present disclosure. Detailed Implementation
[0080] To enable those skilled in the art to better understand the technical solutions of this disclosure, the disclosure will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0081] First, refer to Figure 3 This describes an example electronic device for implementing the apparatus and methods of the embodiments of this disclosure.
[0082] like Figure 3 As shown, the electronic device 200 includes one or more processors 210, one or more storage devices 220, one or more input devices 230, one or more output devices 240, etc., and these components are interconnected via a bus system 250 and / or other forms of connection mechanisms. It should be noted that... Figure 3 The components and structures of the electronic devices shown are merely exemplary and not limiting; other components and structures may be used as needed.
[0083] The processor 210 can be a neural network processor composed of chips with a multi-core architecture, or a single central processing unit (CPU), or a central processing unit plus a multi-core neural network processor array, or other forms of processing unit with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 200 to perform the desired functions.
[0084] Storage device 220 may include one or more computer program products, which may include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory may include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory may include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions may be stored on the computer-readable storage medium, which a processor may execute to implement the client functions (implemented by the processor) in the embodiments of this disclosure described below, and / or other desired functions. Various applications and various data may also be stored in the computer-readable storage medium, such as various data used and / or generated by the applications.
[0085] The input device 230 may be a device used by a user to input commands, and may include one or more of a keyboard, mouse, microphone, and touch screen.
[0086] The output device 240 can output various information (such as images or sounds) to the outside (e.g., a user) and may include one or more of a display, a speaker, etc.
[0087] Below, we will refer to Figure 4 A method for compiling a multitasking program according to an embodiment of the present disclosure is described.
[0088] For example, such as Figure 4 As shown, a multi-tasking program compilation method S100 includes:
[0089] S110. Determine the initial synchronization instruction insertion point for each task.
[0090] Specifically, they can be combined together Figure 5 In this step, the task can be a neural network (NN), meaning each task represents a neural network. Of course, tasks can also be other network structures; this embodiment is not limited in this regard. When the task is a neural network, the initial synchronization instruction insertion point of that neural network can be determined based on the data flow of each neural network. That is, the complete result data output point of a certain layer of the neural network is the initial synchronization instruction insertion point, such as... Figure 5As shown, N neural networks (NN1 to NN) N This allows us to obtain N independent initial synchronization instruction insertion points.
[0091] S120. For each initial synchronization instruction insertion point, generate the initial program segment within each synchronization segment.
[0092] Specifically, in this step, it is combined with Figure 5 For each neural network, based on the initial synchronization instruction insertion point of that neural network, multiple initial synchronization segments are divided, and a program segment within each initial synchronization segment is generated as an initial program segment. When there are N neural networks, N sets of independent initial program segments can be obtained. Of course, other methods can also be used to generate the initial program segments within each synchronization segment separately, and this embodiment does not limit this.
[0093] S130. Calculate the computational cost required for each of the initial program segments.
[0094] Specifically, in this step, it is combined with Figure 5 When there are N neural networks, the computational cost required for each initial program segment in each neural network is calculated. The computational cost required for each initial program segment can be tabulated to obtain N sets of corresponding tables of computational costs required for the initial program segments. Of course, those skilled in the art can also express the required computational cost in other ways, and this embodiment is not limited in this regard.
[0095] S140. Based on the amount of computation required for each initial program segment, all initial program segments are redistributed among the computing cores of the computing core group, so that each computing core performs the same amount of computation. Based on the result of the redistribution, a new program segment and a final synchronization instruction insertion point are generated for each computing core.
[0096] Specifically, in this step, it can be combined with Figure 5 Multiple idle computing cores are identified from a pool of computing cores. Some or all of these idle cores are selected to form a computing core group. Computing cores within the same group share the same synchronization signal. At this point, depending on the computational load required for each initial program segment, the initial program segments can be redistributed within the computing core group by merging or splitting segments of the same task; alternatively, initial program segments of different tasks can be combined. This embodiment does not limit the redistribution. Through this redistribution, each computing core in the group can perform the same amount of computation. Based on the redistribution scheme, a new program segment and the final synchronization instruction insertion point can be generated for each computing core.
[0097] S150. Based on each of the final synchronization instruction insertion points, insert synchronization instructions into the corresponding new program segment to generate the final program segment of the computing core group.
[0098] Specifically, in this step, as in step S140 above, after reallocating each initial program segment, some initial synchronization instruction insertion points may be deleted, some initial synchronization instruction insertion points may be merged, or some new synchronization instruction insertion points may be added, thereby obtaining the final synchronization instruction insertion points, inserting synchronization instructions for each new program segment, and updating and optimizing each new program segment after the synchronization instructions are inserted, thereby generating the final program segment of the computing core group.
[0099] The multi-task program compilation method of this embodiment can run multiple tasks within the same computing core group, thereby saving the number of chips in a multi-task system and reducing costs. Furthermore, this embodiment can optimize the computing resources of the computing cores, maximizing the effective computing power of each core and thus maximizing the total computing power of the chip, thereby accelerating task completion and saving power consumption. Simultaneously, the entire multi-task program segment generation process can be automatically completed by an automatic compiler, greatly reducing the programming difficulty for users.
[0100] In some alternative implementations, the step of reallocating all the initial program segments in the computing core group according to the computational cost required for each initial program segment includes:
[0101] Based on the computing power of each computing core in the computing core group and the amount of computation required for each initial program segment, all the initial program segments are redistributed to each computing core.
[0102] For example, suppose there are two computing cores, each with a computing power of 4 TO (Tera Operations), namely the first computing core and the second computing core, which need to complete two tasks. Each task includes two initial program segments. The two initial program segments in the first task require 1 TO and 4 TO of computing power, respectively. The two initial program segments in the other task require 1 TO and 2 TO of computing power, respectively. Then, the initial program segments requiring 1 TO of computing power in the first task, 1 TO of computing power in the second task, and 2 TO of computing power in the second task can all be allocated to the first computing core. The initial program segments requiring 4 TO of computing power in the first task can be allocated to the second computing core, so that the computing power of the first computing core and the second computing core are both 4 TO, thereby achieving a balance of computing power between the two computing cores.
[0103] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0104] Based on the computing power of the computing cores, the initial program segments belonging to different tasks can be allocated to the same computing core, thereby merging some initial synchronization instruction insertion points. Alternatively, based on the computing power of the computing cores, multiple initial program segments of one task can be allocated to the same computing core, thereby deleting some initial synchronization instruction insertion points. Furthermore, based on the computing power of the computing cores, at least one initial program segment of at least one task can be split and allocated to different computing cores, thereby adding new synchronization instruction insertion points, and so on.
[0105] To illustrate the above allocation process in detail, a specific example is provided below.
[0106] Take, for example, a chip with 6 computing cores each having 1 TOPS (Tera Operations Per Second) of computing power, computing two three-layer neural networks with only 3 computing cores idle.
[0107] like Figure 6 As shown, the chip contains six computing cores, C1 to C6. Among them, computing cores C1, C2, and C3 are in an idle and available state, and they are in the same group, sharing a common synchronization signal generation circuit, Core_ctrl_G1. Computing cores C4, C5, and C6 are in a busy and unavailable state, belonging to another group.
[0108] The neural networks CNN1 and CNN2, which require computation, each have three layers. Their structures and the computational requirements of each layer are as follows: Figure 7 As shown. For the three layers of CNN1, the computational cost of each layer is 1TO, 1TO, and 2TO respectively; for the three layers of CNN2, the computational cost of each layer is 1TO, 1TO, and 3TO respectively.
[0109] The compilation process of a multitasking program is as follows:
[0110] like Figure 8a As shown, for each neural network, the initial synchronization instruction insertion point is determined based on its data flow. Since there are two neural networks in this example, two independent sets of initial synchronization instruction insertion points are obtained. The first set is SP. 1_0 SP 1_1 SP 1_2 and SP 1_3 The second group consists of SP. 2_0 SP 2_1 SP 2_2 and SP 2_3 Its schematic diagram is as follows Figure 8a As shown.
[0111] like Figure 8b As shown, for each neural network, based on each initial synchronization instruction insertion point, initial program segments are generated within each synchronization segment located between two initial synchronization instruction insertion points. Since there are two neural networks in this example, two independent sets of initial program segments are obtained. The first set of initial program segments are P... 1_1 P 1_2 and P 1_3 The second set of initial program segments are P. 2_1 P 2_2 and P 2_3 Its schematic diagram is as follows Figure 8b As shown.
[0112] Calculate the computational cost of each initial program segment in both groups. The computational costs of each initial program segment in the first group are shown in Table 1 below:
[0113] Table 1
[0114] CNN1 Required computation (TO) <![CDATA[P 11 ]]> 1 <![CDATA[P 12 ]]> 1 <![CDATA[P 13 ]]> 2
[0115] The computational requirements for each initial program segment in the second group are shown in Table 2 below:
[0116] Table 2
[0117] CNN2 Required computation (TO) <![CDATA[P 21 ]]> 1 <![CDATA[P 22 ]]> 1 <![CDATA[P 23 ]]> 3
[0118] Based on the computational requirements of each initial program segment, the initial program segments of each neural network are redistributed among the computational kernels of the kernel group. During redistribution, efforts are made to place a complete initial program segment on a single computational kernel to avoid splitting data and program segments. There are several redistribution methods; in this example, the initial program segment requiring the largest computational cost is first identified, which in this case is the third initial program segment P of CNN2. 2_3 The initial program segment P 2_3 The required computation is the largest, at 3TO, therefore, the initial program segment P can be... 2_3 Assigning it to a computational core, such as C3, the remaining initial program segments can be combined in various ways. For example, the P of CNN1 can be used... 1_1 and P 1_2 P with CNN2 2_1 Assign P to computation kernel C1, and transfer P to CNN1. 1_3 And CNN2's P 2_2 The computational core C2 is assigned to the core, and the final allocation results are shown in Table 3 below:
[0119] Table 3
[0120] Computation kernel Initial program segment C1 <![CDATA[P 11 ,P 12 ,P 21 ]]> C2 <![CDATA[P 13 ,P 22 ]]> C3 <![CDATA[P 23 ]]>
[0121] In this way, each computing core has a 3TO load, ensuring that each core in the core group performs the same amount of computation, achieving the best utilization of the chip's computing power. The initial program segment allocation diagram on each computing core at this point is as follows: Figure 9a As shown.
[0122] Due to CNN1's P 1_1 and P 1_2 All were allocated to computational core C1, therefore, P 1_1 and P 1_2 Data transfer between them can take place within the same synchronization segment, with the initial synchronization instruction insertion point SP. 1_1 It can be deleted. Initial synchronization instruction insertion point SP 1_0 and SP 2_0 They can be merged into the final synchronization instruction insertion point SP0 and the initial synchronization instruction insertion point SP. 1_2 and SP 2_1 They can be merged into the final synchronization instruction insertion point SP1 and the initial synchronization instruction insertion point SP. 1_3 and SP 2_2 They can be merged into the final synchronization instruction insertion point SP2 and the initial synchronization instruction insertion point SP. 2_3 This then becomes the final synchronization instruction insertion point SP3. Based on this, if... Figure 9b As shown, the final computational core allocation scheme can be obtained. Based on this scheme, new program segments and final synchronization instruction insertion points can be generated. According to each final synchronization instruction insertion point, synchronization instructions can be inserted into each new program segment. After the synchronization instructions are inserted, each new program segment is updated and optimized, thereby generating the final program segments for the computational core group.
[0123] like Figure 9b As shown, the result OUT1 of CNN1 will be output by computation kernel C2, and the result OUT2 of CNN2 will be output by computation kernel C3. The time-domain representation of the computational pipeline of the two CNNs is shown below. Figure 10 As shown. By Figure 10 As can be seen, all cores are running at full capacity, which greatly improves the chip's effective computing power.
[0124] Another aspect of this disclosure, such as Figure 11 As shown, a multi-tasking program compilation apparatus 100 is provided. This apparatus 100 can be applied to the methods described above. Specific details not mentioned in the following description can be found in the relevant preceding descriptions and will not be repeated here. The apparatus 100 includes:
[0125] The determination module 110 is used to determine the initial synchronization instruction insertion point for each task.
[0126] The generation module 120 is used to generate the initial program segment in each synchronization segment for each initial synchronization instruction insertion point.
[0127] Calculation module 130 is used to calculate the computational amount required for each of the initial program segments;
[0128] The allocation module 140 is used to reallocate all the initial program segments in each computing core of the computing core group according to the amount of computation required for each initial program segment, so that each computing core performs the same amount of computation, and generate a new program segment and a final synchronization instruction insertion point for each computing core based on the result of the reallocation.
[0129] The insertion module 150 is used to insert synchronization instructions into the corresponding new program segment according to each of the final synchronization instruction insertion points, thereby generating the final program segment of the computing core group.
[0130] The multi-task program compilation apparatus of this disclosure can run multiple tasks within the same computing core group, thereby saving the number of chips in a multi-task system and reducing costs. Furthermore, this embodiment can optimize the computing resources of the computing cores, maximizing the effective computing power of each core and thus maximizing the total computing power of the chip, thereby accelerating task completion and saving power consumption. Simultaneously, the entire process of generating multi-task program segments can be automatically completed by the automatic compiler, greatly reducing the programming difficulty for users.
[0131] In some alternative implementations, the step of reallocating all the initial program segments in the computing core group according to the computational cost required for each initial program segment includes:
[0132] The allocation module 140 is specifically used for:
[0133] Based on the computing power of each computing core in the computing core group and the amount of computation required for each initial program segment, all the initial program segments are redistributed to each computing core.
[0134] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0135] The allocation module 140 is specifically used for:
[0136] Based on the computing power of the computing core, the initial program segments belonging to the different tasks are allocated to the same computing core.
[0137] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0138] The allocation module 140 is specifically used for:
[0139] Based on the computing power of the computing core, multiple initial program segments of one of the tasks are assigned to the same computing core.
[0140] In some optional implementations, the step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the amount of computation required for each of the initial program segments includes:
[0141] The allocation module 140 is specifically used for:
[0142] Based on the computing power of the computing core, at least one initial program segment of at least one of the tasks is split into multiple sub-initial program segments, and the split sub-initial program segments are assigned to different computing cores.
[0143] In some alternative implementations, the task is a neural network, and the determining module 110 is used to determine the initial synchronization instruction insertion point for each task, including:
[0144] The determining module 110 is specifically used to determine the initial synchronization instruction insertion point of each neural network based on the data flow in each neural network.
[0145] Another aspect of this disclosure provides a multi-core chip including at least one set of computing cores, the chip being used to perform the methods described above.
[0146] The multi-core chip in this embodiment can be used to execute the methods described above, which can save the number of chips in a multi-tasking system, reduce costs, optimize the computing resources of the computing cores, maximize the effective computing power of each computing core, accelerate task completion, save power consumption, and greatly reduce the programming difficulty for users.
[0147] Another aspect of this disclosure provides an electronic device comprising:
[0148] One or more processors;
[0149] A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method described above.
[0150] Another aspect of this disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, enables the implementation of the method described above.
[0151] The computer-readable storage medium may be included in the apparatus or device disclosed herein, or it may exist independently.
[0152] The computer-readable storage medium can be any tangible medium that contains or stores a program, and can be an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device. More specific examples include, but are not limited to, connections having one or more wires, portable computer disks, hard disks, optical fibers, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.
[0153] The computer-readable storage medium may also include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code, specific examples of which include, but are not limited to, electromagnetic signals, optical signals, or any suitable combination thereof.
[0154] It is understood that the above embodiments are merely exemplary embodiments used to illustrate the principles of this disclosure, and this disclosure is not limited thereto. For those skilled in the art, various modifications and improvements can be made without departing from the spirit and substance of this disclosure, and these modifications and improvements are also considered to be within the scope of protection of this disclosure.
Claims
1. A method for compiling a multi-tasking program, characterized in that, The method includes: Determine the initial synchronization instruction insertion point for each task; For each initial synchronization instruction insertion point, an initial program segment is generated within each synchronization segment. Calculate the computational cost required for each of the initial program segments; Based on the computational amount required for each initial program segment, all initial program segments are redistributed among the computing cores of the computing core group, such that each computing core performs the same amount of computation. Based on the result of the redistribution, a new program segment and a final synchronization instruction insertion point are generated for each computing core. Based on each of the final synchronization instruction insertion points, a synchronization instruction is inserted into the corresponding new program segment to generate the final program segment of the computing core group; The task is a neural network, and determining the initial synchronization instruction insertion point for each task includes: Based on the data flow in each neural network, the initial synchronization instruction insertion point for each neural network is determined; the complete result data output point of a certain layer of the neural network is the initial synchronization instruction insertion point.
2. The method according to claim 1, characterized in that, The step of redistributing all the initial program segments among the computing cores of the computing core group according to the computational amount required for each initial program segment includes: Based on the computing power of each computing core in the computing core group and the amount of computation required for each initial program segment, all the initial program segments are redistributed to each computing core.
3. The method according to claim 2, characterized in that, The step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computing power required by each initial program segment includes: Based on the computing power of the computing core, the initial program segments belonging to different tasks are allocated to the same computing core.
4. The method according to claim 2 or 3, characterized in that, The step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computing power required by each initial program segment includes: Based on the computing power of the computing core, multiple initial program segments of one of the tasks are assigned to the same computing core.
5. The method according to claim 2 or 3, characterized in that, The step of reallocating all the initial program segments to each of the computing cores based on the computing power of each computing core in the computing core group and the computing power required by each initial program segment includes: Based on the computing power of the computing core, at least one initial program segment of at least one of the tasks is split into multiple sub-initial program segments, and the split sub-initial program segments are assigned to different computing cores.
6. A multi-tasking program compilation device, characterized in that, The device includes: The determination module is used to determine the initial synchronization instruction insertion point for each task. The generation module is used to generate the initial program segment in each synchronization segment for each initial synchronization instruction insertion point. The calculation module is used to calculate the computational amount required for each of the initial program segments; The allocation module is used to reallocate all the initial program segments among the computing cores of the computing core group according to the amount of computation required for each initial program segment, so that each computing core performs the same amount of computation, and generate a new program segment and a final synchronization instruction insertion point for each computing core based on the result of the reallocation. An insertion module is used to insert synchronization instructions into the corresponding new program segment according to each of the final synchronization instruction insertion points, thereby generating the final program segment of the computing core group; The task is a neural network, and determining the initial synchronization instruction insertion point for each task includes: Based on the data flow in each neural network, the initial synchronization instruction insertion point for each neural network is determined; the complete result data output point of a certain layer of the neural network is the initial synchronization instruction insertion point.
7. A multi-core chip, comprising at least one computing core group, characterized in that, At least one of the said computational cores is used to perform the method of any one of claims 1 to 5; or, At least one of the computing cores comprises the apparatus of claim 6.
8. An electronic device, characterized in that, include: One or more processors; A storage unit for storing one or more programs that, when executed by one or more processors, enable the one or more processors to implement the method according to any one of claims 1 to 5.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it can implement the method according to any one of claims 1 to 5.