Method and system for performing parallel computing
By designing a coupling architecture between task manager and multi-core core in the CNN system, using the coordinated work of processing units, interfaces and reduction elements, the problem of inefficiency of CNN in parallel computing is solved, and more efficient parallel computing and reduction capabilities are achieved. .
Patent Information
- Application Number
- CN201980056385.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-07-30
- Filing Date
- 2019-08-20
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2039-08-20
AI Technical Summary
The existing convolutional neural network (CNN) architecture design in parallel computing is difficult to effectively utilize the parallelism of multi-core processors, resulting in inefficient computing.
Design a system that includes a task manager and multiple cores. The core is coupled with the task manager. It generates the output feature map of the parallel computing task set through processing units, interfaces and reduction elements, and manages the computing task's calculation task through barrier instructions and recovery instructions. Parallel execution to realize parallel collection and reduction of parallel computing tasks.
It improves the parallelism of CNN operations, enhances the distribution efficiency of calculation and reduction, and improves overall performance, especially the parallel execution capabilities in the convolutional calculation and collection and reduction stages.
Smart Images

Figure CN112639726B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This disclosure claims the benefit of priority to U.S. Provisional Application No. 62 / 724,539, filed on August 29, 2018, and U.S. Patent Application No. 16 / 526,597, filed on July 30, 2019, which are incorporated herein by reference in their entirety. Background Art
[0003] Convolutional Neural Networks (CNNs) are a type of neural network that is widely used in machine learning. CNNs are used in a variety of applications including image processing, speech recognition, game playing, robotics, etc. The design of an efficient architecture for CNN operations and mapping the CNN operations into the architecture can be very important. Summary of the invention
[0004] An embodiment of the present disclosure provides a system for performing parallel computing. The system may include: a task manager; and multiple cores, which are coupled to the task manager and configured to respectively perform parallel computing task sets based on instructions from the task manager, wherein each of the multiple cores also includes: a processing unit, which is configured to generate a first output feature map corresponding to a first computing task in the parallel computing task set; an interface, which is configured to receive one or more instructions from the task manager to collect external output feature maps corresponding to the parallel computing task set from other cores in the multiple cores; and a reduced simple element, which is configured to generate a reduced feature map based on the first output feature map and the received external output feature map.
[0005] The embodiment of the present disclosure also provides a method for executing a parallel computing task set at a core of a plurality of cores coupled to a task manager. The method may include: generating, by a processing unit of the core, a first output feature map corresponding to a first computing task in the parallel computing task set; receiving, by an interface of the core, one or more instructions from the task manager to collect external output feature maps corresponding to the parallel computing task set from other cores of the plurality of cores; generating, by a reduction element of the core, a reduced feature map based on the first output feature map and the received external output feature map.
[0006] The embodiment of the present disclosure also provides a method for managing tasks by a task manager coupled with multiple cores. The method may include: instructing a core among multiple cores to generate a first output feature map corresponding to a first computing task in a parallel computing task set, wherein after generating the first output feature map, the core issues a barrier instruction to the task manager and stops the generation of a second output feature map corresponding to a second computing task; receiving barrier instructions from other cores among multiple cores; after receiving the barrier instruction, sending a recovery instruction to the core to resume the generation of the second output feature map; selecting a core as a reduction core to receive an external output feature map corresponding to a parallel computing task set from other cores, so as to generate a reduced feature map based on the first output feature map and the external output feature map, wherein the generation of the second output feature map occurs in parallel with the generation of the reduced feature map.
[0007] Additional features and advantages of the disclosed embodiments will be explained in part in the following description, and in part will be obvious from the description, or can be learned through practice of the embodiments. The features and advantages of the disclosed embodiments can be realized and obtained by the elements and combinations set forth in the claims.
[0008] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosed embodiments, as claimed. BRIEF DESCRIPTION OF THE DRAWINGS
[0009] Embodiments and various aspects of the present disclosure are illustrated in the following detailed description and accompanying drawings.The various features shown in the drawings are not drawn to scale.
[0010] Figure 1A Illustration of an exemplary convolutional neural network (CNN) operation.
[0011] Figure 1B Figure 3.1 shows the division of CNN operations.
[0012] Figure 1C Illustration of an example procedure for training a convolutional neural network.
[0013] Figure 2A An exemplary neural network processing architecture according to an embodiment of the present disclosure is illustrated.
[0014] Figure 2B A schematic diagram illustrating an exemplary cloud system incorporating a neural network processing architecture according to an embodiment of the present disclosure.
[0015] Figure 3 The diagram illustrates a structure of a system for performing parallel computing according to an embodiment of the present disclosure.
[0016] Figure 4An exemplary procedure for performing parallel computing according to an embodiment of the present disclosure is illustrated.
[0017] Figure 5 FIG. 1 is a flowchart illustrating an exemplary method for executing a set of parallel computational tasks on a set of matrices by a plurality of cores coupled to a task manager according to an embodiment of the present disclosure.
[0018] Figure 6 FIG. 1 is a flowchart illustrating an exemplary method for managing tasks by a task manager coupled to multiple cores according to an embodiment of the present disclosure. DETAILED DESCRIPTION
[0019] Certain aspects of the disclosure are described in further detail below.In the event of a conflict with terms and / or definitions incorporated by reference, the terms and definitions provided herein shall control.
[0020] The technology described in the present disclosure has one or more of the following technical effects. In some implementations, the technology described in the present disclosure provides a system for performing parallel computing, including: a task manager; and multiple cores, which are coupled to the task manager and configured to respectively execute parallel computing task sets based on instructions from the task manager, wherein each of the multiple cores also includes: a processing unit, which is configured to generate a first output feature map corresponding to a first computing task in the parallel computing task set; an interface, which is configured to receive one or more instructions from the task manager to collect external output feature maps corresponding to the parallel computing task set from other cores in the multiple cores; a reduced element, which is configured to generate a reduced feature map based on the first output feature map and the received external output feature map. This task manager allows the processing unit to process matrices for the current iteration and other components (e.g., interfaces, memory units, and reduced elements) to generate reduced feature maps in parallel based on the results of previous iterations. Therefore, the parallelism of, for example, CNN operations can be further utilized.
[0021] As used herein, the terms "comprises," "comprising," or any other variations thereof, are intended to cover a non-exclusive inclusion such that a process, method, composition, article, or apparatus that includes a list of elements includes not only those elements but may include other elements not expressly listed or inherent to such process, method, composition, article, or apparatus. The term "exemplary" is used in the sense of an "example" rather than an "ideal."
[0022] Figure 1A An exemplary convolutional neural network (CNN) operation is shown. In this exemplary operation, the input activations of a layer are structured as a set of two-dimensional (2D) input feature maps across many (e.g., C) channels. Each of the two-dimensional input feature maps can be referred to as a channel. Figure 1AAs shown in , the input data 102 (eg, image) of one channel has a size of H×W. Therefore, the size of a two-dimensional (2D) input feature atlas may be H×W×C.
[0023] Each channel may be convolved with a 2D filter of size S×R from a stack of filters 104. One filter 104 may be used for one channel accordingly. The filter 104 may slide over the input feature map, and a point may be generated from the convolution operation of the filter 104 (including weights) and a portion of the input feature map covered by the filter 104. By sliding the filter 104 over the feature map and repeating the convolution operation, a feature output map 106 of size H'×W' may be generated. Figure 1A As shown in , there are K filter banks 104, resulting in K dimensions of the H'×W' sized feature output map 106. In other words, the size of the feature output map 106 is H'×W'×K.
[0024] In a CNN model, such operations may be repeated for multiple layers. During these operations, the feature output map 106 of the previous layer is used as input data for the next layer. In other words, the feature output map 106 of size H'×W'×K in the previous layer may be used as input data 102 in the next layer. It should be appreciated that the size values (e.g., H', W', and K) of the feature output map 106 in the previous layer may be used as the size values (e.g., H, W, and C) of the input data 102 in the next layer.
[0025] Due to the parallelism of CNN training, the CNN operations of CNN training can be distributed among multiple processing cores in K dimensions. Figure 1B Graph the partitioning of CNN operations. In some embodiments, CNN operations can be partitioned and distributed among several processing cores. For example, Figure 1B As shown in , input data 102 of size H×W×C may be divided into, for example, four parts 112, 114, 116, and 18, and distributed to four processing cores (e.g., cores 132, 134, 136, and 138). Thus, each processing core is responsible for ¼ of the input data 102. Filters of size S×R×C are also distributed across the four processing cores 132, 134, 136, and 138. Thus, in Figure 1BIn the stage of parallel computation of , each of the four processing cores 132, 134, 136, and 138 may generate intermediate feature maps in parallel. After the four processing cores 132, 134, 136, and 138 generate intermediate feature maps, the first core (e.g., core 132) of the four processing cores 132, 134, 136, and 138 may collect the generated intermediate feature maps and reduce them to a single feature map in the stage of collection and reduction. It should be appreciated that the single feature map generated in the stage of collection and reduction is the output of the first group of the K filter groups 104 in the first layer.
[0026] Figure 1C An exemplary procedure 130 for training a convolutional neural network is illustrated. The procedure 130 shows the connection between the output of the first layer L and the input of the second layer L+1. For example, in the exemplary procedure 130, in an iteration of the first layer L, an output R may be generated. L And use it as input to the second layer L+1 iterations.
[0027] As discussed above, the output of the previous layer can be the input of the next layer. Therefore, the above two stages can be repeated up to K times. During the iteration, the collection and reduction rotate among all cores. For example, the second core performs the second round of collection and reduction, the third core and the fourth core perform the third and fourth rounds respectively, and return to the first core again for the fifth round. Return to reference Figure 1B , the iteration may produce an output 120 comprising K feature maps distributed across 4 cores in an interleaved manner.
[0028] Embodiments of the present disclosure provide systems and methods for parallel computing, such as CNN operations. The disclosed embodiments distribute computation and reduction to different components of the core, allowing computation and reduction to be performed in parallel. For example, the computation stage may use an arithmetic and logic unit (ALU), while the collection and reduction stages may use memory, interfaces, and reduction primitives. Therefore, iterations may be pipelined to enhance performance.
[0029] Figure 2A An exemplary parallel computing architecture 200 according to an embodiment of the present disclosure is shown. Figure 2A As shown in FIG. 2 , the architecture 200 may include a chip communication system 202, a host memory 204, a memory controller 206, a direct memory access (DMA) unit 208, a joint test action group (JTAG) / test access port (TAP) controller 210, a peripheral interface 212, a bus 214, a global memory 216, etc. It should be appreciated that the chip communication system 202 may perform algorithmic operations (e.g., machine learning operations) based on the transferred data.
[0030] The chip communication system 202 may include a global manager 2022 and a plurality of cores 2024. The global manager 2022 may include at least one task manager to coordinate with one or more cores 2024. Each task manager may be associated with a core matrix 2024 that provides synaptic / neuron circuits (e.g., neural networks) for parallel computing. For example, Figure 2A The top layer of the processing elements of the core may provide circuits representing the input layer to the neural network, while the second layer of the core may provide circuits representing the hidden layer of the neural network. In some embodiments, the chip communication system 202 may be implemented as a neural network processing unit (NPU), a graphics processing unit (GPU), or another heterogeneous accelerator unit. Figure 2A As shown in , the global manager 2022 may include two task managers to coordinate with the two core arrays.
[0031] The core 2024 may include, for example, one or more processing elements, each of which includes a single instruction multiple data (SIMD) architecture, which includes one or more processing units configured to perform one or more operations (e.g., multiplication, addition, multiply-accumulate, etc.) based on instructions received from the global manager 2022. In order to perform operations on the transmitted data packets, the core 2024 may include one or more processing elements for processing information in the data packets. Each processing element may include any number of processing units. In some embodiments, the core 2024 may be considered as a tile or the like.
[0032] The host memory 204 may be an off-chip memory such as a memory of a host CPU. For example, the host memory 204 may be a DDR memory (e.g., DDR SDRAM), etc. The host memory 204 may be configured to store a large amount of data at a slower access speed than an on-chip memory integrated into one or more processors as a higher level cache.
[0033] The memory controller 206 may manage reading and writing of data to and from specific memory blocks within the global memory 216 having on-chip memory blocks (e.g., 4 blocks of 8GB second generation high bandwidth memory (HBM2)) used as main memory. For example, the memory controller 206 may manage reading / writing data from external chip communication system 202 (e.g., from DMA unit 208 or a DMA unit corresponding to another NPU) or from within the chip communication system 202 (e.g., from local memory in core 2024 via a 2D grid controlled by a task manager through global manager 2022). In addition, although in Figure 2AOne memory controller is shown in 210 , but it should be appreciated that more than one memory controller may be provided in architecture 200. For example, there may be one memory controller for each memory block (eg, HBM2) within global memory 216 .
[0034] The memory controller 206 may generate memory addresses and initiate memory read or write cycles. The memory controller 206 may include several hardware registers that may be written to and read from by one or more processors. The registers may include a memory address register, a byte count register, one or more control registers, and other types of registers. These registers may specify some combination of the source, the destination, the direction of the transfer (reading from an input / output (I / O) device or writing to an I / O device), the size of the transfer unit, the number of bytes to be transferred in a burst, and / or other typical features of a memory controller.
[0035] DMA unit 208 may assist in transferring data between host memory 204 and global memory 216. Additionally, DMA unit 208 may assist in transferring data between multiple NPUs (e.g., NPU 202). DMA unit 208 may allow off-chip devices to access both on-chip and off-chip memory without causing CPU interruptions. Thus, DMA unit 208 may also generate memory addresses and initiate memory read or write cycles. DMA unit 208 may also include several hardware registers that may be written to and read by one or more processors, including a memory address register, a byte count register, one or more control registers, and other types of registers. These registers may specify some combination of source, destination, direction of transfer (read from or write to an input / output (I / O) device), size of transfer unit, and / or number of bytes to be transferred in one burst. It should be appreciated that architecture 200 may include a second DMA unit that may be used to transfer data between other neural network processing architectures to allow multiple neural network processing architectures to communicate directly without involving a host CPU.
[0036] The JTAG / TAP controller 210 may specify a dedicated debug port that implements a serial communication interface (e.g., a JTAG interface) for low-overhead access to the NPU without direct external access to the system address and data buses. The JTAG / TAP controller 210 may also have an on-chip test access interface (e.g., a TAP interface) that implements a protocol to access a set of test registers that expose various parts of the chip logic level and device capabilities.
[0037] Peripheral interface 212 (such as a PCIe interface), if present, serves as (and typically is) an inter-chip bus, providing communications between architecture 200 and other devices.
[0038] Bus 214 includes intra-chip buses and inter-chip buses. The intra-chip bus connects all internal components to each other as required by the system architecture. Although not all components are connected to every other component, all components do have some connection to other components with which they need to communicate. The inter-chip bus connects the NPU to other devices such as off-chip memory or peripherals. Typically, if there is a peripheral interface 212 (e.g., an inter-chip bus), the bus 214 is only related to the intra-chip bus, but in some implementations, it can still be related to specialized inter-bus communications.
[0039] The chip communication system 202 may be configured to perform neural network based operations.
[0040] The architecture 200 may also include a host unit 220. The host unit 220 may be one or more processing units (e.g., an X86 central processing unit). In some embodiments, a host system having the host unit 220 and the host memory 204 may include a compiler (not shown). A compiler is a program or computer software that converts computer code written in a programming language into NPU instructions to create an executable program. In machine learning applications, a compiler may perform various operations, such as preprocessing, lexical analysis, parsing, semantic analysis, converting an input program into an intermediate representation, code optimization, and code generation, or a combination thereof.
[0041] In some embodiments, the compiler that generates the NPU instructions may be on the host system, which pushes the commands to the chip communication system 202. Based on these commands, each task manager may assign any number of tasks to one or more cores (e.g., core 2024). Some of these commands may instruct the DMA unit 208 to load instructions (generated by the compiler) and data from the host memory 204 into the global memory 216. The loaded instructions may then be distributed to each core that is assigned a corresponding task, and one or more cores may process the instructions.
[0042] Figure 2B A schematic diagram of an exemplary cloud system 230 incorporating a neural network processing architecture 200 is illustrated according to an embodiment of the present disclosure.
[0043] like Figure 2B As shown in , cloud system 230 can provide artificial intelligence (AI) capabilities to cloud services and can include multiple computing servers (e.g., 232 and 234). In some embodiments, computing server 232 can be incorporated into, for example, Figure 2A For simplicity and clarity, in Figure 2B The parallel computing architecture 200 is shown in a simplified manner.
[0044] With the help of the parallel computing architecture 200, the cloud system 700 can provide extended AI capabilities such as image recognition, facial recognition, translation, 3D modeling, etc.
[0045] It should be appreciated that the parallel computing architecture 200 can be deployed to other forms of computing devices. For example, the neural network processing architecture 200 can also be integrated into computing devices such as smart phones, tablets, and wearable devices.
[0046] In addition, although Figure 2A-2B A parallel computing architecture is shown in , but it should be appreciated that any accelerator that provides the ability to perform parallel computing may be used.
[0047] Figure 3 FIG. 3 is a block diagram of a system 300 for performing parallel computing according to an embodiment of the present disclosure. The system 300 may be Figure 2A The invention is a subsystem of the architecture 200 of the embodiment of the present invention, and includes a plurality of cores (e.g., cores 2024, 2026, 2028, and 2030) and a task manager 310 coupled to the plurality of cores. The plurality of cores may be configured to respectively perform a parallel computing task set on a matrix set as input data according to an instruction from the task manager 310.
[0048] In some embodiments, core 2024 may include computing unit 302, interface 304, memory unit 306, and logic unit 308. It should be appreciated that cores 2026, 2028, 2030, etc. may include similar or identical structures of core 2024, and a description thereof is omitted herein.
[0049] The computing unit 302 may include one or more ALUs (e.g., as multipliers, adders, or multiply-accumulators) configured to perform one or more computing tasks (e.g., convolution computing) on a portion of a matrix (e.g., a first matrix or a second matrix in a matrix set). For example, the computing unit 302 may process a first matrix to generate a first output feature map corresponding to a first computing task in a parallel computing task set. It should be appreciated that in some embodiments, a matrix (e.g., a first matrix) may include only one row of data. The result of the convolution computing generated by the computing unit 302 of the core 2024 may be stored locally or retrieved by another core (e.g., core 2026, 2028, or 2030). The locally stored result may be referred to as a local result. For example, the first output feature map may be stored locally.
[0050] Interface 304 may be configured to communicate data with task manager 230 and other cores in the same group. In some embodiments, interface 304 may be configured to receive one or more instructions from task manager 310 to collect external output feature maps corresponding to the parallel computing task set from other cores among the plurality of cores.
[0051] The interface 304 may be controlled by a task manager (e.g., task manager 310). For example, the task manager 310 may provide instructions to the interface 304 to enable the core 2024 to perform convolution calculations. The interface 304 may also communicate with other cores (e.g., cores 2026, 2028, and 2030) to receive the results of the convolution calculations assigned to these cores. The results received from other cores may be referred to as external output feature maps. After collecting the external output feature maps, the core 2024 may further perform reduction on the collected external output feature maps and local results (e.g., the first output feature map).
[0052] In addition to actively obtaining external output feature maps from other cores, interface 304 may also be passive and allow task manager 310 to instruct other cores to send external output feature maps to core 2024, which is responsible for performing the collection and reduction.
[0053] The local memory unit 306 may be configured to store local results and retrieved external output feature maps. The local memory unit 306 may include multiple memory blocks using a unified addressing space. In some embodiments, the higher bits of the unified addressing space may be associated with the identification of the core (e.g., core ID). In addition, the memory of each core may perform collection and reduction independently of the computing unit 302. And the local memory unit 306 may have a dedicated port for collection and reduction. For example, the local memory unit 306 may include a first port for sending and receiving data (e.g., local results or external output feature maps) with another core and a second port for cooperating with the computing unit 302 on reduction. Therefore, collection and reduction can be performed in parallel with the convolution calculations performed at each core.
[0054] The reduction unit 308 may be configured to generate a reduced feature map (e.g., having a dimension of H'×W') based on the first output feature map and the received external output feature map. For example, the first output feature map and the received external output feature map may be reduced to the reduced feature map by pooling. Pooling may reduce the first output feature map and the received external output feature map by calculating the maximum value or average value of a region of the feature map.
[0055] Task manager 310 may coordinate multiple cores (e.g., cores 2024, 2026, 2028, and 2030). For example, task manager 310 may include a DMA engine (in the example of FIG. 1 ) configured to instruct multiple cores to perform parallel computations (including convolution computations, collections, and reductions). Figure 3 In some embodiments, the task manager 310 may be configured to instruct the computing unit 302 of the core to generate a second output feature map corresponding to the second computing task and to instruct the reduction unit 308 of the core to simultaneously generate a reduced feature map.
[0056] Because the reduced feature map is generated based on the output feature maps (including the first output feature map and the external output feature maps generated by other cores), the core waits until all external output feature maps are generated. Therefore, if a new output feature map is generated before receiving an earlier output feature map for reduction, the earlier output feature map can be replaced with the new output feature. Therefore, when the first output feature map is generated, the computing unit 302 can issue a barrier instruction to the task manager 310 and stop the generation of the second output feature map. In this way, the output feature map can be retained in the local memory unit 306 until the output feature map is used for reduction.
[0057] The task manager 310 may then receive a barrier instruction from each of the plurality of cores. After receiving the barrier instruction from each of the plurality of cores, it indicates to the task manager 310 that the feature graphs (including the first output feature graph and the external output feature graph) are ready for reduction. The core selected by the task manager 310 for reducing the first output feature graph and the external output feature graph may then receive the external output feature graph from other cores and reduce these simplified feature graphs. On the other hand, each of the plurality of cores may continue to generate a new set of feature graphs (e.g., the second output feature graph). Therefore, after receiving the barrier instruction from each of the plurality of cores, the task manager 310 may send a recovery instruction to the core to resume the generation of the second output feature graph.
[0058] Figure 4 An exemplary procedure 400 for performing parallel computations according to an embodiment of the present disclosure is illustrated.
[0059] and Figure 1C Compared to the program 130 of FIG. 400 , the program 400 divides the K iterations into a low-level program 402 and a high-level program 404. As discussed above, the convolution calculation task can be performed by the processing unit, while the collection and reduction can be performed by the memory, interface, and reduction unit. Therefore, the convolution calculation and the collection and reduction can be performed in parallel. By modifying the program 130 to the form of this program 400, the parallelism between the convolution calculation and the collection and reduction can be utilized.
[0060] can be used on n cores (e.g. Figure 3 The low-level program 402 is executed in parallel on each of the four cores (shown in FIG. 4 ). In each loop of the low-level program 402, the nth core may perform its corresponding convolution calculation by calling the function “compute(n)”. After the execution of the function “compute(n)” is completed, the nth core may issue, for example, a barrier instruction to the task manager. For example, referring to Figure 3, when the processing of the first matrix is completed, the computing unit 302 of the core 2024 may execute the function "signal(i,1)" to issue a barrier instruction to the DMA engine of the task manager 310. In response to the barrier instruction, the task manager 310 knows the convolution calculation completed on the nth core. It should be appreciated that the task manager 310 may receive barrier instructions from all cores associated with the task manager 310 (e.g., including cores 2024, 2026, 2028, and 2030). For example, the task manager 310 may determine whether the barrier instructions from all cores have been received. When the barrier instructions from all cores have been received, the task manager 310 may determine that the convolution calculation of the first iteration is completed.
[0061] In response to the received barrier instruction, the task manager 310 may instruct the nth core to stop the convolution calculation. For example, the task manager 310 may send a "wait" instruction to the nth core to cause the nth core to execute the function wait(i,n), so that the convolution calculation may be stopped. Although the convolution calculation is stopped on the nth core, the nth core may still wait for further instructions from the task manager 310.
[0062] It should be appreciated that the results of the convolution computations (e.g., intermediate feature maps) may be stored in local memory of each core (e.g., local memory unit 306).
[0063] In some embodiments, after barrier instructions have been received from all cores, the high-level program 404 may be executed. In some embodiments, the high-level program 404 may be executed on, for example, the task manager 310. For example, when the task manager determines that the convolution calculation of the first iteration is complete after receiving barrier instructions from all cores, the task manager 310 may select one of the cores (e.g., core 2024) and provide a "collect&reduce" instruction to the core to collect data (e.g., intermediate feature maps) from other cores and reduce them into one feature map, which may be used as an input to the next layer. For example, the core 2024 may retrieve the intermediate feature map from the local memory of each core in association with the task manager 310 and store the retrieved intermediate feature map in the local memory unit 306 of the core 2024. Therefore, after the intermediate feature maps have been retrieved from the other cores, the core 2024 may have all the intermediate feature maps for further processing.
[0064] In addition to sending the "collect&reduce" instruction, after having received the barrier instruction from all cores, the task manager 310 may send a "resume" instruction to all cores to resume the convolution computation for the second iteration. As discussed above, each core may include multiple processing units and reduction elements. Therefore, the collection and reduction of the first iteration and the convolution computation of the second iteration may be performed in parallel. This allows a synchronization scheme between the task manager and the cores, thereby establishing dependencies between all cores when cross-core data exchange is required.
[0065] In some embodiments, the compiler may be responsible for generating and sending "wait" instructions, "collect & reduce" instructions, and "resume" instructions to each core. It should be appreciated that the "wait" instructions and "collect & reduce" instructions may be compiled into hardware instructions.
[0066] Table 1 below also illustrates the process 400 in Figure 3 As discussed above, in the core 2024, the computation unit 302 may perform convolution computations, and the interface 304, the local memory unit 306, and the reduction simple unit 308 may perform collection and reduction. Therefore, in Table 1, the interface, the local memory unit, and the reduction simple unit may be collectively referred to as an R&C unit.
[0067]
[0068] Table 1
[0069] As shown in Table 1, in the i-th iteration (i.e., the first row), each core (e.g., core 2024, 2026, 2028, or 2030) performs a convolution calculation on a set of matrices to generate a feature map, and signals, for example, a barrier instruction to a task manager (e.g., Figure 3 The result of the convolution calculation may be stored locally in each core. After the task manager receives all barrier instructions from all four cores, the task manager may generate a "resume" instruction and a "wait" instruction and issue them to the processing unit and R&C unit of each core, respectively. Upon receiving the "resume" instruction, the processing unit may perform the convolution calculation of the i+1th iteration. At the same time, the R&C unit of core 2024 may collect feature maps from other cores and perform reduction on the collected feature maps. As shown in Table 1, the convolution calculation of the i+1th iteration and the reduction of the feature map of the i-th iteration are performed in parallel.
[0070] In addition, referring to Table 1, when the number of cores associated with the task manager is N and the number of matrix sets to be processed is K, for the kth iteration, the (k%N)th core among the cores may act as a reduction core, which retrieves external results associated with the matrix set from other cores and generates a reduced matrix. k is a positive integer less than or equal to K. It should be appreciated that other cores may further transmit local results to the (k%N)th core. "k%N" represents the remainder after dividing k by N. For example, as shown in Table 1, when N=4 and k=3, the 3rd core (e.g., core 320) will retrieve external results and generate a reduced matrix
[0071] In Table 1, among the cores, the first core (e.g., core 2024) generates a first reduced matrix, the second core (e.g., core 2026) generates a second reduced matrix, the third core (e.g., core 2028) generates a third reduced matrix, and the fourth core (e.g., core 2030) generates a fourth reduced matrix, and the first reduced matrix, the second reduced matrix, the third reduced matrix, and the fourth reduced matrix can be combined into a reduced matrix set in an interleaved manner.
[0072] Figure 5 1 shows a method 500 for executing a parallel computing task set on a set of matrices by multiple cores coupled to a task manager according to an embodiment of the present disclosure. The method 500 may be performed by, for example, Figure 3 The method 500 may be implemented by one of the cores 2024, 2026, 2028 and 2030. The method 500 may include the following steps.
[0073] In step 502, the processing unit of the core may generate a first output feature map corresponding to a first computing task in the parallel computing task set. The first output feature map may be generated by performing a convolution process on a first matrix in the matrix set. The processing unit of the core may also generate a second output feature map corresponding to a second computing task. When the first output feature map is generated, the core may issue a barrier instruction to the task manager and stop the generation of the second output feature map. It should be appreciated that each core may issue its own barrier instruction to the task manager.
[0074] In step 504, the interface of the core may receive one or more instructions from the task manager to collect external output feature maps corresponding to the parallel computing task set from other cores in the plurality of cores. It should be appreciated that each of the plurality of cores may perform processing on a portion of the matrix set and generate an output feature map.
[0075] In step 506, the core's reducer may generate a reduced feature map based on the first output feature map and the external output feature map. As discussed above, the reduced feature map may be used as input for the next iteration of the parallel computing task.
[0076] Among the multiple cores, the first core may generate a first reduced output feature map, the second core may generate a second reduced output feature map, the third core may generate a third reduced output feature map, the fourth core may generate a fourth reduced output feature map, and the first reduced output feature map, the second reduced output feature map, the third reduced output feature map, and the fourth reduced output feature map are combined into a reduced output feature map set in an interleaved manner.
[0077] In some embodiments, the task manager may determine whether a barrier instruction is received from each of the plurality of cores. In response to determining that a barrier instruction is received from each of the plurality of cores, the task manager may then send a recovery instruction to each of the plurality of cores to resume generation of the second output feature map.
[0078] In addition, the task manager may also select a reduction core among the plurality of cores to receive the external output feature map and generate the reduced feature map in parallel with the generation of the second output feature map. For example, the number of cores may be N, the number of matrix sets may be K, and for the kth iteration, the reduction core is the (k%N)th core among the plurality of cores. It should be appreciated that k is a positive integer less than or equal to K.
[0079] Figure 6 600 is illustrated for managing tasks by a task manager coupled to multiple cores according to an embodiment of the present disclosure. The method 600 may be performed by, for example, Figure 3 The method 600 may be implemented by the task manager 310. The method 600 may include the following steps.
[0080] In step 602, the task manager 310 may instruct a core (e.g., core 2024) among the plurality of cores to generate a first output feature map corresponding to a first computing task among the parallel computing task set. For example, the first output feature map may be generated by performing a convolution process on a data matrix. As discussed above, when the first output feature map is generated on the core, the core may issue a barrier instruction to the task manager 310 and may stop the generation of a second output feature map corresponding to a second computing task. In some embodiments, the task manager 310 may stop processing based on the barrier instruction.
[0081] Thus, in step 604, the task manager 310 may receive barrier instructions from other cores of the plurality of cores. It should be appreciated that the barrier instructions sent by the cores in step 602 are also received by the task manager 310. The barrier instructions may act as a synchronization scheme.
[0082] In step 606, the task manager 310 may send a resume instruction to the core to resume the generation of the second output feature map. After receiving all barrier instructions from all multiple cores, the task manager 310 may determine that the first computing task has been completed on each of the multiple cores and send a resume instruction to each core to resume the generation of the second output feature map.
[0083] In step 608, the task manager 310 may select a core as a reduction core to receive an external output feature map corresponding to a parallel computing task set from other cores to generate a reduced feature map based on the first output feature map and the external output feature map. The generation of the second output feature map occurs in parallel with the generation of the reduced feature map. For example, after receiving a barrier instruction from all cores in the plurality of cores, the task manager 310 may issue an instruction to the selected reduction core to collect external output feature maps from cores other than the reduction core to generate a reduced feature map. Because the generation of the second output feature map is also resumed in step 606, the generation of the second output feature map and the generation of the reduced feature map may occur in different orders or in parallel.
[0084] The embodiments of the present disclosure also provide a computer program product, which may include a non-transitory computer-readable storage medium storing computer-readable program instructions for causing a processor to execute the above method.
[0085] A computer-readable storage medium may be a tangible device that can store instructions for use by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of computer-readable storage media includes the following: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as a punch card or a raised structure in a groove on which a record is recorded, and any suitable combination of the foregoing.
[0086] The computer-readable program instructions for performing the above method can be assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data or source code or object code written in any combination of one or more programming languages (comprising object-oriented programming languages) and conventional process programming languages. The computer-readable program instructions can be performed completely on a computer system as an independent software package, or partly on a first computer and partly on a second computer away from the first computer. In the latter scenario, the second remote computer can be connected to the first computer by a network comprising any type of a local area network (LAN) or a wide area network (WAN).
[0087] Computer readable program instructions may be provided to a processor of a computer or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus create means for implementing the above-described methods.
[0088] The flowcharts and diagrams in the figure illustrate the architecture, functionality and operation of the possible implementation of the device, method and computer program product according to various embodiments of this specification. In this regard, the blocks in the flowchart or figure may represent software programs, code segments or parts, and the code includes one or more executable instructions for implementing specific functions. It should also be noted that the functions indicated in each frame may not occur in the order indicated in the figure in some alternative implementations. For example, depending on the functionality involved, two frames shown in succession can actually be executed substantially simultaneously, or sometimes these frames can be executed in reverse order. It should also be noted that each frame of the figure and / or flowchart and the combination of each frame in the figure and flowchart can be implemented by a combination of a dedicated hardware-based system or dedicated hardware and computer instructions that performs the specified function or behavior.
[0089] It should be appreciated that certain features of this specification described in the context of separate embodiments for clarity may also be provided in combination in a single embodiment. Conversely, various features of this specification described in the context of a single embodiment for brevity may also be provided individually or in any suitable subcombination or as appropriate in any other described embodiment of this specification. Certain features described in the context of various embodiments should not be considered essential features of those embodiments unless the embodiment does not function without those elements.
[0090] Although the present specification has been described in conjunction with the specific embodiments of the present specification, it is apparent that many alternatives, modifications and variations will be apparent to those skilled in the art. For example, although some embodiments are described using the processing of input data matrices as examples, the described systems and methods can be applied to any parallel computing task. Therefore, it is intended to include all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.
Claims
1. A system for performing parallel computing, include: task manager; and A plurality of cores, the plurality of cores being coupled to the task manager and configured to respectively execute parallel computing task sets based on instructions from the task manager, wherein each of the plurality of cores further comprises: A processing unit, the processing unit being configured to generate a first output feature map corresponding to a first computing task in the parallel computing task set; the first output feature map being generated by performing a convolution process on a first matrix in the matrix set; an interface configured to receive one or more instructions from the task manager to collect external output feature maps corresponding to the set of parallel computing tasks from other cores of the plurality of cores; A reduction unit is configured to generate a reduced feature map based on the first output feature map and the received external output feature map.
2. The system of claim 1, wherein the task manager is configured to instruct the core to simultaneously perform generation of a second output feature map corresponding to a second computing task and generation of the reduced feature map.
3. The system according to claim 2, wherein when the first output feature map is generated, the processing unit is further configured to issue a barrier instruction to the task manager and stop the generation of the second output feature map. 4 . The system of claim 3 , wherein the task manager is further configured to receive the barrier instruction from each of the plurality of cores. 5 . The system of claim 4 , wherein in response to determining that the barrier instruction is received from each of the plurality of cores, the task manager is configured to send a resume instruction to the core to resume generation of the second output feature map.
6. The system of claim 2, wherein the task manager is further configured to select a reduction core among the plurality of cores to receive the external output feature map and generate the reduced feature map in parallel with generation of the second output feature map.
7. The system of claim 6, wherein the first output feature map is generated by performing a convolution process on a first matrix in a matrix set.
8. The system of claim 7, wherein the number of cores is N, the number of matrix sets is K, and for the kth iteration, the reduced core is the (k%N)th core among the plurality of cores, where k is a positive integer less than or equal to K; in, k%N represents the remainder after dividing k by N.
9. The system according to claim 1, in, Among the multiple cores, the first core generates a first reduced output feature map, the second core generates a second reduced output feature map, the third core generates a third reduced output feature map, and the fourth core generates a fourth reduced output feature map, and the first reduced output feature map, the second reduced output feature map, the third reduced output feature map, and the fourth reduced output feature map are combined into a reduced output feature map set in an interleaved manner.
10. The system of claim 2, wherein the storage unit comprises a first port for generating the second output feature map and a second port for generating the reduced output feature map.
11. A method for executing a set of parallel computing tasks at a core of a plurality of cores coupled to a task manager, include: Generating, by a processing unit of the core, a first output feature map corresponding to a first computing task in the parallel computing task set; The first output feature map is generated by performing a convolution process on a first matrix in the matrix set; receiving, by an interface of the core, one or more instructions from the task manager to collect external output feature maps corresponding to the set of parallel computing tasks from other cores of the plurality of cores; A reduced feature map is generated by a reduced element of the core based on the first output feature map and the received external output feature map.
12. The method according to claim 11, further comprising: include: The generation of a second output feature map corresponding to a second computing task and the generation of the reduced feature map are simultaneously performed by the processing unit of the core.
13. The method according to claim 12, further comprising: include: In response to the first output feature map being generated, issuing a barrier instruction to the task manager, and Stop generating the second output feature map. 14 . The method of claim 13 , wherein the task manager is further configured to determine whether the barrier instruction is received from each of the plurality of cores. 15 . The method of claim 14 , wherein in response to determining that the barrier instruction is received from each of the plurality of cores, the task manager is configured to send a resume instruction to the core to resume generation of the second output feature map.
16. The method according to claim 12, in, A reduction core among the plurality of cores is selected by the task manager to receive the external output feature map and generate the reduced feature map in parallel with generation of the second output feature map.
17. The method according to claim 16, wherein the first output feature map is generated by performing a convolution process on a first matrix in a matrix set.
18. The method of claim 17, wherein the number of cores is N, the number of matrix sets is K, and for the kth iteration, the reduced core is the (k%N)th core among the plurality of cores, wherein k is a positive integer less than or equal to K; in, k%N represents the remainder after dividing k by N.
19. The method of claim 11, wherein among the plurality of cores, a first core generates a first reduced output feature map, a second core generates a second reduced output feature map, a third core generates a third reduced output feature map, and a fourth core generates a fourth reduced output feature map, and the first reduced output feature map, the second reduced output feature map, the third reduced output feature map, and the fourth reduced output feature map are combined into a reduced output feature map set in an interleaved manner.
20. A method for managing tasks by a task manager coupled to a plurality of cores, include: Instructing a core among the plurality of cores to generate a first output feature map corresponding to a first computing task in the parallel computing task set, wherein after the first output feature map is generated, the core issues a barrier instruction to the task manager and stops generating a second output feature map corresponding to a second computing task; the first output feature map is generated by performing a convolution process on a first matrix in the matrix set; receiving the barrier instruction from other cores of the plurality of cores; After receiving the barrier instruction, sending a resume instruction to the core to resume generation of the second output feature map; The core is selected as a reduction core to receive the external output feature map corresponding to the parallel computing task set from other cores to generate a reduced feature map based on the first output feature map and the external output feature map, wherein The generation of the second output feature map occurs in parallel with the generation of the reduced feature map.
