Computing systems, methods for data broadcasting and data reduction, and storage media

Abstract

Embodiments of this disclosure relate to a computing system and a method and storage medium for data broadcasting and data reduction. The computing system includes multiple computing cores disposed in one or more computing devices, one or more on-chip networks respectively located in the one or more computing devices, and a memory. Each computing core in each computing device is connected to a corresponding on-chip network for communication with each other via the on-chip network. The memory includes multiple memory objects created for input data objects to be processed. Each memory object stores data about the data object to be processed, and the multiple memory objects are configured to use the same offset address to store the same data. Each memory object is configured to allow access only to the computing core associated with that memory object. This reduces the number of data transfers required during data processing, thereby increasing the speed of data processing.

Description

Technical Field

[0001] The embodiments of this disclosure generally relate to the field of information processing, and more specifically to a computing system and a method and storage medium for performing data broadcasting and data reduction. Background Technology

[0002] In multi-core computing systems, deep learning tasks are batched and processed in parallel across multiple cores. Specifically, in each iteration of a deep neural network, the input data object (e.g., an input matrix) is divided into multiple equal parts, and then each part undergoes independent forward and backward propagation operations on multiple cores to calculate its gradient. After each iteration, the gradients are combined, the parameters of the deep neural network are updated, and then the next iteration begins.

[0003] Data reduction and data broadcasting are two frequently used processes in batch processing of deep learning tasks. In traditional approaches, a single memory object is created for the input data object to be processed. This memory object is shared by all computation cores. Therefore, when broadcasting or reducing related data of the data object is required, various related data need to be transferred back and forth between the relevant computation cores multiple times, resulting in long latency and high bandwidth consumption of the computation cores, leading to serious performance problems.

[0004] For example, in a traditional scheme, when performing data reduction, all computing cores in the reduction group need to send the corresponding data generated by them and required to participate in the reduction operation to a specific computing core in the group. After receiving the data, the specific computing core uses its reduction calculation unit to perform reduction calculations on the received data to obtain the corresponding reduction calculation results, and sends a reduction completion message to all computing cores in the reduction group. After receiving the reduction completion message, all computing cores in the reduction group also need to send a reduction calculation result read request to the specific computing core through the read network. The reduction calculation result read request must include the address of the corresponding computing core. After receiving the read requests, the specific computing core then broadcasts a read response including the reduction calculation result to all computing cores in the reduction group through the read network. In addition to including the reduction calculation result, the read response must also include the addresses of all computing cores in the reduction group. For example, during data broadcasting, all computational cores in the broadcast group need to send a data read request, including its address, to a specific computational core via the read network. Upon receiving the read request, the specific computational core broadcasts a read response, including the corresponding data, to all computational cores in the broadcast group via the read network. This read response, in addition to the data, also includes the addresses of all computational cores in the broadcast group. In this reduction and broadcasting process, because multiple round trips of data transmission are required, the speed is slow, the latency is long, and each computational core consumes a certain amount of bandwidth to complete the processing. Furthermore, since the read process uses the read network, which is used relatively frequently in batch processing of deep learning tasks, it can easily lead to an imbalance in the read and write bandwidth between computational cores. Summary of the Invention

[0005] To address the aforementioned problems, this disclosure provides a computing system, a method for data broadcasting and data reduction, and a storage medium that helps reduce the number of data transmissions required during data processing, reduce latency and bandwidth consumption, thereby improving data processing speed.

[0006] According to a first aspect of this disclosure, a computing system is provided, comprising a plurality of computing cores disposed in one or more computing devices, one or more on-chip networks respectively located in the one or more computing devices, and a memory, wherein the number of the one or more computing devices is less than the number of the plurality of computing cores, and each computing core in each computing device is connected to a corresponding on-chip network for communicating with each other through the on-chip network; the memory includes a plurality of memory objects created for input data to be processed, each memory object storing data about the data to be processed, and the plurality of memory objects being configured to use the same offset address to store the same data; the number of the plurality of memory objects is the same as the number of the plurality of computing cores, and each memory object is configured to allow access only by the computing core associated with the memory object.

[0007] According to a second aspect of this disclosure, a method for broadcasting data is provided, the method being performed by a computing system according to a first aspect of this disclosure, the method comprising: for each computing core, if new data about the data object to be processed is generated locally during processing of the data object to be processed, sending the same first instruction to multiple computing cores in a broadcast group to which the computing core belongs, storing the new data at a specified offset address, the first instruction including a group identifier of the broadcast group, the new data to be broadcast to the multiple computing cores, and the offset address; and at each of the multiple computing cores, in response to receiving the first instruction, storing the new data at the offset address in an associated memory object for subsequent processing.

[0008] In a third aspect of this disclosure, a method for performing data reduction is provided, the method being performed by a computing system according to a first aspect of this disclosure, the method comprising: for each computing core, if new data about the data object to be processed is generated locally during processing of the data object to be processed, sending the same first instruction to multiple computing cores in a broadcast group to which the computing core belongs, storing the new data at a specified offset address, the first instruction including a group identifier of the broadcast group, the new data to be broadcast to the multiple computing cores, and the offset address; and at each of the multiple computing cores, in response to receiving the first instruction, storing the new data at the offset address in an associated memory object for subsequent processing.

[0009] In a fourth aspect of this disclosure, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the method according to a second or third aspect of this disclosure.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0011] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0012] Figure 1 A schematic diagram of a computing system 100 according to some embodiments of the present disclosure is shown.

[0013] Figure 2 A schematic diagram of a computing system 200 according to other embodiments of the present disclosure is shown.

[0014] Figure 3 A schematic diagram of the arrangement of the computing core according to an embodiment of the present disclosure is shown.

[0015] Figure 4 A flowchart of a method 400 for broadcasting data according to an embodiment of the present disclosure is shown.

[0016] Figure 5 A flowchart of a method 500 for performing data reduction according to an embodiment of the present disclosure is shown. Detailed Implementation

[0017] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0018] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0019] As mentioned above, in the traditional approach, only a single memory object is created in memory for the input data object to be processed. This memory object is shared by all computing cores. Therefore, when broadcasting or reducing the data related to the data object to be processed, the relevant data needs to be transferred back and forth between the relevant computing cores multiple times, resulting in long latency and serious performance problems.

[0020] To at least partially address one or more of the aforementioned problems and other potential issues, exemplary embodiments of this disclosure propose a computing system for data processing, including multiple computing cores disposed in one or more computing devices, one or more on-chip networks respectively located in the one or more computing devices, and a memory, wherein the number of the one or more computing devices is less than the number of the multiple computing cores, and each computing core in each computing device is connected to a corresponding on-chip network for communication with each other through the on-chip network; the memory includes multiple memory objects created for input data objects to be processed, each memory object storing data about the data object to be processed, and the multiple memory objects are configured to use the same offset address to store the same data; the number of multiple memory objects is the same as the number of multiple computing cores, and each memory object is configured to allow access only to the computing core associated with the memory object. In this way, it helps to reduce the number of data transfers required during data processing, reduce latency and bandwidth consumption, thereby improving the speed of output processing.

[0021] Figure 1 A schematic diagram of a computing system 100 according to some embodiments of the present disclosure is shown. The computing system 100 may include memory (i.e., such as...) Figure 1 The system memory 110 shown), one or more on-chip networks 120, and multiple computing cores (i.e., such as...) Figure 1 The computational kernels shown are 1 to n. In this disclosure, n can be any integer greater than or equal to 2.

[0022] Computational cores 1 through n can be located in one or more computing devices, and the number of computing devices is less than the number of computing cores. For example, ... Figure 3 As shown, the multiple computing cores can be respectively disposed in computing device 1, computing device 2, computing device 3, and computing device 4, with each computing device having one or more computing cores (for example, each computing device having computing cores 1 to m, where m is an integer greater than or equal to 1). It should be noted that although... Figure 3The illustration shows computing cores arranged across four computing devices; however, this is merely an example. In actual use, the computing system 100 may include more or fewer computing devices, and the number of computing cores included in each computing device may be the same or different. Computing devices 1-4 include, but are not limited to, chips packaged with multiple wafers, such as central processing unit chips (CPU chips), graphics processing unit chips (GPU chips), artificial intelligence chips (AI chips), high-performance computing chips (HPC chips), etc. When a computing device includes multiple wafers, the individual computing cores disposed on the computing devices may be arranged on different wafers. In this disclosure, different computing devices can communicate with each other via a bus, which may be implemented, for example, through a high-speed I / O bus or Blink.

[0023] One or more Network On-Chip (NOC) 120s are located in one or more of the aforementioned computing devices, meaning that each of the one or more computing devices includes one NOC 120. Each NOC 120 is used to interconnect or connect the various computing cores in the corresponding computing device. In this disclosure, each computing core in each computing device is connected to its corresponding NOC to communicate with each other through the NOC. Furthermore, the NOCs in the different computing devices of the computing system 100 are interconnected via a bus, enabling the various computing cores located on these different computing devices to communicate with each other and collaboratively complete related data processing operations. NOCs typically include read networks and write networks. Read networks are primarily used, for example, to read data from one computing core, while write networks are primarily used, for example, to write (e.g., store) data from one computing core to another.

[0024] Each computational core can be configured to perform one or more general or custom data processing operations. For example, a computational core can be configured to perform logical, logarithmic, exponential, multiplicative, comparison, trigonometric, matrix operations, and / or other suitable general data processing operations. Alternatively or additionally, a computational core can also be configured to perform neural network model training and inference, phylogenetic inference, genome sequencing, climate modeling, weather forecasting, video / audio processing, and / or other suitable custom operations. A computational core may include a reduction engine (also referred to as a reduction computation unit, not shown in the figure). A reduction engine can be configured to perform reduction operations (hereinafter also referred to as reduction computation), which include, but are not limited to, addition and multiplication.

[0025] In this disclosure, the multiple computing cores included in the computing system 100 can be divided into one or more computing core groups. Each computing core group can be a broadcast group or a reduction group. In some implementations, computing core groups can be divided according to the computing device where each computing core is located, that is, computing cores located on the same computing device are divided into one computing core group. Thus, in Figure 3 In the example shown, four core groups can be defined: core group 1 corresponding to computing device 1, core group 2 corresponding to computing device 2, core group 3 corresponding to computing device 3, and core group 4 corresponding to computing device 4. In other implementations, core groups can be defined based on the computing node where each core resides; that is, multiple cores located on the same computing node are grouped into one core group. Typically, a single computing node can include multiple computing devices, and each computing device can have one or more cores. Assuming that in… Figure 3 In this embodiment, computing devices 1 and 2 belong to the first computing node, and computing devices 3 and 4 belong to the second computing node. Therefore, when dividing the computing core groups according to the computing nodes, two computing core groups can be defined: the computing cores in computing devices 1 and 2 belong to one computing core group, and the computing cores in computing devices 3 and 4 belong to another computing core group. In this disclosure, each computing core group is assigned a corresponding group identifier to distinguish it from other computing core groups, thereby making it easier to determine the scope of computing cores that perform operations such as broadcasting or reducing data.

[0026] The memory included in the computing system 100 (i.e., system memory 110) may include multiple memory objects created for input data objects to be processed (e.g., a data matrix of an input image to be processed, etc.), one memory object for each computing core, for example... Figure 1 The illustrated embodiment uses memory objects 1 to n. Each created memory object should store data about the data object to be processed, such as each data point in the data matrix of the image to be processed. The memory objects are configured to use the same offset address to store the same data. For example, the offset address for storing a particular data point of the data object to be processed in memory object 1, memory object 2, and memory object n are all the same.

[0027] In this disclosure, the number of memory objects created should be the same as the number of computing cores. For example, in Figure 1The system includes n computing cores, therefore n memory objects should be created, one memory object for each computing core. In this disclosure, each memory object is configured to be accessed only by the computing core associated with that memory object among the multiple computing cores in the computing system 100. For example, memory object 1 is accessed only by computing core 1, memory object 2 is accessed only by computing core 2, and memory object n is accessed only by computing core n.

[0028] Although Figure 1 In the illustrated embodiment, multiple memory objects created for the input data object to be processed are all created on the system memory 110. For example, memory object 1 associated with computing core 1, memory object 2 associated with computing core 2, and memory object n associated with computing core n are all created on the system memory 110. However, in practical applications, if a computing core includes on-chip memory and that on-chip memory has sufficient storage space to create memory objects for the input data object to be processed, then the memory object associated with that computing core can be created in the on-chip memory of that computing core, instead of in the system memory. For example, by way of example only, if computing core 1 in the figure includes on-chip memory and that on-chip memory has sufficient storage space to create memory objects for the input data object to be processed, then memory object 1 associated with computing core 1 can be created in the on-chip memory of computing core 1, instead of in the system memory 110. The same applies to other computing cores.

[0029] In this disclosure, system memory 110 is also connected to one or more on-chip networks 120, enabling each computing core to access various memory objects created in system memory 110 via a corresponding on-chip network 120. In some embodiments, system memory 110 is connected to various on-chip networks 120 respectively disposed in these computing devices via a bus between the computing devices.

[0030] In this disclosure, each memory object occupies a storage segment of a corresponding memory. Each created memory object has corresponding configuration information. This configuration information may include the base address allocated to the corresponding memory object, and may also include the length, width, and height of the storage segment occupied by the corresponding memory object. The configuration information of each memory object is provided only to the computational core associated with that memory object among multiple computational cores, so that the associated computational core can determine the specific storage address of the corresponding data based on the corresponding base address and the offset address received through the instruction.

[0031] In such Figure 1 In the illustrated embodiment, the computing system 100 may further include multiple cache memories (e.g., such as...). Figure 1The cache memory shown is 1 to n. In some implementations, the cache memory can be, for example, a level 2 cache. The number of cache memories is the same as the number of computing cores, for example in... Figure 1 It includes n computing cores, and therefore correspondingly has n cache memories. Each computing core is connected to a cache memory local to that core. For example, such as Figure 1 As shown, computing core 1 is connected to a local cache memory 1, computing core 2 is connected to a local cache memory 2, and computing core n is connected to a local cache memory n. In this disclosure, the local cache memory of a computing core is closer to the computing core than other cache memories in the computing system 100. Logical pages of memory objects associated with any computing core are mapped to the cache memory connected to that computing core, allowing the computing core to access the associated memory object through the corresponding cache memory. For example, as... Figure 1 As shown, the logical pages of memory object 1 associated with computing core 1 are mapped to cache memory 1 connected to computing core 1, the logical pages of memory object 2 associated with computing core 2 are mapped to cache memory 2 connected to computing core 2, and the logical pages of memory object n associated with computing core n are mapped to cache memory n connected to computing core n.

[0032] Of course, in some embodiments, the computing system 100 may not include cache memories 1 to n, and the computing cores 1 to n may directly access the associated memory objects in the system memory 110.

[0033] Figure 2 A schematic diagram of a computing system 200 according to other embodiments of the present disclosure is shown. For example... Figure 2 As shown, the computing system 200 includes computing cores 1 to n and one or more on-chip networks 220, and each computing core includes on-chip memory, i.e., as shown in the figure. Figure 1 The on-chip memories 1 to n are shown, where n is an integer greater than or equal to 2. Computational cores 1 to n are connected to one or more on-chip networks 220. Computational cores 1 to n may also be located in one or more computing devices, and the number of computing devices is less than the number of computing cores. In this disclosure, each computing core located in each computing device is connected to the on-chip network of that computing device. Furthermore, the one or more computing devices can communicate with each other via a bus, and the on-chip networks of different computing devices are interconnected via this bus, thereby enabling the computing cores located on different computing devices in the computing system 200 to communicate with each other to collaboratively complete related data processing operations. Figure 2In the illustrated embodiment, multiple memory objects are created for the input data object to be processed (e.g., a data matrix of the input image to be processed, etc.) and are respectively created on the on-chip memory of the associated computing core. Specifically, the memory object associated with computing core 1 is created on on-chip memory 1 of computing core 1, the memory object associated with computing core 2 is created on on-chip memory 3 of computing core 3, and the memory object associated with computing core n is created on on-chip memory n of computing core n. However, it should be noted that the condition for creating memory objects on the on-chip memory of the computing core is that the on-chip memory has sufficient storage space to store the various data associated with the data object to be processed; otherwise, it is impossible to create suitable memory objects thereon. Other details regarding computing cores 1 to n, on-chip network 220, and memory objects, etc., can be combined with the above. Figure 1 The contents of computing cores 1 to n, on-chip network 120, and memory objects described in the embodiments are the same, and will not be repeated here.

[0034] In addition, although Figure 1 In the embodiments shown, the memory objects associated with each computing core are created in system memory, and... Figure 2 In the illustrated embodiments, memory objects associated with each computing core are created separately in the on-chip memory of the respective computing core. However, in other embodiments, the computing system may include both system memory and at least one on-chip memory included in at least one of the plurality of computing cores. Therefore, in these embodiments, memory objects associated with at least one computing core may be created separately in the on-chip memory of the respective computing core (provided that the on-chip memory has sufficient storage space), while memory objects associated with each of the other computing cores are created in system memory.

[0035] Through the above technical solution, multiple memory objects are created for different computing cores for the data object to be processed. These memory objects are identical except for their base addresses; for example, they store the same data, and the offset address of each data item is the same across all memory objects. Therefore, in this disclosure, when processing the data object, each computing core can access its associated memory object based on the same offset address for each data item, without needing to transfer data back and forth between different computing cores multiple times. This significantly reduces waiting time and improves data processing efficiency.

[0036] Figure 4A flowchart of a method 400 for broadcasting data according to an embodiment of the present disclosure is shown. Method 400 may be performed in the aforementioned computing system 100 or 200. It should be understood that method 400 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0037] In step 402, for each computing core, if new data about the data object to be processed is generated locally during the processing of the data object to be processed, the same first instruction to store the new data at the specified offset address is sent to multiple computing cores in the broadcast group to which the computing core belongs.

[0038] In this disclosure, the first instruction may include a group identifier for the broadcast group, the new data to be broadcast to the multiple compute cores mentioned in step 402, and the specified offset address. The group identifier indicates the broadcast group, i.e., which compute core group the new data should be broadcast to. In this disclosure, the broadcast group refers to the compute core group to which the compute core to which the broadcast operation needs to be performed in step 402 belongs. The offset address indicates where the new data should be stored in the associated memory objects. The first instruction may also include a broadcast enable flag to indicate that the first instruction involves data broadcasting. In this implementation, the first instruction may be, for example, store.group0.broadcast addr0,data0, which indicates that new data data0 is broadcast to the compute cores in the broadcast group group0 and stored at the offset address addr0 of the associated memory objects.

[0039] In some implementations, after new data (e.g., the new data mentioned in step 402) is generated by the corresponding computing core, it is usually first registered in a register or cache of that computing core to await further processing. Therefore, the first instruction may also involve retrieving the new data from that register or cache. In such an implementation, the first instruction may be, for example: store.group0.broadcast addr0,register_N, which indicates that the new data (e.g., data0) registered in the register_N of the corresponding computing core is broadcast to each computing core in the broadcast group group0, and stored at the offset address addr0 of each associated memory object.

[0040] In some implementations, where new data does not need to be broadcast to all compute cores included in the broadcast group, the first instruction may further include a broadcast mode identifier, which indicates the broadcast mode to be used. In this disclosure, the broadcast mode is used to indicate the source compute core initiating the first instruction and the multiple destination compute cores in the corresponding broadcast group (i.e., the broadcast group identified by the group identifier included in the first instruction) that should receive the new data. In such an implementation, the first instruction may be, for example: store.group0.pattern0.broadcast addr0,register_N, which indicates that new data (e.g., data0) registered in the register_N of the corresponding compute core is broadcast from the source compute core indicated by the broadcast mode pattern0 to the respective destination compute cores in the broadcast group group0 indicated by the broadcast mode pattern0, and stored at the offset address addr0 of the respective associated memory object.

[0041] In this disclosure, since the first instruction in step 402 involves data transmission and storage, the new data can be broadcast to multiple computing cores in the on-chip network that are related to subsequent processing of the data object to be processed via the write network.

[0042] In step 404, at each of the plurality of computing cores mentioned in step 402, in response to receiving the first instruction, the new data is stored at the offset address of the associated memory object for subsequent processing.

[0043] For example, new data can be stored at that offset address in the associated memory object via the corresponding on-chip memory or the corresponding cache memory, depending on whether the associated memory object is created on the corresponding on-chip memory or on system memory.

[0044] Upon receiving the first instruction, each computing core in the broadcast group stores the received new data at the offset address of its associated memory object for subsequent processing.

[0045] By employing the above techniques, during data broadcasting, new data can be written to all relevant computing cores using a single instruction specifying a single offset address, eliminating the need for multiple round trips. This reduces latency and bandwidth consumption by each computing core, thereby improving broadcast efficiency. Furthermore, in this disclosure, the broadcasting is accomplished through the write network within the on-chip network. Since the write network is used less frequently than the read network, this helps reduce the imbalance between the write and read networks.

[0046] Figure 5 A flowchart of a method 500 for data reduction according to an embodiment of the present disclosure is shown. Method 500 may be performed in the aforementioned computing system 100 or 200. It should be understood that method 500 may also include additional boxes not shown and / or the boxes shown may be omitted, and the scope of the present disclosure is not limited in this respect.

[0047] In step 502, for each computing core, if new data that needs to participate in the reduction calculation is generated locally during the calculation of the data object to be processed, the same second instruction is sent to multiple computing cores in the reduction group to which the computing core belongs to perform the reduction calculation and store the reduction calculation result at the specified offset address.

[0048] In this disclosure, since the second instruction in step 502 involves data transmission and storage, the new data can be broadcast to the computing core and multiple related computing cores via the write network in the on-chip network for reduction calculations to be performed separately.

[0049] In this disclosure, the second instruction may include a group identifier of the reduction group mentioned in step 502, the new data to be involved in the reduction calculation, and the offset address. The group identifier is the identifier of the reduction group, indicating that the reduction calculation needs to be performed among data generated by multiple computation cores in the reduction group. In this disclosure, the reduction group may be a computation core group to which the computation core that generated the new data in step 502 belongs. The offset address indicates where the reduction calculation result should be stored in the associated memory object. The second instruction may also include a broadcast enable flag to indicate that the operation involves data broadcasting. In this implementation, the second instruction may be, for example, reduce.group0.broadcastaddr0,data0, which indicates that the new data data0 is broadcast to the computation cores in the reduction group group0 for reduction calculation to be performed at these computation cores and the reduction calculation result is stored at the offset address addr0 of the associated memory object.

[0050] In some implementations, after new data (e.g., the new data in step 502) is generated by the corresponding computation core, it is usually first registered in a register or buffer of that computation core to await further processing. Therefore, before broadcasting the new data based on the second instruction for reduction computation, the new data needs to be retrieved from the register or buffer. In this implementation, the second instruction may be, for example, reduce.group0.broadcast addr0, register_N. This second instruction indicates that the new data (e.g., data0) registered in the register_N of the corresponding computation core is broadcast to each computation core in the reduction group group0, so that reduction computation can be performed at these computation cores and the reduction computation result can be stored at the offset address addr0 of the associated memory object.

[0051] In some implementations, where the new data does not require reduction computation in all computation cores of the reduction group indicated by the group identifier in the second instruction, the second instruction may also include a broadcast mode identifier. The broadcast mode identifier indicates the broadcast mode to be used. Each broadcast mode indicates the source computation core initiating the second instruction and the multiple destination computation cores in the reduction group that should participate in the corresponding reduction computation. In this implementation, the second instruction may be, for example: reduce.group0.pattern0.broadcast addr0,register_N, which indicates that new data (e.g., data0) registered in register N of the corresponding computation core is broadcast from the source computation core (i.e., the corresponding computation core) indicated by broadcast mode pattern0 to the various destination computation cores in the multiple computation cores of reduction group group0 indicated by broadcast mode pattern0, so that reduction computation can be performed at these computation cores and the reduction computation result can be stored at the offset address addr0 of the associated memory object.

[0052] In this disclosure, a similar second instruction is executed at each of the multiple computational cores in the reduction group mentioned in step 502, except that the data sent by these computational cores via the second instruction is different, and this new data is generated by the respective computational core.

[0053] In step 504, at each of the plurality of computing cores mentioned in step 502, in response to receiving a plurality of new data respectively from the plurality of computing cores, the reduction calculation is performed on the plurality of new data to obtain the corresponding reduction calculation result, and the reduction calculation result is stored at the offset address in the associated storage object for subsequent processing.

[0054] In this disclosure, reduction calculations may include, for example, addition, multiplication, and other calculations. The specific reduction calculation to be performed may also be specified, for example, in a corresponding second instruction. Each computation core may perform the reduction calculation on its respective reduction calculation unit.

[0055] In addition, the same reduction calculation needs to be performed at each computation core that needs to participate in the reduction calculation, and the corresponding reduction calculation result will also be stored at the specified offset address.

[0056] In this disclosure, the computation result may be stored at the offset address in the associated memory object, for example, via the corresponding on-chip memory or the corresponding cache memory, depending on whether the associated memory object is created on the corresponding on-chip memory or on the system memory.

[0057] Through the above technical solution, since multiple identical memory objects are created for the data objects to be processed, with one memory object per computing core, the data to be involved in the reduction calculation can be broadcast to each relevant computing core via the same second instruction at the same offset address. This allows the reduction calculation to be performed at each relevant computing core in the reduction group to obtain the corresponding reduction result. Furthermore, the obtained reduction result can be stored at the same offset address of each relevant memory object, eliminating the need for multiple round trips. This reduces latency and bandwidth consumption by each computing core, improving broadcast efficiency. Additionally, in this disclosure, the broadcast is accomplished through the write network in the on-chip network. Since the write network is used less frequently than the read network, this helps reduce the imbalance between the write and read networks.

[0058] This disclosure relates to methods, apparatus, systems, electronic devices, computer-readable storage media, and / or computer program products. A computer program product may include computer-readable program instructions for performing various aspects of this disclosure.

[0059] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination thereof. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0060] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge computing devices. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to computer-readable storage media within the respective computing / processing device.

[0061] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0062] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0063] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0064] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0065] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0066] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or technical improvements to the embodiments in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A computing system comprising a plurality of computing cores disposed in one or more computing devices, one or more on-chip networks respectively located in the one or more computing devices, and a memory, wherein the number of the one or more computing devices is less than the number of the plurality of computing cores. Each computing core in the computing device is connected to a corresponding on-chip network to communicate with each other through the on-chip network; The memory includes multiple memory objects created for the input data object to be processed, each memory object storing data about the data object to be processed, and the multiple memory objects are configured to use the same offset address to store the same data; the memory includes one of the following three: system memory; Multiple on-chip memories in multiple computing cores; Or system memory and at least one on-chip memory included in at least one of the plurality of computing cores; The number of the plurality of memory objects is the same as the number of the plurality of computing cores, and each memory object is configured to allow access only to the computing cores associated with the memory object.

2. The computing system of claim 1, wherein each memory object created has corresponding configuration information, the configuration information including a base address assigned to the corresponding memory object, and the configuration information of each memory object is provided only to the computing core associated with the memory object among the plurality of computing cores.

3. The computing system according to claim 2, wherein each memory object occupies a storage segment of the corresponding memory, and the configuration information further includes the length, width and height of the storage segment occupied by each memory object.

4. The computing system of claim 1, wherein the memory includes system memory, the system memory is also connected to the one or more on-chip networks, and the plurality of memory objects are created on the system memory.

5. The computing system of claim 4, wherein the computing system further comprises a plurality of cache memories, the number of the plurality of cache memories being the same as the number of the plurality of computing cores, each computing core being connected to one of the plurality of cache memories located locally on the computing core, and logical pages of memory objects associated with any computing core being mapped to the cache memory connected to the computing core, so that the computing core can access the associated memory objects through the cache memory.

6. The computing system of claim 1, wherein the memory comprises a plurality of on-chip memories respectively included in the plurality of computing cores, and a memory object associated with each computing core is created on the on-chip memory of the respective computing core.

7. The computing system of claim 1, wherein the memory comprises system memory and at least one on-chip memory included in at least one of the plurality of computing cores. The memory object associated with at least one of the plurality of computing cores is created on the on-chip memory of the respective computing core; Memory objects associated with the other computing cores besides the at least one of the plurality of computing cores are created on the system memory.

8. The computing system according to claim 1, wherein the plurality of computing cores are divided into a plurality of computing core groups, and each computing core group has a corresponding group identifier.

9. The computing system according to claim 8, wherein the plurality of computing cores are divided into a plurality of computing core groups according to the computing device or computing node in which they are located.

10. The computing system of claim 1, wherein the one or more computing devices communicate with each other via a bus.

11. A method for broadcasting data, the method being performed in a computing system according to any one of claims 1-10, the method comprising: For each computing core, if new data about the data object to be processed is generated locally during the processing of the data object to be processed, the same first instruction is sent to multiple computing cores in the broadcast group to which the computing core belongs to store the new data at a specified offset address. The first instruction includes the group identifier of the broadcast group, the new data to be broadcast to the multiple computing cores, and the offset address. as well as At each of the plurality of computing cores, in response to receiving the first instruction, the new data is stored at the offset address in the associated memory object for subsequent processing.

12. The method of claim 11, wherein storing the new data at the offset address in the associated memory object comprises: The new data is stored at the offset address in the associated memory object via the corresponding on-chip memory or the corresponding cache memory.

13. The method of claim 11, wherein the first instruction further includes a broadcast mode identifier, the broadcast mode identifier indicating a broadcast mode to be used, the broadcast mode being used to indicate a source computing core that initiates the first instruction and a plurality of destination computing cores in the plurality of computing cores in the broadcast group that should receive the new data.

14. A method for performing data reduction, said method being performed in a computing system according to any one of claims 1-10, said method comprising: For each computing core, if new data that needs to participate in the reduction calculation is generated locally during the calculation of the data object to be processed, the same second instruction is sent to multiple computing cores in the reduction group to which the computing core belongs to perform the reduction calculation and store the reduction calculation result at the specified offset address. The second instruction includes the group identifier of the reduction group, the new data that needs to participate in the reduction calculation, and the offset address. as well as At each of the plurality of computing cores, in response to receiving a plurality of new data from the plurality of computing cores respectively, the reduction calculation is performed on the plurality of new data to obtain a corresponding reduction calculation result, and the reduction calculation result is stored at the offset address in the associated storage object for subsequent processing.

15. The method of claim 14, wherein the second instruction further includes a broadcast mode identifier indicating a broadcast mode to be used, the broadcast mode being used to indicate the source computing core that initiates the second instruction and the plurality of destination computing cores in the reduction group that should participate in the reduction computation.

16. The method of claim 14, wherein storing the reduction calculation result at the offset address in the associated storage object comprises: The reduction calculation result is stored at the offset address in the associated memory object through the corresponding on-chip memory or the corresponding cache memory.

17. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 11-13.

18. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 14-16.

Images