Universal graphics processor, computing method, computing device, medium, and program product

Abstract

The invention relates to a universal graphics processor, a computing method, a computing device, a medium and a program product. The general-purpose graphics processor comprises a first number of first-type computing cores, wherein the first-type computing cores at least comprise tensor computing units; the second type of calculation cores at least comprise vector calculation units and loading and storage units, and the second number is smaller than or equal to the first number; and the task scheduling unit is configured to issue the current task to the second type of calculation core for execution in response to the confirmation that the current task is the communication task or the vector calculation task and the state of the second type of calculation core meets a predetermined condition. According to the invention, even in a communication scene, the chip resource utilization rate and the economic benefit can be improved.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application generally relate to the field of artificial intelligence, and more particularly to a general-purpose graphics processor, a computing method for the general-purpose graphics processor, a computing device, a computer readable storage medium, and a computer program product. BACKGROUND

[0002] Conventional general-purpose graphics processors (GPGPUs) usually adopt a non-discriminatory computing core architecture for chip design, each computing core of which is integrated with a vector arithmetic logic unit (ALU), a tensor arithmetic logic unit, and a load-store unit. Such design has significant defects in typical artificial intelligence (AI) workloads: AI tasks contain both computation-intensive operations (requiring tensor ALUs) and communication-intensive operations (load-store instructions relying only on memory semantics, such as “ld / store” instructions). In the above conventional GPGPU chip, the tensor ALU usually occupies 20%-30% of the chip area of a single computing core, and the area ratio is in a continuous growth trend. However, when the GPGPU executes communication tasks, the tensor ALU is in a completely idle and cannot be utilized state. This will result in a serious waste of chip area resources, which will directly lead to a limited number of computing cores and deterioration of overall power-performance-area (PPA) indicators.

[0003] In summary, the conventional general-purpose graphics processor has the disadvantage that the tensor ALU in the communication scenario is in a completely idle state, which reduces the chip resource utilization and economic benefits. SUMMARY

[0004] The present application provides a general-purpose graphics processor, a computing method for the general-purpose graphics processor, a computing device, a computer readable storage medium, and a computer program product, which can improve the chip resource utilization and economic benefits even in the communication scenario.

[0005] According to a first aspect of the present application, a general-purpose graphics processor is provided. The general-purpose graphics processor comprises: a first number of first type computing cores, the first type computing cores comprising at least a tensor computing unit; a second number of second type computing cores, the second type computing cores comprising at least a vector computing unit and a load and store unit, the second number being less than or equal to the first number; and a task scheduling unit configured to, in response to confirming that a current task is a communication task or a vector computation task, and a state of the second type computing cores satisfies a predetermined condition, assign the current task to the second type computing cores for execution.

[0006] In some embodiments, the state of the second type of computing core satisfies predetermined conditions including: at least one second type of computing core is in an idle state.

[0007] In some embodiments, the communication task and / or vector computation task are associated with a predetermined identifier.

[0008] In some embodiments, the second quantity is configured such that the total communication bandwidth of the second type of computing cores is greater than or equal to the peak communication demand of a predetermined artificial intelligence workload.

[0009] In some embodiments, the task scheduling unit is further configured to: in response to confirming that the current task is a tensor computation task, distribute the current task to a first type of computation core for execution.

[0010] In some embodiments, the ratio of the first quantity to the second quantity is 5:1.

[0011] In some embodiments, the first quantity is 40 and the second quantity is 8.

[0012] In some embodiments, the general-purpose graphics processor further includes: direct memory access, configured to send a drive command for direct memory access in response to determining that a computing core has completed data production for a subtask of a computing task, such that direct memory access only moves data for the one subtask, the computing core including a first type of computing core and a second type of computing core, the computing task including multiple subtasks.

[0013] In some embodiments, the direct memory access is driven off-core or on-core by a first type of computing core and / or a second type of computing core.

[0014] In some embodiments, the second type of computing core is composed of a first type of computing core that does not meet the predetermined functional requirements.

[0015] According to a second aspect of the present invention, a computation method for a general-purpose graphics processing unit (GPU) is also provided. The GPU includes computation cores, which include a first type of computation core and a second type of computation core. The method includes: confirming the task type of a current task; in response to confirming that the task type of the current task is a first type of task, assigning the current task to a first type of computation core, the first type of computation core including at least a tensor computation unit; and in response to confirming that the task type of the current task is a second type of task, assigning the current task to a second type of computation core, the second type of computation core including at least a vector computation unit and a loading and storage unit.

[0016] In some embodiments, the first type of task is a tensor computation task, and the second type of task is a communication task or a vector computation task. In response to confirming that the task type of the current task is the second type of task, sending the current task to the second type of computation core includes: in response to confirming that the current task is a communication task or a vector computation task, and that the state of the second type of computation core meets a predetermined condition, sending the current task to the second type of computation core for execution.

[0017] In some embodiments, the computation method for a general-purpose graphics processor further includes: in response to determining that a computing core has completed data production for a subtask of a computation task, sending a drive command for direct memory access so that direct memory access only moves data for the one subtask, wherein the computing core includes a first type of computing core and a second type of computing core, and the computation task includes multiple subtasks.

[0018] According to a third aspect of the invention, a computing device is also provided. The computing device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the computing device to perform the method of the second aspect of the invention.

[0019] According to a fourth aspect of the present invention, a computer-readable storage medium is also provided. The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method of the second aspect of the present invention.

[0020] According to a fifth aspect of the present invention, a computer program product is also provided, comprising a computer program that, when executed by a machine, performs the method of the second aspect of the present invention.

[0021] This invention utilizes a second type of computing core to perform communication or vector tasks, and a first type of computing core to perform tensor computation tasks; thereby, it can make full use of the chip area resources of a general-purpose graphics processor, make full use of the inter-chip bandwidth of a general-purpose graphics processor, conveniently optimize chip load, and improve chip resource utilization and economic efficiency even in communication scenarios.

[0022] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0023] The above and other features, advantages, and aspects of the various embodiments of the present invention will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements.

[0024] Figure 1 A schematic diagram of a general-purpose graphics processor according to an embodiment of the present invention is shown.

[0025] Figure 2 A schematic diagram of a computational core according to some embodiments of the present invention is shown.

[0026] Figure 3 A schematic diagram of a computational core according to other embodiments of the present invention is shown.

[0027] Figure 4 A schematic diagram of the layout of a general-purpose graphics processor according to some embodiments of the present invention is shown.

[0028] Figure 5 A schematic diagram of the layout of a general-purpose graphics processor according to other embodiments of the present invention is shown.

[0029] Figure 6 A flowchart of a computation method for a general-purpose graphics processor according to an embodiment of the present invention is shown.

[0030] Figure 7 A schematic diagram of an architecture for computing core-driven direct memory access according to some embodiments of the present invention is shown.

[0031] Figure 8 A schematic diagram of an instruction-driven direct memory access architecture within a computing core according to some embodiments of the present invention is shown.

[0032] Figure 9 A schematic diagram of an instruction-driven direct memory access architecture within a computing core according to other embodiments of the present invention is shown.

[0033] Figure 10 A schematic diagram of a typical instruction-driven direct memory access architecture is shown.

[0034] Figure 11 A schematic diagram of a typical microcontroller-driven direct memory access architecture is shown.

[0035] Figure 12 The diagram illustrates a computing apparatus for implementing a computing method for a general-purpose graphics processor according to an embodiment of the present invention.

[0036] In the various figures, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0037] Preferred embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While preferred embodiments of the invention are shown in the drawings, it should be understood that the invention can be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that the invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0038] The term "comprising" and its variations as used herein signify open inclusion, i.e., "including but not limited to". Unless otherwise stated, the term "or" means "and / or". The term "based on" means "at least partially based on". The terms "one example embodiment" and "one embodiment" mean "at least one example embodiment". The term "another embodiment" means "at least one additional embodiment". The terms "first", "second", etc., may refer to different or the same objects.

[0039] As described above, the shortcomings of traditional general-purpose graphics processors are that the tensor ALU is completely idle in communication scenarios, which reduces chip resource utilization and economic efficiency.

[0040] Existing graphics processor architectures, including those that include Direct Memory Access (DMA), employ instruction sets or microcontrollers to drive direct memory access. (See reference...) Figure 10 This diagram illustrates a typical instruction-driven direct memory access architecture 1000. In this architecture, a central processing unit (CPU) (not shown) issues instructions to a device 1010, such as a graphics processing unit (GPU), for example, to an instruction processor 1012. Specifically, the instruction may include a source data address, a destination data address, and the data size. Subsequently, the instruction processor 1012 transmits the instruction via a bus to the direct memory access 1014 (e.g., in...). Figure 10 In the diagram, the command processor 1012 and the direct memory access 1014 are marked with shaded text to instruct the command processor 1012 to send instructions to the direct memory access 1014 via the bus, so as to drive the direct memory access 1014 to move data.

[0041] refer to Figure 11 This diagram illustrates a typical microcontroller-driven direct memory access architecture 1100. In this architecture, a microcontroller 1112 is used to drive direct memory access 1114 to move data during producer-consumer synchronization between the CPU (not shown) and GPU 1110. Specifically, the microcontroller 1112 runs firmware 1118, which detects whether the compute core 1116 has completed data production, and then sends direct memory access data moving commands to direct memory access 1114 (e.g., in...).Figure 11 In the diagram, microcontroller 1112 and direct memory access 1114 are marked with shaded text to indicate that microcontroller 1112 sends direct memory access data transfer commands to direct memory access 1114.

[0042] Research has found that in traditional direct memory access (DMI) driver methods, since hundreds or thousands of parallel subtasks typically run on the computing core, and the typical driver method requires all subtasks to be completed before data transfer, there is a significant overall latency, resulting in wasted bandwidth. Furthermore, traditional chips often use a direct memory access engine (DMA engine) to reduce memory movement between computing cores. While the DMA engine is small, it can only perform data transfer and cannot effectively achieve a generalized integration of communication and computation. For example, when using data from different computing cores (e.g., computing core A and computing core B) for computation, traditional chips might first move the data from computing core A to the high-bandwidth memory (HBM) of computing core B, and then the tensor computation unit of computing core B would retrieve the data from its HBM. In this process, communication and computation are separated, making the process cumbersome.

[0043] Furthermore, the interconnect bandwidth of chips is growing rapidly every year, with current mainstream chips even exceeding 1000 GB / s. This means that to fully utilize the communication bandwidth, more computing cores need to be dedicated to communication. For example, a computing core with a bandwidth of 128 GB / s theoretically requires eight cores to fully utilize 1000 GB / s bandwidth. In the undifferentiated computing core architecture of traditional general-purpose graphics processors, this means that while many cores are performing communication tasks, their tensor computation units are idle, failing to balance bandwidth and tensor computation unit utilization efficiency, resulting in serious resource waste.

[0044] To at least partially address one or more of the aforementioned problems and other potential issues, an exemplary embodiment of the present invention proposes a general-purpose graphics processing unit (GPU). In this GPU, a first number of first-type computing cores are configured, each first-type computing core including at least a tensor computation unit; a second number of second-type computing cores are configured, each second-type computing core including at least a vector computation unit and a loading and storage unit, the second number being less than or equal to the first number; and a task scheduling unit is configured to, in response to confirming that the current task is a communication task or a vector computation task, and that the state of the second-type computing cores meets predetermined conditions, distribute the current task to the second-type computing cores for execution. Therefore, communication tasks or vector tasks are executed using the second-type computing cores, while tensor computation tasks are executed using the first-type computing cores; thereby, the chip area resources of the GPU can be fully utilized, the inter-chip bandwidth of the GPU can be fully utilized, chip load can be easily optimized, and even in communication scenarios, chip resource utilization and economic efficiency can be improved.

[0045] The following will combine Figure 1 , Figure 2 and Figure 3 A general-purpose graphics processor 100 is described in the embodiments of the present invention. Figure 1 A schematic diagram of a general-purpose graphics processor 100 according to an embodiment of the present invention is shown. Figure 1 As shown, the general-purpose graphics processor 100 also includes at least: a first number of first-type computing cores 102, a second number of second-type computing cores 104, and a task scheduling unit 106.

[0046] Regarding the first type of computational core 102, it includes at least a tensor computation unit capable of performing tensor computation tasks. In some embodiments, the first type of computational core 102 may also be as follows: Figure 2 The computing core 200 is shown. In some embodiments, the first type of computing core 102 is, for example, a "full-featured computing core," or a "full-size computing core" or a "large core." (See reference...) Figure 2The computing core 200 includes at least a tensor core 210, a vector core 220, and a load-store unit 230. The load-store unit 230 further includes, for example, a global shared memory (GSM) / Level 1 cache 232. In some embodiments, the first type computing core 102 is selected for performing communication or vector computing tasks at a lower priority than the second type computing core 104. It should be understood that, for example, the first type computing core 102 can also be used to perform communication or vector computing tasks even when no idle second type computing core 104 is available. It should be understood that in some embodiments, the first type computing core 102 may not include the load-store unit 230 in the computing core 200.

[0047] The second type of computing core 104 includes at least a vector computing unit and a loading and storage unit, and is capable of performing communication tasks or vector computing tasks. In some embodiments, the size of the second type of computing core 104 is smaller than that of the first type of computing core 102. The second type of computing core 104 is referred to, for example, as a "small core". The second number of the second type of computing core 104 is less than or equal to the first number of the first type of computing core 102.

[0048] Regarding the location of the second type computing core 104 within the general-purpose graphics processor 100, in some embodiments, refer to... Figure 4 and Figure 5 This is, for example, located at one end of the general-purpose graphics processor 100 near other devices (e.g., a remote GPU) with which it communicates, to facilitate communication with other devices. It should be understood that, in addition to configuring a first type of computational core including tensor computation units, the general-purpose graphics processor also configures a second type of computational core for performing communication or vector computation tasks, facilitating direct chip interconnect communication, for example, connecting independently packaged chips over a very short distance to achieve high-bandwidth and low-latency data exchange. In some embodiments, such interconnection can occur between chips in the same package or adjacent packages. Furthermore, as described above, configuring the second type of computational core at one end of the general-purpose graphics processor near an adjacent general-purpose graphics processor with which it communicates further facilitates chip-to-chip interconnect communication.

[0049] In some embodiments, the second type of computing core 104 may be as follows: Figure 3 The calculation core shown is 300. (Reference) Figure 3The computing core 300 includes at least a vector computing unit 310 and a load and store unit 320. The load and store unit 320 includes global shared memory / Level 1 cache 322. Since the computing core 300 does not include a tensor computing unit compared to the computing core 200, the computing core 300 has a smaller size and occupies a smaller area. Compared to the indiscriminate computing core architecture of traditional general-purpose graphics processors, by using the computing core 300 as a second type of computing core in a general-purpose graphics processor according to an embodiment of the present invention, the computing core 300 can utilize its smaller size and be flexibly deployed in the limited space of the general-purpose graphics processor. This type of computing core 300 also allows more computing cores to be arranged in the same area, thereby replacing the redundant tensor computing units in the computing cores of traditional general-purpose graphics processors, and allowing for the reallocation of area resources. This, in turn, makes efficient use of the area resources of the general-purpose graphics processor.

[0050] In some embodiments, the second type of computing core 104 may also be a first type of computing core that fails to meet predetermined functional conditions due to manufacturing processes, such as an "isolated full-size cell." For example, it could be a first type of computing core that fails to meet predetermined functional conditions and is harvested after chip manufacturing. Thus, by using a first type of computing core that fails to meet predetermined functional conditions as the second type of computing core 104, functionally flawed computing cores in a general-purpose graphics processor can be fully utilized.

[0051] Regarding the layout of the first type computing core 102 and the second type computing core 104 in the general-purpose graphics processor 100, please refer to the following combination, for example. Figure 4 and Figure 5 The illustrated embodiments will not be repeated here.

[0052] Regarding the task scheduling unit 106, it is configured to, in response to confirming that the current task is a communication task or a vector computation task, and that the state of the second type of computing core 104 meets predetermined conditions, assign the current task to the second type of computing core 104 for execution. In some embodiments, the task scheduling unit 106 is further configured to, in response to confirming that the current task is a tensor computation task, assign the current task to the first type of computing core 102 for execution. The scheduling strategy of the task scheduling unit 106 is dynamically managed, for example, by a hardware task allocator, to facilitate automatic switching based on real-time load.

[0053] Regarding the predetermined conditions, for example, at least one second-type computing core is in an idle state.

[0054] Regarding communication tasks or vector computation tasks, these tasks are associated with predetermined identifiers, for example. In some embodiments, the predetermined identifier indicates that the communication task or vector computation task is only sent to the second type of computing core 104. In other embodiments, the predetermined identifier indicates that the communication task or vector computation task is preferentially sent to the second type of computing core 104, and if all second type of computing cores 104 are not idle, it is sent to the first type of computing core 102. In this way, the second type of computing core 104 can focus on communication tasks or vector computation tasks, while the first type of computing core 102 can focus on tensor computation-intensive tasks.

[0055] As described above, a drawback of traditional general-purpose graphics processors (GPUs) is that the tensor ALUs in the computing cores are completely idle in communication scenarios, reducing chip resource utilization and economic efficiency. In contrast, the general-purpose GPU according to embodiments of the present invention configures a first number of first-type computing cores (which at least include tensor computing units), a second number of second-type computing cores (which at least include vector computing units and loading and storage units), and a task scheduling unit. It is configured to, in response to confirmation that the current task is a communication task or a vector computing task, and that the state of the second-type computing cores meets predetermined conditions, distribute the current task to the second-type computing cores for execution. The present invention can utilize the second-type computing cores to execute communication tasks or vector tasks, and utilize the first-type computing cores to execute tensor computing tasks. This enables the full utilization of communication bandwidth using fewer full-size computing cores, effectively achieving a generalized integration of communication and computing, and further making full use of the chip area resources and inter-chip bandwidth of the general-purpose GPU, conveniently optimizing chip load, and improving chip resource utilization and economic efficiency even in communication scenarios. Furthermore, compared to the traditional approach described above that uses a Direct Memory Access (DMA) engine for only data movement, the general-purpose graphics processor according to embodiments of the present invention integrates communication and vector computing functions within the second type of computing core. Therefore, it is more advantageous for operations such as allReduce in aggregate communication. Moreover, the overall power consumption-performance-area (PPA) of the general-purpose graphics processor according to embodiments of the present invention is superior to that of traditional general-purpose graphics processors with undifferentiated computing core architectures, achieving functional improvements with minimal hardware modifications. This provides a high-performance, low-cost solution for AI scenarios with high communication ratios.

[0056] In some embodiments, the general-purpose graphics processor 100 further includes: direct memory access (not shown), configured to send a drive command for direct memory access in response to determining that a computing core has completed data production for a subtask of a computing task, such that direct memory access only moves data for the one subtask, the computing core including a first type of computing core and a second type of computing core, the computing task including multiple subtasks.

[0057] Regarding direct memory access, it is located, for example, in a first-type computing core 102 and / or a second-type computing core 104. In some embodiments, it is, for example, an instruction-driven asynchronous DMA unit within the core, thereby enabling data production and data transfer interleaving at a finer granularity. In some embodiments, the general-purpose graphics processor 100, in response to determining that a computing core has completed data production for a subtask of a computing task, sends a drive command for direct memory access so that direct memory access moves only the data of said subtask, said computing core including the first-type computing core 102 and the second-type computing core 104, said computing task including multiple subtasks.

[0058] For example, a computational task is matrix multiplication.

[0059] Regarding subtasks, such as parallel subtasks in a computation task.

[0060] Regarding the driving method of direct memory access, in some embodiments, direct memory access is driven by a computing core including a first type computing core 102 and / or a second type computing core 104. In other embodiments, direct memory access is included within a computing core, and the direct memory access within this core can be driven by instructions within the computing core. See below for reference. Figure 7 , Figure 8 and Figure 9 The details will not be elaborated here.

[0061] The following will combine Figure 4 and Figure 5 The layout of a general-purpose graphics processor according to an embodiment of the present invention is described. Figure 4 A schematic diagram of the layout of a general-purpose graphics processor 400 according to some embodiments of the present invention is shown. Figure 5 A schematic diagram of the layout of a general-purpose graphics processor 500 according to other embodiments of the present invention is shown.

[0062] refer to Figure 4 The general-purpose graphics processor 400 includes a first number of first-type computing cores 410 and a second number of second-type computing cores 420. The second number is less than or equal to the first number.

[0063] Regarding the first type of computing core 410, its example is as described above. Figure 2 The calculation core shown is 200.

[0064] Regarding the second type of computing core 420, it may be constructed, for example, from a first type of computing core that does not meet predetermined functional requirements. These predetermined functional requirements are, for example, requirements concerning computing performance. In the above embodiment, the second type of computing core 420 is comparable in size to the first type of computing core 410. The second type of computing core 420 is at least capable of functioning as a communication core, performing communication tasks. By using a first type of computing core that does not meet predetermined functional requirements as the second type of computing core, functionally flawed computing cores in general-purpose graphics processors can be fully utilized. It should be understood that the second type of computing core 420 may also be... Figure 3 The small core or simplified core shown is even smaller.

[0065] like Figure 4 As shown, the general-purpose graphics processor 400 can interact with a remote GPU via a second number of second-type computing cores 420, and can also communicate with a bus 430, which in turn can communicate with high-bandwidth memory (HBM) 440. As described above, the layout of the general-purpose graphics processor 400 enables efficient execution of communication tasks, facilitating full utilization of the inter-chip bandwidth of the general-purpose graphics processor and optimizing chip load. This provides a high-performance, low-cost solution for AI scenarios with high communication ratios. Test data shows that if the bandwidth of each second-type computing core 420 is 128GB / s, then four second-type computing cores 420 can support 512GB / s of peer-to-peer (P2P) bandwidth.

[0066] refer to Figure 5 In the layout of the general-purpose graphics processor 500, there is a first number of first-type computing cores 510 and a second number of second-type computing cores 520. The first-type computing cores 510 are, for example, as described above. Figure 2 The computational core 200 is shown. Regarding the second type of computational core 520, it is, for example, as described above. Figure 3The computational core 300 is shown. In some embodiments, since the second type of computational core 520 does not include a tensor computation unit compared to the first type of computational core 510, the second type of computational core 520 has a smaller size than the first type of computational core 510, and is referred to as a "small core" or "simple core". It should be understood that the second type of computational core 520 can at least serve as a communication core to perform communication tasks. The general-purpose graphics processor 500 can send data to a remote GPU via a second number of second type of computational cores 520, and can also communicate with a bus 530, which can communicate with high-bandwidth memory 540. Because the second type of computational core 520 has a smaller size, it can be flexibly placed in space-constrained locations within the general-purpose graphics processor 500, such as in corner locations; furthermore, more second type of computational cores 520 can be arranged in the same area. As described above, the second type of computational core 520 can efficiently perform communication tasks, which is beneficial for fully utilizing the inter-chip bandwidth of the general-purpose graphics processor and optimizing chip load. For example, if each Type II compute core 520 has a bandwidth of 128 GB / s, six Type II compute cores 520 can be configured in the same space in the rightmost column, thus supporting a P2P bandwidth of 768 GB / s. Therefore, Type II compute cores 520 bring significant benefits to communication. Assuming the same bandwidth of 768 GB / s is covered, a conventional general-purpose graphics processor has an undifferentiated compute core architecture, with each compute core being a conventional compute core including tensor computation units and having a bandwidth of 128 GB / s. A conventional general-purpose graphics processor would require six conventional compute cores for communication, thus occupying a larger area, while reducing the usable area of ​​the compute cores mainly used for tensor computation. Furthermore, the tensor computation units in these six conventional compute cores are also limited, thus wasting computing resources.

[0067] In some embodiments, the second number is configured such that the total communication bandwidth of the second type of computing cores is greater than or equal to the peak communication demand of a predetermined artificial intelligence workload. In some embodiments, the ratio of the first number to the second number is 5:1. Test data shows that at this ratio, the total communication bandwidth can better cover the peak communication demand of a typical artificial intelligence workload. For example, the first number is 40 and the second number is 8 (this configuration is referred to as the "40+8 core configuration"). Test data shows that in this configuration, the area utilization of the general-purpose graphics processor is improved by 5%, further increasing the computing density and enabling efficient algorithms that integrate communication and computing.

[0068] Regarding total communication bandwidth, it indicates, for example, the maximum bandwidth allowed for communication between computing cores. In some embodiments, the AI ​​product manual describes it as "bidirectional bandwidth." As mentioned earlier, in the undifferentiated computing core architecture of traditional general-purpose graphics processors, if full utilization of communication bandwidth is desired, more computing cores need to be dedicated to communication. This means that while a large number of computing cores are performing communication tasks, their tensor computation units are idle, resulting in serious resource waste and an inability to balance bandwidth and tensor computation unit utilization efficiency. In contrast, the general-purpose graphics processor of this invention utilizes a second type of computing core to perform communication or vector tasks and a first type of computing core to perform tensor computation tasks; thereby, it can fully utilize the chip area resources of the general-purpose graphics processor, fully utilize the inter-chip bandwidth of the general-purpose graphics processor, conveniently optimize chip load, and improve chip resource utilization and economic efficiency even in communication scenarios.

[0069] The following will combine Figure 6 , Figure 7 , Figure 8 and Figure 9 This invention describes a computational method for a general-purpose graphics processor according to an embodiment of the present invention. Figure 6 A flowchart of a calculation method 600 for a general-purpose graphics processor 100 according to an embodiment of the present invention is shown. It should be understood that method 600 can, for example, be implemented in... Figure 12 The method 600 is executed at the general-purpose computing device 1200 described. The method may also include additional actions not shown and / or the actions shown may be omitted; the scope of the invention is not limited in this respect.

[0070] At step 602, the computing device 1200 confirms the task type of the current task.

[0071] Regarding task types, in some embodiments, task types include a first task type and a second task type. In some embodiments, the first type task is a tensor computation task, and the second type task is a communication task or a vector computation task. In some embodiments, the first type task and the second type task are associated with predetermined identifiers to indicate their types. The general-purpose graphics processor 100 determines the task type of the current task, for example, based on the predetermined identifiers associated with the current task.

[0072] In step 604, in response to confirming that the task type of the current task is a first type of task, the computing device 1200 sends the current task to a first type of computing core, the first type of computing core including at least a tensor computing unit.

[0073] The first type of computational core includes at least a tensor computation unit. In some embodiments, the first type of computational core includes at least a tensor computation unit, a vector computation unit, and a loading and storage unit. For details regarding the first type of computational core, please refer to the above description; further details will not be repeated here.

[0074] At step 606, in response to confirming that the task type of the current task is a second type of task, the computing device 1200 sends the current task to a second type of computing core, the second type of computing core including at least a vector computing unit and a loading and storage unit.

[0075] The second type of computing core includes at least a vector computation unit and a load and storage unit. For details regarding the second type of computing core, please refer to the previous section on second-type computing cores; further details will not be repeated here.

[0076] The method for assigning the current task to the second type of computing core includes, for example, assigning the current task to the second type of computing core for execution if the computing device 1200 confirms that the current task is a communication task or a vector computing task and the state of the second type of computing core meets predetermined conditions.

[0077] Regarding the state of the second type of computing core satisfying a predetermined condition, for example: the state of the second type of computing core satisfies a predetermined condition.

[0078] In some embodiments, method 600 further includes: in response to determining that a computing core has completed data production for a subtask of a computing task, sending a drive command for direct memory access so that direct memory access only moves data for the one subtask, wherein the computing core includes a first type of computing core and a second type of computing core, and the computing task includes multiple subtasks. The following will combine... Figures 7-9 The detailed architecture of the driver for direct memory access will not be elaborated here.

[0079] This invention utilizes a second type of computing core to perform communication or vector tasks, and a first type of computing core to perform tensor computation tasks; thereby, it can make full use of the chip area resources of a general-purpose graphics processor, make full use of the inter-chip bandwidth of a general-purpose graphics processor, conveniently optimize chip load, and improve chip resource utilization and economic efficiency even in communication scenarios. Figure 12 The illustration schematically shows a computing device implementing a computing method for a general-purpose graphics processor according to an embodiment of the present invention. Figure 12As shown, the computing device 1200 may have one or more processing units, including dedicated processing units such as a graphics processing unit (GPU), a field-programmable gate array (FPGA), an application-specific integrated circuit (ASIC), or a general-purpose computing on graphics processing unit (GPGPU). The computing device 1200 may include, for example, a task type confirmation unit 1202 for the current task, a first-type computing core task assignment unit 1204, and a second-type computing core task assignment unit 1206.

[0080] Regarding the task type confirmation unit 1202 for the current task, it is used to confirm the task type of the current task.

[0081] Regarding the first type of computing core task distribution unit 1204, it is used to distribute the current task to the first type of computing core in response to confirming that the task type of the current task is a first type of task. The first type of computing core includes at least a tensor computing unit.

[0082] Regarding the second type of computing core task distribution unit 1206, it is used to distribute the current task to the second type of computing core in response to confirming that the task type of the current task is a second type of task. The second type of computing core includes at least a vector computing unit and a loading and storage unit.

[0083] The following will combine Figure 7 , Figure 8 and Figure 9 This describes the architecture for driving direct memory access according to embodiments of the present invention. Specifically, direct memory access can be... Figure 1 The first type computing core 102 and / or the second type computing core 104 of the general-purpose graphics processor shown are driven outside or inside the core. Figure 7 A schematic diagram of an architecture 700 for computing core-driven direct memory access according to some embodiments of the present invention is shown. Figure 8 A schematic diagram of an instruction-driven direct memory access architecture 800 within a computing core according to some embodiments of the present invention is shown. Figure 9 A schematic diagram of an architecture 900 for instruction-driven direct memory access within a computational core according to other embodiments of the present invention is shown. It should be understood that architectures 700, 800, and 900 are used, for example, for... Figure 1 The general-purpose graphics processor 100 is described. The scope of the invention is not limited in this respect.

[0084] As mentioned above, refer to Figure 10 and Figure 11 Existing graphics processing units (GPUs) with direct memory access architectures employ instruction or microcontroller-driven direct memory access. Since hundreds or thousands of parallel subtasks typically run on the computation core, and the typical driving method requires all subtasks to complete synchronously before data transfer, there is a significant overall latency, resulting in wasted bandwidth.

[0085] In contrast, Figures 7 to 9 The present invention illustrates a driving method for direct memory access according to an embodiment of the present invention. Specifically, direct memory access can be driven outside or inside the core by a first type of computing core and / or a second type of computing core, which has less latency and can achieve fine-grained latency hiding of computation and communication.

[0086] refer to Figure 7 In architecture 700, a compute core 712 drives a direct memory access (DMI) 714 to move data. Specifically, the DMI 714 is located outside the compute core 712. When the compute core 712 completes the data production of a subtask, it can send a drive command for DMI to drive the DMI 714 to move the data for that subtask. This allows for finer-grained latency hiding in computation and communication, enabling finer-grained producer-consumer data synchronization. The compute core 712 can be, for example, a first-type compute core or a second-type compute core.

[0087] refer to Figure 8 This diagram illustrates a schematic of a computation core-driven direct memory access architecture 800 according to some embodiments of the present invention. In architecture 800, a computation core 810 includes a vector computation unit 811, a load and store unit 812, and a tensor computation unit 813. The load and store unit 812 further includes a global shared memory / Level 1 cache 814 and a direct memory access unit 815. In some embodiments, the instruction-driven direct memory access unit 815 within the computation core 810 can asynchronously retrieve tensor data from the computation core 810. In some embodiments, in response to determining that the computation core 810 has completed data production for a subtask of a computation task, a driving command is sent for the direct memory access unit 815 so that the direct memory access unit 815 only fetches data from said subtask. In some embodiments, the direct memory access unit 815 within the computation core 810 is driven, for example, by the tensor computation unit 813. In other embodiments, the direct memory access unit 815 within the computation core 810 is driven, for example, by the vector computation unit 811 (e.g., in...). Figure 8In the diagram, the vector computing unit 811 and the direct memory access 815 are marked with shaded text to indicate the direct memory access 815 within the computing core 810 driven by the vector computing unit 811. This allows for more granular implementation of the interweaving of data production and data movement.

[0088] In some embodiments, Figure 8 The computing core 810 shown is a first-type computing core.

[0089] refer to Figure 9 This illustrates a schematic diagram of a computation core-driven direct memory access architecture 900 according to other embodiments of the present invention. In architecture 900, the computation core 910 includes a vector computation unit 911 and a load and store unit 912, excluding, for example, a tensor computation unit. The load and store unit 912 further includes a global shared memory / Level 1 cache 913 and a direct memory access unit 914. The instruction-driven direct memory access unit 914 within the computation core 910 is capable of asynchronously acquiring tensor data within the computation core 910. In some embodiments, in response to determining that the computation core 910 has completed data production for a subtask of a computation task, a drive command is sent for the direct memory access unit 914 so that the direct memory access unit 914 only handles the data for that subtask. In some embodiments, the direct memory access unit 914 within the computation core 910 is driven, for example, by the vector computation unit 911 (e.g., in...). Figure 9 In this design, the vector computation unit 911 and the direct memory access unit 914 are marked with shaded text to indicate that the direct memory access unit 914 is driven by the vector computation unit 911. This allows for finer-grained implementation of the interweaving of data production and data movement. Furthermore, since the computation core 910 does not include a tensor computation unit, the architecture 900 can further reduce its size. In some embodiments, Figure 9 The computing core 910 shown is a second-type computing core. By employing the above methods, this invention can fully utilize the chip area resources of a general-purpose graphics processing unit (GPU) and the inter-chip bandwidth of the GPU, conveniently optimizing chip load. Even in communication scenarios, it can improve chip resource utilization and economic efficiency. Furthermore, by configuring direct memory access, it can drive direct memory access to move data between subtasks, achieving finer-grained latency hiding in computation and communication, enabling finer-grained producer-consumer data synchronization, and realizing a generalized integration of communication and computation.

[0090] The various processes and procedures described above, such as method 600, can be executed at a computing device. This computing device may include, for example, at least one processor (at least one graphics processor and at least one central processing unit); and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor. In some embodiments, method 600 may be implemented as a computer software program or program product tangibly contained in a machine-readable medium. In some embodiments, part or all of the computer program may be loaded and / or installed on the computing device via read-only memory (ROM) and / or a communication unit. When the computer program is loaded into random-access memory (RAM) and executed by the GPU and CPU, one or more actions of method 600 described above can be performed.

[0091] This invention can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of the invention. The computer-readable storage medium may be a tangible device capable of holding and storing instructions used by an instruction execution device. The computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof.

[0092] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to various computing / processing devices, or downloaded via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network, to an external computer or external storage device. Various aspects of the invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0093] These computer-readable program instructions can be provided to the central processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the central processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of a flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of a flowchart and / or block diagram.

[0094] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those shown in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0095] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this application can be achieved, and this is not limited herein.

[0096] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors.

Claims

1. A general-purpose graphics processor, characterized in that, include: A first number of first-type computational cores, wherein the first-type computational cores include at least tensor computational units; A second number of second-type computing cores, the second-type computing cores including at least a vector computing unit and a loading and storage unit, the second number being less than or equal to the first number; as well as The task scheduling unit is configured to, in response to confirming that the current task is a communication task or a vector computation task, and that the state of the second type of computing core meets predetermined conditions, assign the current task to the second type of computing core for execution. Among them, the state of the second type of computing core satisfies the predetermined conditions, including: at least one second type of computing core is in an idle state.

2. The general-purpose graphics processor according to claim 1, characterized in that, The communication task and / or vector calculation task are associated with a predetermined identifier.

3. The general-purpose graphics processor according to claim 1, characterized in that, The second quantity is configured such that the total communication bandwidth of the second type of computing cores is greater than or equal to the peak communication demand of a predetermined artificial intelligence workload.

4. The general-purpose graphics processor according to claim 1, characterized in that, The task scheduling unit is further configured to: In response to the confirmation that the current task is a tensor computation task, the current task is assigned to the first type of computation core for execution.

5. The general-purpose graphics processor according to claim 1, characterized in that, The ratio of the first quantity to the second quantity is 5:

1.

6. The general-purpose graphics processor according to claim 1, characterized in that, The first quantity is 40, and the second quantity is 8.

7. The general-purpose graphics processor according to claim 1, characterized in that, Also includes: Direct memory access is configured to, in response to determining that a computing core has completed data production for a subtask in a computing task, send a driver command for direct memory access so that direct memory access only moves data for said subtask, said computing core including a first type computing core and a second type computing core, said computing task including multiple subtasks.

8. The general-purpose graphics processor according to claim 7, characterized in that, The direct memory access is driven outside or inside the core by the first type of computing core and / or the second type of computing core.

9. The general-purpose graphics processor according to claim 1, characterized in that, The second type of computing core is composed of the first type of computing core that does not meet the predetermined functional requirements.

10. A calculation method for a general-purpose graphics processor, characterized in that, The general-purpose graphics processor includes computing cores, which include a first type of computing core and a second type of computing core; the method includes: Confirm the task type of the current task; In response to confirming that the current task is a first-type task, the current task is assigned to a first-type computational core, wherein the first-type computational core includes at least tensor computation units; and In response to the confirmation that the current task is a second type of task, the current task is sent to a second type of computing core, which includes at least a vector computing unit and a loading and storage unit.

11. The method according to claim 10, characterized in that, The first type of task is a tensor computation task, and the second type of task is a communication task or a vector computation task. In response to confirming that the current task is a second type of task, the current task is sent to the second type of computation core, including: In response to the confirmation that the current task is a communication task or a vector computing task, and that the status of the second type of computing core meets the predetermined conditions, the current task is assigned to the second type of computing core for execution.

12. The method according to claim 11, characterized in that, Also includes: In response to determining that a computing core has completed data production for a subtask in a computing task, a driver command for direct memory access is sent so that direct memory access only moves data for the one subtask. The computing core includes a first type of computing core and a second type of computing core, and the computing task includes multiple subtasks.

13. A computing device, characterized in that, include: At least one processor; as well as A memory that is communicatively connected to the at least one processor; in The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 10-12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a machine, performs the method according to any one of claims 10-12.

15. A computer program product, characterized in that, Includes a computer program, which, when executed by a machine, performs the method according to any one of claims 10-12.

Images