AI computing platform, method and ai cloud computing system

Abstract

The application relates to the technical field of artificial intelligence computing, and discloses an AI computing platform, an AI computing method and an AI cloud computing system. The AI computing platform comprises at least one computing component, wherein the computing component comprises a processor configured to initiate a computing task and decompose the computing task into a plurality of ordered subtasks according to stored network topology information tables; a plurality of near-memory computing modules, each of the plurality of near-memory computing modules is connected to the processor and each of the plurality of near-memory computing modules is connected to each other, wherein each of the plurality of near-memory computing modules is configured to implement different operation types, and each of the plurality of near-memory computing modules completes one or more of the plurality of subtasks according to the operation type implemented by the near-memory computing module. The application can reduce the unified scheduling load of the processor, has good universality and scalability, and can be applied to single-server, multi-server and cloud scenarios.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence computing technology, and in particular to an AI computing platform, method and AI cloud computing system. Background Technology

[0002] In existing technologies, when a single server containing multiple AI computing modules processes complex tasks, the conventional approach is as follows: Figure 1 As shown, the CPU centrally schedules resources in the NCM (Near Computing Memory), and all data exchanges between NCMs are forwarded by the CPU. This architecture makes the CPU a bottleneck in processing power and interface communication capabilities. When applications expand to more complex multi-server or cloud computing scenarios, the aforementioned bottlenecks limit processing resources, hinder flexible cloud deployment, and result in poor portability and scalability, thus preventing efficient computing and hindering efficient application in complex computing scenarios. Summary of the Invention

[0003] The purpose of this application is to provide an AI computing platform, method, and AI cloud computing system that fully leverages the advantages of segmented computing, reduces the unified scheduling load of the processor, and has good versatility and scalability, and is applicable to single-server, multi-server, and cloud scenarios.

[0004] This application discloses an AI computing platform, including: at least one computing component, wherein the computing component includes:

[0005] A processor, configured to initiate a computational task and decompose the computational task into several ordered subtasks based on a stored network topology information table; and

[0006] A plurality of near-memory computing modules are connected in pairs to the processor and in pairs to each other. Each of the plurality of near-memory computing modules is configured to implement different types of operations, and each of the plurality of near-memory computing modules completes one or more of the plurality of subtasks according to the types of operations it implements.

[0007] In one embodiment, the network topology information table includes: the IP address of the host where the processor is located, the index of the near-memory computing module, the types of operations supported by the near-memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules.

[0008] In one embodiment, the processor is further configured to generate a data packet based on the decomposed plurality of ordered subtasks, and transmit the data packet to the near-memory computing module that processes the subtask ranked first in the processing order of the subtasks. The data packet includes: the number of near-memory computing modules required to process the data packet, a list of near-memory computing modules required to process the data packet, a packet header cyclic redundancy check, a payload length, and a payload.

[0009] In one embodiment, the near-memory computing module is further configured to process the corresponding subtasks to generate processed data packets upon receiving the data packets, and send the processed data packets to the next near-memory computing module or return them to the processor according to the near-memory computing module list.

[0010] In one embodiment, the list of near-memory computing modules includes the IP address, index, supported operation types, and load rate of the host where the near-memory computing module that processes each subtask is located.

[0011] In one embodiment, the processor is further configured to implement operation types different from the plurality of near-memory computing modules, wherein the processor and the plurality of near-memory computing modules complete one or more of the plurality of subtasks according to the operation types they implement.

[0012] In one embodiment, the processors of the computing components are connected via a bus.

[0013] This application also discloses an AI-based computing method, including:

[0014] The processor initiates a computing task and decomposes the computing task into several ordered subtasks according to the stored network topology information table;

[0015] The processor generates data packets based on the decomposed ordered subtasks and routes the data packets to the near-memory computing module that processes the subtask ranked first, according to the processing order of the subtasks; and

[0016] When the near-memory computing module receives the data packet, it processes the corresponding subtask to generate the processed data packet, and routes the processed data packet to the near-memory computing module that processes the next subtask until all subtasks are completed, and then routes it to the processor connected to the near-memory computing module that processes the last subtask.

[0017] In one embodiment, the plurality of near-memory computing modules are connected in pairs to the processor, and the plurality of near-memory computing modules are also connected in pairs to each other, wherein each of the plurality of near-memory computing modules is configured to implement different types of operations.

[0018] In one embodiment, the network topology information table includes: the IP address of the host where the processor is located, the index of the near-memory computing module, the types of operations supported by the near-memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules.

[0019] In one embodiment, the data packet includes: the number of near-memory computing modules required to process the data packet, a list of near-memory computing modules required to process the data packet, a packet header cyclic redundancy check, a payload length, and a payload.

[0020] In one embodiment, the list of near-memory computing modules includes the IP address, index, supported operation types, and load rate of the host where the near-memory computing module that processes each subtask is located.

[0021] In one embodiment, the processor is configured to implement a different type of operation than the plurality of near-memory computing modules, wherein the processor and the plurality of near-memory computing modules complete one or more of the plurality of subtasks according to the type of operation they each implement.

[0022] This application also discloses an AI cloud computing system, including: a cloud computing center, and the AI ​​computing platforms described above, wherein the cloud computing center is connected to the plurality of AI computing platforms.

[0023] In this application, the processor decomposes the computation task into several ordered subtasks based on a network topology information table and generates data packets based on the Segmented Computation Protocol for Near Memory (SC4NCM). Upon receiving the data packets, each near-memory computing module processes the corresponding subtask according to the types of operations it can perform. This approach fully leverages the advantages of segmented computing while reducing the processor's unified scheduling load and possesses good versatility and scalability. Therefore, this application is applicable to single-server, multi-server, and cloud scenarios, providing an efficient cloud-native computing platform architecture for deploying cloud-native applications on AI servers. Furthermore, because the SC4NCM protocol's data packets can flexibly define various computational functions, it can also be applied to computing modules such as GPGPUs and FPGAs, and can be extended to more complex computational applications such as machine training.

[0024] The specification of this application contains numerous technical features distributed across various technical solutions. Listing all possible combinations of these technical features (i.e., technical solutions) would make the specification excessively lengthy. To avoid this problem, the various technical features disclosed in the above-described invention, the various technical features disclosed in the following embodiments and examples, and the various technical features disclosed in the accompanying drawings can be freely combined to form various new technical solutions (all of which should be considered as described in this specification), unless such a combination of technical features is technically infeasible. For example, one example discloses feature A+B+C, and another example discloses feature A+B+D+E. Features C and D are equivalent technical means that serve the same function, and technically only one needs to be used; they cannot be used simultaneously. Feature E can technically be combined with feature C. Therefore, the solution A+B+C+D should not be considered as described because it is technically infeasible, while the solution A+B+C+E should be considered as described. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of a computing server using existing technology.

[0026] Figure 2 This is a schematic diagram of the structure of an AI computing platform according to an embodiment of this application.

[0027] Figure 3 This is a schematic diagram of a data packet according to an embodiment of this application.

[0028] Figure 4 This is a schematic diagram of the structure of a cloud-native AI computing system according to an embodiment of this application.

[0029] Figure 5 This is a schematic flowchart of an AI calculation method according to an embodiment of this application.

[0030] Figure 6 This is a flowchart illustrating the completion of a computational task according to an embodiment of this application. Detailed Implementation

[0031] In the following description, many technical details are presented to help the reader better understand this application. However, those skilled in the art will understand that the technical solutions claimed in this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0032] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0033] This application discloses an AI computing platform comprising at least one computing component, each computing component including a processor and a plurality of near-memory computing modules. The plurality of near-memory computing modules are connected in pairs to the processor, and are also connected in pairs to each other. The processor initiates computing tasks and decomposes the computing tasks into a plurality of ordered subtasks according to a stored network topology information table. Each of the plurality of near-memory computing modules is configured to implement different operation types, and each of the plurality of near-memory computing modules completes one or more of the plurality of subtasks according to its implemented operation type.

[0034] In one embodiment, the computing module of the AI ​​computing platform includes a near-memory computing module (NCM), which is a type of AI chip implementation. In other embodiments, the computing module of the AI ​​computing platform may also include computing modules such as GPGPU and FPGA, which are suitable for more complex computing applications such as machine training.

[0035] like Figure 2 As shown, this example illustrates an AI computing platform comprising two computing components, each including a processor and four near-memory computing modules. It's worth noting that the number of computing components in the AI ​​computing platform and the number of near-memory computing modules in each component can be configured according to actual needs; this embodiment does not impose any limitations on this. Figure 2 As shown, in each computing component, four near-memory computing modules are connected to the processor in pairs, and the four near-memory computing modules are also connected to each other in pairs. The processor and the four near-memory computing modules can be connected via PCIe. In one embodiment, the processors of each computing component can be connected via a bus. For example, Figure 2 The two processors are connected via a UPI (Ultra Path Interconnect) bus. In one embodiment, the processors and the near-memory computing module can be configured to perform different types of operations. For example, in... Figure 2 In the middle, processor The processor can perform logical operations on J1 and J2, the near-memory computing module ② can perform logical operations on A1 and A2, the near-memory computing module ③ can perform logical operations on B1 and B2, and so on. It should be understood that the types of operations implemented by the processor and the near-memory computing modules can be set according to requirements, and this embodiment does not impose any restrictions on this.

[0036] Figure 2 processor in The system stores a network topology information table. In one embodiment, the network topology information table includes: the IP address of the host where the processor resides, the index of the nearest memory computing module, the types of operations (OPs) supported by the nearest memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules. Table 1 shows an example of a network topology information table.

[0037]

[0038] Table 1 Network Topology Information

[0039] In one embodiment, the processor decomposes the computational task into several ordered subtasks based on network topology information (i.e., while decomposing the computational task into several subtasks, it also generates the processing order of each subtask), generates data packets, and transmits the data packets to the near-memory computing module that processes the subtask ranked first, according to the processing order of the subtasks. Upon receiving the data packet, the near-memory computing module processes the corresponding subtask to generate a processed data packet, and sends the processed data packet to the near-memory computing module that processes the next subtask, and so on. When all subtasks are completed, the near-memory computing module that processes the last subtask returns the processed data packet to the processor. Figure 3 As shown, in one embodiment, the data packet may include the number of near-memory computing modules required to process the data packet, a list of near-memory computing modules required to process the data packet, a cyclic redundancy check (CRC) header, a payload length, and the payload itself. In another embodiment, the data packet may also include a data packet number. The data packet number is primarily used to indicate the order of the data packets, ensuring the reliability of data packet transmission. The list of near-memory computing modules required to process the data packet includes detailed information for each near-memory computing module, as detailed below. The cyclic redundancy check header ensures correct data communication. The payload length is the length of the payload to be processed by the near-memory computing modules; this payload length is refreshed after each near-memory computing module completes its corresponding computational operation.

[0040] like Figure 2 As shown, for example, the processor The computational task is decomposed into multiple subtasks, which perform the following operations on relevant data: A1, C2, B1, J1, K2, E1, and H2, respectively, and are completed sequentially. Then, the processor determines the near-memory computing module responsible for each subtask based on the network topology information table and generates a near-memory computing module list. In one embodiment, the near-memory computing module list includes the IP address, index, supported operation type, and load rate of the host where the near-memory computing module sequentially processes each subtask. After processing the corresponding subtask and generating the processed data packet, the near-memory computing module can determine the near-memory computing module responsible for the next subtask based on this list. Table 2 below shows a near-memory computing module list in one embodiment.

[0041]

[0042]

[0043] Table 2 List of Near-Memory Computing Modules

[0044] In one embodiment, the processor does not specifically participate in the processing of subtasks; instead, each subtask is handled by a separate near-memory computing module. In another embodiment, after initiating a computing task, the processor can also participate in the processing of subtasks; that is, all subtasks are processed by the processor and the various near-memory computing modules. In this case, the processor can be configured to implement a different type of operation than the various near-memory computing modules. The processor and the various near-memory computing modules complete one or more of the various subtasks according to their respective implemented operation types. It should be noted that the type of operation implemented by the processor is optional.

[0045] Embodiments of this application disclose an AI cloud computing system, such as Figure 4 As shown in the diagram. The AI ​​cloud computing system includes a cloud computing center and multiple AI computing platforms, with the cloud computing center connected to the multiple AI computing platforms. The structure of the AI ​​computing platforms is as described above and will not be repeated in this embodiment.

[0046] In this embodiment of the application, all computing resources consisting of schedulable computing modules in the cloud computing system application are maintained in the form of a network topology information table in the form of "edges" (point-to-point connections), which is used by the processor to decompose the computing task into subtasks when initiating a computing task.

[0047] An embodiment of this application discloses an AI calculation method. Figure 5 A flowchart of an AI calculation method 500 in one embodiment is shown. The method 500 includes the following steps:

[0048] Step 502: The processor initiates a computing task and decomposes the computing task into several ordered subtasks according to the stored network topology information table. The network topology information table includes: the IP address of the host where the processor is located, the index of the near-memory computing module, the types of operations supported by the near-memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules.

[0049] Step 504: The processor generates data packets based on the decomposed ordered subtasks and routes the data packets to the near-memory computing module that processes the subtask ranked first according to the processing order of the subtasks.

[0050] The plurality of near-memory computing modules are connected in pairs to the processor, and the plurality of near-memory computing modules are also connected in pairs to each other, wherein each of the plurality of near-memory computing modules is configured to implement different types of operations.

[0051] In one embodiment, the data packet may include the number of near-memory computing modules required to process the data packet, a list of near-memory computing modules required to process the data packet, a cyclic redundancy check header, a payload length, and a payload. In another embodiment, the data packet may also include a data packet number. In one embodiment, the list of near-memory computing modules includes the IP address, index, supported operation types, and load rate of the host where the near-memory computing module that processes each subtask is located.

[0052] Step 506: When the near-memory computing module receives the data packet, it processes the corresponding subtask to generate the processed data packet, and routes the processed data packet to the near-memory computing module that processes the next subtask until all subtasks are completed, and then routes it to the processor connected to the near-memory computing module that processes the last subtask.

[0053] Furthermore, the processor can also be used to implement different types of operations than the plurality of near-memory computing modules, wherein the processor and the plurality of near-memory computing modules complete one or more of the plurality of subtasks according to the operation types they implement. That is, the processor can also participate in the processing of subtasks, just like the near-memory computing modules.

[0054] It should be noted that the data packets mentioned in this application are data packets defined based on the Near Memory Segmented Computation Protocol (SC4NCM), that is, SC4NCM data packets.

[0055] In one embodiment using a single-machine, multi-GPU environment as an example, Figure 6 This is a flowchart of the computation task processing according to this embodiment of the application. The task processing flow is as follows:

[0056] (1) The CPU initiates a computing task and decomposes the computing task into several ordered subtasks based on the network topology information table. For example, the several ordered subtasks are as follows: A1->C2->B1->J1->K2->E1->H2.

[0057] (2) The CPU constructs an SC4NCM data packet based on the decomposed subtasks processed in an ordered manner and routes it to the initial node (i.e., the near-memory computing module that processes the subtask with the first order).

[0058] (3) The node (near-memory computing module or CPU that processes the subtask) completes the corresponding segmented computation according to the OP type in the SC4NCM data packet protocol, generates the processed SC4NCM data packet, and routes it to the next node. The segmented computation mentioned in the embodiments of this application is a type of distributed computing.

[0059] (4) Repeat step (3) until all computational subtasks are completed and the final generated SC4NCM data packet is returned to the processor. The data path is as follows: Figure 6 As shown by the curve in the middle.

[0060] Similarly, this protocol can also be used in scenarios such as multi-server or cloud server, and has good versatility and scalability.

[0061] It should be noted that in this patent application, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one" does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element. In this patent application, if it refers to performing an action according to an element, it means performing the action at least according to that element, including two cases: performing the action only according to that element, and performing the action according to that element and other elements. Expressions such as "multiple," "repeatedly," and "various" include two, two times, two kinds, and more than two, more than two times, and more than two kinds.

[0062] All references to this specification are considered to be incorporated integrally into the disclosure of this application so that they can serve as the basis for modifications if necessary. Furthermore, it should be understood that the above descriptions are merely preferred embodiments of this specification and are not intended to limit the scope of protection of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the scope of protection of one or more embodiments of this specification.

[0063] In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

Claims

1. An AI computing platform, characterized in that, include: At least one computing component, wherein the computing component includes: A processor, configured to initiate a computational task and decompose the computational task into several ordered subtasks based on a stored network topology information table; and A plurality of near-memory computing modules, each of which is connected to the processor, and any two of which are connected to each other. Each of the plurality of near-memory computing modules is configured to implement different types of operations, and each of the plurality of near-memory computing modules completes one or more of the plurality of subtasks according to the types of operations it implements. The processor is further configured to generate data packets based on the near-memory segmented computing protocol SC4NCM according to the decomposed ordered subtasks, and route the data packets to the near-memory computing module for processing the subtask ordered first according to the processing order of the subtasks; the data packets include: a list of near-memory computing modules required to process the data packets; The near-memory computing module is also used to process the corresponding subtask when the data packet is received to generate a processed data packet, and to send the processed data packet to the near-memory computing module for processing the next subtask according to the near-memory computing module list, wherein the near-memory computing module for processing the last subtask returns the processed data packet it generates to the processor.

2. The AI ​​computing platform according to claim 1, characterized in that, The network topology information table includes: the IP address of the host where the processor is located, the index of the near-memory computing module, the types of operations supported by the near-memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules.

3. The AI ​​computing platform according to claim 1, characterized in that, The data packet also includes: the number of near-memory computing modules required to process the data packet, the packet header cyclic redundancy check, the payload length, and the payload.

4. The AI ​​computing platform according to claim 1, characterized in that, The list of near-memory computing modules includes the IP address, index, supported operation types, and load rate of the host where the near-memory computing module that processes each subtask is located.

5. The AI ​​computing platform according to claim 1, characterized in that, The processor is also used to implement different types of operations than the plurality of near-memory computing modules, wherein the processor and the plurality of near-memory computing modules complete one or more of the plurality of subtasks according to the types of operations they implement.

6. The AI ​​computing platform according to claim 1, characterized in that, The processors of each computing component are connected via a bus.

7. An AI calculation method, characterized in that, The method is applied to a computing component, which includes a processor and a plurality of near-memory computing modules. The processor is connected to each of the plurality of near-memory computing modules, and any two near-memory computing modules are connected to each other. Each of the plurality of near-memory computing modules is configured to implement different types of operations. The calculation method includes: The processor initiates a computing task and decomposes the computing task into several ordered subtasks according to the stored network topology information table; The processor generates data packets based on the decomposed ordered subtasks using the SC4NCM (Segmented Computation Protocol for Near Memory) near-memory computing protocol, and routes these data packets to the near-memory computing module that processes the first subtask according to their processing order. The data packets include: a list of near-memory computing modules required to process the data packets; and... When the near-memory computing module receives the data packet, it processes the corresponding subtask to generate the processed data packet and routes the processed data packet to the near-memory computing module that processes the next subtask until all subtasks are completed. The near-memory computing module that processes the last subtask returns the processed data packet it generates to the connected processor.

8. The AI ​​calculation method according to claim 7, characterized in that, The network topology information table includes: the IP address of the host where the processor is located, the index of the near-memory computing module, the types of operations supported by the near-memory computing module, the load rate, the number of adjacent memory computing modules, and the types of operations supported by the adjacent memory computing modules.

9. The AI ​​calculation method according to claim 7, characterized in that, The data packet also includes: the number of near-memory computing modules required to process the data packet, the packet header cyclic redundancy check, the payload length, and the payload.

10. The AI ​​calculation method according to claim 7, characterized in that, The list of near-memory computing modules includes the IP address, index, supported operation types, and load rate of the host where the near-memory computing module that processes each subtask is located.

11. The AI ​​calculation method according to claim 7, characterized in that, The processor is used to implement different types of operations than the plurality of near-memory computing modules, wherein the processor and the plurality of near-memory computing modules complete one or more of the plurality of subtasks according to the types of operations they implement.

12. An AI cloud computing system, characterized in that, It includes: a cloud computing center, and several AI computing platforms as described in any one of claims 1-6, wherein the cloud computing center is connected to the several AI computing platforms.

Images