Intelligent computing cluster based on multi-core CPU-NPU cooperation

CN118939425APending Publication Date: 2024-11-12CHINA SHIP SCIENTIFIC RESEARCH CENTER

Patent Information

Application Number
CN202410989075.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-12

Smart Images

  • Figure CN118939425A_ABST
    Figure CN118939425A_ABST
Patent Text Reader

Abstract

The invention discloses an intelligent computing cluster based on multi-core CPU-NPU cooperation, and relates to the field of artificial intelligence clusters, the intelligent computing cluster comprises a plurality of computing nodes deployed based on a bare metal communication framework and a distributed memory, all solving cores of each multi-core CPU integrated in each computing node are divided into a plurality of logic partitions, and each logic partition is divided into a plurality of logic partitions; the solving core contained in each logic partition is matched and bound with one NPU in the same computing node to form a hybrid computing unit; the cluster manager uses the hybrid computing unit as a minimum scheduling unit to allocate the computing task, and the NPU in the hybrid computing unit allocated to the computing task and the locally matched and bound solving core cooperatively execute the computing task, so that the resource consumption of cross-node communication in the multi-core CPU-NPU task collaboration process can be reduced; therefore, the bandwidth and time consumption of data exchange are reduced, the execution efficiency of the calculation task is improved, and the task execution time consumption is reduced.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence clusters, and in particular to an intelligent computing cluster based on multi-core CPU-NPU collaboration. Background Art

[0002] With the rapid development and widespread application of big data and artificial intelligence technologies, the demand for computing power in various industries is also growing rapidly. Traditional single hardware devices are difficult to meet such large-scale computing power requirements. In the complex context of the times, using multiple hardware devices to build intelligent computing clusters to provide high computing power has become the current development direction.

[0003] For example, more and more industries are constantly combining artificial intelligence models with various businesses. The training of artificial intelligence models has a large computing power requirement, so it is often necessary to use intelligent computing clusters to implement it. At present, most intelligent computing clusters used for artificial intelligence model training adopt a CPU-NPU node architecture. Multiple CPUs (Central Processing Units) and multiple NPUs (Neural Processing Units) are integrated in the intelligent computing cluster. Sockets are used between CPUs and NPUs to provide a network connection method. When executing a training task of an artificial intelligence model, several corresponding CPUs and NPUs are allocated to the training task according to the task and scheduling strategy. The allocated CPU performs data preprocessing on a large amount of training data before starting the iterative update of model parameters, and then the allocated NPU implements model training and iterative optimization. However, the CPU and NPU allocated to a training task are automatically allocated according to the task and scheduling strategy. There is often cross-core, cross-node, and even cross-partition data interaction between the CPU and NPU, resulting in unreasonable resource allocation and unnecessary communication delay, which in turn affects the overall computing efficiency. Summary of the invention

[0004] In response to the above problems and technical requirements, this application proposes an intelligent computing cluster based on multi-core CPU-NPU collaboration. The technical solution of this application is as follows:

[0005] An intelligent computing cluster based on multi-core CPU-NPU collaboration, the intelligent computing cluster includes multiple computing nodes and distributed storage deployed based on a bare metal communication framework, and the multiple computing nodes and distributed storage are connected to each other through a network;

[0006] Each computing node integrates several multi-core CPUs and several NPUs. All the solution cores of each multi-core CPU are divided into multiple logical partitions, and each logical partition includes several solution cores. A logical partition of each multi-core CPU is matched and bound with an NPU in the same computing node to form a hybrid computing unit.

[0007] A scheduling management system is deployed on the hardware resources of one of the computing nodes to form a cluster manager. The cluster manager obtains the corresponding task data from the distributed memory according to the received computing tasks, and assigns the computing tasks and their corresponding task data to the corresponding hybrid computing units. The NPU in the hybrid computing unit and the matching bound solving core jointly execute the computing tasks according to the assigned task data.

[0008] A further technical solution is that a logical partition of each multi-core CPU is matched and bound with an NPU in the same computing node through identification and communication via a PCIe switch.

[0009] Its further technical solution is that when the resource requirements of a computing task exceed the computing resources provided by a hybrid computing unit, the cluster manager distributes the computing task and its corresponding task data to multiple hybrid computing units according to the resource requirements of the computing task, and the multiple hybrid computing units interact with each other to execute the computing tasks in parallel, and the total computing resources provided by the multiple hybrid computing units meet the resource requirements of the computing task.

[0010] Its further technical solution is that when there are multiple hybrid computing units in an idle state and whose solution cores belong to the same multi-core CPU and the total computing resources provided meet the resource requirements of the computing task, the cluster manager will allocate the computing task and its corresponding task data to multiple hybrid computing units formed by matching and binding with the same multi-core CPU; otherwise, the cluster manager will allocate the computing task and its corresponding task data to hybrid computing units formed by matching and binding with multiple multi-core CPUs.

[0011] Its further technical solution is that when there are multiple hybrid computing units in an idle state and whose solution cores belong to multiple multi-core CPUs in the same computing node, and the total computing resources provided meet the resource requirements of the computing task, the cluster manager will allocate the computing task and its corresponding task data to multiple hybrid computing units formed by matching and binding multiple multi-core CPUs in the same computing node; otherwise, the cluster manager will allocate the computing task and its corresponding task data to the hybrid computing units formed in multiple computing nodes.

[0012] A further technical solution is that the cluster manager distributes computing tasks and their corresponding task data to hybrid computing units in multiple computing nodes according to the distribution principle of distributing to the least number of computing nodes.

[0013] A further technical solution is that the scheduling management system deployed on the cluster manager is implemented based on Slurm.

[0014] A further technical solution is that the cluster manager stores an execution log of each computing task, and the execution log of each computing task records the execution tracing information of the computing task.

[0015] Its further technical solution is that different computing nodes are connected through a parameter sharing network, and a communication connection is established between the computing nodes and the distributed storage through a task storage network; the cluster manager obtains the task data of the computing tasks from the distributed storage via the task storage network, and distributes the computing tasks and their task data to the corresponding hybrid computing units via the parameter sharing network.

[0016] A further technical solution is that the number of solution cores of the multi-core CPUs included in any two hybrid computing units is the same or different.

[0017] The beneficial technical effects of this application are:

[0018] The present application discloses an intelligent computing cluster based on multi-core CPU-NPU collaboration. The intelligent computing cluster is deployed based on a bare metal communication framework to support customized access control between CPU, NPU and memory. The solution core of a single multi-core CPU is logically divided through a neighbor binding strategy, and is assigned to a single NPU in the same partition and node for binding. A smaller and tighter hybrid computing unit integrating CPU and NPU is constructed. The hybrid computing unit is used as the minimum scheduling unit to implement diversified computing tasks such as data preprocessing and model training that are adapted to different hardware requirements. To a certain extent, the resource consumption of cross-node communication in the multi-core CPU-NPU task collaboration process is reduced, thereby reducing the bandwidth and time consumption of data exchange, improving the execution efficiency of computing tasks, and reducing the time consumption of task execution.

[0019] The scheduling management system deployed on the cluster manager of the intelligent computing cluster adopts a multi-core CPU-NPU collaborative optimization strategy for task scheduling. When allocating computing tasks, it allocates NPUs according to resource requirements and uses the CPUs that match and bind to them to solve the core. When allocating tasks, the same multi-core CPU is used first, followed by different multi-core CPUs in the same computing node, and finally different multi-core CPUs in different computing nodes, so as to make full use of the resources in the local Socket as much as possible and reduce unnecessary delays caused by cross-Socket applications.

[0020] The scheduling management system deployed on the cluster manager of the intelligent computing cluster is based on Slurm deployment to realize cluster management and job scheduling functions. It uses a non-virtualized solution to further improve the traceability of information transmission when the intelligent computing cluster processes sensitive information, avoids problems such as information tracing difficulties in the virtualization process, and is suitable for reliable computing of sensitive information or data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 It is a schematic diagram of the architecture of an intelligent computing cluster according to an embodiment of the present application.

[0022] Figure 2 The diagram is a schematic diagram of the matching binding structure of a multi-core CPU and an NPU in an example. DETAILED DESCRIPTION

[0023] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0024] This application discloses an intelligent computing cluster based on multi-core CPU-NPU collaboration, please refer to Figure 1 The schematic diagram of the architecture of the intelligent computing cluster shown in the figure includes multiple computing nodes and distributed memories deployed based on a bare metal communication framework. The multiple computing nodes and distributed memories establish access connections with each other through the network. Each computing node is integrated with several multi-core CPUs and several NPUs. The number and configuration of the multi-core CPUs and NPUs integrated in each computing node can be the same or different. The method deployed based on the bare metal communication framework can support customized access control between multi-core CPUs, NPUs and distributed memories, and provides convenient conditions for realizing multi-core CPU-NPU collaboration. In one embodiment, Figure 1 As shown, different computing nodes are connected through a parameter sharing network, and a communication connection is established between the computing nodes and the distributed storage through a task storage network.

[0025] The intelligent computing cluster of this application introduces a multi-core CPU-NPU neighbor binding strategy. Please refer to Figure 2 The diagram shows the neighbor binding of a single multi-core CPU and an NPU. All the solving cores of each multi-core CPU are divided into multiple logical partitions, each logical partition includes several solving cores, and the number of solving cores contained in each logical partition can be equal or unequal. A logical partition of each multi-core CPU is matched and bound with an NPU in the same computing node to form a hybrid computing unit. Please refer to Figure 1 In one embodiment, a logical partition of each multi-core CPU is identified and communicated with an NPU in the same computing node through a PCIe switch to achieve matching and binding.

[0026] In one embodiment, the intelligent computing cluster of the present application is built based on the Kunpeng 920 CPU and the Ascend 910 NPU, and each computing node contains 4 Kunpeng 920 CPUs and 8 Ascend 910 NPUs. The 48 solution cores contained in each Kunpeng 920 CPU are divided into two logical partitions, and each logical partition includes 24 solution cores. The 24 solution cores of a logical partition of each Kunpeng 920 CPU are matched and bound to the only Ascend 910 NPU in the same partition and the same computing node to form a hybrid computing unit. Thus, each Kunpeng 920 CPU is matched and bound to two Ascend 910 NPUs respectively to form two hybrid computing units, and a total of 8 hybrid computing units are formed in each computing node. Figure 2 Take this as an example. The multi-core CPU and NPU in each computing node may have different core numbers and configuration quantities. In practice, the multi-core CPU and NPU in each computing node can be reasonably allocated according to configuration requirements and characteristics. Figure 2 This is just an example. Depending on the actual resources, the number of solution cores of the multi-core CPU included in any two hybrid computing units may be the same or different.

[0027] A scheduling management system is deployed on the hardware resources of one of the computing nodes to form a cluster manager, and the integrated manager is used to schedule the working process of the intelligent computing cluster. In one embodiment, the scheduling management system deployed on the cluster manager is implemented based on Slurm, and the scheduling management system implemented based on Slurm has three key functions: first, it can allocate exclusive and / or non-exclusive access rights to computing resources in the computing node for computing tasks within a period of time; second, it can provide framework support for starting, executing and monitoring work on the allocated node set; third, it can determine the occupancy of computing resources by managing the queue of pending work.

[0028] The cluster manager obtains the task data required for the computing task from the distributed storage according to the received computing task, and distributes the computing task and its corresponding task data to the corresponding hybrid computing unit. When the computing nodes and distributed storage are efficiently organized through the parameter sharing network and the task storage network, the cluster manager obtains the task data of the computing task from the distributed storage via the task storage network, and distributes the computing task and its task data to the corresponding hybrid computing unit via the parameter sharing network.

[0029] Since the scheduling management system based on Slurm can determine the occupancy of computing resources by managing the queue of pending work, the state of each hybrid computing unit can be determined. The state of each hybrid computing unit is busy or idle. The busy state indicates that the hybrid computing unit is performing other computing tasks, and the idle state indicates that the hybrid computing unit is not performing other computing tasks. When the cluster manager allocates computing tasks and their corresponding task data to hybrid computing units, it allocates them to the hybrid computing units in the idle state. The NPU in the allocated hybrid computing unit and the matching bound solution core will collaboratively perform the computing tasks according to the allocated task data.

[0030] This application uses a hybrid computing unit as the minimum unit for task scheduling to perform task allocation. The CPU solution core and NPU in the same hybrid computing unit have pre-implemented neighbor binding to achieve efficient communication within the same computing node. The NPU can make full use of the resources in the local Socket to collaborate with the CPU solution core, thereby reducing unnecessary delays caused by cross-Socket applications and improving cross-Socket communication efficiency and bandwidth utilization.

[0031] When the computing resources provided by a hybrid computing unit can meet the resource requirements of the computing task, the computing task can be directly assigned to a hybrid computing unit. However, it is difficult to complete a computing task by a single hybrid computing unit. When the resource requirements of the computing task exceed the computing resources provided by a hybrid computing unit, the cluster manager assigns the computing task and its corresponding task data to multiple hybrid computing units according to the resource requirements of the computing task. These multiple hybrid computing units exchange data to execute the computing task in parallel, and the total computing resources provided by multiple hybrid computing units meet the resource requirements of the computing task. When the same computing task is assigned to multiple hybrid computing units, there will be cross-Socket communication between different hybrid computing units, including two types of cross-Socket communication: one is the communication between different hybrid computing units in the same computing node, and the other is the communication between hybrid computing units in different computing nodes.

[0032] In order to minimize the delay caused by cross-Socket communication, when there are multiple hybrid computing units in idle state and whose solution cores belong to the same multi-core CPU and provide total computing resources that meet the resource requirements of computing tasks, the cluster manager will assign computing tasks and their corresponding task data to multiple hybrid computing units formed by matching and binding the same multi-core CPU, otherwise the cluster manager will assign computing tasks and their corresponding task data to hybrid computing units formed by matching and binding multiple multi-core CPUs. That is, the cluster manager will give priority to assigning the same computing task to the hybrid computing unit formed by the same multi-core CPU, which can effectively avoid cross-CPU communication losses.

[0033] In the case where it is necessary to assign computing tasks and their corresponding task data to hybrid computing units formed by matching and binding multiple multi-core CPUs, when there are multiple hybrid computing units in an idle state and whose solution cores belong to multiple multi-core CPUs in the same computing node, and the total computing resources provided meet the resource requirements of the computing tasks, the cluster manager assigns the computing tasks and their corresponding task data to multiple hybrid computing units formed by matching and binding multiple multi-core CPUs in the same computing node, otherwise the cluster manager assigns the computing tasks and their corresponding task data to hybrid computing units formed in multiple computing nodes. That is, the cluster manager preferentially assigns the same computing task to the hybrid computing units in the same computing node, which can effectively avoid communication losses caused by cross-computing nodes. When the computing task runs only on a single computing node, the hybrid computing unit organizes the multi-core CPU and NPU to work together in an orderly manner, quickly assigns the CPU core used for task solving and the bound NPU for concurrent computing, and realizes efficient collaborative tuning of multi-core CPU-NPU, shortens the path of local socket data transmission, optimizes the local memory access mode and improves bandwidth utilization.

[0034] When it is necessary to distribute computing tasks and their corresponding task data to hybrid computing units within multiple computing nodes, the cluster manager distributes the computing tasks and their corresponding task data to the hybrid computing units within multiple computing nodes according to the distribution principle of distributing them to the least number of computing nodes. That is, the cluster manager distributes the computing tasks and their corresponding task data to multiple computing nodes as concentratedly as possible to minimize the communication loss caused by cross-computing nodes.

[0035] Therefore, according to the different resource requirements of the computing tasks and the different states of different hybrid computing units in the intelligent computing cluster, the hybrid computing unit actually used to execute a computing task includes a multi-core CPU and an NPU, or includes a multi-core CPU and multiple NPUs, or includes multiple multi-core CPUs and multiple NPUs.

[0036] Experimental verification shows that under the same YOLO example and data set computing tasks, the calculation time when automatically allocating CPU and NPU according to the traditional method is 8.5 hours, while the training time is shortened to 7.8 hours after using the method of this application, and the overall performance is improved by more than 8%.

[0037] In addition, the traditional approach is to build resource pools through virtualization of hardware resources, so as to provide computing services to the outside world to achieve efficient use of resources. However, in some special industries or fields, users need to ensure that the input and output data and models are traceable and protected within the complete processing flow. The means of virtualization, while providing a relatively convenient environment, add considerable difficulty to the tracking of sensitive information. The scheduling management system implemented based on Slurm in this application can dynamically update and store the execution information of each computing task as an execution log, which records the execution traceability information of the computing task, including the execution start time, execution end time, execution time of the computing task, the identification of the hardware device to which the hybrid computing unit that executes the computing task belongs, and the intermediate data generated by various execution processes, etc. The execution log makes the input and output data and models traceable and protected within the complete processing flow, avoiding the problems of information traceability difficulties in the virtualization process, and is suitable for reliable calculation of sensitive information or data.

[0038] The above is only a preferred embodiment of the present application, and the present application is not limited to the above embodiments. It is understood that other improvements and changes directly derived or associated by those skilled in the art without departing from the spirit and concept of the present application should be considered to be included in the protection scope of the present application.

Claims

1. An intelligent computing cluster based on multi-core CPU-NPU collaboration, characterized in that: The intelligent computing cluster includes multiple computing nodes and distributed storage deployed based on a bare metal communication framework, and the multiple computing nodes and distributed storage are connected to each other through a network; Each computing node integrates several multi-core CPUs and several NPUs. All the solution cores of each multi-core CPU are divided into multiple logical partitions, and each logical partition includes several solution cores. A logical partition of each multi-core CPU is matched and bound with an NPU in the same computing node to form a hybrid computing unit. A scheduling management system is deployed on the hardware resources of one of the computing nodes to form a cluster manager. The cluster manager obtains corresponding task data from the distributed memory according to the received computing task, and assigns the computing task and its corresponding task data to the corresponding hybrid computing unit. The NPU in the hybrid computing unit and the matching bound solving core jointly execute the computing task according to the assigned task data.

2. The intelligent computing cluster according to claim 1, characterized in that: A logical partition of each multi-core CPU is identified and communicated with an NPU in the same computing node through a PCIe switch to achieve matching and binding.

3. The intelligent computing cluster according to claim 1, characterized in that: When the resource requirements of the computing task exceed the computing resources provided by a hybrid computing unit, the cluster manager distributes the computing task and its corresponding task data to multiple hybrid computing units according to the resource requirements of the computing task. The multiple hybrid computing units interact with each other to execute the computing task in parallel. The total computing resources provided by the multiple hybrid computing units meet the resource requirements of the computing task.

4. The intelligent computing cluster according to claim 3, characterized in that: When the total computing resources provided by multiple hybrid computing units that are in an idle state and whose solution cores belong to the same multi-core CPU meet the resource requirements of the computing task, the cluster manager allocates the computing task and its corresponding task data to multiple hybrid computing units formed by matching and binding the same multi-core CPU; otherwise, the cluster manager allocates the computing task and its corresponding task data to hybrid computing units formed by matching and binding multiple multi-core CPUs.

5. The intelligent computing cluster according to claim 4, characterized in that: When there are multiple hybrid computing units in an idle state and whose solution cores belong to multiple multi-core CPUs in the same computing node, and the total computing resources provided meet the resource requirements of the computing task, the cluster manager allocates the computing task and its corresponding task data to multiple hybrid computing units formed by matching and binding multiple multi-core CPUs in the same computing node; otherwise, the cluster manager allocates the computing task and its corresponding task data to hybrid computing units formed in multiple computing nodes.

6. The intelligent computing cluster according to claim 5, characterized in that: The cluster manager distributes the computing tasks and their corresponding task data to the hybrid computing units in the plurality of computing nodes according to the distribution principle of distributing to the least number of computing nodes.

7. The intelligent computing cluster according to claim 1, characterized in that: The scheduling management system deployed on the cluster manager is implemented based on Slurm.

8. The intelligent computing cluster according to claim 7, characterized in that: The cluster manager stores an execution log of each computing task, and the execution log of each computing task records execution tracing information of the computing task.

9. The intelligent computing cluster according to claim 1, characterized in that: Different computing nodes are connected through a parameter sharing network, and a communication connection is established between the computing nodes and the distributed storage through a task storage network; the cluster manager obtains the task data of the computing tasks from the distributed storage via the task storage network, and distributes the computing tasks and their task data to the corresponding hybrid computing units via the parameter sharing network.

10. The intelligent computing cluster according to claim 1, characterized in that: The number of solution cores of the multi-core CPUs included in any two hybrid computing units may be the same or different.

Citation Information

Patent Citations

  • Method for constructing slurm scheduling parallel computing cluster

    CN114745385A

  • High-expansion node system based on CPU-NPU cooperation and training method

    CN116074179A

  • Inference task scheduling method and device and computer equipment

    CN116841706A

  • NPU fault-tolerant scheduling system of computer cluster

    CN117632444A

  • Multi-processor system

    US20030009626A1

Cited By

  • NPU time division multiplexing real-time reasoning system and scheduling method

    CN119668811A

  • Task scheduling method and device, chip, equipment and storage medium

    CN121210071A