A large-scale heterogeneous brain-like cluster software stack fusion method and system

Abstract

The present invention discloses a large-scale heterogeneous brain-like cluster software stack fusion method and system, which realizes efficient collaboration of heterogeneous computing nodes through embedded calls, hierarchical calls and fusion of software stacks; adopts pooling layer resource scheduling and communication delay hiding technology to improve cluster communication performance; introduces resource mapping and task scheduling strategies to enhance the adaptability and flexibility of the system. At the same time, virtualization technology is introduced to pool heterogeneous hardware resources into dynamically allocable and schedulable virtual resources, thereby improving resource utilization. At the accelerator level, a code operation and communication abstraction mechanism with non-disassembled dimensions is proposed, and the computing and communication tasks of the accelerator are abstracted into three types: signaling messages, data, and control, so as to simplify the programming and management complexity of heterogeneous accelerators. Through means such as equipment commissioning and performance analysis, the reliable operation of the system is guaranteed, the execution efficiency of tasks in large-scale heterogeneous clusters is improved, and the efficient operation of computing and communication capabilities is achieved.

Description

Technical Field

[0001] The present invention relates to the field of large-scale heterogeneous brain-like cluster computing, and in particular to a large-scale heterogeneous brain-like cluster software stack fusion method and system. Background Art

[0002] With the rapid development of artificial intelligence and big data technology, the demand for high-performance computing is growing. In order to cope with increasingly complex computing tasks, researchers have begun to explore the emerging field of brain-like computing. Brain-like computing is a computing model that draws on the structure and function of biological nervous systems and uses large-scale parallel and distributed information processing mechanisms to achieve efficient solutions to complex problems.

[0003] In order to support brain-like computing, people have proposed the idea of ​​building large-scale heterogeneous brain-like clusters. The so-called heterogeneous brain-like cluster refers to a distributed computing system composed of multiple different types of computing nodes, which may include general-purpose CPUs, GPUs, and specially designed brain-like chips. By organizing these heterogeneous computing resources into a unified cluster architecture, their respective computing advantages can be brought into play, while improving the overall performance of the system.

[0004] However, existing large-scale heterogeneous brain-like clusters still face many challenges and problems in practical applications.

[0005] First, the diversity of heterogeneous computing nodes has greatly increased the complexity of cluster systems. Different types of nodes have significant differences in architecture, instruction set, programming model, etc. How to achieve efficient collaboration between them is a key issue. Existing cluster systems lack a unified software stack and programming framework, which makes application development and migration difficult.

[0006] Secondly, large-scale heterogeneous brain-like clusters have very high requirements for communication and synchronization between nodes. Brain-like computing usually involves a large amount of data exchange and frequent synchronization operations, which poses severe challenges to the communication performance of the cluster. Existing cluster systems still have a lot of room for improvement in communication efficiency and latency optimization, making it difficult to fully tap the computing potential of brain-like chips.

[0007] Thirdly, the resource management and task scheduling issues of heterogeneous brain-like clusters also need to be solved urgently. In a complex heterogeneous environment, how to dynamically allocate and schedule computing resources according to the characteristics of computing tasks and the load conditions of the system to achieve the optimization of system performance is a complex combinatorial optimization problem. Existing cluster scheduling algorithms and strategies are not intelligent enough to meet the needs of brain-like computing.

[0008] In addition, the reliability and maintainability of brain-like cluster systems also face challenges. Due to the large scale and complex composition of the system, how to effectively monitor, diagnose and maintain the cluster hardware and software, and how to ensure that the system can maintain normal operation when some nodes or links fail, are issues that need to be addressed urgently. Summary of the invention

[0009] The embodiments of the present invention provide a large-scale heterogeneous brain-like cluster software stack fusion method and system, which effectively solves the difficulties faced by existing heterogeneous brain-like clusters through embedded calling, hierarchical calling and fusion of software stacks.

[0010] According to one aspect of the present invention, a large-scale heterogeneous brain-like cluster software stack fusion method is provided, the method comprising:

[0011] Configure each component to provide communication library functions and distributed cluster computing acceleration libraries, and achieve software stack integration from the communication layer and acceleration layer.

[0012] For the acceleration card device after the cluster power-on self-test is completed, the data routing table or message routing table of the acceleration card device is configured in a layer-by-layer progressive manner using the routing configuration tool based on the transmission write function of the communication protocol between the PCIe port and the master control server and the message ring mechanism;

[0013] Combined with the brain-like chip controller in the accelerator card device, the algorithm mapping is completed, and the SPCP (Step Phase Core Protocol) timing control diagram of the brain-like chip is completed in combination with the algorithm running timing, and the basic functions of the cluster host and the accelerator card device are used based on the SPCP timing control diagram of the brain-like chip to generate the corresponding microcode;

[0014] Use the device debugger to perform pre-maintenance and status tracking on cluster hardware accelerator devices, and use the performance analyzer to understand the performance of user software multi-tenants in real time;

[0015] Using the pooling layer as an interface module between the super manager and the mapper, and as an interface module between the super manager and the task job, in combination with the user scheduling layer and the corresponding layer to delay hidden communication and calculation;

[0016] The compiler is used to convert the user's coding model into cluster host software and resource lists, and the resource lists are transmitted to the mapper. The mapper combines the status of the existing accelerator card device and maps the layout to the actual physical unit of the corresponding accelerator card device through mapping core algorithms and optimization strategies.

[0017] In some embodiments, the brain-like chip controller includes a microcode controller and a clock-precision emulator. The microcode controller is used to complete the assembly of the brain-like chip service package, and the clock-precision emulator is used to perform precise timing control of the brain-like chip based on the microcode.

[0018] In some implementations, using a compiler to convert the user's coding model into a cluster host software and resource list includes:

[0019] The compiler is used to convert the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

[0020] In some implementations, before configuring a data routing table or a message routing table for the accelerator card device, the method further includes the following steps:

[0021] Connect the accelerator card and the service board according to the network topology of the brain-like cluster system;

[0022] The host configures the PCIe port to communicate with the host control server through the relevant data and message area.

[0023] In some implementations, after configuring the PCIe port through the host to communicate with the host server through the relevant data and message area, the method further includes:

[0024] The master control server configures a routing table of a source module for communicating with the PCIe port through a BAR (Base Address Register) space of PCIe, and the routing table belongs to the data domain;

[0025] After the source module for the communication between the PCIe port and the main control server is configured, the data routing tables of other ports are configured through the communication between the PCIe port and the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Ports). The routing configuration channel of the PCIe port is implemented in the form of the corresponding channel (Channel) of the host system bus (Bus) of the main control server. The data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

[0026] In some implementations, after the source module for communicating between the PCIe port and the master control server is configured, the method further includes the following steps:

[0027] According to the configuration file generated by the compiler, configure the macro instruction processor code of each accelerator card device, the configuration part of the actuator module and the brain-like chip;

[0028] The host initiates an enabling action, which is driven and implemented in a data stream driven manner.

[0029] According to one aspect of the present invention, a large-scale heterogeneous brain-like cluster software stack fusion system is provided, the system comprising:

[0030] A compiler, used to convert the user's coding model into cluster host software and resource lists;

[0031] The host is used to configure the data and message areas related to the communication between the PCIe port and the main control server, and configure the routing table of the source module for the communication between the PCIe port and the main control server through the base address register space;

[0032] Routing configuration tool, used to configure the data routing table or message routing table of the accelerator card device after the cluster power-on self-test is completed;

[0033] A brain-like chip controller, used to complete the SPCP (StepPhase Core Protocol) timing control diagram of the brain-like chip in combination with algorithm mapping and operation timing, and generate corresponding microcode based on the timing control diagram;

[0034] Device debugger, used to perform pre-maintenance and status tracking of cluster hardware accelerator devices;

[0035] Performance analyzer, used to understand the performance of multi-tenants of user software in real time;

[0036] The pooling layer is used as an interface module between the super manager and the mapper and task job, and is used in conjunction with the user scheduling layer and the corresponding layer to delay hidden communication and computation;

[0037] The mapper is used to receive the resource list and map the resource list to the corresponding actual physical unit through mapping core algorithm and optimization strategy in combination with the status of the existing accelerator card device.

[0038] In some embodiments, the brain-like chip controller includes:

[0039] Microcode controller, used to complete the construction of brain-like chip business packages;

[0040] A clock-precision simulator is used to perform precise timing control on the brain-like chip based on the microcode.

[0041] In some implementations, the compiler converts the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

[0042] In some embodiments, after the source module for the PCIe port to communicate with the main control server is configured, the data routing tables of other ports are configured through the PCIe port to communicate with the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Port), wherein the routing configuration channel of the port is implemented in the form of the corresponding channel (Channel) of the system bus (Bus), the data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

[0043] According to the technical solution of the present invention, there are at least the following beneficial effects:

[0044] 1. Improving the collaborative efficiency of heterogeneous brain-inspired clusters

[0045] The large-scale heterogeneous brain-like cluster software stack fusion solution proposed in the present invention realizes the fusion of software stacks at the communication layer and the acceleration layer, so that different types of computing nodes can work together more efficiently. The unified software stack and programming framework simplifies the application development and transplantation process, reduces the programming complexity in heterogeneous environments, and improves the availability and ease of use of the cluster system.

[0046] (II) Optimizing cluster communication performance and unleashing the computing potential of brain-like chips

[0047] The present invention adopts technologies such as pooling layer and delay hiding, which can realize the superposition and parallelism of distributed cluster computing and communication, and effectively optimize the communication efficiency of the cluster system. Among them, the pooling layer is used as the interface between the super manager and other modules, combined with the user scheduling layer and the corresponding layer, to achieve communication and computing delay hiding, and reduce the impact of communication overhead on computing performance. At the same time, the present invention also further improves the communication efficiency of the cluster through means such as routing configuration and performance analysis, and gives full play to the computing potential of brain-like chips.

[0048] 3. Enhance the intelligence level of resource management and task scheduling

[0049] The present invention introduces intelligent resource mapping and task scheduling strategies, which enhance the adaptability and flexibility of heterogeneous brain-like clusters. The compiler converts the user's coding model into cluster host software and resource lists, and then the mapper uses the core algorithm and optimization strategy to map the resources to the actual physical units according to the status of the existing accelerator card device, thereby realizing the dynamic allocation and optimized scheduling of computing resources. In addition, this intelligent scheduling method can adaptively adjust resource configuration according to system load and task characteristics, thereby improving the overall performance and efficiency of the cluster.

[0050] 4. Ensuring the reliability and maintainability of heterogeneous brain-like clusters

[0051] The present invention adopts means such as equipment debugging and performance analysis to improve the reliability and maintainability of heterogeneous brain-like clusters. Among them, the equipment debugger performs pre-maintenance and status tracking on the cluster hardware, which can timely discover and diagnose potential faults and reduce the unavailability time of the system; the performance analyzer can monitor the running status and performance indicators of user software in real time, providing a basis for system optimization and problem location. These measures effectively ensure the stable operation and continuous optimization of heterogeneous brain-like clusters.

[0052] In summary, the large-scale heterogeneous brain-like cluster software stack fusion method and system proposed in the present invention have significant advantages in improving cluster collaboration efficiency, improving communication performance, enhancing resource scheduling, and ensuring system reliability, and are of great significance to the large-scale application and technological progress of brain-like computing.

[0053] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the specification and, together with the description, serve to explain the principles of the specification.

[0055] Figure 1 A schematic flow chart of a large-scale heterogeneous brain-like cluster software stack fusion method according to an embodiment of the present invention is shown;

[0056] Figure 2 A complete set of signaling messages and data stream handshake synchronization communication scheme in the longitudinal dimension according to an exemplary embodiment of the present invention is shown;

[0057] Figure 3 A specific schematic diagram showing a signaling message and a data stream handshake synchronization party communication scheme according to an exemplary embodiment of the present invention;

[0058] Figure 4 A schematic diagram of a process of using a large-scale heterogeneous brain-like cluster according to an exemplary embodiment of the present invention is shown;

[0059] Figure 5 A schematic diagram of the architecture design of a heterogeneous brain-like cluster computing platform according to an exemplary embodiment of the present invention is shown. DETAILED DESCRIPTION

[0060] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0061] Figure 1 At least one embodiment of the present invention shows a large-scale heterogeneous brain-like cluster software stack fusion method. Figure 1 As shown, the method includes the following steps 101 to 106.

[0062] In step 101, each component is configured to provide communication library functions and a distributed cluster computing acceleration library, so as to realize the software stack fusion from two aspects: the communication layer and the acceleration layer.

[0063] The components in a large-scale heterogeneous brain-like cluster mainly refer to different types of computing nodes in the cluster, including general-purpose CPUs, GPUs, and specially designed brain-like chips. These heterogeneous computing nodes have significant differences in architecture, instruction set, programming model, etc. According to this embodiment, efficient collaboration between them can be achieved through the fusion of software stacks.

[0064] At the communication level, this embodiment encapsulates the underlying details of communication between heterogeneous nodes by configuring library functions that provide communication functions for each component, so that different components can easily exchange data and synchronize operations. Through the configuration of communication library functions, efficient communication links can be established between heterogeneous nodes.

[0065] At the acceleration level, this embodiment configures a distributed cluster computing acceleration library for heterogeneous clusters. These acceleration libraries can encapsulate commonly used parallel computing modes and algorithms, provide simple and easy-to-use programming interfaces, and enable users to easily develop and deploy applications on heterogeneous clusters. Through the configuration of the acceleration library, heterogeneous nodes can choose to give full play to their respective computing advantages according to their own computing characteristics and capabilities.

[0066] In step 102, for the acceleration card device after the cluster power-on self-test is completed, the routing configuration tool is used based on the transmission write function of the PCIe port and the communication protocol of the main control server, and with the help of the message ring (Message-Ring) mechanism, a layer-by-layer progressive method is used to configure the data routing table or the message routing table of the acceleration card device.

[0067] This embodiment utilizes the routing configuration tool, through the transmission write function of the PCIe port to communicate with the main control server, and with the help of the layer-by-layer progressive method of the message ring mechanism, realizes the configuration of the data routing table and the message routing table of the acceleration card device to establish an efficient data communication and message transmission mechanism.

[0068] The PCIe (Peripheral Component Interconnect Express) port is a high-speed serial computer expansion system bus standard used to connect accelerator card devices on the motherboard. The PCIe port communicates with the master server and is the interface for data transmission between the accelerator card device and the motherboard. It supports high-bandwidth, low-latency communication. The large-scale heterogeneous brain-like cluster according to the present invention uses the PCIe port to communicate with the master server as the main channel for routing configuration and data transmission between the accelerator card devices.

[0069] The message ring mechanism is a message transmission mechanism based on a ring topology, which is used for efficient data exchange and synchronization between devices in a heterogeneous cluster. In the message ring mechanism, devices are organized into a logical ring structure, and each device has a unique identifier and fixed predecessor / successor nodes. Devices communicate by passing messages, and messages are forwarded between nodes in turn according to the ring topology until they reach the target device.

[0070] This embodiment uses a message ring mechanism to configure layer by layer in a progressive manner. According to the present invention, first, the routing configuration tool can communicate with the master control server through the PCIe port to configure the routing table of the device directly connected to the host (i.e., the first layer); then, these devices pass the routing configuration information to the devices in the next layer through the message ring mechanism; and so on, until the routing tables of all accelerator card devices are configured. This progressive approach can simplify the configuration process, reduce configuration time, and improve configuration reliability.

[0071] In step 103, the algorithm mapping is completed in combination with the brain-like chip controller in the acceleration card device, and the SPCP (Step Phase Core Protocol) timing control diagram of the brain-like chip is completed in combination with the algorithm running timing. Based on the SPCP timing control diagram of the brain-like chip, the basic functions of the cluster host and the acceleration card device are used to generate the corresponding microcode.

[0072] In a heterogeneous brain-like cluster system, a brain-like chip is an acceleration device specifically used for brain-like computing. The user's algorithm can be mapped to the brain-like chip, and the corresponding control instructions and data streams can be generated to give full play to the computing power of the brain-like chip.

[0073] The brain-like chip controller is a hardware module specifically used to control and manage brain-like chips. It is responsible for converting the control instructions and data streams of the upper-level software into low-level instructions and data formats that the brain-like chip can understand and execute, and coordinating the interaction between the brain-like chip and other hardware modules (such as memory, interconnection, etc.). The brain-like chip controller is a key component for programming and controlling brain-like chips.

[0074] According to this embodiment, the algorithm mapping is completed in combination with the brain-like chip controller. For example, the brain-like chip controller can map the user's algorithm representation (such as a calculation graph, a mathematical formula, etc.) into a set of basic operations of the brain-like chip (such as neuron activation, synaptic connection, etc.), and determine the execution order and data dependency of these operations.

[0075] In this embodiment, the SPCP timing control diagram of the brain-like chip is used to describe and control the time sequence and rhythm of the brain-like chip when executing the algorithm, that is, the running timing.

[0076] Microcode is an intermediate code form between hardware circuits and software instructions. Microcode is usually used to control complex hardware circuits, such as processors, accelerators, etc. In this embodiment, the microcode is generated according to the SPCP timing control diagram of the brain-like chip, and is used to directly control the operation of the brain-like chip. The microcode can include all the low-level control information required for the brain-like chip to execute the algorithm, such as data reading and writing, neuron activation, synaptic weight update, etc.

[0077] This embodiment combines a brain-like chip controller to complete algorithm mapping and the generation of a timing control diagram, and generates microcode based on the timing control diagram, thereby achieving efficient programming and control of the brain-like chip.

[0078] In step 104, the device debugger is used to perform pre-maintenance and status tracking on the cluster hardware acceleration card device, and the performance analyzer is used to grasp the performance of the multi-tenant user software in real time.

[0079] In the operation process of the heterogeneous brain-like cluster system, in order to ensure the stability, reliability and high performance of the system, it is necessary to continuously monitor and maintain the cluster hardware acceleration card devices and grasp the operation status and performance bottlenecks of the user software in real time. This embodiment introduces a device debugger and a performance analyzer to achieve pre-maintenance and status tracking of cluster hardware acceleration card devices, as well as performance analysis and optimization of multi-tenant user software.

[0080] The device debugger is a special tool for debugging and testing cluster hardware accelerator card devices. The device debugger can be connected to the test interface of the accelerator card device to read, modify and monitor the internal status of the accelerator card device. At the same time, the device debugger can also inject various test data and control signals into the accelerator card device to verify the functional correctness and performance boundaries of the accelerator card device. Through the device debugger, the hardware accelerator card device can be fully tested and pre-maintained before the cluster system is running, improving the stability and reliability of the system.

[0081] A performance analyzer is a tool used to analyze and optimize software performance. By inserting probes on the critical path of the user software or using hardware performance counters, the performance analyzer can collect various performance data during the software operation, such as the number of function calls, execution time, cache hit rate, memory access pattern, etc. By collecting and analyzing these performance data, the performance analyzer can find performance bottlenecks and hot code in the software, and give optimization suggestions to help developers perform targeted performance tuning of the software.

[0082] In a heterogeneous brain-like cluster system, software applications of multiple users can run simultaneously on the same physical cluster, forming a multi-tenant resource sharing model. Each tenant has its own independent software stack and operating environment, but they share the underlying hardware resources and communication network. The user software multi-tenant model can improve the utilization of cluster resources and reduce the cost of use for a single user. However, resource isolation and performance interference between multiple tenants also bring challenges to system management and optimization.

[0083] According to this embodiment, by using the device debugger for pre-maintenance and status tracking, and using the performance analyzer to implement performance analysis and optimization of multi-tenant user software, the reliability and availability of the cluster system are improved while maximizing the performance potential of heterogeneous hardware and multi-tenant software.

[0084] In step 105, the pooling layer is used as an interface module between the super manager and the mapper, and as an interface module between the super manager and the task job, in combination with the user scheduling layer and the corresponding layer to delay hidden communication and calculation.

[0085] In heterogeneous brain-like cluster systems, in order to achieve efficient task scheduling and load balancing, it is necessary to establish a flexible middle layer between user applications and underlying hardware resources. This embodiment introduces the pooling layer as the core component of the middle layer, which plays an important interface and adaptation role between the super manager, mapper and task work. At the same time, the pooling layer is also used in combination with the user scheduling layer and the corresponding layer, and the overlap of communication and calculation is achieved through delay hiding technology, thereby improving the overall performance of the system.

[0086] In this embodiment, the pooling layer is a middleware layer located between the user application and the cluster resource management system. The main function of the pooling layer is to aggregate, convert and distribute the task requests submitted by the user and the allocation requests of the hardware resources to achieve decoupling and adaptation between the upper and lower layers.

[0087] The super manager is the global resource management and scheduling center of the heterogeneous brain-like cluster system. The super manager is responsible for receiving task requests submitted by users and assigning tasks to appropriate computing nodes for execution based on the current state of the system and the predetermined strategy. At the same time, the super manager is also responsible for monitoring the operating status of the entire cluster, such as the health of the nodes, the execution progress of the tasks, the usage of resources, etc., and dynamically adjusts the scheduling strategy of the tasks and the allocation plan of the resources based on this information to ensure the efficient operation and rapid response of the system.

[0088] The mapper is a dedicated module responsible for mapping tasks to resources. The mapper receives task allocation requests from the super manager and calculates the optimal task-to-resource mapping solution based on the type, scale, priority and other attributes of the task, as well as the current available hardware resources, in order to minimize resource idleness and conflicts while meeting task performance requirements, and to achieve load balancing and maximize system efficiency.

[0089] Task work is an application or service submitted by users that needs to be executed on a heterogeneous brain-like cluster system. Task work usually consists of a set of related computing, storage, communication and other operations that work together to complete a specific function or goal. Task work can be of different types, such as batch jobs, interactive queries, real-time control, data analysis, etc., and each type has different requirements for system resources and performance.

[0090] The user scheduling layer is a task submission and monitoring interface for end users. The user scheduling layer provides a set of easy-to-use APIs (Application Programming Interface) or command line tools that allow users to submit and manage their own tasks in a declarative or interactive manner. For example, users can specify the type, input data, execution parameters, priority, deadline and other attributes of the task through the scheduling layer, and can also query the current status, execution progress, resource usage and other information of the task.

[0091] The corresponding layer is an adaptation layer located between the pooling layer and the cluster physical resources. The main function of the corresponding layer is to convert the logical tasks and resource mapping requests generated by the pooling layer into physical instructions and configuration parameters that can be directly understood and executed by the underlying hardware and operating system. Specifically, the corresponding layer needs to interact with the drivers and runtime environments of different types of heterogeneous resources (such as CPU, GPU, brain-like chips, FPGA, etc.), such as OpenMP, CUDA, TensorFlow, MPI, etc., to load task code and data into the specified computing unit, start and monitor the execution of tasks, and recycle and clean up resources after the tasks are completed. The corresponding layer shields the underlying details and differences of heterogeneous resources and provides a unified resource abstraction and control interface for the pooling layer.

[0092] In this embodiment, latency hiding is mainly achieved through the pooling layer, that is, the computing operation of the task is overlapped with the communication operation as much as possible, so that the calculation can continue while waiting for the data to arrive, thereby hiding the impact of communication delay on performance. Specifically, when scheduling tasks, the pooling layer can pre-analyze the data dependencies and communication modes of the tasks, and assign independent tasks to different computing nodes so that they can be executed in parallel; at the same time, the pooling layer will trigger the transmission and exchange of data in advance before the task starts executing, so that communication and computing can overlap. In this way, even when the task is waiting for data to arrive, the node can switch to other ready tasks to continue computing, thereby hiding the communication delay and improving performance.

[0093] According to this embodiment, by introducing a pooling layer between the super manager, mapper and task work, and combining it with the user scheduling layer and the corresponding layer, this embodiment achieves efficient scheduling and flexible adaptation of heterogeneous brain-like cluster systems.

[0094] In step 106, the compiler is used to convert the user's coding model into cluster host software and a resource list, and the resource list is transmitted to the mapper. The mapper combines the status of the existing accelerator card device and maps the layout to the actual physical unit of the corresponding accelerator card device through mapping core algorithms and optimization strategies.

[0095] This embodiment uses a compiler and a mapper to automatically convert and map the user's coding model to the physical resources of the heterogeneous brain-like cluster system, achieving seamless adaptation and efficient execution between the user application and the cluster hardware.

[0096] The coding model refers to the source code written by the user in a specific programming language or framework (such as Python, TensorFlow, etc.) that describes the brain-like computing application.

[0097] According to this embodiment, a compiler is used to convert the user's coding model into target code for execution on a heterogeneous brain-like cluster system, namely, cluster host software.

[0098] According to this embodiment, a compiler is also used to convert the user's coding model into a resource list, that is, a list of various computing, storage and communication resources required during the execution of the coding model.

[0099] A mapper is a tool specifically used to map the logical resource requirements in a resource list to actual physical resources. The mapper finds a mapping solution that meets resource constraints and optimizes task performance and system efficiency as much as possible by comprehensively considering the status and capabilities of devices such as computing nodes, storage nodes, and network topology in the current cluster system, as well as the heterogeneous characteristics and limitations between different hardware. According to this embodiment, the compiler transmits the resource list to the mapper, and the mapper maps the logical resource requirements to the actual physical unit based on the mapping core algorithm and optimization strategy, that is, specifies which CPU core, GPU stream processor, brain-like chip, etc. each computing task or data object should be executed on.

[0100] The compiler and mapper according to this embodiment realize the automatic conversion and optimized mapping of user coding models to heterogeneous cluster physical resources, improve the portability, scalability and execution efficiency of brain-like applications in heterogeneous cluster environments, and lower the threshold for user development and deployment.

[0101] According to the large-scale heterogeneous brain-like cluster software stack fusion method of this embodiment, efficient collaboration of heterogeneous computing nodes is achieved through embedded calls, hierarchical calls and fusion of software stacks; pooling layer and delay hiding technology are used to improve cluster communication performance; intelligent resource mapping and task scheduling strategies are introduced to enhance the adaptability and flexibility of the system; at the same time, through means such as equipment debugging and performance analysis, the reliable operation of the system is guaranteed, thereby effectively solving the difficulties and challenges faced by existing heterogeneous brain-like clusters.

[0102] In some embodiments, the brain-like chip controller includes a microcode controller and a clock-precision emulator. The microcode controller is used to complete the assembly of the brain-like chip service package, and the clock-precision emulator is used to perform precise timing control of the brain-like chip based on the microcode.

[0103] According to this implementation, the programmability and flexibility of brain-like chips can be improved, the precise control of the execution timing of brain-like chips can be enhanced, the functional verification and performance tuning of brain-like chips can be facilitated, and the integration and management of brain-like chips in a heterogeneous cluster environment can be facilitated.

[0104] In some implementations, using a compiler to convert the user's coding model into cluster host software and a resource list includes: using the compiler to convert the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

[0105] According to this implementation, by adopting the classic compilation process of "syntax interpretation--linking--compiling", the standardization and reliability of the conversion process from the user coding model to the cluster software are guaranteed, and the conversion from the user coding to the cluster software is automatically completed by the compiler, which greatly simplifies the user's development work. The resource list generated according to this implementation can accurately describe the various resources required for the coding model to run in the cluster environment, providing a key basis for the scheduling and optimization decision of the cluster management system.

[0106] In some implementations, before configuring a data routing table or a message routing table for the accelerator card device, the method further includes the following steps:

[0107] Connect the accelerator card and the service board according to the network topology of the brain-like cluster system;

[0108] The host configures the PCIe port to communicate with the host control server through the relevant data and message area.

[0109] In some implementations, after configuring the PCIe through the host to communicate with the host server through the relevant data and message area, the method further includes:

[0110] The master control server configures the routing table of the source module of the PCIe port communicating with the master control server through the BAR (Base Address Register) of PCIe, and the routing table belongs to the data domain;

[0111] After the source module for the communication between the PCIe port and the main control server is configured, the data routing tables of other ports are configured through the communication between the PCIe port and the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Ports). The routing configuration channel of the PCIe port is implemented in the form of the corresponding channel (Channel) of the host system bus (Bus) of the main control server. The data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

[0112] In some implementations, after the source module for communicating between the PCIe port and the master control server is configured, the method further includes the following steps:

[0113] According to the configuration file generated by the compiler, configure the macro instruction processor code of each accelerator card device, the configuration part of the executor and the brain-like chip;

[0114] The host initiates an enabling action, which is driven and implemented in a data stream driven manner.

[0115] According to the initialization and configuration process of the large-scale heterogeneous brain-like cluster of the above-mentioned implementation mode, the hardware-level interconnection and initialization are realized by connecting the accelerator card and the service board according to the network topology of the brain-like cluster system, and configuring the relevant data and message areas; the host configures the routing tables at all levels, including the source module routing table for the PCIe port to communicate with the main control server, the data routing table and the message routing table of other ports and pseudo-ports, so as to realize the data and message paths between the accelerator card devices in the cluster, and ensure the efficient data transmission and communication in the cluster; the compiler generates the configuration file, configures the macro instruction processor code, the executor and the brain-like chip, realizes the flexible configuration and programming of the cluster hardware at the software level, and improves the availability and adaptability of the cluster; and the host initiates the enabling action and realizes the operation of the cluster in a data flow-driven manner, so as to improve the cluster work efficiency and simplify the control logic.

[0116] Figure 2 A complete set of signaling messages and data stream handshake synchronization communication scheme in the vertical dimension according to an exemplary embodiment of the present invention is shown. Figure 3 A specific schematic diagram of the signaling message and data stream handshake synchronization communication scheme is shown.

[0117] like Figure 2 and 3 As shown, the signaling message and message stream handshake synchronization communication scheme includes the following steps:

[0118] 1. Connect the required accelerator cards and service boards according to the brain-like cluster system network topology;

[0119] 2. The host configures the PCIe port to communicate with the host control server related data and message area;

[0120] 3. The host configures the routing table of the source module that communicates with the main control server through the PCIe BAR space (the routing table belongs to the data domain);

[0121] 4. After the source module for the PCIe port to communicate with the master server is configured, it can work normally. The data routing table of other ports is configured through this interface, where the routing configuration channel of the port is implemented in the corresponding channel of the bus (the routing table belongs to the data domain);

[0122] 5. After the source module for the PCIe port to communicate with the main control server is configured, it can work normally. The data routing table of other pseudo ports is configured through this interface, where the routing configuration channel of the port is implemented in the corresponding channel of the bus (the routing table belongs to the data domain);

[0123] 6. After the source module for the PCIe port to communicate with the master server is configured, it can work normally. The message routing table of other pseudo ports is configured through this interface, where the routing configuration channel of the port is implemented in the corresponding channel of the bus (the routing table belongs to the message domain);

[0124] 7. After completing step 4, the Pseudo Sink of the indirect board can receive the configuration information of the host and execute steps 3-5 in sequence;

[0125] 8. Configure the macro processor code of each accelerator card device according to the configuration file generated by the compiler;

[0126] 9. Follow the compiler's generated configuration file and the executor's configuration section;

[0127] 10. Configure the host according to the configuration file generated by the Tianji compiler.

[0128] 11. The host initiates the enabling action and drives it according to the data flow driving method.

[0129] Figure 4 A schematic diagram of a process of using a large-scale heterogeneous brain-like cluster according to an exemplary embodiment of the present invention is shown. Figure 4 The process shown includes determining the scenario application and determining the algorithm network architecture. After the scenario application is determined, the algorithm implementation is completed and enters the automatic compiler to determine the communication and tasks, and use the compiler to generate configuration files based on the above algorithm tasks and communications.

[0130] In the other step, the cluster is powered on and the cluster is configured on the right side. After completion, the cloud software stack and background management program are installed, and user registration is completed at the same time.

[0131] After the above steps are completed, the host software will be run to run the host executable programs and the accelerator executable programs. At this time, the performance analysis step can be entered.

[0132] The present invention also discloses a large-scale heterogeneous brain-like cluster software stack fusion system, the system comprising:

[0133] A compiler, used to convert the user's coding model into cluster host software and resource lists;

[0134] The host is used to configure the relevant data and message areas of PCIe, and configure the routing table of the source module of the PCIe port communicating with the main control server through the base address register space;

[0135] Routing configuration tool, used to configure the data routing table or message routing table of the accelerator card device after the cluster power-on self-test is completed;

[0136] A brain-like chip controller, used to complete the SPCP (StepPhase Core Protocol) timing control diagram of the brain-like chip in combination with algorithm mapping and operation timing, and generate corresponding microcode based on the timing control diagram;

[0137] Device debugger, used to perform pre-maintenance and status tracking of cluster hardware accelerator devices;

[0138] Performance analyzer, used to understand the performance of multi-tenants of user software in real time;

[0139] The pooling layer is used as an interface module between the super manager and the mapper and task job, and is used in conjunction with the user scheduling layer and the corresponding layer to delay hidden communication and computation;

[0140] The mapper is used to receive the resource list and map the resource list to the corresponding actual physical unit through mapping core algorithm and optimization strategy in combination with the status of the existing accelerator card device.

[0141] In some embodiments, the brain-like chip controller includes:

[0142] Microcode controller, used to complete the construction of brain-like chip business packages;

[0143] A clock-precision simulator is used to perform precise timing control on the brain-like chip based on the microcode.

[0144] In some implementations, the compiler converts the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

[0145] In some embodiments, the system further includes an accelerator card and a service board for connecting according to a brain-like cluster system network topology.

[0146] In some embodiments, after the source module for the PCIe port to communicate with the main control server is configured, the data routing tables of other ports are configured through the PCIe port to communicate with the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Port), wherein the routing configuration channel of the port is implemented in the form of the corresponding channel (Channel) of the system bus (Bus), the data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

[0147] In some embodiments, the system further comprises:

[0148] A macro processor, whose code is configured by a generated configuration file of the compiler;

[0149] The executor is configured in part by a configuration file generated by the compiler.

[0150] The details and beneficial effects of this embodiment and implementation method are described above and will not be repeated here.

[0151] Figure 5 The schematic diagram of the architecture design of a heterogeneous brain-inspired cluster computing platform according to an exemplary embodiment of the present invention is shown. The architecture integrates three types of processors: BIC (Brain-Inspired Chip), GPU, and NPU (Neural Processing Unit). The software of the architecture includes four parts, namely, the first part K8s (Kubernetes) cloud and containerization, the second part system application and communication library, the third part software stack of the acceleration chip, and the fourth part resource allocation and scheduling.

[0152] The software architecture according to the present invention adopts a composite heterogeneous cluster model, which can provide a unified programming framework and scheduling interface for applications at the top level, and can abstract and manage diverse heterogeneous hardware (brain-like chips, GPUs, etc.) through containers, runtimes, compilers and other middleware at the bottom level, giving full play to the computing advantages of different hardware while shielding users from the complexity of heterogeneous environments.

[0153] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the data processing device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0154] The above is a description of a specific embodiment of the specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recorded in the claims can be performed in an order different from that in the embodiments and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0155] Although this specification includes many specific implementation details, these should not be interpreted as limiting the scope of any invention or the scope of protection claimed, but are mainly used to describe the features of the specific embodiments of specific inventions. Certain features described in multiple embodiments in this specification may also be implemented in combination in a single embodiment. On the other hand, the various features described in a single embodiment may also be implemented separately in multiple embodiments or in any suitable sub-combination. In addition, although features may work in certain combinations as described above and even initially claim protection, one or more features from the claimed combination may be removed from the combination in some cases, and the claimed combination may point to a sub-combination or a variation of a sub-combination.

[0156] Similarly, although operations are depicted in a particular order in the accompanying drawings, this should not be understood as requiring that these operations be performed in the particular order shown or performed sequentially, or requiring that all illustrated operations be performed to achieve the desired results. In some cases, multitasking and parallel processing may be advantageous. In addition, the separation of various system modules and components in the above-described embodiments should not be understood as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product, or packaged into multiple software products.

[0157] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve the desired results. In addition, the processes depicted in the drawings do not necessarily require the particular order or sequential order shown to achieve the desired results. In some implementations, multitasking and parallel processing may be advantageous.

[0158] The above description is merely a preferred embodiment of one or more embodiments of the present specification and is not intended to limit one or more embodiments of the present specification. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of one or more embodiments of the present specification shall be included in the scope of protection of one or more embodiments of the present specification.

Claims

1. A large-scale heterogeneous brain-like cluster software stack fusion method, characterized in that: The method comprises: Configure each component to provide communication library functions and distributed cluster computing acceleration libraries, and achieve software stack integration from the communication layer and acceleration layer. For the acceleration card device after the cluster power-on self-test is completed, the data routing table or message routing table of the acceleration card device is configured in a layer-by-layer progressive manner using the routing configuration tool based on the transmission write function of the communication protocol between the PCIe port and the master control server and the message ring mechanism; Combined with the brain-like chip controller in the accelerator card device, the algorithm mapping is completed, and the SPCP (Step Phase Core Protocol) timing control diagram of the brain-like chip is completed in combination with the algorithm running timing, and the basic functions of the cluster host and the accelerator card device are used based on the SPCP timing control diagram of the brain-like chip to generate the corresponding microcode; Use the device debugger to perform pre-maintenance and status tracking on cluster hardware accelerator devices, and use the performance analyzer to understand the performance of user software multi-tenants in real time; Using the pooling layer as an interface module between the super manager and the mapper, and as an interface module between the super manager and the task job, in combination with the user scheduling layer and the corresponding layer to delay hidden communication and calculation; The compiler is used to convert the user's coding model into cluster host software and resource lists, and the resource lists are transmitted to the mapper. The mapper combines the status of the existing accelerator card device and maps the layout to the actual physical unit of the corresponding accelerator card device through mapping core algorithms and optimization strategies.

2. The method according to claim 1, characterized in that The brain-like chip controller includes a microcode controller and a clock-precision simulator. The microcode controller is used to complete the assembly of the brain-like chip service package, and the clock-precision simulator is used to perform precise timing control on the brain-like chip based on the microcode.

3. The method according to claim 1, characterized in that Use the compiler to convert the user's coding model into a list of cluster host software and resources, including: The compiler is used to convert the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

4. The method according to claim 1, characterized in that: Before configuring a data routing table or a message routing table for the accelerator card device, the method further includes: Connect the accelerator card and the service board according to the network topology of the brain-like cluster system; The host configures the PCIe port to communicate with the host control server through the relevant data and message area.

5. The method according to claim 4, characterized in that After configuring the PCIe port through the host to communicate with the relevant data and message area of ​​the main control server, the method further includes: The master control server configures a routing table of a source module for communicating with the PCIe port through a BAR (Base Address Register) space of PCIe, and the routing table belongs to the data domain; After the source module for the communication between the PCIe port and the main control server is configured, the data routing tables of other ports are configured through the communication between the PCIe port and the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Ports). The routing configuration channel of the PCIe port is implemented in the form of the corresponding channel (Channel) of the host system bus (Bus) of the main control server. The data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

6. The method according to claim 5, characterized in that After the source module for communicating between the PCIe port and the main control server is configured, the method further includes the following steps: According to the configuration file generated by the compiler, configure the macro instruction processor code of each accelerator card device, the configuration part of the actuator module and the brain-like chip; The main control server host initiates the enabling action and drives the implementation in a data stream driven manner.

7. A large-scale heterogeneous brain-like cluster software stack fusion system, characterized by: The system comprises: A compiler, used to convert the user's coding model into cluster host software and resource lists; The host is used to configure the data and message areas related to the communication between the PCIe port and the main control server, and configure the routing table of the source module for the communication between the PCIe port and the main control server through the base address register space; Routing configuration tool, used to configure the data routing table or message routing table of the accelerator card device after the cluster power-on self-test is completed; A brain-like chip controller, which is used to complete the SPCP (Step Phase Core Protocol) timing control diagram of the brain-like chip in combination with the algorithm mapping and the running timing, and to generate corresponding microcode based on the timing control diagram; Device debugger, used to perform pre-maintenance and status tracking of cluster hardware accelerator devices; Performance analyzer, used to understand the performance of multi-tenants of user software in real time; The pooling layer is used as an interface module between the super manager and the mapper and task job, and is used in conjunction with the user scheduling layer and the corresponding layer to delay hidden communication and computation; The mapper is used to receive the resource list and map the resource list to the corresponding actual physical unit through mapping core algorithm and optimization strategy in combination with the status of the existing accelerator card device.

8. The system according to claim 7, characterized in that The brain-like chip controller comprises: Microcode controller, used to complete the construction of brain-like chip business packages; A clock-precision simulator is used to perform precise timing control on the brain-like chip based on the microcode.

9. The system according to claim 7, characterized in that The compiler converts the user's coding model into cluster host software and the resource list in a syntax interpretation-linking-compiling manner.

10. The system according to claim 7, characterized in that After the source module for the communication between the PCIe port and the main control server is configured, the data routing tables of other ports are configured through the communication between the PCIe port and the main control server, as well as the data routing tables and message routing tables of other pseudo ports (Pseudo Port), wherein the routing configuration channel of the port is implemented in the form of the corresponding channel (Channel) of the system bus (Bus), the data routing tables of other ports and the data routing table of the pseudo port belong to the data domain, and the message routing table of the pseudo port belongs to the message domain.

Images