Processing architecture for super-large scale brain-like intelligent heterogeneous fusion

Abstract

The invention provides a processing architecture for super-large-scale brain-like intelligent heterogeneous fusion, which refers to a function partition and processing loop mechanism of a brain, and abstracts, isolates and schedules physical resources of a brain-like platform through a virtualization technology and a scheduling deployment tool. Furthermore, a calculation model and a calculation task provided by the user are deployed in a brain-like system, so that the flexibility, the safety and the hardware resource utilization rate of platform application are improved; on the basis of a platform integration development environment, rich software and hardware interfaces and a rapid design method, open application ecology is achieved, interactive communication, scheduling, integration and the like are carried out through simple, flexible and accurately defined module interfaces aiming at different applications by designing a coarse-grained and loosely-coupled integration architecture, and the complexity of underlying programming is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, and in particular to a processing architecture for super-large brain-like intelligent heterogeneous fusion. BACKGROUND

[0002] With the development of artificial intelligence technology, brain-like computing as a new computing paradigm poses new challenges to existing computing architectures due to its complexity, openness and heterogeneity.

[0003] The advanced cognitive functions, computing algorithms and chip designs of brain-like intelligence rely more on the linkage of multiple heterogeneous computing resources to complete the whole process development. Traditional brain-like computing systems face the contradiction between hardware scalability and brain scale simulation, low efficiency of CPU / GPU / BPU heterogeneous resource collaboration, and the defect of fragmented algorithm-chip collaborative development process, which leads to problems such as limited communication bandwidth, insufficient resource utilization, long development cycle, etc. in traditional computing architecture when supporting trillion-level neural synapse scale, making it difficult to meet the needs of multi-modal and multi-level brain processing paradigm, especially in supporting super-large brain computing and multiple computing resource fusion.

[0004] Therefore, there is an urgent need for a new computing architecture to realize efficient fusion of brain-like computing and multiple heterogeneous computing resources and provide strong hardware support for multi-modal and multi-level new processing paradigm. SUMMARY

[0005] The present application provides a processing architecture for super-large brain-like intelligent heterogeneous fusion to overcome the defects of traditional computing architecture in supporting trillion-level neural synapse scale, such as limited communication bandwidth, insufficient resource utilization, long development cycle, etc., which makes it difficult to meet the needs of multi-modal and multi-level brain processing paradigm, especially in supporting super-large brain computing and multiple computing resource fusion.

[0006] In a first aspect, the present application provides a processing architecture for super-large brain-like intelligent heterogeneous fusion, which includes a heterogeneous hardware layer arranged from bottom to top, an operating system and driver layer, a development tool and programming interface layer, a computing framework and cluster management layer, and an application and interaction layer. The heterogeneous hardware layer is configured to provide a hardware interface to realize access and collaborative invocation of heterogeneous computing resources. The operating system and driver layer is configured to provide a running environment and a scheduling interface between the heterogeneous hardware layer and the upper layer software to shield physical differences, realize real-time resource allocation and cross-chip data sharing. The development tool and programming interface layer is configured to provide a software interface, receive user algorithms and model descriptions, and complete conversion, optimization and mapping to generate heterogeneous computing tasks that can be scheduled by the running environment. The computing framework is combined with a cluster management layer, which is used for unified scheduling and management of heterogeneous computing resources of multiple nodes, realizes distributed execution of tasks, load balancing and cross-node elastic expansion; The application and interaction layer is used for providing an application running environment and an interaction interface, converting a user request into an executable task and issuing it to the lower layer, and visually presenting the returned result, realizing deployment of brain-like intelligent applications and bidirectional interaction between man and machine.

[0007] In one of the embodiments, the heterogeneous hardware layer includes: A hardware abstraction interface is used to provide a standard connection port for external computing units and brain-like chips, and to provide a corresponding hardware description file; A heterogeneous interconnection bus is used to establish a unified address space and a shared consistency domain between the computing units and the brain-like chips, and to provide a data exchange channel supporting cross-chip data interaction.

[0008] In one of the embodiments, the operating system and driver layer includes: An operating system is used to provide a running environment; A brain-like chip driver is used to complete online management and state monitoring of brain-like hardware resources in cooperation with the operating system; A heterogeneous resource manager is used to obtain load and energy efficiency information of each computing unit in real time, dynamically determine a target execution device according to a task type, generate a scheduling instruction, and issue it to the operating system and the brain-like chip driver, so as to realize on-demand allocation, migration and rescheduling of computing power resources; A memory structure is used to share a memory space between cross-brain-like chips, and to support automatic conversion of pulse events and tensor data. In one of the embodiments, the operating system and driver layer further includes: A container runtime is used to perform virtualization deployment and life cycle management of application workloads based on container technology after receiving a task scheduling instruction, and to return real-time resource usage information to the heterogeneous resource manager.

[0009] In one of the embodiments, the development tool and programming interface layer includes: A programming interface is used to provide a standardized calling entry to an upper-layer application, receive a user algorithm and model description, and generate a calling request conforming to an internal intermediate representation; A brain-like chip software development kit includes operator compilation optimization and resource scheduling monitoring; The operator compilation optimization is used to convert operators and computation graphs in the calling request into an intermediate representation for heterogeneous hardware according to target hardware capabilities, and to perform operator-level and graph-level optimization on the computation graph; Resource scheduling monitoring, for monitoring the load and energy efficiency of each computing unit in real time during task execution, and triggering remapping or task migration instructions when resource bottlenecks or load drifts are detected, so as to adjust the overall performance and energy efficiency of the system; Mapping and optimization tool library, for dividing the compiled and optimized computation graph into task subgraphs according to the hardware capability description, selecting the optimal execution device for each subgraph according to the load type and bandwidth, energy consumption indicators, and inserting necessary data migration and synchronization instructions to form a task device mapping table that can be executed. In one of the embodiments, the brain-like chip software development kit further comprises: Brain-like learning algorithm library, for providing reusable brain-like learning rules and plasticity algorithms for the programming interface or the operator compilation optimization to call; Basic brain model library, for storing pre-stored brain network templates for reuse and development.

[0010] In one of the embodiments, the computing framework and cluster management layer comprises: Brain-like computing framework, for generating cross-node executable graphs based on unified high-level abstraction and providing synchronization primitives; Heterogeneous cluster plug-in, for completing task arrangement and elastic expansion and contraction between heterogeneous nodes in combination with the extended scheduling plug-in; Dynamic task scheduler, for implementing task migration and rescheduling according to the running period load state. In one of the embodiments, the computing framework and cluster management layer further comprises: Monitoring and optimization toolbox, for providing energy efficiency analysis and event delay visualization, and providing optimization basis for the heterogeneous cluster plug-in and the dynamic task scheduler. In one of the embodiments, the application and interaction layer comprises: Interaction interface, for receiving user requests or real-time data streams and parsing their semantics, triggering task generation downward; and feeding back execution status to the user end; Brain-like integrated development environment, for providing a low-code configuration interface and an interactive three-dimensional network editor, so that the user can drag and build a spiking neural network model and render a spiking propagation animation in real time, and generate a model description conforming to the unified programming interface specification after the model is confirmed; Task issuing unit, for generating heterogeneous computing tasks that can be scheduled by the lower layer running environment according to the model description output by the brain-like integrated development environment and the running parameters received by the interaction interface, and issuing the tasks to the computing framework and cluster management layer; Result visualization unit, for parsing and graphically displaying the pulse event stream, tensor result and performance indicators returned by the lower layer, providing multi-dimensional views such as time, space and energy efficiency, and transmitting operable feedback back to the interaction interface. In one embodiment, the application and interaction layer further comprises: An application template library for storing preset brain simulation and edge intelligence model templates for calling by the brain-like integrated development environment to accelerate prototype verification and development iteration; A brain-computer interface toolkit for providing electroencephalogram-pulse code conversion and closed-loop stimulation control interface to expand input and output channels of the interaction interface and realize brain-computer integrated interaction.

[0011] The processing architecture for super-large brain-like intelligent heterogeneous fusion has at least the following advantages: The present application learns from the functional partitioning and processing loop mechanism of the brain, abstracts, isolates and schedules the physical resources of the brain-like platform through virtualization technology and scheduling and deployment tools, and further deploys the computing models and computing tasks provided by the user on the brain-like system to improve the flexibility, security and hardware resource utilization of platform applications; relying on the platform integrated development environment, rich software and hardware interfaces and rapid design methods, an open application ecosystem is realized, through the design of a coarse-grained, loosely-coupled integrated architecture, different applications are interacted, communicated, scheduled and integrated through simple, flexible and precisely defined module interfaces, reducing the complexity of bottom programming. BRIEF DESCRIPTION OF DRAWINGS

[0012] Figure 1 is a structural block diagram of the processing architecture in one embodiment; Figure 2 is a structural block diagram of the heterogeneous hardware layer in one embodiment; Figure 3 is a structural block diagram of the operating system and driver layer in one embodiment; Figure 4 is a structural block diagram of the heterogeneous hardware layer in one embodiment; Figure 5 is a structural block diagram of the development tool and programming interface layer in one embodiment; Figure 6 is a structural block diagram of the application and interaction layer in one embodiment. DETAILED DESCRIPTION

[0013] The embodiments of the present application will be described in detail below with specific reference to specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the disclosure of the present specification. The present application can also be implemented or applied by means of other different specific embodiments, and the details in the present specification can be modified or changed based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0014] Some example embodiments of the present application are described for the purpose of illustration, it being understood that the present application can be implemented in other ways not specifically shown in the drawings.

[0015] For the purpose of illustration, some example embodiments of the present application are described, it being understood that the present application can be implemented in other ways not specifically shown in the drawings. Figure 1 In an example embodiment, the present application provides a processing architecture for super-large brain-like intelligent heterogeneous fusion, specifically comprising: a heterogeneous hardware layer arranged from bottom to top, an operating system and driver layer, a development tool and programming interface layer, a computing framework and cluster management layer, and an application and interaction layer. Wherein, control instructions, model descriptions and scheduling information are transmitted along the path from top to bottom to complete task issuance and resource mapping; the computing results and state monitoring information generated by the heterogeneous hardware layer are uploaded layer by layer along the path from bottom to top to realize result feedback and visualization; each layer encapsulates data and control signals through standardized interfaces to ensure consistent semantics, low-latency collaboration across layers, thereby realizing unified management and efficient operation of super-large brain-like intelligent heterogeneous computing resources.

[0016] The heterogeneous hardware layer is configured to provide a hardware interface to realize access and collaborative invocation of heterogeneous computing resources.

[0017] Specifically, the heterogeneous computing resources include computing units and brain-like chips. The computing unit is a core hardware module in a computer system that performs mathematical operations, logical processing, and data operations, and is used to convert input data into meaningful output results. For example, the computing unit includes multi-architecture central processing units (CPUs), graphics processing units (GPUs), neural network processing units (NPUs), and intelligent processors (BPUs); wherein the CPU includes instruction set architectures such as x86, ARM, RISC-V, etc. The above computing units work collaboratively, for example, using CPU scheduling and GPU and NPU accelerators, to form a heterogeneous computing system.

[0018] The brain-like chip is a special hardware that simulates the structure and working mode of the biological brain, processes information through the mechanism of bionic neurons (Neuron) and synapses (Synapse), and is fundamentally different from the traditional von Neumann architecture CPU / GPU. Its goal is to achieve intelligent computing with high energy efficiency, low latency, and adaptive learning. The heterogeneous hardware layer of the present application supports mainstream neuromorphic chips such as Intel Loihi, IBM TrueNorth, and SpiNNaker.

[0019] The operating system and driver layer is configured to provide a running environment and scheduling interface between the heterogeneous hardware layer and the upper layer software, to mask the physical differences, realize real-time resource allocation and cross-chip data sharing.

[0020] Specifically, the operating system and the driver layer are provided with a unified kernel environment, a device driver set and a heterogeneous scheduling interface. When starting, the operating system and the driver layer complete device enumeration and capability registration for the multi-architecture computing unit, create processes or threads for the upper-layer task through a name space isolation and resource quota mechanism and establish a virtual address mapping, thereby forming a running environment that can mask hardware differences. Meanwhile, the operating system and the driver layer also utilize a built-in real-time scheduler and a cross-chip shared memory mechanism to maintain a unified address space and a consistency domain between the computing units, support direct memory access and automatic conversion of pulse event-tensor data. When the upper layer submits a heterogeneous computing task, the layer dynamically allocates the computing power of each neuromorphic computing unit according to the load type and the power consumption threshold, and triggers task migration or rescheduling when the load changes or a fault occurs, finally realizing real-time allocation of heterogeneous hardware resources and low-latency data sharing.

[0021] The development tool and programming interface layer is used for providing a software interface, receiving a user algorithm and model description and completing conversion, optimization and mapping, and generating a heterogeneous computing task that can be scheduled by the running environment.

[0022] Specifically, the development tool and programming interface layer includes a set of software interface mechanisms for receiving algorithm logic or model description from the user end, converting the algorithm logic or model description into a unified intermediate representation after syntax analysis, performing operator-level and graph-level optimization in combination with a target hardware capability description to eliminate redundant calculations, adjust storage layout and improve parallelism, dividing subgraphs between multiple computing units through a mapping strategy, inserting data format conversion nodes and generating routing and synchronization instructions, and finally outputting a heterogeneous computing task package compatible with the lower-layer running environment. Exemplarily, the heterogeneous computing task package includes binary code that can be directly scheduled and executed, a resource allocation table and performance constraint parameters, thereby realizing end-to-end automatic deployment of a user model to heterogeneous hardware.

[0023] The computing framework and cluster management layer is used for uniformly scheduling and managing heterogeneous computing resources of multiple nodes, realizing distributed execution of tasks, load balancing and cross-node elastic expansion.

[0024] Specifically, after receiving the heterogeneous computing task package issued by the development tool and programming interface layer, the computing framework and cluster management layer first divide the task graph into parallel subtasks according to the node-level hardware capability description and generate a resource application list; then, through the cluster resource view, the dynamic arrangement of subtasks among multiple nodes is completed, the utilization rate, power consumption and communication bandwidth of each node are monitored in real time, and when the load is uneven or the node state changes, migration, replication or elastic expansion operations are triggered to maintain the global throughput rate and service time delay requirements; in this process, the computing framework and cluster management layer also maintain the cross-node consistency domain and fault recovery log to ensure the correctness and continuity of distributed execution, and finally realize the unified management, load balancing and on-demand elastic expansion of multi-node heterogeneous computing resources.

[0025] The application and interaction layer is used to provide an application running environment and an interaction interface, convert user requests into executable tasks and issue them to the lower layer, and visually present the returned results, so as to realize the deployment and bidirectional interaction of brain-like intelligent applications.

[0026] Specifically, the application and interaction layer is provided with an application running container and a human-computer interaction interface, which on the one hand collects user requests or real-time data streams through a graphical interface, a Web end or external sensing and control devices, and parses the requests into model parameters, performance constraints and execution instructions; on the other hand, converts the parsed information into executable task descriptions conforming to the semantics of the development tool and programming interface layer, and issues them to the lower layer together with visual identifiers; further, when the lower layer returns a pulse event stream, a tensor result or performance statistics, the application and interaction layer renders and visually displays them in multiple dimensions, and supports users to adjust parameters or issue control instructions in real time through the interaction interface, so as to realize the rapid deployment, running monitoring and bidirectional interaction of brain-like intelligent applications.

[0027] The above processing architecture for super-large-scale brain-like intelligent heterogeneous fusion abstracts, isolates and schedules the physical resources of the brain-like platform through virtualization technology and scheduling and deployment tools, and further deploys the computing models and computing tasks provided by users on the brain-like system, thereby improving the flexibility, security and hardware resource utilization rate of platform applications; relying on the platform integrated development environment, rich software and hardware interfaces and rapid design methods, an open application ecosystem is realized, through the design of a coarse-grained, loosely-coupled integrated architecture, different applications are interacted, communicated, scheduled and integrated through simple, flexible and precisely defined module interfaces, thereby reducing the complexity of bottom programming.

[0028] Please refer to Figure 2 Optionally, the heterogeneous hardware layer comprises a hardware abstraction interface and a heterogeneous interconnection bus.

[0029] A hardware abstraction interface is configured to provide a standard connection port for an external computing unit and a brain-like chip, and provide a corresponding hardware description file.

[0030] Specifically, the hardware abstraction interface provides a standardized hardware description file to define the physical architecture and functional characteristics of the chip, and realizes unified access of the upper application to the heterogeneous brain-like hardware. The hardware description file refers to a structured text used to describe the neuron model, synaptic connection topology, pulse coding protocol and memory mapping rule of the chip, thereby shielding the physical differences of different chips and converting into a unified logical view. Exemplarily, the hardware description file includes XML, YAML, etc. With this scheme, the application has good compatibility and can support plug-and-play of multiple mainstream neuromorphic chips; at the same time, a new chip only needs to extend the description file template without modifying the upper application framework, and has good scalability; in addition, the application reduces a large amount of hardware adaptation code, reduces the algorithm transplantation time, and greatly improves the transplantation efficiency.

[0031] A heterogeneous interconnection bus is configured to establish a unified address space and a shared consistency domain between the computing unit and the brain-like chip, and provide a data exchange channel supporting cross-chip data interaction.

[0032] Specifically, the heterogeneous interconnection bus integrates multiple high-speed protocols and supports dynamic protocol switching, realizes low-delay communication between chips, and collaborative computing between traditional computing units and brain-like chips. Exemplarily, the heterogeneous interconnection bus of the application supports CXL, NVLink and other protocols, realizes physical-level collaboration between traditional chips and brain-like chips by automatically identifying device types and selecting the optimal communication mode.

[0033] Please refer to Figure 3 Optionally, the operating system and the driver layer include: an operating system, a brain-like chip driver, a heterogeneous resource manager and a memory structure.

[0034] The operating system is configured to provide a running environment.

[0035] Specifically, the operating system is an interface between the computing system hardware and the user, configured to provide a unified process management, thread scheduling and interrupt processing interface to the upper software, and to perform address mapping and device enumeration on the lower multi-type computing unit, thereby shielding the physical differences and building a general running environment at the system level. It provides a basic environment for the operation of the brain-like chip driver program, and provides event-driven microsecond-level task scheduling for the brain-like chip.

[0036] The brain-like chip driver is configured to complete online management and state monitoring of brain-like hardware resources in cooperation with the operating system.

[0037] Specifically, the brain-like chip driver performs register-level control, event packet transmission and timing synchronization on the computing unit, maps the pulse event data semantics into uniformly addressable data objects, and completes the online management and state monitoring of brain-like hardware resources in cooperation with the operating system. The brain-like chip driver interacts with the hardware through the interface provided by the operating system.

[0038] The heterogeneous resource manager is configured to acquire the load and energy efficiency information of each computing unit in real time, dynamically determine the target execution device according to the type of the task, generate a scheduling instruction, and deliver the scheduling instruction to the operating system and the brain-like chip driver, so as to realize on-demand allocation, migration and rescheduling of computing power resources.

[0039] The memory structure is configured to share the memory space across the brain-like chips, and support automatic conversion of pulse events and tensor data. Specifically, the memory structure establishes a shared address space and a consistency domain between the general-purpose computing unit and the brain-like chip, supports direct memory access across the chips, and supports bidirectional automatic conversion of pulse event data and tensor data, thereby realizing high-bandwidth and low-latency data sharing.

[0040] The above components work together to enable the operating system and the driver layer to provide a unified running environment and a scheduling interface between the heterogeneous hardware layer and the upper-layer software, and realize real-time resource allocation of heterogeneous computing power and cross-chip data sharing. Optionally, the operating system and the driver layer further include a container runtime.

[0041] The container runtime is configured to perform virtualization deployment and lifecycle management on the application workloads based on the container technology after receiving the task scheduling instruction, and return real-time resource usage information to the heterogeneous resource manager.

[0042] Specifically, the container runtime is the core component of the container technology, responsible for creating, managing and running containers. It provides the basic execution environment and application isolation mechanism for containers. The operating system provides the basic running environment for the container runtime, including the file system, network interface and device driver. The container runtime uses the resource isolation mechanism of the operating system to realize the isolation and resource management of containers, supports heterogeneous hardware acceleration, such as integrating the NVIDIA Container Toolkit and the brain-like chip driver.

[0043] For details, please refer to Figure 4 Optionally, the development tool and programming interface layer includes a programming interface, a brain-like chip software development kit and a mapping and optimization tool library.

[0044] The programming interface is configured to provide a standardized calling entry for the upper-layer application, receive user algorithms and model descriptions, and generate a calling request conforming to the internal intermediate representation.

[0045] Exemplarily, the programming interface of the present application adopts a two-layer architecture design of a physical abstraction layer (low-level API) and a declarative programming layer (high-level API), wherein the low-level API directly operates hardware primitives such as pulse neurons and synapses of the brain-like chip. The high-level API is compatible with the declarative programming interface of frameworks such as PyNN and BindsNET.

[0046] The brain-like chip software development kit has a model loading interface, is used to acquire a predefined high-level model template through the model loading interface, and call operator compilation optimization to convert the high-level model template into an instruction set executable by a neuromorphic computing unit, and adjust an execution strategy in cooperation with resource scheduling and monitoring during a running period to guarantee deployment and running of the model. Exemplarily, an embodiment of the present application adopts an SDK tool kit, which is a tool set for developing and deploying brain-like chip applications and provides full-process support from model design to hardware deployment for developers. The SDK calls an operator compilation optimization tool to convert a high-level model into a hardware executable instruction set; the SDK can load a model in a basic brain-like model library and convert it into a format executable by the brain-like chip; and the SDK cooperates with a resource scheduling and monitoring tool to ensure efficient running of the model on the brain-like chip.

[0047] The brain-like chip software development kit includes operator compilation optimization and resource scheduling and monitoring.

[0048] The operator compilation optimization is used to convert operators and a computation graph in a calling request into an intermediate representation for heterogeneous hardware according to target hardware capability, and perform operator-level and graph-level optimization on the computation graph.

[0049] Specifically, the operator compilation optimization compiles and optimizes operators in a model into a format recognizable by the brain-like chip; in addition, the operator compilation optimization can also dynamically adjust an optimization strategy according to resource usage provided by the resource scheduling and monitoring.

[0050] The resource scheduling and monitoring is used to monitor load and energy efficiency states of each computing unit in real time during task running, and trigger a remapping or task migration instruction when detecting resource bottlenecks or load drifts, so as to adjust overall performance and energy efficiency of the system.

[0051] Specifically, the resource scheduling and monitoring manages and monitors computing resources of the brain-like chip to ensure efficient use of the resources. Main functions thereof include resource management, task scheduling, performance monitoring, and fault detection.

[0052] The mapping and optimization tool library is used to divide a computation graph after compilation and optimization into task subgraphs, select optimal execution devices for each subgraph according to load types, bandwidth, and energy consumption indicators, and insert necessary data migration and synchronization instructions to form a task device mapping table executable for execution.

[0053] In particular, the mapping and optimization tool library faces the model deployment process of special hardware, and provides support and optimization for the running of brain-like or other models in the system in the form of a tool chain. Further, the mapping and optimization tool library includes hardware resource description, model partitioning, routing generation tools, and dynamic fault tolerance tools. The tool library will set constraints for the optimization problem to be solved under the hardware resource description in the markup language. Subsequently, the model-to-hardware core allocation problem under the multi-core system will be partitioned by a graph theory algorithm tool, and the inter-chip communication density will be reduced by optimizing the mapping of sub-networks to computing chips. The routing generation tool realizes the trade-off between the minimum generation and the shortest path under the breadth-first search of load balancing, and further solves the interconnection communication problem between computing cores by taking into account the transmission delay of the critical path and the global and local communication burden of the system. Finally, the dynamic fault tolerance tool provides a dynamic reconstruction and routing adaptation mechanism in the case of local hardware failure, realizes fault awareness and positioning through hardware self-checking, and realizes device replacement through reconstruction and routing path change of the computing module of the adjacent chip, thereby improving the robustness and fault tolerance capability of the system.

[0054] Optionally, the brain-like chip software development kit unit further includes: A brain-like learning algorithm library for providing reusable brain-like learning rules and plasticity algorithms for programming interface or operator compilation optimization calls.

[0055] In particular, the brain-like learning algorithm library unit is a set of algorithm implementations for brain-like intelligence, supporting training and optimization of models. The algorithms include unsupervised learning, reinforcement learning, small sample learning, etc. The algorithms in the brain-like learning algorithm library can be applied to the models in the basic brain-like model library to realize training and optimization of the models. Illustratively, the algorithms in the brain-like learning algorithm library include at least one of a spike network learning algorithm, a DNN conversion, an offline gradient learning, an online plasticity learning, a model optimization algorithm, a quantization algorithm, and an optimization problem solving algorithm.

[0056] A basic brain-like model library for storing pre-defined brain-like network templates for reuse and development.

[0057] In particular, the basic brain-like model library unit provides pre-defined brain-like models to support rapid development and verification. Illustratively, the brain-like models include, for example, LIF, Izhikevich neuron models, synaptic models such as STDP, plasticity models, and network topologies such as full connection, convolution.

[0058] See Figure 5 Optionally, the computing framework and cluster management layer includes: a brain-like computing framework, a heterogeneous cluster plug-in, and a dynamic task scheduler.

[0059] A brain-inspired computing framework for generating cross-node executable graphs based on unified high-level abstractions and providing synchronization primitives.

[0060] Specifically, the brain-inspired computing framework provides extensive modeling and analysis tools to support the development of complex brain-inspired intelligent systems. Illustratively, PyNN is a general-purpose neural network modeling interface that supports multiple simulators (including NEST, Brian2, NEURON), enabling users to run and validate models on different platforms. By integrating PyNN, the brain-inspired computing framework can leverage its cross-platform support and standardized interface, allowing users to more easily run models on different simulators, improving development efficiency and flexibility. Through high-level abstraction modeling based on PyNN, the brain-inspired system is adapted to the code writing and model organization and deployment of neuro-morphic simulators such as NEURON, NEST, and Brian2.

[0061] A heterogeneous cluster plugin for task arrangement and elastic scaling between heterogeneous nodes in conjunction with an extended scheduling plugin.

[0062] Illustratively, the heterogeneous cluster plugin includes a Kubernetes plugin and a Slurm plugin, where the Kubernetes plugin supports brain-inspired chips as custom resources to achieve hybrid workload scheduling, and the Slurm plugin extends job queue strategies to distinguish between real-time pulse tasks and batch training tasks. Through the above plugins, the cluster management supports fine-grained hybrid task scheduling.

[0063] A dynamic task scheduler for implementing task migration and rescheduling according to the load state at runtime.

[0064] Specifically, the dynamic task scheduler allocates hardware resources according to load types, including compute-intensive and event-driven. Optionally, the computing framework and cluster management layer further include a monitoring and optimization toolbox.

[0065] The monitoring and optimization toolbox provides energy efficiency analysis and event delay visualization to provide optimization basis for the heterogeneous cluster plugin and the dynamic task scheduler. Further, the monitoring and optimization toolbox includes an energy efficiency analyzer and a real-time task tracker; where the energy efficiency analyzer is used to compare the performance per watt of different hardware; and the real-time task tracker is used to visualize the propagation delay of pulse events among distributed nodes. Please refer to Figure 6 Optionally, the application and interaction layer includes an interaction interface, a brain-inspired integrated development environment, a task issuing unit, and a result visualization unit.

[0066] The interaction interface is used to receive user requests or real-time data streams and parse their semantics, triggering task generation downward; at the same time, the execution status is fed back to the user end.

[0067] A brain-like integrated development environment (IDE) is used to provide a low-code configuration interface and an interactive three-dimensional network editor, so that a user can drag and build a spiking neural network model and render a spiking propagation animation in real time, and automatically generate a model description conforming to a unified programming interface specification after the model is confirmed. Further, the brain-like integrated development environment includes an interactive 3D brain network editor and a low-code interface. The interactive 3D brain network editor is used to drag and build an SNN and render a spiking propagation animation in real time; and the low-code interface is used to automatically generate PyNN code through visual configuration, such as dragging components or filling in forms, to complete the setting of neuron types and learning rules.

[0068] Further, the brain-like integrated development environment can combine a variety of brain-like algorithm function libraries to automatically inherit and share tools and semantics embedded in the algorithms. According to application requirements, corresponding modules can be independently configured and called to realize custom brain-like system functions on the development side and facilitate performance benchmark testing; by integrating a Cloud IDE cloud development environment, a virtualization solution for underlying special hardware and a scheduling matching solution for computing resources are provided, so that online transplantation is completed on the cloud to reduce reliance on local resources; by providing a shared interface, supporting general intelligent algorithms, and being able to quickly access application-side data, algorithms, peripheral intelligent devices, and virtual peripherals, etc., Github and other open source program sharing and visual arrangement tools are provided for service and resource management and scheduling, and further service component integration and standardized interface opening are provided.

[0069] A task issuing unit is configured to generate a heterogeneous computing task that can be scheduled by a lower layer running environment according to a model description output by the brain-like integrated development environment and running parameters received by an interactive interface, and issue the task to a computing framework and a cluster management layer. A result visualization unit is configured to analyze and graphically display a pulse event stream, a tensor result, and a performance indicator returned by the lower layer, provide multi-dimensional views of time, space, energy efficiency, etc., and transmit an operable feedback back to the interactive interface. Optionally, the application and interaction layer further includes an application template library and a brain-computer interface toolkit.

[0070] The application template library is configured to store preset brain simulation and edge intelligence model templates for calling by the brain-like integrated development environment to accelerate prototype verification and development iteration. Illustratively, the application template library includes preset brain simulation templates and edge intelligence application templates; the preset brain simulation templates include a hippocampus memory model and a visual cortex pulse coding; and the edge intelligence application templates include an event camera-based unmanned aerial vehicle obstacle avoidance SNN.

[0071] The brain-computer interface toolkit is used to provide electroencephalogram-pulse encoding conversion and closed-loop stimulation control interface, expand input and output channels of interactive interface, and realize brain-computer integrated interaction. Exemplarily, the brain-computer interface toolkit feeds back the SNN output to an external nerve stimulation device. Further, the application and interaction layer can also access a brain-like intelligent application through the interactive interface or the brain-computer interface toolkit, and perform brain simulation.

[0072] The processing architecture for super-large brain-like intelligent heterogeneous fusion described above adopts a hierarchical and modular architecture, takes a brain-like chip and a computing system as a core, realizes multi-scale hardware expansion through a software and hardware system and an on-chip network interface, and realizes a certain scale and hierarchical computing capability.

[0073] Locally, the processing chip and the local storage chip are tightly coupled, modular computing, storage and communication are realized, and a processing module taking a reconfigurable unit as a core is constructed.

[0074] Globally, a super-large hardware scalable system architecture is realized, the processing module is expanded to a multi-chip processing module, a multi-module computing card and a multi-computing card processing system through the on-chip network interface, and a multi-scale hardware computing and development scheme is provided for a complex time-varying and multi-modal multi-level brain-like processing paradigm.

[0075] Based on architecture fusion, an Event-Driven processing pipeline is realized through domain programming or visual editing, functional components are instantiated based on analysis of processing task and data dependency, multi-type heterogeneous resource dynamic allocation and event-triggered processing are realized, and flexibility of task flow and resource utilization efficiency are greatly improved.

[0076] Specifically, a super-large plastic brain-like chip distributed computing network is established through multi-mode on-chip routing, multi-channel parallel software and hardware programming reconfiguration, data and task mapping scheduling, high-concurrency pulse firing during simulation and the like of a large-scale distributed plastic brain-like chip are realized, a real computing power support platform with a scale of hundreds of millions of neurons and hundreds of billions of synapses is established, RTL-level brain-like chip circuit design and brain-like algorithm model software are supported for collaborative compilation, loading and testing, and a brain-like chip prototype is tested.

[0077] Further, the application learns from the functional partitioning and processing loop mechanism of the brain, realizes a heterogeneous fusion scheduling system for multi-type hardware resources, realizes pooling and EDA development of heterogeneous resources and micro-service of a brain-like algorithm model development environment based on virtualization technology; an EDA development service based on CPU virtualization is provided, which provides an auxiliary design tool and a development environment for a hardware circuit, supports seamless migration and verification of an EDA design process to a platform; a model development service based on GPU virtualization is provided, which provides a common framework and a development environment for a software algorithm, and supports rapid iterative development and mapping execution of an algorithm model to a platform.

[0078] Finally, the application abstracts, isolates and schedules the physical resources of the brain-like platform through virtualization technology and scheduling deployment tools, and further deploys the computing model and computing task provided by the user on the brain-like system through the compiler and mapping tool library, improves the flexibility, security and hardware resource utilization rate of the platform application; relying on the platform integrated development environment, rich hardware and software interface and rapid design method, an open application ecosystem is realized, through designing a coarse-grained and loosely-coupled integrated architecture, different applications are interacted, communicated, scheduled and integrated through simple, flexible and precisely defined module interfaces, and the complexity of bottom programming is reduced.

[0079] Those of ordinary skill in the art can understand that all or part of the processes in the above embodiments can be completed by a computer program instructing related hardware, and the computer program can be stored in a non-volatile computer readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments of the method. In the embodiments provided by the present application, any reference to the memory, database or other medium can include at least one of non-volatile and volatile memories. The non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical storage, high-density embedded non-volatile memory, resistive memory (ReRAM), magnetoresistive random access memory (Magnetoresistive Random Access Memory, MRAM), ferroelectric memory (Ferroelectric Random Access Memory, FRAM), phase change memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory can include random access memory (Random Access Memory, RAM) or external cache memory, etc. As an illustration but not limitation, the RAM can be in various forms, such as static random access memory (Static Random Access Memory, SRAM) or dynamic random access memory (Dynamic Random Access Memory, DRAM), etc. The database involved in the embodiments provided by the present application can include at least one of a relational database and a non-relational database. The non-relational database can include a distributed database based on a block chain, etc., without being limited thereto. The processor involved in the embodiments provided by the present application can be a general-purpose processor, a central processing unit, a graphics processing unit, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., without being limited thereto.

[0080] Any technical features in the above embodiments can be combined, and for the sake of brevity, not all possible combinations are described above, however, as long as the combinations of technical features do not have contradictions, they shall be considered within the scope of the present disclosure.

[0081] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it shall not be understood as a limitation on the patent scope of the present application. It shall be pointed out that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these shall be within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the appended claims.

Claims

1. A processing architecture for ultra-large scale brain-like intelligent heterogeneous fusion, characterized in that, The processing architecture comprises a bottom-up heterogeneous hardware layer, an operating system and driver layer, a development tool and programming interface layer, a computing framework and cluster management layer, and an application and interaction layer; The heterogeneous hardware layer is configured to provide a hardware interface to realize access and collaborative calling of heterogeneous computing resources; The operating system and driver layer is configured to provide a running environment and a scheduling interface between the heterogeneous hardware layer and the upper-layer software, to shield physical differences, realize real-time resource allocation, and share cross-chip data; The development tool and programming interface layer is configured to provide a software interface, receive user algorithms and model descriptions, and complete conversion, optimization and mapping to generate a heterogeneous computing task that can be scheduled by the running environment; The computing framework and cluster management layer is configured to uniformly schedule and manage the heterogeneous computing resources of multiple nodes, to realize distributed execution of tasks, load balancing, and cross-node elastic expansion; The application and interaction layer is configured to provide an application running environment and an interaction interface, convert user requests into executable tasks, and issue the executable tasks to the lower layer and visually present the returned results, to realize deployment of brain-like intelligent applications and bidirectional interaction between humans and machines.

2. The processing architecture of claim 1, wherein, The heterogeneous hardware layer comprises: a hardware abstraction interface configured to provide a standard connection port for external computing units and brain-like chips, and provide a corresponding hardware description file; a heterogeneous interconnection bus configured to establish a unified address space and a shared consistency domain between the computing units and the brain-like chips, and provide a data exchange channel supporting cross-chip data interaction.

3. The processing architecture of claim 1, wherein, The operating system and driver layer comprises: an operating system configured to provide a running environment; a brain-like chip driver configured to cooperate with the operating system to complete online management and state monitoring of brain-like hardware resources; a heterogeneous resource manager configured to obtain load and energy efficiency information of each computing unit in real time, dynamically determine a target execution device according to a task type, generate a scheduling instruction, and issue the scheduling instruction to the operating system and the brain-like chip driver, to realize on-demand allocation, migration and rescheduling of computing power resources; a memory structure configured to share a memory space between cross-brain-like chips, and support automatic conversion of pulse events and tensor data.

4. The processing architecture of claim 3, wherein, The operating system and driver layer further comprises: a container runtime configured to, after receiving a task scheduling instruction, perform virtualization deployment and life cycle management on application workloads based on container technology, and return real-time resource usage information to the heterogeneous resource manager.

5. The processing architecture of claim 1, wherein, The development tool and programming interface layer comprises: a programming interface configured to provide a standardized calling entry to an upper-layer application, receive user algorithms and model descriptions, and generate a calling request conforming to an internal intermediate representation; a brain-like chip software development kit comprising operator compilation optimization and resource scheduling monitoring; The operator compilation optimization is configured to convert operators and computation graphs in the calling request into an intermediate representation for heterogeneous hardware according to target hardware capabilities, and perform operator-level and graph-level optimization on the computation graphs; The resource scheduling monitoring is configured to monitor load and energy efficiency states of each computing unit in real time during task running, and trigger a remapping or task migration instruction when a resource bottleneck or load drift is detected, to adjust overall performance and energy efficiency of the system. A mapping and optimization tool library is configured to refer to a hardware capability description, divide a compiled and optimized computation graph into task subgraphs, select optimal execution devices for each subgraph according to load types and bandwidth and energy consumption indexes, and insert necessary data migration and synchronization instructions to form a task device mapping table that can be executed.

6. The processing architecture of claim 5, wherein, The brain-like chip software development kit further includes: A brain-like learning algorithm library is configured to provide reusable brain-like learning rules and plasticity algorithms for the programming interface or the operator compilation and optimization to call; A basic brain-like model library is configured to store preset brain-like network templates for reuse and development.

7. The processing architecture of claim 1, wherein, The computing framework and cluster management layer includes: A brain-like computing framework is configured to generate a cross-node executable graph based on a unified high-level abstraction and provide synchronization primitives; A heterogeneous cluster plug-in is configured to complete task arrangement and elastic expansion and contraction between heterogeneous nodes in combination with an extended scheduling plug-in; A dynamic task scheduler is configured to implement task migration and rescheduling according to a running state.

8. The processing architecture of claim 7, wherein, The computing framework and cluster management layer further includes: A monitoring and optimization tool box is configured to provide energy efficiency analysis and event delay visualization to provide optimization basis for the heterogeneous cluster plug-in and the dynamic task scheduler.

9. The processing architecture of claim 4, wherein, The application and interaction layer includes: An interaction interface is configured to receive user requests or real-time data streams and parse semantics thereof, and trigger task generation downward; meanwhile, execution status is fed back to a user end; A brain-like integrated development environment is configured to provide a low-code configuration interface and an interactive three-dimensional network editor, so that a user can drag and build a pulse neural network model and render a pulse propagation animation in real time, and a model description conforming to a unified programming interface specification is automatically generated after the model is confirmed; A task issuing unit is configured to generate a heterogeneous computing task that can be scheduled by a lower-layer running environment according to a model description output by the brain-like integrated development environment and running parameters received by the interaction interface, and issue the task to the computing framework and cluster management layer; A result visualization unit is configured to parse and graphically display a pulse event stream, a tensor result and a performance index returned by a lower layer, provide multi-dimensional views such as time, space and energy efficiency, and transmit an operable feedback back to the interaction interface.

10. The processing architecture of claim 1, wherein, The application and interaction layer further includes: An application template library is configured to store preset brain simulation and edge intelligence model templates for the brain-like integrated development environment to call, so as to accelerate prototype verification and development iteration; A brain-computer interface tool kit is configured to provide electroencephalogram-pulse coding conversion and closed-loop stimulation control interfaces for the interaction interface to expand input and output channels and realize brain-computer integrated interaction.

Images