Multi-card cluster heterogeneous scheduling method and system based on event bus

Abstract

The invention discloses a multi-card cluster heterogeneous scheduling method and system based on an event bus, and relates to the technical field of distributed computing and resource management. The method comprises the steps that functions of heterogeneous computing hardware resources are packaged into a plurality of independent services, and each service provides a standardized interface and hides bottom layer hardware details; the computing task is disassembled into a service calling chain formed by a plurality of services according to a preset sequence; receiving, distributing and processing various events generated during system operation through an event bus, wherein the events at least comprise a node calling event, a service scheduling event and an abnormal event; wherein the event is subjected to asynchronous processing by utilizing a dynamically telescopic thread pool, and the node calling event is subjected to flow control. According to the method, the problems that a static and tight coupling mode is usually adopted in a traditional resource management method, upper-layer application development difficulty is large, portability is poor, task reconstruction cost is high, and the method is difficult to adapt to dynamically-changing calculation loads and complex heterogeneous environments are solved.

Description

Technical Field

[0001] This invention relates to the field of distributed computing and resource management technology, and in particular to a multi-card cluster heterogeneous scheduling method and system based on an event bus. Background Technology

[0002] In recent years, the diversity and complexity of intelligent application algorithms have placed higher demands on computing systems. Utilizing various types of heterogeneous computing components to build dedicated computing systems has become a mainstream solution to meet the performance, energy efficiency, and other requirements of different applications. However, the deployment and implementation of such heterogeneous computing platforms face significant challenges.

[0003] First, the diversity of accelerator types presents significant challenges to platform integration. Different hardware architectures and substantial differences in software and hardware stacks make compatibility and collaborative operation a thorny issue when integrating multiple accelerators on the same platform. Second, the varying computing resource requirements of intelligent applications necessitate efficient matching and scheduling between diverse application needs and diverse hardware resources; therefore, achieving this efficient matching and scheduling is a core challenge.

[0004] Traditional resource management methods typically employ static and tightly coupled approaches, which make upper-layer application development difficult, have poor portability, and incur high task refactoring costs, making them unsuitable for dynamically changing computing loads and complex heterogeneous environments.

[0005] Therefore, there is an urgent need in this field for a heterogeneous resource scheduling solution that can shield underlying hardware differences, achieve flexible and efficient scheduling, and possess high reliability. Summary of the Invention

[0006] Purpose of the invention: To provide a multi-card cluster heterogeneous scheduling method and system based on an event bus, so as to at least solve one of the problems existing in the prior art.

[0007] Technical solution: A multi-card cluster heterogeneous scheduling method based on event bus, comprising: The functionality of heterogeneous computing hardware resources is encapsulated into multiple independent services, each service providing a standardized interface and hiding the underlying hardware details; and the computing task is decomposed into a service call chain consisting of multiple services in a predetermined order. The system receives, distributes, and processes various events generated during system operation via an event bus. These events include at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used to process the events asynchronously, and flow control is applied to the node call events. Based on the service call chain, the execution of the service is driven by the event bus to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

[0008] Preferably, based on the service call chain, the execution of the service is driven by an event bus to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources, including: Receive heartbeat, resource utilization and throughput data reported by the service, and submit corresponding events to the event bus based on preset trigger conditions; By subscribing to and processing events from the event bus, at least one of the following operations can be performed: service fault tolerance, fault migration, dynamic scaling, load balancing, and service circuit breaking, to ensure the stability of task operation.

[0009] As a preferred option, service fault tolerance and failover are implemented, including: When the first service heartbeat times out, submit a heartbeat timeout event to the event bus; Trigger service fault tolerance logic, and prioritize finding available resources for fault tolerance on the node where the first service is located; If the fault tolerance of the node fails, the fault migration logic is triggered to migrate the task of the first service to other nodes in the cluster for execution. After successful fault tolerance or migration, the load balancing logic is triggered to generate a new service call sequence.

[0010] Preferably, performing the dynamic scaling includes: When the system load is detected to meet the preset scaling conditions, a scaling event is submitted to the event bus; Trigger dynamic scaling logic to automatically increase or decrease the number of instances of a specified service, where the number of parallel instances of a single-node service is dynamically adjusted within a user-specified range.

[0011] Preferably, the load balancing is implemented using a weighted round-robin algorithm, including: Different weights are assigned to services based on the processing capacity of the nodes where the services reside. A service call sequence is generated based on the weights; According to the service call sequence, service requests are distributed to each service instance.

[0012] Preferably, it also includes: providing name addressing functionality between services, specifically including: Deploy a name service on each compute node to manage the mapping between service names and actual addresses; Services are bound or connected through virtual addresses identified by service names; When a service is bound to a virtual address, the name service assigns it an actual address and registers it. When a service connects to a virtual address, the name service resolves it and returns the actual address to establish peer-to-peer communication.

[0013] As a preferred option, the service also includes name addressing functionality between services, and further includes: A single host server maintains information about all nodes in the cluster. The node information is synchronized to all name services, enabling connections to be established between the name services. Coordinate with all name services to complete cross-node service name resolution and service launch notification.

[0014] As a preferred option, this also includes: configuration management steps: Configuration parameters stored in the configuration center service in key-value pair format; The service subscribes to its own configuration change events after startup; When the corresponding configuration parameters in the configuration center change, the relevant service is notified through the event bus. Trigger the parameter update logic within the service.

[0015] As a preferred option, it also includes: The platform data plane provides a unified data receiving or sending capability to the framework, shielding the differences in underlying hardware interfaces. This includes: external services requesting device handles from the platform data plane through inter-process communication, and using these handles to send and receive data. The platform management plane provides the framework with cross-device service distribution and deployment capabilities, including: the master node distributes service packages and configuration files to target device nodes through the platform management plane, and starts or stops the service.

[0016] Preferably, the computation task is decomposed into a service call chain consisting of multiple services in a predetermined order, including: The service call chain is described using a directed acyclic graph (DAG), where nodes in the DAG represent services and directed edges represent the execution order and data dependencies between services. The DAG is sorted using a topological sorting algorithm to determine the order in which tasks are executed.

[0017] To achieve the above objectives, according to another aspect of this application, a multi-card cluster heterogeneous scheduling system based on an event bus is provided.

[0018] The event bus-based multi-card cluster heterogeneous scheduling system according to this application includes: The service abstraction module is used to encapsulate the functions of heterogeneous computing hardware resources into multiple independent services, each service providing a standardized interface and hiding the underlying hardware details; and to decompose computing tasks into a service call chain consisting of multiple services in a predetermined order. The event handling module is used to receive, distribute, and process various events generated during system runtime via the event bus. These events include at least node call events, service scheduling events, and exception events. The module utilizes a dynamically scalable thread pool to process events asynchronously and performs flow control on the node call events. The task scheduling and execution module is used to drive the execution of the service through the event bus based on the service call chain, so as to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

[0019] To achieve the above objectives, according to another aspect of this application, an electronic device is provided, the electronic device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores a computer program executable by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to execute the event bus-based multi-card cluster heterogeneous scheduling method according to any one of the present invention.

[0020] To achieve the above objectives, according to another aspect of this application, a computer-readable storage medium is provided, wherein computer instructions are stored therein, the computer instructions being configured to cause a processor to execute and implement the event bus-based multi-card cluster heterogeneous scheduling method of any one of the present invention.

[0021] Beneficial Effects: In this embodiment, a multi-card cluster heterogeneous scheduling method based on an event bus is adopted. This method encapsulates the functionality of heterogeneous computing hardware resources into multiple independent services, each providing a standardized interface and hiding underlying hardware details. Furthermore, computing tasks are decomposed into service call chains composed of multiple services in a predetermined order. The event bus receives, distributes, and processes various events generated during system runtime, including at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used for asynchronous event processing, and flow control is applied to the node call events. Based on the service call chain, the event bus drives the execution of the services to complete the scheduling of computing tasks and achieve dynamic management and binding of heterogeneous computing hardware resources. This achieves automatic task partitioning, dynamic migration, and load balancing among multiple types of computing cards, thereby improving the overall system operating efficiency and stability. It also solves the technical problems of traditional resource management methods, which typically employ static and tightly coupled approaches, resulting in high difficulty in upper-layer application development, poor portability, high task refactoring costs, and difficulty in adapting to dynamically changing computing loads and complex heterogeneous environments. Attached Figure Description

[0022] Figure 1This is a framework diagram of the card group parallel scheduling middleware according to an embodiment of this application; Figure 2 This is a diagram of the card group parallel scheduling middleware components according to an embodiment of this application; Figure 3 This is a flowchart illustrating a multi-card cluster heterogeneous scheduling method based on an event bus according to an embodiment of this application; Figure 4 This is a schematic diagram of the structure of a multi-card cluster heterogeneous scheduling system based on an event bus according to an embodiment of this application; and Figure 5 This is a schematic diagram of the structure of an electronic device based on the event bus-based multi-card cluster heterogeneous scheduling method according to an embodiment of this application. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this application described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Furthermore, the terms "installation," "setup," "equipped with," "connection," "linking," and "socketing" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances.

[0026] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0027] like Figure 3 As shown in the embodiment of the present invention, a multi-card cluster heterogeneous scheduling method based on an event bus is provided, the method comprising the following steps S101 to S103: Step S101: Encapsulate the functions of heterogeneous computing hardware resources into multiple independent services, each service providing a standardized interface and hiding the underlying hardware details; and decompose the computing task into a service call chain consisting of multiple services in a predetermined order. Specifically, the service abstraction module is configured to abstract and encapsulate the functions of various heterogeneous computing hardware resources in the system into multiple independent service units. Each service has a clear interface and functional definition, and communicates and interacts through standardized protocols. At the same time, the module decomposes and abstracts complex computing tasks into a series of schedulable services, forming a service call chain composed of multiple services in a predetermined order.

[0028] According to an embodiment of the present invention, preferably, the computational task is decomposed into a service call chain consisting of multiple services in a predetermined order, including: The service call chain is described using a directed acyclic graph (DAG), where nodes in the DAG represent services and directed edges represent the execution order and data dependencies between services. The DAG is sorted using a topological sorting algorithm to determine the order in which tasks are executed.

[0029] Specifically, computational tasks are abstracted as a set of services scheduled in a predetermined order. These services are organized into a pipeline driven by data flow through a streaming computing model. The scheduling order is described using a directed acyclic graph and defined through a JSON configuration file.

[0030] Step S102: Receive, distribute, and process various events generated during system operation via an event bus. These events include at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used to process the events asynchronously, and flow control is applied to the node call events. Specifically, the event bus module, as the system's message hub, is configured to receive, distribute, and process various events generated during system operation. These events include node call events, service scheduling events, and exception / error events. The event bus module maintains a dynamically scalable thread pool to asynchronously process various events and perform flow control on node call events that are generated at a high rate to prevent request backlog from causing service crashes.

[0031] Step S103: Based on the service call chain, drive the execution of the service through the event bus to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

[0032] Specifically, the task scheduling module is communicatively connected to the event bus module and the service abstraction module. It is configured to drive the execution and combination of services by publishing and subscribing to events on the event bus based on the service call chain, thereby completing the scheduling of computing tasks. Through the management of services and resource mapping, it realizes the unified management and dynamic binding of underlying heterogeneous computing hardware resources.

[0033] According to an embodiment of the present invention, preferably, based on the service call chain, the execution of the service is driven by an event bus to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources, including: Receive heartbeat, resource utilization and throughput data reported by the service, and submit corresponding events to the event bus based on preset trigger conditions; By subscribing to and processing events from the event bus, at least one of the following operations can be performed: service fault tolerance, fault migration, dynamic scaling, load balancing, and service circuit breaking, to ensure the stability of task operation.

[0034] Specifically, the status monitoring unit is configured to receive data such as heartbeat, resource utilization, and throughput reported by the service, and use this data to determine the service status and trigger corresponding scheduling events. It receives the metric data pushed by the service through the OpenTelemetry interface and uses a trigger mechanism to submit events to the event bus when conditions are met.

[0035] The service high availability unit is configured to perform at least one of the following operations—service fault tolerance, fault migration, dynamic scaling, load balancing, and service circuit breaking—by subscribing to and processing events from the event bus module, in order to ensure the stability of task operation.

[0036] According to an embodiment of the present invention, preferably, performing service fault tolerance and failover includes: When the first service heartbeat times out, submit a heartbeat timeout event to the event bus; Trigger service fault tolerance logic, and prioritize finding available resources for fault tolerance on the node where the first service is located; If the fault tolerance of the node fails, the fault migration logic is triggered to migrate the task of the first service to other nodes in the cluster for execution. After successful fault tolerance or migration, the load balancing logic is triggered to generate a new service call sequence.

[0037] According to an embodiment of the present invention, preferably, performing the dynamic scaling includes: When the system load is detected to meet the preset scaling conditions, a scaling event is submitted to the event bus; Trigger dynamic scaling logic to automatically increase or decrease the number of instances of a specified service, where the number of parallel instances of a single-node service is dynamically adjusted within a user-specified range.

[0038] According to an embodiment of the present invention, preferably, the load balancing is implemented using a weighted round-robin algorithm, including: Different weights are assigned to services based on the processing capacity of the nodes where the services reside. A service call sequence is generated based on the weights; According to the service call sequence, service requests are distributed to each service instance.

[0039] According to an embodiment of the present invention, preferably, it further includes: providing inter-service name addressing functionality, specifically including: Deploy a name service on each compute node to manage the mapping between service names and actual addresses; Services are bound or connected through virtual addresses identified by service names; When a service is bound to a virtual address, the name service assigns it an actual address and registers it. When a service connects to a virtual address, the name service resolves it and returns the actual address to establish peer-to-peer communication.

[0040] According to an embodiment of the present invention, preferably, providing inter-service name addressing functionality further includes: A single host server maintains information about all nodes in the cluster. The node information is synchronized to all name services, enabling connections to be established between the name services. Coordinate with all name services to complete cross-node service name resolution and service launch notification.

[0041] Specifically, the communication management module is configured to solve the inter-service communication problem through a name addressing mechanism. This module includes a name service deployed on each computing node, which is used to allocate addresses to services, manage the mapping between service names and addresses, resolve service names and publish service addresses. Services communicate with each other through virtual addresses that start with a preset address. To support cross-node communication, a host server is also included, which manages all nodes in the system and synchronizes node information to all name services, thereby realizing name services across the entire network.

[0042] According to an embodiment of the present invention, preferably, it further includes: a configuration management step: Configuration parameters stored in the configuration center service in key-value pair format; The service subscribes to its own configuration change events after startup; When the corresponding configuration parameters in the configuration center change, the relevant service is notified through the event bus; Trigger the parameter update logic within the service.

[0043] Specifically, the configuration center module stores storage configurations in the form of key-value pairs or files, and also has the function of updating service parameters during the task. After the service starts, it subscribes to configuration change events by default, and when the configuration changes, it notifies the corresponding service through the event bus.

[0044] According to an embodiment of the present invention, preferably, it further includes: The platform data plane provides a unified data receiving or sending capability to the framework, shielding the differences in underlying hardware interfaces. This includes: external services requesting device handles from the platform data plane through inter-process communication, and using these handles to send and receive data. The platform management plane provides the framework with cross-device service distribution and deployment capabilities, including: the master node distributes service packages and configuration files to target device nodes through the platform management plane, and starts or stops the service.

[0045] Specifically, the platform data plane provides the framework with a unified data receiving or sending capability, shielding the differences in underlying hardware interfaces; external services use the platform data plane to perform device initialization and data sending and receiving through inter-process communication (IPC).

[0046] The platform management plane provides the framework with cross-device service distribution and deployment capabilities as well as hardware resource reclamation capabilities; the master node distributes and starts services through the platform management plane.

[0047] As can be seen from the above description, this application achieves the following technical effects: In this embodiment, a multi-card cluster heterogeneous scheduling approach based on an event bus is adopted. This involves encapsulating the functionality of heterogeneous computing hardware resources into multiple independent services, each providing a standardized interface and hiding underlying hardware details. Computational tasks are decomposed into service call chains composed of these services in a predetermined order. The event bus receives, distributes, and processes various events generated during system runtime, including at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used for asynchronous event processing, and flow control is applied to the node call events. Based on the service call chains, the event bus drives the execution of the services to schedule computational tasks and achieve dynamic management and binding of heterogeneous computing hardware resources. This achieves automatic task partitioning, dynamic migration, and load balancing across multiple types of computing cards, thereby improving the overall system efficiency and stability. Furthermore, it solves the technical problems of traditional resource management methods, which typically employ static and tightly coupled approaches, resulting in high difficulty in upper-layer application development, poor portability, high task refactoring costs, and difficulty adapting to dynamically changing computational loads and complex heterogeneous environments.

[0048] To better understand this application, its working principle will be explained in detail below: like Figure 1 As shown, the SIM parallel scheduling middleware, guided by a service abstraction model, performs SIM resource management and task scheduling. Utilizing computational task abstraction, the middleware decomposes and abstracts tasks into a series of schedulable services. Service scheduling enables computational task scheduling, while service management and resource mapping facilitate the management and scheduling of computational resources. The middleware primarily consists of a resource management module, a computational task abstraction module, a computational task scheduling module, and a toolchain.

[0049] The resource management module manages the card cluster resources using a service-based approach. Guided by a service abstraction model, the module uses a unified interface encapsulation to shield the underlying hardware implementation and specific details, enabling unified management of heterogeneous hardware resources.

[0050] The computation task abstraction module breaks down computation tasks into a set of small, highly maintainable services. Each service focuses on a specific business function and provides services to the outside world using a uniformly encapsulated data input / output interface. Furthermore, based on a data flow mechanism, the task execution process can be equated to a data processing flow, and services interact through a lightweight communication mechanism. This abstraction provides a service-oriented hardware resource scheduling mechanism for service scheduling.

[0051] The computing task scheduling module maps hardware resources to services through the service composition and mapping module, enabling resource scheduling on a service-by-service basis, and then completing computing tasks through the chaining of services. Meanwhile, the dynamic task scheduling module provides task scheduling strategies in the form of plug-ins.

[0052] like Figure 2 As shown, the toolchain provides tools for visually breaking down and configuring complete task business processes, as well as tools for generating and configuring specific services after the breakdown.

[0053] Specifically, after the system starts, the task management module (part of the task scheduling module) completes the startup of each sub-module (such as the event bus, configuration center, and status monitoring) and the initialization of exception management. Simultaneously, the middleware loads runtime configuration files to initialize service scheduling and the task call chain. The task management module distributes services based on the task packages generated by the task building tool. The master node distributes the service executable files, dependency libraries, configuration files, etc., to the target device nodes through the platform management interface and starts the services.

[0054] After a service starts, tasks are scheduled in a predetermined order. A task is defined using a DAG (Directed Acyclic Graph), where nodes represent services and edges represent scheduling order. These services are scheduled sequentially as tasks. Simultaneously, the scheduler can create multiple task DAGs, enabling parallel computation of tasks, with services within different tasks working in parallel in a pipelined manner.

[0055] It's important to know that a task can be nested into a DAG. For example, a large task can be represented as a large DAG, and one of its steps can also be a DAG.

[0056] Services act as a bridge between applications and hardware resources, possessing well-defined functions and interfaces. By abstracting and defining the functions and resources within the system, they provide independent, specific functionalities for complex applications at the upper layer, while shielding the differences in underlying heterogeneous hardware at the lower layer.

[0057] Taking a specific drone target detection task as an example, this task is horizontally divided into an RTSP decoding service, an inference service, an image rendering service, and an RTSP encoding and streaming service. Each service is completed using specific hardware; for example, the inference service can be bound to an NPU, and the image rendering service is bound to a CPU. The decomposed task is described through a JSON configuration file, which defines the type, parallelism, list of runnable devices, subordinate nodes, and custom parameters for each service.

[0058] The event bus is the core of service scheduling, used to handle various events generated during task execution and service scheduling. The event bus maintains a dynamically scalable thread pool to asynchronously process events generated throughout the scheduling process.

[0059] Service fault tolerance process: When the status monitoring module detects a service heartbeat timeout, it submits a heartbeat timeout event to the event bus. The service high availability unit subscribes to this event and triggers the service fault tolerance logic. The system first searches for available resources on the current node for fault tolerance; if this fails, it triggers fault migration, migrating the service to other nodes. Upon success, it triggers the load balancer to generate a new service call sequence.

[0060] The dynamic scaling process is as follows: When the status monitoring module detects that the system load meets the conditions, it submits a scaling event to the event bus. The service high availability unit subscribes to this event and automatically increases or decreases the number of instances of the specified service.

[0061] The computing resource abstraction module provides a unified encapsulation of underlying heterogeneous hardware resources. The device abstraction layer adapts to the underlying hardware using a unified heterogeneous hardware adaptation framework. Horizontally, it is divided by function (such as decoding, inference, and encoding), with each function providing a unified abstract interface for services in the form of a dynamic link library.

[0062] Taking the inference function as an example, the inference service statically links the device abstraction library (DAL) during compilation, calling its interfaces for device management, model execution, etc. The service specifies the hardware type through parameters, without needing to concern itself with the underlying calls. The DAL dynamically loads the HAL library for the specified hardware at runtime, thereby using that hardware for inference, thus decoupling the service's business logic from the underlying hardware.

[0063] The system provides task building tools and service generation tools to improve ease of use.

[0064] Task building tool: Provides users with a graphical interface to complete the configuration and generation of tasks by dragging and dropping service components, configuring device binding, defining processes through connections, and finally outputting task software packages.

[0065] Service generation tool: Provides developers with a visual interface to generate service code templates by selecting templates, configuring parameters, and defining data exchange formats. After implementing the core business logic, developers can use this tool to compile and generate the service's executable package.

[0066] This application also has the following beneficial effects: 1. Thorough decoupling and high flexibility: Through the service abstraction model, hardware resources, computing tasks and scheduling logic are separated; combined with the event-driven architecture of the event bus, loose coupling of system components is achieved, making the system easier to maintain, expand and reuse, and greatly improving the system's flexibility and scalability.

[0067] 2. High availability and stability: The built-in comprehensive service governance mechanism (service fault tolerance, fault migration, dynamic scaling, load balancing, and service circuit breaking) can automatically respond to abnormal situations such as node failures and load fluctuations, significantly improving the stability and reliability of long-term operation in complex heterogeneous environments.

[0068] 3. High resource utilization: The pipelined execution model based on data flow, combined with asynchronous processing and flow control of the event bus, as well as load balancing and dynamic scaling strategies, can fully tap and utilize the computing potential of heterogeneous hardware, reduce resource idle time and data exchange waiting time, and achieve the optimization of overall resource utilization efficiency.

[0069] 4. Convenient development and deployment: Through a unified resource abstraction interface and visual building tools (task building tools and service generation tools), the complexity of the underlying hardware, cross-device communication details, and learning costs of dedicated APIs are shielded from application deployment personnel and service developers, supporting the rapid construction and deployment of scenario applications, which greatly reduces the development threshold and workload.

[0070] like Figure 5As shown, to achieve the above objectives, according to another aspect of this application, a multi-card cluster heterogeneous scheduling system based on an event bus is provided. This multi-card cluster heterogeneous scheduling system based on an event bus includes: Service abstraction module 401 is used to encapsulate the functions of heterogeneous computing hardware resources into multiple independent services, each service providing a standardized interface and hiding the underlying hardware details; and to decompose computing tasks into a service call chain consisting of multiple services in a predetermined order. Specifically, the service abstraction module is configured to abstract and encapsulate the functions of various heterogeneous computing hardware resources in the system into multiple independent service units. Each service has a clear interface and functional definition, and communicates and interacts through standardized protocols. At the same time, the module decomposes and abstracts complex computing tasks into a series of schedulable services, forming a service call chain composed of multiple services in a predetermined order.

[0071] The event handling module 402 is used to receive, distribute, and process various events generated during system operation via the event bus. The events include at least node call events, service scheduling events, and exception events. The event handling module uses a dynamically scalable thread pool to process the events asynchronously and performs flow control on the node call events. Specifically, the event bus module, as the system's message hub, is configured to receive, distribute, and process various events generated during system operation. These events include node call events, service scheduling events, and exception / error events. The event bus module maintains a dynamically scalable thread pool to asynchronously process various events and perform flow control on node call events that are generated at a high rate to prevent request backlog from causing service crashes.

[0072] The task scheduling and execution module 403 is used to drive the execution of the service through the event bus based on the service call chain, so as to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

[0073] Specifically, the task scheduling and execution module is communicatively connected to the event bus module and the service abstraction module. It is configured to drive the execution and combination of services by publishing and subscribing to events on the event bus based on the service call chain, thereby completing the scheduling of computing tasks. Through the management of services and resource mapping, it realizes the unified management and dynamic binding of underlying heterogeneous computing hardware resources.

[0074] As can be seen from the above description, this application achieves the following technical effects: In this embodiment, a multi-card cluster heterogeneous scheduling approach based on an event bus is adopted. This involves encapsulating the functionality of heterogeneous computing hardware resources into multiple independent services, each providing a standardized interface and hiding underlying hardware details. Computational tasks are decomposed into service call chains composed of these services in a predetermined order. The event bus receives, distributes, and processes various events generated during system runtime, including at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used for asynchronous event processing, and flow control is applied to the node call events. Based on the service call chains, the event bus drives the execution of the services to schedule computational tasks and achieve dynamic management and binding of heterogeneous computing hardware resources. This achieves automatic task partitioning, dynamic migration, and load balancing across multiple types of computing cards, thereby improving the overall system efficiency and stability. Furthermore, it solves the technical problems of traditional resource management methods, which typically employ static and tightly coupled approaches, resulting in high difficulty in upper-layer application development, poor portability, high task refactoring costs, and difficulty adapting to dynamically changing computational loads and complex heterogeneous environments.

[0075] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0076] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0077] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as event bus-based multi-card cluster heterogeneous scheduling methods.

[0078] In some embodiments, the event bus-based multi-card cluster heterogeneous scheduling method can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the event bus-based multi-card cluster heterogeneous scheduling method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to execute the event bus-based multi-card cluster heterogeneous scheduling method by any other suitable means (e.g., by means of firmware).

[0079] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0080] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0081] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0082] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0083] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LA), wide area networks (WA), blockchain networks, and the Internet.

[0084] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0085] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0086] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A multi-card cluster heterogeneous scheduling method based on an event bus, characterized in that, include: The functionality of heterogeneous computing hardware resources is encapsulated into multiple independent services, each of which provides a standardized interface and hides the underlying hardware details. Furthermore, the computational task is decomposed into a service call chain consisting of multiple services in a predetermined order; The system receives, distributes, and processes various events generated during system operation via an event bus. These events include at least node call events, service scheduling events, and exception events. A dynamically scalable thread pool is used to process the events asynchronously, and flow control is applied to the node call events. Based on the service call chain, the execution of the service is driven by the event bus to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

2. The method according to claim 1, characterized in that, Based on the service call chain, the execution of the services is driven by an event bus to schedule computing tasks and achieve dynamic management and binding of heterogeneous computing hardware resources, including: Receive heartbeat, resource utilization and throughput data reported by the service, and submit corresponding events to the event bus based on preset trigger conditions; By subscribing to and processing events from the event bus, at least one of the following operations can be performed: service fault tolerance, fault migration, dynamic scaling, load balancing, and service circuit breaking, to ensure the stability of task operation.

3. The method according to claim 2, characterized in that, Perform service fault tolerance and failover, including: When the first service heartbeat times out, submit a heartbeat timeout event to the event bus; Trigger service fault tolerance logic, and prioritize finding available resources for fault tolerance on the node where the first service is located; If the fault tolerance of the node fails, the fault migration logic is triggered to migrate the task of the first service to other nodes in the cluster for execution. After successful fault tolerance or migration, the load balancing logic is triggered to generate a new service call sequence.

4. The method according to claim 2, characterized in that, Performing the dynamic scaling includes: When the system load is detected to meet the preset scaling conditions, a scaling event is submitted to the event bus; Trigger dynamic scaling logic to automatically increase or decrease the number of instances of a specified service, where the number of parallel instances of a single-node service is dynamically adjusted within a user-specified range.

5. The method according to claim 2, characterized in that, The load balancing is implemented using a weighted round-robin algorithm, including: Different weights are assigned to services based on the processing capacity of the nodes where the services reside. A service call sequence is generated based on the weights; According to the service call sequence, service requests are distributed to each service instance.

6. The method according to claim 1, characterized in that, It also includes: providing name addressing functionality between services, specifically including: Deploy a name service on each compute node to manage the mapping between service names and actual addresses; Services are bound or connected through virtual addresses identified by service names; When a service is bound to a virtual address, the name service assigns it an actual address and registers it. When a service connects to a virtual address, the name service resolves it and returns the actual address to establish peer-to-peer communication.

7. The method according to claim 6, characterized in that, In addition to providing name addressing functionality between services, it also includes: A single host server maintains information about all nodes in the cluster. The node information is synchronized to all name services, enabling connections to be established between the name services. Coordinate with all name services to complete cross-node service name resolution and service launch notification.

8. The method according to claim 1, characterized in that, Also includes: Configuration management steps: Configuration parameters stored in the configuration center service in key-value pair format; The service subscribes to its own configuration change events after startup; When the corresponding configuration parameters in the configuration center change, the relevant service is notified through the event bus; Trigger the parameter update logic within the service.

9. The method according to claim 1, characterized in that, Also includes: The platform data plane provides a unified data receiving or sending capability to the framework, shielding the differences in underlying hardware interfaces. This includes: external services requesting device handles from the platform data plane through inter-process communication, and using these handles to send and receive data. The platform management plane provides the framework with cross-device service distribution and deployment capabilities, including: the master node distributes service packages and configuration files to target device nodes through the platform management plane, and starts or stops the service.

10. The method according to claim 1, characterized in that, The computational task is broken down into a service call chain consisting of multiple services in a predetermined order, including: The service call chain is described using a directed acyclic graph (DAG), where nodes in the DAG represent services and directed edges represent the execution order and data dependencies between services. The DAG is sorted using a topological sorting algorithm to determine the order in which tasks are executed.

11. A multi-card cluster heterogeneous scheduling system based on an event bus, characterized in that, include: The service abstraction module is used to encapsulate the functionality of heterogeneous computing hardware resources into multiple independent services. Each service provides a standardized interface and hides the underlying hardware details. Furthermore, the computational task is decomposed into a service call chain consisting of multiple services in a predetermined order; The event handling module is used to receive, distribute, and process various events generated during system runtime via the event bus. These events include at least node call events, service scheduling events, and exception events. The module utilizes a dynamically scalable thread pool to process events asynchronously and performs flow control on the node call events. The task scheduling and execution module is used to drive the execution of the service through the event bus based on the service call chain, so as to complete the scheduling of computing tasks and realize the dynamic management and binding of heterogeneous computing hardware resources.

12. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the event bus-based multi-card cluster heterogeneous scheduling method according to any one of claims 1-10.

13. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that cause the processor to execute the event bus-based multi-card cluster heterogeneous scheduling method as described in any one of claims 1-10.

Images