Embedded multi-board card cluster scheduling distribution method and system

Abstract

The invention provides an embedded multi-board card cluster scheduling distribution method and system, and relates to the technical field of cluster scheduling distribution, and the method comprises the steps: connecting a task management unit to recognize a to-be-processed task; the to-be-processed tasks transmitted by the task management unit are scheduled and allocated according to a scheduling algorithm to obtain a plurality of task queues; executing the corresponding task queues in the multi-board card processing module, and sending a plurality of load state monitoring data of the multi-board card processing module to a task management unit; and the task management unit performs load balancing analysis on the plurality of load state monitoring data and adjusts the plurality of task queues. The technical problems that in the prior art, due to the fact that a cluster scheduling algorithm is single, the stability of a cluster system is insufficient, and load balancing of a multi-embedded board card is unstable can be solved, task classification management, multi-algorithm scheduling and load balancing are carried out through the cluster task management scheduler, and the stability of the whole cluster system is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of cluster scheduling and allocation, and in particular to an embedded multi-board cluster scheduling and allocation method and system. Background Art

[0002] Cluster management and scheduling in embedded systems is a core technology in distributed systems, encompassing the allocation and management of hardware and software resources, as well as system performance, stability, and scalability. Most existing cluster systems use a single scheduling algorithm, resulting in poor adaptability to diverse tasks and loads, particularly with difficulty making efficient adjustments to dynamic load fluctuations. In particular, when resource utilization is uneven, clusters are prone to resource waste and task delays. Due to inflexible scheduling algorithms, resource scheduling and task allocation can be imbalanced, leading to resource overload on some nodes and idleness on others. This not only reduces resource utilization efficiency but can also cause system crashes or task failures, compromising cluster stability. Furthermore, in multi-card clusters, load distribution across different cards can be uneven, with some cards being overused while others are underutilized. This can lead to overloading of some cards, impacting task efficiency, increasing latency, and potentially even causing system bottlenecks or crashes.

[0003] In summary, the existing technology has technical problems such as insufficient cluster system stability and unstable load balancing of multiple embedded boards due to a single cluster scheduling algorithm. Summary of the Invention

[0004] The present application provides an embedded multi-board cluster scheduling allocation method and system, which is used to solve the technical problems existing in the prior art, such as insufficient cluster system stability and unstable load balancing of multiple embedded boards due to a single cluster scheduling algorithm.

[0005] In view of the above problems, the present application provides an embedded multi-board cluster scheduling and allocation method and system.

[0006] In the first aspect, the present application provides an embedded multi-board cluster scheduling and allocation method, which is implemented by an embedded multi-board cluster scheduling and allocation system, wherein the embedded multi-board cluster scheduling and allocation method includes: connecting a task management unit to identify tasks to be processed; sending the tasks to be processed to a task scheduling unit, and the task scheduling unit scheduling and allocating the tasks to be processed transmitted by the task management unit according to a scheduling algorithm to obtain multiple task queues; executing corresponding task queues in a multi-board processing module according to the multiple task queues, and sending multiple load status monitoring data of the multi-board processing module to the task management unit, wherein the multiple task queues correspond to the multi-board processing modules; the task management unit performs load balancing analysis on the multiple load status monitoring data and adjusts the multiple task queues.

[0007] Optionally, the task management unit identifies multiple load indicators corresponding to the multi-board processing module based on the multiple load status monitoring data; performs load balancing analysis on the multiple load indicators to obtain a load balancing coefficient; if the load balancing coefficient is less than a preset balancing threshold, identifies a high-load board with a load indicator greater than or equal to a first preset load indicator, and sends a task scheduling instruction to the task scheduling unit; and updates the task queue corresponding to the high-load board according to the task scheduling instruction.

[0008] Optionally, load status monitoring data corresponding to each board processing module is obtained, wherein the load status monitoring data includes at least CPU usage, memory usage, IO throughput, task execution time and board temperature; after normalizing the various data of the load status monitoring data, a weight coefficient layer is configured; weight calculation is performed on the various data of the load status monitoring data according to the weight coefficient layer, and multiple load indicators are output.

[0009] Optionally, determine the task queue corresponding to the high-load board, the task queue includes K tasks, where K is the total number of tasks in the task queue; obtain K busyness corresponding to the K tasks, identify the first task among the K busynesses, the first task being the task with the highest busyness among the K busynesses; migrate the first task out of the task queue, obtain the updated task queue of the high-load board, and release the identification status.

[0010] Optionally, a low-load board card with a load index less than or equal to a second preset load index is identified, wherein the second preset load index is less than the first preset load index; the first task is reallocated to the task queue of the low-load board card, and the load status monitoring data of the low-load board card is updated.

[0011] Optionally, the remaining computing resources of the low-load board are obtained; the compatibility between the remaining computing resources of the low-load board and the computing resources required for the first task is analyzed to obtain a fitness score; the target low-load board is located according to the fitness score result, the first task is reallocated to the target low-load board, and the task queue of the target low-load board is updated.

[0012] Optionally, multiple load status monitoring data of the multi-board processing module are sent to the task management unit; wherein the task management unit establishes a communication connection with the multi-board processing module through a server client model to synchronously update the multiple load status monitoring data.

[0013] Optionally, the server-client model includes a preset update cycle, and the multi-board processing module feeds back the plurality of load status monitoring data to the task management unit according to the preset update cycle.

[0014] In the second aspect, the present application also provides an embedded multi-board cluster scheduling and allocation system for executing the embedded multi-board cluster scheduling and allocation method as described in the first aspect, wherein the embedded multi-board cluster scheduling and allocation system includes: a task identification module for connecting to the task management unit to identify tasks to be processed; a scheduling and allocation module for sending the tasks to be processed to the task scheduling unit, and the task scheduling unit schedules and allocates the tasks to be processed transmitted by the task management unit according to the scheduling algorithm to obtain multiple task queues; a task execution module for executing corresponding task queues in the multi-board processing module according to the multiple task queues, and sending multiple load status monitoring data of the multi-board processing module to the task management unit, wherein the multiple task queues correspond to the multi-board processing modules; a load balancing analysis module for the task management unit to perform load balancing analysis on the multiple load status monitoring data and adjust the multiple task queues.

[0015] One or more technical solutions provided in this application have at least the following technical effects or advantages:

[0016] The system comprises a cluster task management unit, a task scheduling unit, and a task scheduling unit. The task scheduling unit schedules and allocates the tasks transmitted by the task management unit according to a scheduling algorithm to obtain a plurality of task queues. The system executes corresponding task queues in the multi-board processing module according to the plurality of task queues, and sends a plurality of load status monitoring data of the multi-board processing module to the task management unit, wherein the plurality of task queues correspond to the multi-board processing module. The task management unit performs load balancing analysis on the plurality of load status monitoring data and adjusts the plurality of task queues. In other words, the system comprises a cluster task management scheduler including the task management unit, the task scheduling unit, and the multi-board processing module, and performs task classification management, multiple algorithm scheduling, and load balancing. Specifically, the system comprises the task scheduling unit using a scheduling algorithm to allocate tasks, generating a plurality of task queues, collecting and analyzing the load status monitoring data in real time and performing load balancing analysis, dynamically adjusting the task queues, migrating overloaded board tasks to boards with lower loads, achieving load balancing, and significantly improving the stability of the cluster system under large-scale or high-load conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in this application or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely exemplary, and a person of ordinary skill in the art can obtain other drawings based on the provided drawings without creative work.

[0018] Figure 1 This is a flow chart of the embedded multi-board cluster scheduling and allocation method of this application;

[0019] Figure 2 This is a structural diagram of the embedded multi-board cluster scheduling and allocation system of this application.

[0020] Description of reference numerals: task identification module 11 , scheduling and allocation module 12 , task execution module 13 , load balancing analysis module 14 . DETAILED DESCRIPTION

[0021] This application provides an embedded multi-board cluster scheduling and allocation method and system to solve the technical problems in the prior art of insufficient cluster system stability and unstable load balancing of multiple embedded boards due to a single cluster scheduling algorithm. Through a cluster task management scheduler containing a task management unit, a task scheduling unit, and a multi-board processing module, task classification management, multiple algorithm scheduling, and load balancing are performed. Specifically, the task scheduling unit uses a scheduling algorithm to allocate tasks, generates multiple task queues, collects and analyzes load status monitoring data in real time and performs load balancing analysis, dynamically adjusts task queues, and migrates overloaded board tasks to boards with lower loads to achieve load balancing, significantly improving the stability of the cluster system under large-scale or high-load conditions.

[0022] Example 1, as Figure 1 As shown, the present application provides an embedded multi-board cluster scheduling and allocation method, wherein the embedded multi-board cluster scheduling and allocation method is applied to an embedded multi-board cluster scheduling and allocation system, and the embedded multi-board cluster scheduling and allocation method specifically includes the following steps:

[0023] S100: The connection task management unit identifies a task to be processed.

[0024] Specifically, a cluster task management scheduler is designed to manage and schedule tasks on multiple computing nodes (such as embedded boards) in a cluster, effectively distribute tasks to multiple nodes, ensure balanced resource utilization, improve cluster system efficiency, and track task execution status. The cluster task management scheduler consists of a task management unit, a task scheduling unit, and a multi-board processing module. The task management unit is responsible for managing the entire task lifecycle. It is responsible for receiving, updating, and tracking task status information (such as whether the task is completed or timed out), as well as collecting task execution results for each board and summarizing and counting the execution information of all tasks. The task scheduling unit is responsible for assigning tasks to each board according to different scheduling algorithms and determining which board executes which task. The multi-board processing module is mainly responsible for statistics and timely reporting of the status information of each board's tasks, executing the tasks assigned to it by the task scheduling unit, reporting the status information of its own task execution (such as task start time, end time, execution results, resource consumption, etc.) in real time, and reporting possible errors or failure information.

[0025] The task management unit establishes a connection with the cluster system, receiving and identifying pending tasks—tasks that have not yet begun execution or are awaiting scheduling. The task management unit transmits these pending tasks to the task scheduling unit, which schedules and allocates them based on their resource requirements, priority, and the current cluster load, identifying multiple task queues. The task management unit continuously monitors the status of these pending tasks until they complete or fail. Upon completion, the task management unit updates the task status and collects statistics on task execution. By automating the task classification and identification process, the task management unit can quickly and accurately identify pending tasks and prepare for subsequent scheduling.

[0026] S200: Sending the pending tasks to a task scheduling unit. The task scheduling unit schedules and allocates the pending tasks transmitted by the task management unit according to a scheduling algorithm to obtain a plurality of task queues.

[0027] Specifically, after identifying the pending tasks, the task management unit will send this task information to the task scheduling unit, and assign the pending tasks to different boards or nodes in the cluster for processing according to a certain scheduling algorithm, ensuring the balanced distribution of tasks among different nodes, optimizing the utilization of cluster resources, avoiding overload or waste of resources, and scheduling and allocating pending tasks according to different scheduling algorithms (such as FIFO, FCFS, CFS, etc.). Each scheduling algorithm has its own specific characteristics and is suitable for different scenarios and needs. In other words, tasks are assigned according to the current load of each board to ensure that the load of all nodes in the cluster is relatively balanced, thereby improving the efficiency of the overall cluster system. Through the scheduling algorithm, the pending tasks are divided into multiple task queues. Each task queue corresponds to a board or computing node, and the tasks in the task queue will be executed by the corresponding board or node.

[0028] FIFO scheduling is a scheduling algorithm based on the first-in-first-out principle. Tasks or processes are scheduled in the order they enter the queue. The task that enters the queue earliest will be executed first, and subsequent tasks will be queued in order until the previous task is completed. FCFS scheduling is similar to FIFO. FCFS scheduling also schedules tasks in the order in which they arrive. However, FCFS is a first-come, first-served principle and is usually used in a wider range of scenarios, including job scheduling and process scheduling. In job scheduling, jobs are scheduled according to the order in which they arrive in the backup queue; in process scheduling, they are scheduled according to the order in which processes arrive in the ready queue. CFS scheduling is a scheduling policy in the Linux kernel. Its design goal is to achieve completely fair CPU time allocation. The priority of a task is calculated based on the virtual runtime of each task (the number of time slices it runs in). The priority of a task changes dynamically, and tasks with shorter virtual runtimes are executed first to ensure fair resource allocation.

[0029] Depending on the scheduling algorithm, the task scheduling unit assigns tasks to multiple task queues, assigning tasks to different task queues based on factors such as priority, execution time, and resource requirements. CFS scheduling may prioritize short tasks, while FIFO and FCFS schedule tasks in the order they arrive. By assigning pending tasks to multiple task queues based on scheduling algorithms (such as load balancing, shortest job first, and round-robin), the task scheduling unit ensures efficient utilization of cluster resources and balanced task execution.

[0030] S300: Execute corresponding task queues in the multi-board processing module respectively according to the multiple task queues, and send multiple load status monitoring data of the multi-board processing module to the task management unit, wherein the multiple task queues correspond to the multi-board processing modules.

[0031] The task management unit establishes a communication connection with the multi-board processing module through a server-client model to synchronously update the multiple load status monitoring data.

[0032] The server-client model includes a preset update cycle, and the multi-board processing module feeds back the plurality of load status monitoring data to the task management unit according to the preset update cycle.

[0033] Specifically, the task scheduling unit sends the assigned multiple task queues to the corresponding multi-board processing modules, and each multi-board processing module is responsible for executing the tasks in the corresponding task queue. Each board processing module executes tasks according to the assigned task queue, monitors its own load status in real time, and promptly feeds back to the task management unit so as to monitor the system load in real time and ensure load balancing of each board. Each character queue and board processing module has a one-to-one correspondence. Load status monitoring data refers to the operating status information of each multi-board processing module when executing tasks, including indicators such as CPU utilization, memory usage, IO throughput, task execution time, and board temperature.

[0034] The task management unit establishes a communication connection with the multi-board processing module through the server-client model. The task management unit acts as a server and receives load status monitoring data from multiple boards. Each multi-board processing module acts as a client and regularly (according to a preset update cycle) sends its load status monitoring data to the task management unit. According to the preset update cycle, the multi-board processing module feeds back the load monitoring data to the task management unit, helping the task management unit understand the load status of each board, thereby adjusting the allocation of task queues to achieve load balancing. The server-client model is a common client-server communication model. The server acts as a service provider, managing and providing resources; the client acts as a requester, requesting services from the server. The multi-board processing module acts as a client, the task management unit acts as a server, and the client requests and feeds back information about the task execution process to the server.

[0035] The preset update cycle refers to the time interval for data synchronization between the task management unit and the multi-board processing module. It is usually a time interval pre-set according to demand. The update cycle should not be too long. Failure to update data for a long time will cause serious lag in information and fail to reflect the current real situation; it should not be too short. Frequent data updates will consume a lot of system resources, resulting in reduced efficiency.

[0036] By using the server-client model, the task management unit can receive load status monitoring data from multiple board processing modules in real time and update it synchronously according to the preset update cycle, so that the task management unit can schedule and optimize tasks according to the load conditions of each board, thereby achieving system load balancing and improving resource utilization efficiency.

[0037] S400: The task management unit performs load balancing analysis on the plurality of load status monitoring data and adjusts the plurality of task queues.

[0038] Furthermore, the present application S400 includes:

[0039] The task management unit identifies multiple load indicators corresponding to the multi-board processing module based on the multiple load status monitoring data; performs load balancing analysis on the multiple load indicators to obtain a load balancing coefficient; if the load balancing coefficient is less than a preset balancing threshold, identifies a high-load board with a load indicator greater than or equal to a first preset load indicator, and sends a task scheduling instruction to the task scheduling unit; and updates the task queue corresponding to the high-load board according to the task scheduling instruction.

[0040] Obtain load status monitoring data corresponding to each board processing module, wherein the load status monitoring data includes at least CPU usage, memory usage, IO throughput, task execution time, and board temperature; normalize the various data of the load status monitoring data, and configure a weight coefficient layer; calculate the weights of the various data of the load status monitoring data according to the weight coefficient layer, and output multiple load indicators.

[0041] Determine a task queue corresponding to the high-load board, the task queue including K tasks, where K is the total number of tasks in the task queue; obtain K busynesses corresponding to the K tasks, identify a first task among the K busynesses, the first task being the task with the highest busyness among the K busynesses; migrate the first task out of the task queue, obtain the updated task queue of the high-load board, and release the identification state.

[0042] Specifically, each board processing module regularly collects its own load status monitoring data, i.e., data generated during operation, including CPU utilization, memory usage, I / O throughput, task execution time, and board temperature. CPU utilization is the percentage of the board CPU's current workload as a percentage of its maximum processing capacity; memory usage is the amount of memory currently used by the board, usually measured in MB or GB; I / O throughput represents the amount of input and output operations completed by the board per second, usually measured in MB / s or IOPS (input and output operations per second); task execution time is the time required for each task to start and complete, usually in seconds; and board temperature refers to the temperature of the board hardware, usually expressed in degrees Celsius, which is used to monitor the health of the board.

[0043] Each load status monitoring data item is normalized, converting it to a range of 0 to 1. For example, if a board's CPU utilization is 85%, with a maximum of 100% and a minimum of 0%, the normalized value is 0.85. Similar processing is performed on all load status data to ensure that all data items are calculated on the same scale. Normalization converts load status data of different dimensions to the same standard range, typically between 0 and 1. This allows different types of load data to be compared and calculated on the same scale.

[0044] After normalization, each load status monitoring data item is assigned a weight coefficient, indicating its influence on the final comprehensive load index. Generally, more important indicators receive higher weights. Based on the configured weight coefficient layer, each item in the load status monitoring data is weighted and summed to calculate the load index for each board processing module. This is used to evaluate the load status of each board processing module.

[0045] A load balancing analysis is performed on the multiple load indicators corresponding to the multi-board processing modules to calculate a load balancing coefficient, which reflects the degree of load balance across the boards in the cluster. The cluster's balance is determined by comparing the combined load indicators of each board. If the load indicators of most boards are similar, the load balancing coefficient is high, indicating a relatively balanced load; conversely, the load balancing coefficient is low. A pre-set balancing threshold is set. If the load balancing coefficient is less than the preset balancing threshold, the current load is considered unbalanced, and analysis of the boards with higher load indicators is required. Task scheduling instructions are sent to the task scheduling unit, requesting that some tasks be migrated from the high-load boards to the lower-load boards to balance the load.

[0046] After receiving the instruction, the task scheduling unit updates the task queue, migrating some tasks from the high-loaded card to the low-loaded card, redistributing tasks to ensure a more balanced overall load distribution. The task queue corresponding to the high-loaded card is determined. This is the queue of all tasks to be processed on this card's processing module, including K tasks, where K is the total number of tasks. The busyness of each task is obtained, which is an indicator of the workload or resource requirements of each task. This can usually be measured by the CPU, memory, I / O bandwidth, and other factors used by the task. Busier tasks consume more resources, which may affect system performance and response time.

[0047] According to K busynesses, the busiest task (i.e., the task with the highest busyness) is selected as the first task, and the first task is migrated out of the task queue of the current high-load board to reduce the burden of the board, thereby improving the load balance of the system. After the migration, the tasks in the task queue are rearranged, and the load of the updated task queue is effectively adjusted. After the migration, the high-load board marked above is unmarked, indicating that its load has been adjusted and a better load balance has been restored. For example, a high-load board contains 5 tasks, and their corresponding busynesses are 40 for task 1, 50 for task 2, 70 for task 3, 30 for task 4, and 60 for task 5. Among them, task 3 has the highest busyness, so it is migrated out of the task queue of the high-load board to reduce the load of the board. The task queue after migration contains task 1, task 2, task 4, and task 5.

[0048] Each board updates the load status of each task in its task queue to the task management unit at regular intervals. The task management unit calculates the total load status of each task queue, finds the task queue with the busiest CPU usage based on the total load status of each task queue, finds the task with the busiest CPU usage from the busiest task queue, and migrates the busiest task from the current task queue, effectively reducing the pressure on high-load boards and avoiding performance bottlenecks caused by task overload on some boards, thereby improving the overall performance and stability of the cluster.

[0049] Furthermore, the present application further comprises the following steps:

[0050] Identify a low-load board card that has a load index less than or equal to a second preset load index, wherein the second preset load index is less than the first preset load index; reallocate the first task to a task queue of the low-load board card, and update load status monitoring data of the low-load board card.

[0051] Specifically, boards with a load balancing coefficient less than or equal to a second preset load indicator are identified as low-load boards. Low-load boards have a lighter load, lower resource utilization, and are typically idle or lightly used. The second preset load indicator is smaller than the first preset load indicator and is used to indicate a low-load board. Specifically, the system calculates the load balancing coefficient for each board. If the coefficient is less than or equal to the second preset load indicator, the board is identified as a low-load board. The second preset load indicator is typically set to a lower value, indicating that the board's resources are being used less frequently.

[0052] After identifying the low-load board, the tasks that were previously migrated from the high-load board are reallocated to the low-load board, distributing the workload to the idle boards. That is, the busiest task (i.e., the first task) is placed in the task queue of the low-load board, further improving the efficiency of resource utilization and avoiding excessive concentration on certain boards. After the tasks are reallocated to the low-load board, the load status monitoring data of the board is updated. At this time, the load of the low-load board increases, so it is necessary to update the monitoring data in a timely manner and recalculate the CPU usage, memory usage, IO throughput and other indicators of the board to ensure that the actual load status of each board is reflected in real time. By allocating to the low-load board, the low-load board can be fully utilized, thereby improving the overall resource utilization efficiency and avoiding the waste of computing resources due to too low load on some boards.

[0053] Furthermore, the present application further comprises the following steps:

[0054] Obtain the remaining computing resources of the low-load board; analyze the compatibility of the remaining computing resources of the low-load board with the computing resources required for the first task to obtain a fitness score, locate the target low-load board according to the fitness score result, reallocate the first task to the target low-load board, and update the task queue of the target low-load board.

[0055] Specifically, after determining the low-load board, the remaining computing resources of the board are obtained, that is, the computing resources that have not been occupied. This is usually obtained by subtracting the occupied computing resources from its total computing resources, which determines whether the board can handle additional tasks. The compatibility of each low-load board with the first task is calculated, which reflects the degree of match between the remaining computing resources of the low-load board and the computing resources required for the first task. The fitness score can be calculated by the ratio of the remaining computing resources of the low-load board to the computing resources required for the first task. The higher the fitness score, the more the remaining computing resources of the low-load board can meet the requirements of the task, and the better the matching degree.

[0056] Based on the fitness score, the target low-load card is located, indicating the low-load card most suitable for handling the first task. The first task is reassigned to the target low-load card, and the target low-load card's task queue is updated. The target low-load card begins processing the newly assigned task, and the load status monitoring data will also change based on the task's execution status.

[0057] Through the fitness scoring mechanism, tasks are assigned to the low-load boards with the most abundant computing resources, thereby ensuring the efficiency of task execution, improving the overall resource utilization efficiency, and avoiding the waste of board resources.

[0058] In summary, the embedded multi-board cluster scheduling and allocation method provided by this application has the following technical effects:

[0059] The system comprises a cluster task management unit, a task scheduling unit, and a task scheduling unit. The task scheduling unit schedules and allocates the tasks transmitted by the task management unit according to a scheduling algorithm to obtain a plurality of task queues. The system executes corresponding task queues in the multi-board processing module according to the plurality of task queues, and sends a plurality of load status monitoring data of the multi-board processing module to the task management unit, wherein the plurality of task queues correspond to the multi-board processing module. The task management unit performs load balancing analysis on the plurality of load status monitoring data and adjusts the plurality of task queues. In other words, the system comprises a cluster task management scheduler including the task management unit, the task scheduling unit, and the multi-board processing module, and performs task classification management, multiple algorithm scheduling, and load balancing. Specifically, the system comprises the task scheduling unit using a scheduling algorithm to allocate tasks, generating a plurality of task queues, collecting and analyzing the load status monitoring data in real time and performing load balancing analysis, dynamically adjusting the task queues, migrating overloaded board tasks to boards with lower loads, achieving load balancing, and significantly improving the stability of the cluster system under large-scale or high-load conditions.

[0060] Example 2: Based on the same inventive concept as the embedded multi-board cluster scheduling and allocation method in the aforementioned Example 1, this application also provides an embedded multi-board cluster scheduling and allocation system, see the attached Figure 2 , the embedded multi-board cluster scheduling and distribution system includes:

[0061] A task identification module 11 is used to connect to the task management unit to identify tasks to be processed; a scheduling allocation module 12 is used to send the tasks to be processed to the task scheduling unit, and the task scheduling unit schedules and allocates the tasks to be processed transmitted by the task management unit according to the scheduling algorithm to obtain multiple task queues; a task execution module 13 is used to execute corresponding task queues in the multi-board processing module according to the multiple task queues, and send multiple load status monitoring data of the multi-board processing module to the task management unit, wherein the multiple task queues correspond to the multi-board processing modules; a load balancing analysis module 14 is used for the task management unit to perform load balancing analysis on the multiple load status monitoring data and adjust the multiple task queues.

[0062] Furthermore, the task execution module 13 in the embedded multi-board cluster scheduling and allocation system is further configured to:

[0063] Sending multiple load status monitoring data of the multi-board processing module to the task management unit; wherein the task management unit establishes a communication connection with the multi-board processing module through a server client model to synchronously update the multiple load status monitoring data.

[0064] Furthermore, the task execution module 13 in the embedded multi-board cluster scheduling and allocation system is further configured to:

[0065] The server-client model includes a preset update cycle, and the multi-board processing module feeds back the plurality of load status monitoring data to the task management unit according to the preset update cycle.

[0066] Furthermore, the load balancing analysis module 14 in the embedded multi-board cluster scheduling and distribution system is further configured to:

[0067] The task management unit identifies multiple load indicators corresponding to the multi-board processing module based on the multiple load status monitoring data; performs load balancing analysis on the multiple load indicators to obtain a load balancing coefficient; if the load balancing coefficient is less than a preset balancing threshold, identifies a high-load board with a load indicator greater than or equal to a first preset load indicator, and sends a task scheduling instruction to the task scheduling unit; and updates the task queue corresponding to the high-load board according to the task scheduling instruction.

[0068] Furthermore, the load balancing analysis module 14 in the embedded multi-board cluster scheduling and distribution system is further configured to:

[0069] Obtain load status monitoring data corresponding to each board processing module, wherein the load status monitoring data includes at least CPU usage, memory usage, IO throughput, task execution time, and board temperature; normalize the various data of the load status monitoring data, and configure a weight coefficient layer; calculate the weights of the various data of the load status monitoring data according to the weight coefficient layer, and output multiple load indicators.

[0070] Furthermore, the load balancing analysis module 14 in the embedded multi-board cluster scheduling and distribution system is further configured to:

[0071] Determine a task queue corresponding to the high-load board, the task queue including K tasks, where K is the total number of tasks in the task queue; obtain K busynesses corresponding to the K tasks, identify a first task among the K busynesses, the first task being the task with the highest busyness among the K busynesses; migrate the first task out of the task queue, obtain the updated task queue of the high-load board, and release the identification state.

[0072] Furthermore, the load balancing analysis module 14 in the embedded multi-board cluster scheduling and distribution system is further configured to:

[0073] Identify a low-load board card that has a load index less than or equal to a second preset load index, wherein the second preset load index is less than the first preset load index; reallocate the first task to a task queue of the low-load board card, and update load status monitoring data of the low-load board card.

[0074] Furthermore, the load balancing analysis module 14 in the embedded multi-board cluster scheduling and distribution system is further configured to:

[0075] Obtain the remaining computing resources of the low-load board; analyze the compatibility of the remaining computing resources of the low-load board with the computing resources required for the first task to obtain a fitness score, locate the target low-load board according to the fitness score result, reallocate the first task to the target low-load board, and update the task queue of the target low-load board.

[0076] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. Figure 1The embedded multi-board cluster scheduling and allocation method and specific examples in Example 1 are also applicable to the embedded multi-board cluster scheduling and allocation system of this embodiment. Through the above detailed description of the embedded multi-board cluster scheduling and allocation method, technical personnel in this field can clearly understand the embedded multi-board cluster scheduling and allocation system of this embodiment, so for the sake of brevity of the specification, it will not be described in detail here.

[0077] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

[0078] Obviously, those skilled in the art may make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the present application and its equivalents, the present application is intended to include these modifications and variations.

Claims

1. An embedded multi-board cluster scheduling and allocation method, characterized in that: include: The connection task management unit identifies the task to be processed; Sending the pending tasks to a task scheduling unit, which schedules and allocates the pending tasks transmitted by the task management unit according to a scheduling algorithm to obtain multiple task queues; Executing corresponding task queues in the multi-board processing modules respectively according to the plurality of task queues, and sending a plurality of load status monitoring data of the multi-board processing modules to the task management unit, wherein the plurality of task queues correspond to the multi-board processing modules; The task management unit performs load balancing analysis on the multiple load status monitoring data and adjusts the multiple task queues.

2. The embedded multi-board cluster scheduling and allocation method according to claim 1, wherein: The task management unit performs load balancing analysis on the plurality of load status monitoring data and adjusts the plurality of task queues, including: The task management unit identifies a plurality of load indicators corresponding to the multi-board processing module according to the plurality of load status monitoring data; Performing load balancing analysis on the multiple load indicators to obtain a load balancing coefficient; If the load balancing coefficient is less than a preset balancing threshold, identifying a high-load board card that is greater than or equal to a first preset load indicator, and sending a task scheduling instruction to the task scheduling unit; The task queue corresponding to the high-load board is updated according to the task scheduling instruction.

3. The embedded multi-board cluster scheduling and allocation method according to claim 2, characterized in that: The task management unit identifies a plurality of load indicators corresponding to the multi-board processing module according to the plurality of load status monitoring data, including: Obtaining load status monitoring data corresponding to each board processing module, wherein the load status monitoring data includes at least CPU usage, memory usage, I / O throughput, task execution time, and board temperature; After normalizing the various data of the load status monitoring data, a weight coefficient layer is configured; The weight calculation is performed on each data of the load status monitoring data according to the weight coefficient layer, and a plurality of load indicators are output.

4. The embedded multi-board cluster scheduling and allocation method according to claim 2, wherein: Updating the task queue corresponding to the high-load board according to the task scheduling instruction includes: Determine a task queue corresponding to the high-load board, wherein the task queue includes K tasks, where K is the total number of tasks in the task queue; Obtaining K busynesses corresponding to the K tasks, and identifying a first task among the K busynesses, where the first task is the task with the highest busyness among the K busynesses; Migrate the first task out of the task queue, obtain the updated task queue of the high-load board, and release the identification state.

5. The embedded multi-board cluster scheduling and allocation method according to claim 4, characterized in that: After migrating the first task out of the task queue, the method further includes: Identifying a low-load board that is less than or equal to a second preset load index, wherein the second preset load index is less than the first preset load index; The first task is reallocated to the task queue of the low-load board, and the load status monitoring data of the low-load board is updated.

6. The embedded multi-board cluster scheduling and allocation method according to claim 5, characterized in that: Before reallocating the first task to the task queue of the low-load board, the method further includes: Obtaining the remaining computing resources of the low-load board; Analyze the compatibility of the remaining computing resources of the low-load board and the computing resources required for the first task to obtain a compatibility score, locate the target low-load board according to the compatibility score result, reallocate the first task to the target low-load board, and update the task queue of the target low-load board.

7. The embedded multi-board cluster scheduling and allocation method according to claim 1, wherein: Sending a plurality of load status monitoring data of the multi-board processing module to the task management unit; The task management unit establishes a communication connection with the multi-board processing module through a server-client model to synchronously update the multiple load status monitoring data.

8. The embedded multi-board cluster scheduling and allocation method according to claim 7, characterized in that: The serverclient model includes a preset update cycle, and the multi-board processing module feeds back the plurality of load status monitoring data to the task management unit according to the preset update cycle.

9. Embedded multi-board cluster scheduling and distribution system, characterized in that: The embedded multi-board cluster scheduling and allocation system is configured to implement the method for scheduling and allocating embedded multi-board clusters according to any one of claims 1 to 8, wherein the method comprises: A task identification module, used to connect to the task management unit to identify tasks to be processed; A scheduling and allocation module, configured to send the pending tasks to a task scheduling unit, wherein the task scheduling unit schedules and allocates the pending tasks transmitted by the task management unit according to a scheduling algorithm to obtain a plurality of task queues; a task execution module, configured to execute corresponding task queues in the multi-board processing module respectively according to the plurality of task queues, and send a plurality of load status monitoring data of the multi-board processing module to the task management unit, wherein the plurality of task queues correspond to the multi-board processing modules; The load balancing analysis module is used for the task management unit to perform load balancing analysis on the multiple load status monitoring data and adjust the multiple task queues.

Images