A gradient scheduling method and system based on a cloud platform

Abstract

The application discloses a gradient scheduling method and system based on a cloud platform, relates to the field of data processing, and comprises the following steps: a recognition module is used for accessing a to-be-processed task list received by a cloud platform central processor in real time, recognizing the attributes of each to-be-processed task, and dividing the to-be-processed task list into a plurality of subtask lists based on the attributes of the to-be-processed tasks; a waiting module is used for receiving all the subtask lists obtained by real-time operation of the recognition module, acquiring the operation results of a monitoring module in real time, and deciding and executing the distribution of each to-be-processed task in each subtask list according to the operation results of the monitoring module; and the application realizes efficient gradient scheduling by intelligently dividing tasks and dynamically matching processing resources, classifies tasks according to attributes, orders the tasks according to priorities, accurately matches the tasks in combination with real-time state scores of processors, and improves the overall task processing efficiency.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, specifically to a gradient scheduling method and system based on a cloud platform. Background Technology

[0002] In industrial data processing scenarios, cloud platforms with integrated processors serve as devices for industrial data processing, undertaking important responsibilities such as data management, analysis, and classification.

[0003] Patent application number 201711343933.0 discloses a cloud-based virtualization instance scheduling system, which includes an API gateway platform, a task scheduling module, and a resource module. The API gateway platform provides a user interface, resource management entry point, and implements full lifecycle management of access authentication, computing resources, and network resources. The task scheduling module calculates, schedules, and allocates network and resources to maximize global resource utilization while meeting user resource needs. The resource module provides the resources and virtualization instances to be allocated. The API gateway platform includes an access authentication module, an access control module, a resource management module, and a network management module. The task scheduling module includes a resource scheduling module and a network scheduling module. The resource scheduling module performs resource creation, deletion, and modification scheduling according to scheduling policies. The network scheduling module solves the north-south and east-west traffic communication problems between physical machines, virtual machines, and containers. When an application entity sends resource creation requests and network creation requests to the API gateway platform... The access control module of the API gateway platform is used to review the sent resource creation requests and network creation requests; call the network creation interface and resource creation interface; forward the network creation request to the network scheduling module through the network creation interface, and the network scheduling module performs network scheduling; forward the resource creation request to the resource scheduling module through the resource creation interface, and the resource scheduling module performs resource scheduling. This application aims to solve the problem that "current mainstream similar technical solutions mainly include ECS and CaaS services, both of which mainly provide hybrid management and network interconnection solutions for single or partial computing virtualization technologies, making it difficult to solve the problems encountered by traditional software or Internet industries in the process of migrating to the cloud. Mainstream cloud computing vendors provide cloud host services, generally supporting both physical machines and virtualization, and network interconnection between physical machines and virtualization requires customization. For enterprise applications with container-like components, it cannot meet their cloud migration needs. CaaS only provides container virtualization technology, which cannot adapt to all business scenarios, and cannot meet the cloud migration needs of applications whose components cannot run in containers."

[0004] However, for industrial data processing scenarios, existing technologies often distinguish tasks based on their type and queue them according to their cloud upload time. This approach is difficult to optimize the overall data processing capabilities of the cloud platform when there are many complex tasks to be processed.

[0005] To address this, a gradient scheduling method and system based on a cloud platform are proposed. Summary of the Invention

[0006] In view of the above-mentioned shortcomings of the existing technology, the present invention provides a gradient scheduling method and system based on a cloud platform, which can effectively solve the problems of the existing technology.

[0007] To achieve the above objectives, the present invention is implemented through the following technical solutions;

[0008] This invention discloses a gradient scheduling system based on a cloud platform, comprising:

[0009] The system comprises the following modules: Identification module, which accesses the real-time task list received by the cloud platform's central processing unit (CPU), identifies the attributes of each task, and divides the task list into several sub-task lists based on these attributes; Waiting module, which receives all sub-task lists obtained by the identification module in real-time, acquires the results of the monitoring module's operation, and uses these results to determine and execute the distribution of tasks in each sub-task list; Monitoring module, which monitors the operating status of each coprocessor below the CPU and obtains a running status score for each coprocessor; Prevention module, which monitors whether each coprocessor is malfunctioning, and when a coprocessor malfunction is detected, applies the waiting module to re-determine and distribute the currently unprocessed and processing tasks of the malfunctioning coprocessor; and Management module, which records the historical running status scores of each coprocessor, generates a coprocessor maintenance queue based on these scores, and synchronously generates processing messages for the task list.

[0010] Furthermore, the identification module is equipped with a storage unit, which is used to store a list of several sub-tasks divided by the identification module during operation;

[0011] The identification module operates in real time. When a new list of tasks to be processed is received, it synchronously identifies the attributes of each task in the list. After the attributes of the tasks to be processed are identified, it forwards each task to the corresponding sub-task list based on the attributes of each task to be processed, so as to update the tasks to be processed in the sub-task list.

[0012] Among them, each task to be processed in the subtask list of each storage unit is based on the corresponding Sort the values ​​in descending order, so that Larger values ​​are listed first. The tasks with smaller values ​​are listed later, and the tasks listed earlier are given priority by the central processing unit.

[0013] In the formula: The current time; The time it takes for the task to be processed to be separated from the task list to the sub-task list by the identification module; The complexity of the task to be processed.

[0014] Furthermore, the attributes of the task to be processed include: basic data processing tasks, parallel computing tasks, and complex logic and AI tasks;

[0015] The basic data processing tasks include: measurement calculation, data management, forwarding and migration control; the parallel computing tasks include: function calculation, data analysis; and the complex logic and AI tasks include: massive data processing, algorithm model training and running.

[0016] The updated pending tasks in the subtask list are based on After sorting in descending order, all tasks are placed after the previously sorted tasks in descending order.

[0017] Furthermore, the complexity of the task to be processed Calculated using the following formula:

[0018] ;

[0019] In the formula: Score based on basic complexity; Score for dynamic complexity; , As weight;

[0020] Among them, weight , The sum of all numbers is 1 and all of them are positive. < Weight , Initial settings =0.4、 =0.6, The larger the value, the greater the difficulty of the task to be processed and the more computing resources are required.

[0021] Furthermore, the central processing unit is provided with several coprocessors at its lower level, and the several coprocessors and the central processing unit form a processor topology. The several coprocessors are configured with no less than three attributes for each type of task to be processed.

[0022] The operating status score of each coprocessor in the monitoring module is obtained by the following formula:

[0023] ;

[0024] In the formula: Rate the coprocessor's operating status; The total workload of the coprocessor currently completing the task; This represents the total time taken for the coprocessor to complete the current task. This refers to the total resources of the coprocessor. This represents the amount of resources currently used by the coprocessor. This represents the total number of tasks currently running in the coprocessor. This is the base value for the estimated processing time of the new task; The amount of data for the new task; This is the estimated processing time per unit of data volume based on historical data;

[0025] Among them, the coprocessor running status score A higher value indicates that the corresponding coprocessor is more suitable as a distribution target for the task to be processed. , The above formula is derived from the last task in the subtask list to which the target coprocessor belongs, and the above formula is used to perform calculations on each coprocessor. Value calculation.

[0026] Furthermore, after each coprocessor completes its runtime status score calculation, it matches the coprocessor with a subtask list based on the attributes of the tasks to be processed, so that each coprocessor set corresponds to a subtask list. Each coprocessor in the coprocessor set is based on... Arrange them in descending order to form a coprocessor queue;

[0027] The process involves matching the first coprocessor in the coprocessor queue with the last task in the subtask list, and then matching the last coprocessor in the coprocessor queue with the first task in the subtask list. This continues until all coprocessors in the coprocessor set have completed their task matching process. Each coprocessor then begins processing its matched task. After any coprocessor completes its matched task, the process refreshes and re-matches the tasks in the subtask list with the coprocessors in the coprocessor set based on the above logic.

[0028] In the above logic, when a set of coprocessors processes a list of subtasks, the task in the middle of the list is processed last.

[0029] Furthermore, the prevention and control module interacts with the self-testing circuits and diagnostic tools configured for each coprocessor in the cloud platform to obtain the coprocessor operation fault judgment results. When a coprocessor operation fault is obtained, the module calculates the fault values ​​of each unprocessed and in-process task. Value, based on The calculation results are used to insert each unprocessed and processing task into the corresponding subtask list, and then the waiting module decides and executes the task distribution.

[0030] Among them, unprocessed and processing tasks are being executed. When calculating the value, Additional information on the time the task remains in the fault coprocessor.

[0031] Furthermore, when the management module records the historical running status scores of each coprocessor, it distinguishes and records each coprocessor based on a line graph. In the line graph corresponding to each coprocessor, the horizontal axis represents the time for obtaining the coprocessor's running status score, and the vertical axis represents the running status score obtained at each time. The management module reads the line graph corresponding to each coprocessor and selects the coprocessors corresponding to the line graphs in which the running status score shows a continuous downward trend as priority maintenance targets, and selects the coprocessors corresponding to the line graphs from which the score decreases most frequently as priority maintenance targets, in order to generate a coprocessor maintenance queue for manual online and offline maintenance.

[0032] The pending task list message includes several sub-task list messages. Each sub-task list message contains: the name of the coprocessor used for processing each pending task, the start timestamp of processing each pending task, the end timestamp of processing each pending task, the stage of processing each pending task by the coprocessor, and the real-time resource consumption of the coprocessor.

[0033] Furthermore, the identification module is internally connected to a storage unit via a wireless network, and is also internally connected to a waiting module and a monitoring module via a wireless network. The monitoring module is also internally connected to a prevention and control module via a wireless network, and is also internally connected to a management module via a wireless network.

[0034] On the other hand, a gradient scheduling method based on a cloud platform includes:

[0035] The system accesses the list of pending tasks received by the cloud platform's central processing unit, identifies the attributes of each task, divides it into several sub-task lists, and sorts the pending tasks in each sub-task list in descending order. It receives the divided sub-task lists, combines the monitoring results of the lower-level coprocessors' running status, decides on and executes the distribution of each task. It monitors the running status of each lower-level coprocessor in real time, calculates the running status score of each coprocessor, and completes the configuration of pending tasks and coprocessors based on the running status score. It monitors the running status of coprocessors; if a fault is detected, it re-includes unprocessed and processing tasks in the task distribution process. It records the historical running status scores of each coprocessor, generates a maintenance queue, and simultaneously generates task processing messages. After a coprocessor completes a task, it triggers a refresh of the re-matching of pending tasks and coprocessors.

[0036] Compared with the known prior art, the technical solution provided by this invention has the following beneficial effects:

[0037] This invention provides a gradient scheduling method and system based on a cloud platform. During execution, this method and system achieve efficient gradient scheduling by intelligently dividing tasks and dynamically matching processing resources. It classifies tasks according to their attributes and sorts them by priority, and combines real-time processor status scoring for accurate matching, thereby improving the overall task processing efficiency. In case of failure, tasks are quickly recycled and redistributed to reduce the impact of interruptions and ensure stability. At the same time, it records the historical status of the processor, prioritizes the maintenance of devices with declining performance, and extends their service life. The task processing adopts a front-to-back cross-matching logic to balance the load and shorten the overall processing time. It can also generate detailed processing messages to provide data support for optimization, comprehensively improving the efficiency, stability, and manageability of task processing on the cloud platform. Attached Figure Description

[0038] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of a gradient scheduling system based on a cloud platform;

[0040] Figure 2 This is a flowchart illustrating a gradient scheduling method based on a cloud platform. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0042] The present invention will be further described below with reference to embodiments.

[0043] Example 1:

[0044] This embodiment presents a gradient scheduling system based on a cloud platform, such as... Figure 1 As shown, it includes:

[0045] The identification module is used to access the list of tasks to be processed received in real time by the central processing unit of the cloud platform, identify the attributes of each task to be processed, and divide the list of tasks to be processed into several sub-task lists based on the attributes of the tasks to be processed.

[0046] The recognition module has an internal storage unit, which is used to store a list of several sub-tasks divided by the recognition module during operation.

[0047] The identification module runs in real time. When a new list of tasks to be processed is received, it synchronously identifies the attributes of each task in the list. After the attributes of the tasks to be processed are identified, the module forwards each task to the corresponding sub-task list based on the attributes of each task to update the tasks to be processed in the sub-task list.

[0048] Among them, each task to be processed in the subtask list of each storage unit is based on the corresponding Sort the values ​​in descending order, so that Larger values ​​are listed first. The tasks with smaller values ​​are listed later, and the tasks listed earlier are given priority by the central processing unit.

[0049] In the formula: The current time; The time it takes for the task to be processed to be separated from the task list to the sub-task list by the identification module; The complexity of the task to be processed;

[0050] The attributes of the tasks to be processed include: basic data processing tasks, parallel computing tasks, and complex logic and AI tasks;

[0051] Basic data processing tasks include: econometric calculations, data management, forwarding, migration and control; parallel computing tasks include: function computation, data analysis; complex logic and AI tasks include: massive data processing, algorithm model training and execution.

[0052] Updated pending tasks in the subtask list based on After sorting in descending order, all tasks are placed after the tasks in the previous descending sort.

[0053] Complexity of tasks to be processed Calculated using the following formula:

[0054] ;

[0055] In the formula: Score based on basic complexity; Score for dynamic complexity; , As weight;

[0056] Among them, weight , The sum of all numbers is 1 and all of them are positive. < Weight , Initial settings =0.4、 =0.6, The larger the value, the greater the difficulty of the task to be processed and the more computing resources are required;

[0057] , The calculation is performed using the following formula:

[0058] ;

[0059] In the formula: The length of the basic operation instruction; This refers to the task structure complexity parameter; The amount of raw data processed by the task; The frequency of interaction between the task and other components or devices; This is the data transmission bandwidth fluctuation coefficient;

[0060] in, The value takes the following order: when there is no nested hierarchy (such as a single measurement calculation), =1, when there are 1-2 levels of nesting (such as direct forwarding after calculation), =1.5; When there are 3 or more levels of nesting (such as multi-step data verification + calculation + forwarding), =2;

[0061] This is the ratio of the real-time transmission bandwidth during task processing to the rated transmission bandwidth of the coprocessor. If the real-time bandwidth is greater than 80% of the rated bandwidth... =1; 50%–80% =1.2; when less than 50%, =1.5 (The lower the bandwidth, the greater the impact of transmission delay on complexity);

[0062] The waiting module is used to receive the list of all subtasks obtained by the identification module in real time, obtain the running results of the monitoring module in real time, and make decisions and execute the distribution of each pending task in each subtask list based on the running results of the monitoring module.

[0063] The monitoring module is used to monitor the operating status of each coprocessor below the central processing unit and obtain the operating status score of each coprocessor.

[0064] The central processing unit (CPU) has several coprocessors at its lower level. These coprocessors, together with the CPU, form a processor topology. Each coprocessor has at least three attribute settings for each type of task to be processed.

[0065] The running status score of each coprocessor in the monitoring module is obtained by the following formula:

[0066] ;

[0067] In the formula: Rate the coprocessor's operating status; The total workload of the coprocessor currently completing the task; This represents the total time taken for the coprocessor to complete the current task. This refers to the total resources of the coprocessor. This represents the amount of resources currently used by the coprocessor. This represents the total number of tasks currently running in the coprocessor. This is the base value for the estimated processing time of the new task; The amount of data for the new task; This is the estimated processing time per unit of data volume based on historical data;

[0068] Among them, the coprocessor running status score A higher value indicates that the corresponding coprocessor is more suitable as a distribution target for the task to be processed. , The above formula is derived from the last task in the subtask list to which the target coprocessor belongs, and the above formula is used to perform calculations on each coprocessor. Value calculation;

[0069] It is important to note that in the above formula, the numerator reflects the remaining processing capacity, while the denominator reflects the processing burden. The score is obtained by the ratio of the two. Under this logic, when the remaining processing capacity is strong (large numerator) and the processing burden is small (small denominator), A high score indicates that the coprocessor is better suited to handle new tasks in the current state and is more suitable to be selected to execute new tasks; conversely, a low score indicates that the coprocessor is not very suitable for handling new tasks.

[0070] After each coprocessor completes its running status score calculation, it matches the coprocessor with the subtask list based on the attributes of the tasks to be processed, so that each coprocessor set corresponds to a subtask list. Each coprocessor in the coprocessor set is based on... Arrange them in descending order to form a coprocessor queue;

[0071] The process involves matching the first coprocessor in the coprocessor queue with the last task in the subtask list, and then matching the last coprocessor in the coprocessor queue with the first task in the subtask list. This continues until all coprocessors in the coprocessor set have completed their task matching process. Each coprocessor then begins processing its matched task. After any coprocessor completes its matched task, the process refreshes and re-matches the tasks in the subtask list with the coprocessors in the coprocessor set based on the above logic.

[0072] Among them, when the coprocessor set processes the subtask list through the above logic, the task to be processed in the middle position of the subtask list is completed last.

[0073] The prevention and control module is used to monitor whether each coprocessor is running faulty. When a coprocessor is found to be running faulty, the module will make a decision and distribute the currently unprocessed and processed tasks of the faulty coprocessor to the waiting module.

[0074] The prevention and control module interacts with the self-test circuits and diagnostic tools configured for each coprocessor in the cloud platform to obtain the coprocessor's operational fault judgment results. When a coprocessor operational fault is detected, the module calculates the fault values ​​for each unprocessed and processing task. Value, based on The calculation results are used to insert each unprocessed and processing task into the corresponding subtask list, and then the waiting module decides and executes the task distribution.

[0075] Among them, unprocessed and processing tasks are being executed. When calculating the value, Including the time the task remained in the fault coprocessor;

[0076] The management module is used to record the historical running status scores of each coprocessor, generate a coprocessor maintenance queue based on the historical running status scores of each coprocessor, and synchronously generate a list of tasks to be processed and processing messages.

[0077] When the management module records the historical running status scores of each coprocessor, it distinguishes and records each coprocessor based on a line graph. In the line graph corresponding to each coprocessor, the horizontal axis represents the time when the coprocessor's running status score is obtained, and the vertical axis represents the running status score obtained at each time. The management module reads the line graph corresponding to each coprocessor and selects the coprocessors corresponding to the line graphs in which the running status score shows a continuous downward trend as priority maintenance targets, and selects the coprocessors corresponding to the line graphs from which the score decreases most frequently as priority maintenance targets, in order to generate a coprocessor maintenance queue for manual online and offline maintenance.

[0078] The pending task list message includes several sub-task list messages. Each sub-task list message contains: the name of the coprocessor used for processing each pending task, the start timestamp of processing each pending task, the end timestamp of processing each pending task, the stage of processing each pending task by the coprocessor, and the real-time resource consumption of the coprocessor.

[0079] The identification module has a storage unit connected via a wireless network, a waiting module and a monitoring module connected via a wireless network, a prevention and control module connected via a wireless network, and a management module connected via a wireless network.

[0080] In this embodiment, the identification module accesses the list of tasks to be processed received in real time by the central processing unit of the cloud platform, identifies the attributes of each task to be processed, and divides the list of tasks to be processed into several sub-task lists based on the attributes of the tasks to be processed. The storage unit synchronously stores the several sub-task lists divided by the identification module. The waiting module receives all the sub-task lists divided by the identification module in real time, obtains the running results of the monitoring module in real time, and makes decisions and executes the distribution of each task to be processed in each sub-task list based on the running results of the monitoring module. Then, the monitoring module monitors the running status of each coprocessor under the central processing unit and obtains the running status score of each coprocessor. The prevention and control module further monitors whether each coprocessor is running faulty. When a coprocessor running fault is detected, the waiting module makes decisions and executes the distribution of the currently unprocessed and processed tasks of the faulty coprocessor. Finally, the management module records the historical running status score of each coprocessor, generates a coprocessor maintenance queue based on the historical running status score of each coprocessor, and synchronously generates a task list processing message.

[0081] In this embodiment, the system can efficiently divide processing tasks, intelligently schedule tasks according to task attributes and priorities to improve processing efficiency, monitor the status of lower-level processors in real time, dynamically match tasks, and ensure the rational use of resources. In the event of a fault, tasks can be quickly transferred to reduce processing interruptions. The system can also record the historical status of processors for timely maintenance, ensuring stable operation and optimizing the overall task processing flow of the cloud platform, thereby improving response speed and reliability.

[0082] Furthermore, for the system in the above embodiments, the following extended designs can be made during the specific implementation stage:

[0083] Dynamic adaptation of task attributes:

[0084] A task attribute self-learning mechanism can be introduced to automatically identify new types of tasks (such as edge computing collaborative tasks and real-time streaming data processing tasks) by analyzing historical task processing data, and generate new sub-task list division rules based on their resource requirements and time consumption characteristics, thus avoiding the lag in manual configuration caused by the addition of new task types.

[0085] Coprocessor scheduling optimization:

[0086] Based on the existing scoring formula, a load balancing correction factor is added. When the task volume of a certain type of subtask list surges, the scoring weight of the corresponding coprocessor set is dynamically adjusted to avoid continuous overload of high-scoring coprocessors. At the same time, coprocessor "capability tags" can be introduced, such as GPU acceleration capabilities for AI tasks and core number adaptability for parallel computing, to make task matching more accurate.

[0087] Intelligent maintenance mechanism:

[0088] To optimize the maintenance queue, a dynamic priority adjustment strategy is implemented. This strategy considers the current load of the coprocessor (e.g., the urgency of processing tasks) and schedules maintenance during off-peak hours while ensuring core tasks are not affected. Simultaneously, maintenance plan suggestions are automatically generated; for example, for coprocessors experiencing continuous performance drops, hardware testing or firmware upgrades are recommended.

[0089] Example 2:

[0090] At the implementation level, based on Example 1, this example refers to... Figure 2 A further detailed description of the gradient scheduling system based on a cloud platform in Example 1 is provided below:

[0091] A gradient scheduling method based on a cloud platform includes:

[0092] Step 1: Access the list of tasks to be processed received by the central processing unit of the cloud platform, identify the attributes of each task and divide them into several sub-task lists, and sort the tasks to be processed in descending order in each sub-task list.

[0093] Step 2: Receive the divided list of subtasks, combine it with the running status monitoring results of the lower-level coprocessors, decide on and execute the distribution of each task;

[0094] Step 3: Monitor the running status of each lower-level coprocessor in real time, calculate the running status score of each coprocessor, and complete the configuration of the task to be processed and the coprocessor based on the running status score;

[0095] Step 4: Monitor the coprocessor's operation. If a fault is detected, re-include unprocessed and processing tasks into the task distribution process.

[0096] Step 5: Record the historical running status score of each coprocessor, generate a maintenance queue, and simultaneously generate task processing messages.

[0097] Step 6: After the coprocessor completes its task, trigger a refresh to re-match the pending tasks with the coprocessor.

[0098] In summary, the methods and systems described in the above embodiments achieve efficient gradient scheduling by intelligently dividing tasks and dynamically matching processing resources during execution. They categorize tasks based on attributes and prioritize them, combining this with real-time processor status scoring for precise matching, thus improving overall processing efficiency. In case of failure, tasks are quickly recovered and reassigned, reducing interruption impact and ensuring stability. Simultaneously, the historical processor status is recorded, prioritizing maintenance of devices with declining performance to extend their lifespan. The use of front-to-back cross-matching logic in task processing balances the load, shortens overall processing time, and generates detailed processing messages, providing data support for optimization. This comprehensively improves the efficiency, stability, and manageability of task processing on the cloud platform.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions will not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A gradient scheduling system based on a cloud platform, characterized in that, include: The identification module is used to access the list of tasks to be processed received in real time by the central processing unit of the cloud platform, identify the attributes of each task to be processed, and divide the list of tasks to be processed into several sub-task lists based on the attributes of the tasks to be processed. The waiting module is used to receive the list of all subtasks obtained by the identification module in real time, obtain the running results of the monitoring module in real time, and make decisions and execute the distribution of each pending task in each subtask list based on the running results of the monitoring module. The monitoring module is used to monitor the operating status of each coprocessor below the central processing unit and obtain the operating status score of each coprocessor. The prevention and control module is used to monitor whether each coprocessor is running faulty. When a coprocessor is found to be running faulty, the module will make a decision and distribute the currently unprocessed and processed tasks of the faulty coprocessor to the waiting module. The management module is used to record the historical running status scores of each coprocessor, generate a coprocessor maintenance queue based on the historical running status scores of each coprocessor, and synchronously generate a list of tasks to be processed and processing messages. The central processing unit has several coprocessors at its lower level. The coprocessors and the central processing unit form a processor topology. The coprocessors have at least three attributes for each type of task to be processed. The operating status score of each coprocessor in the monitoring module is obtained by the following formula: ； In the formula: Rate the coprocessor's operating status; The total workload of the coprocessor currently completing the task; This represents the total time taken for the coprocessor to complete the current task. This refers to the total resources of the coprocessor. This represents the amount of resources currently used by the coprocessor. This represents the total number of tasks currently running in the coprocessor. This is the base value for the estimated processing time of the new task; The amount of data for the new task; This is the estimated processing time per unit of data volume based on historical data; Among them, the coprocessor running status score A higher value indicates that the corresponding coprocessor is more suitable as a distribution target for the task to be processed. , The above formula is derived from the last task in the subtask list to which the target coprocessor belongs, and the above formula is used to perform calculations on each coprocessor. Value calculation; After each coprocessor completes its running status score calculation, it matches the coprocessor with the subtask list based on the attributes of the tasks to be processed, so that each coprocessor set corresponds to a subtask list. Each coprocessor in the coprocessor set is based on... Arrange them in descending order to form a coprocessor queue; The process involves matching the first coprocessor in the coprocessor queue with the last task in the subtask list, and then matching the last coprocessor in the coprocessor queue with the first task in the subtask list. This continues until all coprocessors in the coprocessor set have completed their task matching process. Each coprocessor then begins processing its matched task. After any coprocessor completes its matched task, the process refreshes and re-matches the tasks in the subtask list with the coprocessors in the coprocessor set based on the above logic. In the above logic, when a set of coprocessors processes a list of subtasks, the task in the middle of the list is processed last.

2. The gradient scheduling system based on a cloud platform according to claim 1, characterized in that, The identification module is equipped with a storage unit, which is used to store a list of several sub-tasks divided by the identification module during operation. The identification module operates in real time. When a new list of tasks to be processed is received, it synchronously identifies the attributes of each task in the list. After the attributes of the tasks to be processed are identified, it forwards each task to the corresponding sub-task list based on the attributes of each task to be processed, so as to update the tasks to be processed in the sub-task list. Among them, each task to be processed in the subtask list of each storage unit is based on the corresponding Sort the values ​​in descending order, so that Larger values ​​are listed first. The tasks with smaller values ​​are listed later, and the tasks listed earlier are given priority by the central processing unit. In the formula: The current time; The time it takes for the task to be processed to be separated from the task list to the sub-task list by the identification module; The complexity of the task to be processed.

3. The gradient scheduling system based on a cloud platform according to claim 2, characterized in that, The attributes of the tasks to be processed include: basic data processing tasks, parallel computing tasks, and complex logic and AI tasks. The basic data processing tasks include: measurement calculation, data management, forwarding and migration control; the parallel computing tasks include: function calculation, data analysis; and the complex logic and AI tasks include: massive data processing, algorithm model training and running. The updated pending tasks in the subtask list are based on After sorting in descending order, all tasks are placed after the previously sorted tasks in descending order.

4. A gradient scheduling system based on a cloud platform according to claim 2, characterized in that, The complexity of the task to be processed Calculated using the following formula: ； In the formula: Score based on basic complexity; Score for dynamic complexity; , As weight; Among them, weight , The sum of all numbers is 1 and all of them are positive. < Weight , Initial settings =0.4、 =0.6, The larger the value, the greater the difficulty of the task to be processed and the more computing resources are required.

5. A gradient scheduling system based on a cloud platform according to claim 1, characterized in that, The prevention and control module interacts with the self-testing circuits and diagnostic tools configured for each coprocessor in the cloud platform to obtain the coprocessor operation fault judgment results. When a coprocessor operation fault is obtained, the module calculates the fault values ​​of each unprocessed and processing task. Value, based on The calculation results are used to insert each unprocessed and processing task into the corresponding subtask list, and then the waiting module decides and executes the task distribution. Among them, unprocessed and processing tasks are being executed. When calculating the value, Additional information on the time the task remains in the fault coprocessor.

6. A gradient scheduling system based on a cloud platform according to claim 1, characterized in that, When the management module records the historical running status scores of each coprocessor, it distinguishes and records each coprocessor based on a line graph. In the line graph corresponding to each coprocessor, the horizontal axis represents the time when the coprocessor's running status score is obtained, and the vertical axis represents the running status score obtained at each time. The management module reads the line graph corresponding to each coprocessor and selects the coprocessors corresponding to the line graphs in which the running status score shows a continuous downward trend as priority maintenance targets. It also selects the coprocessors corresponding to the line graphs from which the score decreases most frequently as priority maintenance targets, in order to generate a coprocessor maintenance queue for manual online and offline maintenance. The pending task list message includes several sub-task list messages. Each sub-task list message contains: the name of the coprocessor used for processing each pending task, the start timestamp of processing each pending task, the end timestamp of processing each pending task, the stage of processing each pending task by the coprocessor, and the real-time resource consumption of the coprocessor.

7. A gradient scheduling system based on a cloud platform according to claim 1, characterized in that, The identification module is internally connected to a storage unit via a wireless network. The identification module is also connected to a waiting module and a monitoring module via a wireless network. The monitoring module is connected to a prevention and control module via a wireless network, and the monitoring module is also connected to a management module via a wireless network.

8. A gradient scheduling method based on a cloud platform, wherein the method is an implementation method of a gradient scheduling system based on a cloud platform as described in any one of claims 1-7, characterized in that, include: Step 1: Access the list of tasks to be processed received by the central processing unit of the cloud platform, identify the attributes of each task and divide them into several sub-task lists, and sort the tasks to be processed in descending order in each sub-task list. Step 2: Receive the divided list of subtasks, combine it with the running status monitoring results of the lower-level coprocessors, decide on and execute the distribution of each task; Step 3: Monitor the running status of each lower-level coprocessor in real time, calculate the running status score of each coprocessor, and complete the configuration of the task to be processed and the coprocessor based on the running status score; Step 4: Monitor the coprocessor's operation. If a fault is detected, re-include unprocessed and processing tasks into the task distribution process. Step 5: Record the historical running status score of each coprocessor, generate a maintenance queue, and simultaneously generate task processing messages. Step 6: After the coprocessor completes its task, trigger a refresh to re-match the pending tasks with the coprocessor.

Images