Multi-task distribution and scheduling method for multi-core data stream processor

Abstract

The invention discloses a multi-task allocation and scheduling method for a multi-core data stream processor. The method comprises the following steps: step 1, building a system architecture; 2, task collection and sparseness calculation; step 3, dynamic priority calculation; step 4, implementing a load prediction mechanism; step 5, task scheduling and distribution; and step 6, evaluating and optimizing the system. By introducing a dynamic priority adjustment mechanism, the system can respond to changes of task characteristics and kernel states in real time, the scheduling efficiency is remarkably improved, and the dynamic adaptability ensures that key tasks can preferentially obtain processing resources, so that the processing delay is reduced; through a load prediction mechanism, the load among the cores is effectively balanced, and the situation that some cores are overloaded and other cores are idle is avoided, so that the overall resource utilization rate of the system is improved, dynamic scheduling and load optimization are integrated, the system can keep efficient operation in a complex data stream processing scene, and the throughput and real-time response capability of data processing are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of processor task allocation, and in particular to a multi-task allocation and scheduling method for a multi-core data stream processor. Background Art

[0002] In modern computing, with the surge in data volumes and increasing computing demands, multi-core data stream processors are becoming increasingly popular, particularly in applications such as big data processing, machine learning, and real-time data stream analysis. Multi-core processors significantly improve computing performance through parallel processing, thereby meeting the demand for high throughput and low latency. However, effectively managing and scheduling tasks within these multi-core processors has become a pressing technical challenge. Multi-core processors integrate multiple processing cores on a single physical processor chip. Each core can independently execute instructions, enabling parallel processing. Compared to single-core processors, multi-core processors can achieve higher computing performance and lower power consumption at the same clock frequency.

[0003] However, the existing technology still has major deficiencies, such as:

[0004] Existing scheduling schemes usually rely on static priority allocation and cannot respond to changes in task characteristics and kernel status in real time, resulting in low scheduling efficiency. In multi-core processors, scheduling algorithms fail to effectively balance the load between cores, resulting in resource waste. Summary of the Invention

[0005] The object of the present invention is to provide a multi-task allocation and scheduling method for a multi-core data stream processor to solve the problems raised in the above background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A multi-task allocation and scheduling method for a multi-core data stream processor comprises the following steps:

[0008] Step 1: System architecture construction;

[0009] Step 2: Task collection and sparsity calculation;

[0010] Step 3: Dynamic priority calculation;

[0011] Step 4: Implementation of load prediction mechanism;

[0012] Step 5: Task scheduling and allocation;

[0013] Step 6: System evaluation and optimization.

[0014] Preferably, the step 1: system architecture construction specifically includes:

[0015] Build a scheduling system architecture for a multi-core data stream processor, including the following modules:

[0016] Task management module: responsible for managing all pending tasks and their status;

[0017] Core status monitoring module: real-time monitoring of the load, processing speed, and temperature status of each core;

[0018] Priority calculation module: dynamically calculates priority scores based on kernel status and task characteristics;

[0019] Task allocation module: realizes task allocation and scheduling based on priority scores;

[0020] Historical data recording module: records the historical performance data of each core as the basis for subsequent load prediction.

[0021] Preferably, the step 2: task collection and sparsity calculation specifically includes:

[0022] Collect task information:

[0023] The system receives information about pending tasks from the task management module, including task size, density, and dependencies;

[0024] For each task, the system calculates the sparsity based on its density and the actual task information of the kernel. The formula for setting sparsity is:

[0025]

[0026] Pass the sparsity information to the priority calculation module.

[0027] Preferably, the step three: dynamic priority calculation specifically includes:

[0028] Dynamic priority score calculation:

[0029] The formula for calculating the priority score is set as:

[0030]

[0031] Among them, k1, k2 and k3 are adjustable weight coefficients, which are optimized according to system performance tests;

[0032] Historical performance feedback:

[0033] The real-time monitoring module collects the historical performance data of the kernel and feeds it back to the priority calculation module so that the weight coefficient can be dynamically adjusted when calculating the priority score.

[0034] Preferably, the step 4: implementing the load prediction mechanism specifically includes:

[0035] Data Collection and Analysis:

[0036] Through the historical data recording module, the historical load data of each core is collected and a machine learning model is established for load prediction.

[0037] Real-time load forecasting:

[0038] At regular intervals, the system uses the established model to predict the load of each core within a certain period of time in the future. Based on the current status and historical data, it calculates the expected load and generates a report;

[0039] Prediction result feedback:

[0040] The load prediction results are passed to the task allocation module to determine the load situation of each core and optimize resource allocation.

[0041] Preferably, the step five: task scheduling and allocation specifically includes:

[0042] Task allocation decisions:

[0043] The task allocation module uses the following strategies to schedule tasks based on the real-time priority scores and load forecast results:

[0044] Assign tasks to the highest priority and least loaded cores;

[0045] If the load of a core is higher than the set threshold, its priority is temporarily lowered;

[0046] Task execution and feedback:

[0047] The kernel starts executing tasks, monitors the execution status of tasks in real time, and feeds back the execution data to the task management module and the historical data recording module.

[0048] Preferably, the step six: system evaluation and optimization specifically includes:

[0049] System performance evaluation:

[0050] Regularly evaluate system performance, analyze task completion time, resource utilization, and system response speed, and identify potential bottlenecks;

[0051] Interface and parameter optimization:

[0052] Based on the evaluation results, adjust the weight coefficients k1, k2, and k3 in the priority score calculation formula and the parameters of the load forecasting model to optimize the overall performance.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. By introducing a dynamic priority adjustment mechanism, the system can respond to changes in task characteristics and kernel status in real time, significantly improving scheduling efficiency. This dynamic adaptability ensures that critical tasks receive priority processing resources, thereby reducing processing latency.

[0055] 2. Through the load prediction mechanism, the load between cores is effectively balanced, preventing some cores from being overloaded while others are idle, thereby improving the overall resource utilization of the system. This effect directly solves the problem of resource waste;

[0056] 3. Comprehensive dynamic scheduling and load optimization enable the system to maintain efficient operation in complex data stream processing scenarios, improve data processing throughput and real-time response capabilities, and meet the stringent performance requirements of modern applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0057] Figure 1 It is a schematic diagram of the process of the present invention. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0059] See also Figure 1 , the present invention provides a technical solution:

[0060] A multi-task allocation and scheduling method for a multi-core data stream processor comprises the following steps:

[0061] Step 1: System architecture construction;

[0062] Step 2: Task collection and sparsity calculation;

[0063] Step 3: Dynamic priority calculation;

[0064] Step 4: Implementation of load prediction mechanism;

[0065] Step 5: Task scheduling and allocation;

[0066] Step 6: System evaluation and optimization.

[0067] Step 1: System architecture construction, including:

[0068] Build a scheduling system architecture for a multi-core data stream processor, including the following modules:

[0069] Task management module: responsible for managing all pending tasks and their status;

[0070] Core status monitoring module: real-time monitoring of the load, processing speed, and temperature status of each core;

[0071] Priority calculation module: dynamically calculates priority scores based on kernel status and task characteristics;

[0072] Task allocation module: realizes task allocation and scheduling based on priority scores;

[0073] Historical data recording module: records the historical performance data of each core as the basis for subsequent load prediction.

[0074] The task management module is responsible for receiving all pending tasks submitted from external systems and classifying them by type, size, density, and dependencies. It also records the status of each task in real time, such as pending, in progress, completed, or failed, and provides feedback on the current status of tasks to users or other modules.

[0075] The core status monitoring module continuously monitors the load status of each core, including the number of tasks currently being processed and the processing time required for each task. It regularly collects core performance indicators such as processing speed, temperature, and energy consumption to ensure that the system operates within the design range. If the load of a core exceeds the set threshold, the temperature is too high, or other anomalies are detected, an alarm will be issued in a timely manner.

[0076] The priority calculation module dynamically calculates the priority score of each task based on the core's real-time status, such as load and temperature, and task characteristics, such as sparsity and run time, using a pre-set priority calculation formula. This module continuously optimizes the parameters of the priority calculation model based on historical performance feedback to adapt to the characteristics of different types of tasks and the actual performance of the processor.

[0077] The task allocation module intelligently allocates tasks to cores with lighter loads and higher priorities based on the priority scores provided by the priority calculation module. During task execution, if a core is detected to be overloaded, the system can adjust the task allocation in real time and migrate some tasks to cores with lower loads.

[0078] The historical data recording module records the historical performance data of each core when processing various tasks, such as processing time, success rate, load changes, etc., to provide data support for subsequent load forecasting, regularly generate performance reports, analyze historical data, and identify performance bottlenecks and optimization points.

[0079] The above system architecture, real-time monitoring and dynamic adjustment mechanism enable the system to quickly respond to load changes and adjust task allocation in a timely manner, avoiding task delays and resource waste. Through intelligent task scheduling, it ensures that each core operates within its optimal load range, reducing resource idleness and overload. The continuous monitoring and abnormal status alarm mechanism improves the stability of the system and reduces the risk of failure due to overload or high temperature. The analysis of historical data and the application of load prediction models make the system more efficient when processing complex tasks, improving the overall task completion speed and system performance.

[0080] Step 2: Task collection and sparsity calculation, including:

[0081] Collect task information:

[0082] The system receives information about pending tasks from the task management module, including task size, density, and dependencies;

[0083] Task submission interface:

[0084] Design a standardized task submission interface for the task management module, so that external systems can submit tasks through API or message queue. Each task submission must include the following information:

[0085] Task ID: a unique identifier used to track task status;

[0086] Task type: indicates the nature of the task, such as data processing, computationally intensive tasks, etc.

[0087] Task size: Indicates the computational complexity or data volume of the task, usually expressed in bytes or processing time;

[0088] Density: The data density of the task, which indicates the proportion of useful information in the task;

[0089] Dependency: The dependencies between tasks are represented in a graph or list format to ensure that tasks are executed in the order of dependencies.

[0090] Information Verification:

[0091] In the task management module, an information verification mechanism is set up to perform format and logic verification on the received task information to ensure that all required information is complete and meets expectations. Once the task information passes the verification, the system will add the task to the pending task queue and update the task status to "pending". The receipt status, such as "successfully received task" or "task information incomplete", will be fed back to the task submitter to improve system transparency and user experience.

[0092] For each task, the system calculates the sparsity based on its density and the actual task information of the kernel. The formula for setting sparsity is:

[0093]

[0094] Pass the sparsity information to the priority calculation module.

[0095] Data preparation: For each pending task, obtain the task size, density, and current load information of each core in the system from the task management module;

[0096] In computing: Task size: obtained from task information; Density: provided when the task is submitted, reflecting the proportion of valid information in the task; Kernel load: obtained in real time from the kernel status monitoring module, indicating the workload currently being processed by the kernel;

[0097] Dynamic calculation:

[0098] Since the kernel load can vary, the sparsity calculation should be a dynamic process. The system can set a timer to periodically update the kernel load and recalculate the sparsity of all tasks in the queue accordingly.

[0099] Sparsity information storage:

[0100] The calculated sparsity information and the corresponding task ID are stored in the task management module for subsequent use;

[0101] Information transmission:

[0102] The sparsity information of each task is passed to the priority calculation module for use in subsequent priority assignment.

[0103] Through effective task information collection and sparsity calculation, the system can improve the intelligence level of task scheduling, enhance the overall performance of the system, and reduce the average delay and completion time of tasks. This step enables the system to maintain flexibility and adaptability in a changing task environment through dynamic monitoring and real-time feedback mechanisms, further strengthening the intelligent management of the system. The sparsity calculation of tasks provides an important data basis for subsequent load prediction, making the system more efficient when processing complex tasks, thereby improving the effectiveness and stability of overall task scheduling.

[0104] Step 3: Dynamic priority calculation, including:

[0105] Dynamic priority score calculation:

[0106] The formula for calculating the priority score is set as:

[0107]

[0108] Among them, k1, k2 and k3 are adjustable weight coefficients, which are optimized according to system performance tests;

[0109] In this formula:

[0110] Sparsity: The sparsity calculated in step 2 reflects the effectiveness of the task.

[0111] Kernel running time: indicates the actual time the current kernel uses to process tasks. A long running time may indicate that the kernel is overloaded, affecting its processing capacity.

[0112] Bandwidth parameter: reflects the data transmission capacity between the core and storage or other cores, affecting the execution efficiency of tasks.

[0113] K1, K2, K3: Adjustable weight coefficients used to further optimize the calculation of priority scores.

[0114] Dynamically adjust weight coefficients:

[0115] Real-time feedback mechanism: Based on the core load status and task execution status, the system can adjust the weight coefficient in real time. For example:

[0116] If a core's runtime is significantly higher than other cores, you can lower the value of K2 to reduce the priority impact from that core and encourage the system to assign more tasks to the less loaded cores.

[0117] If the task sparsity is high, you can increase the value of K1 to ensure that these efficient tasks get higher priority;

[0118] Priority score updates:

[0119] Whenever the task state or kernel state changes, the system recalculates the corresponding priority score and updates the task scheduling queue. This process can be set to run periodically, such as once a second, or based on specific events such as task completion, kernel load changes, etc.

[0120] Historical performance feedback:

[0121] The real-time monitoring module collects the historical performance data of the kernel and feeds it back to the priority calculation module so that the weight coefficient can be dynamically adjusted when calculating the priority score.

[0122] Historical performance data collection:

[0123] The real-time monitoring module regularly collects kernel performance data, including but not limited to:

[0124] Time required to complete the task: The time each task takes to execute on each core.

[0125] Failure rate: The failure rate of tasks on different cores.

[0126] Load variation: The load fluctuation of the kernel in different time periods.

[0127] Data analysis and feedback:

[0128] The system analyzes the collected historical performance data to identify how the core behaves under specific conditions.

[0129] For example:

[0130] If a core performs poorly on a particular type of task, the priority or weight factor for that type of task can be adjusted.

[0131] For cores that are often highly loaded, the system can lower their priority scores to encourage tasks to migrate to other cores.

[0132] Dynamically adjust weight coefficients:

[0133] Based on historical performance data, the priority calculation module dynamically adjusts the values of K1, K2, and K3 to ensure that the calculated priority scores are more consistent with the current system's actual conditions. Through this feedback mechanism, the system can continuously optimize itself, thereby improving the accuracy of task scheduling.

[0134] The combination of dynamic priority calculation and historical performance feedback enables the system to achieve self-learning and optimization during execution, thereby continuously improving overall performance. This step enables the system to analyze and adjust in real time through intelligent algorithms, enhancing the adaptability of multi-core processors to complex task environments and improving the intelligence level of scheduling. The dynamic calculation results of priority scores will directly affect task allocation and execution strategies, providing data support for the entire scheduling system, ensuring that the system has good predictability and adjustability while executing efficiently. By optimizing task scheduling, reducing task completion time and improving success rate, it ultimately improves user satisfaction and trust in the system.

[0135] Step 4: Implement the load prediction mechanism, including:

[0136] Data Collection and Analysis:

[0137] Through the historical data recording module, the historical load data of each core is collected and a machine learning model is established for load prediction.

[0138] Historical data collection:

[0139] Using the historical data recording module, the system regularly collects the load data of each core. This data includes:

[0140] Load level: The kernel workload in different time periods, such as CPU usage and memory usage.

[0141] Task execution time: The time required for each task to execute on different cores.

[0142] Task type: The nature and characteristics of the task (such as compute-intensive, IO-intensive, etc.).

[0143] Data preprocessing:

[0144] The collected historical load data is cleaned and preprocessed to ensure its accuracy and validity. The processing steps include:

[0145] Remove incomplete or erroneous data points.

[0146] Normalize the data to facilitate model training and analysis.

[0147] Perform feature engineering as needed to extract features related to load prediction.

[0148] Model selection and construction:

[0149] Select a suitable machine learning model based on the characteristics of the data. Commonly used models include:

[0150] Linear regression: Suitable for situations where the relationship between load and time is relatively linear.

[0151] Time series analysis: Applicable to trend analysis of load data changing over time, such as ARIMA model.

[0152] The selected model is trained using historical data to enable it to identify patterns and trends in load variations.

[0153] Model Evaluation:

[0154] Use methods such as cross-validation to evaluate the performance of the model, ensure the accuracy and reliability of the model in prediction, and select the best model for load forecasting.

[0155] Real-time load forecasting:

[0156] At regular intervals, the system uses the established model to predict the load of each core within a certain period of time in the future. Based on the current status and historical data, it calculates the expected load and generates a report;

[0157] Timing Predictive Execution:

[0158] The system sets a timer to trigger the load prediction process at regular intervals (such as every second). The prediction is made based on the current kernel status and historical data.

[0159] Load forecast calculation:

[0160] Use the established machine learning model to predict the load of each core within a certain period of time in the future. The output of the prediction includes:

[0161] Estimated load of each core: Calculates the load level of each core in the future based on historical data and current status.

[0162] Load Trending: Identify whether the load is increasing or decreasing, and possible load peaks.

[0163] Report Generation:

[0164] The prediction results are organized into reports and provided to the task allocation module to facilitate the subsequent formulation of task allocation strategies.

[0165] Prediction result feedback:

[0166] The load prediction results are passed to the task allocation module to determine the load situation of each core and optimize resource allocation.

[0167] Result delivery:

[0168] The load forecast results are promptly passed to the task allocation module, and the system will dynamically adjust the tasks based on the predicted load conditions.

[0169] Optimize resource allocation:

[0170] Based on the load forecast results, the task allocation module can:

[0171] Identify overloaded cores and reduce their tasks to reduce the load.

[0172] New tasks are assigned first to cores with lower load, ensuring balanced resource utilization.

[0173] Dynamically adjust the scheduling strategy:

[0174] Based on real-time predictions and feedback, the system can dynamically adjust scheduling strategies to improve the efficiency and responsiveness of task execution.

[0175] The implementation of the load prediction mechanism makes the system more intelligent, able to automatically adjust the task scheduling strategy in a dynamic environment, and improve the adaptability of the overall system. Through accurate load prediction, the system can achieve more efficient task scheduling, reduce task waiting time, and improve user experience. The prediction results are not only used for current task allocation, but also provide data support for subsequent system optimization, forming a closed-loop feedback and improving the overall performance of the system. By continuously collecting and analyzing load data, the system can form a self-learning mechanism, gradually improving the accuracy and reliability of load prediction over time.

[0176] Step 5: Task scheduling and allocation, including:

[0177] Task allocation decisions:

[0178] The task allocation module uses the following strategies to schedule tasks based on the real-time priority scores and load forecast results:

[0179] Assign tasks to the highest priority and least loaded cores;

[0180] If the load of a core is higher than the set threshold, its priority is temporarily lowered;

[0181] Task execution and feedback:

[0182] The kernel starts executing tasks, monitors the execution status of tasks in real time, and feeds back the execution data to the task management module and the historical data recording module.

[0183] Task allocation basis:

[0184] The task allocation module schedules tasks based on real-time priority scores and load forecast results. Specific criteria include:

[0185] Real-time priority score: The priority score obtained from the dynamic priority calculation module, reflecting the importance and urgency of the task.

[0186] Load prediction results: The future load estimate obtained from the load prediction module shows the load of each core.

[0187] Task allocation strategy:

[0188] Priority combined with load:

[0189] The system first identifies the current priority score of each core and then makes a decision based on the load prediction results. The task will be assigned to:

[0190] The highest priority and least loaded core: This is the main principle of the priority allocation policy, which aims to ensure that high-priority tasks are processed quickly.

[0191] Load threshold management:

[0192] If the load of a core exceeds a set threshold (for example, reaches 80%), the system will temporarily lower the priority of the core to ensure that the efficiency and quality of task execution are not affected by overload.

[0193] Decision-making process:

[0194] The system sorts all cores and prioritizes cores with high priority scores and low loads for task allocation. If the load on all cores reaches a threshold, the system may consider reallocating currently executing low-priority tasks to prioritize new high-priority tasks.

[0195] Task execution and feedback

[0196] Specific steps: Task execution:

[0197] Once a task is assigned, the kernel begins executing it. The system monitors the execution status of the task in real time to ensure that it proceeds as expected. During execution, the kernel records relevant execution data, including:

[0198] Execution time: The time the task actually runs on the core.

[0199] Resource usage: The CPU, memory and other resources occupied by the kernel during task execution.

[0200] Task status: whether the task was completed successfully, whether an error occurred, etc.

[0201] Data feedback mechanism:

[0202] The kernel feeds real-time execution data back to the task management module and the historical data recording module. This feedback mechanism includes:

[0203] Real-time feedback: During the task execution process, the system can update the task status at any time so that the task management module can adjust the task scheduling strategy in a timely manner.

[0204] Historical records: After execution is completed, the system will summarize the execution data and store it in the historical data recording module for subsequent load forecasting and performance analysis.

[0205] The optimization of task scheduling and allocation will directly improve the performance of the entire multi-core data stream processor, enabling it to handle complex tasks more efficiently and maximize the potential of system resources. By comprehensively considering task priority and load conditions, the system has higher intelligent scheduling capabilities and can better cope with dynamically changing workloads and task requirements. Through the use and feedback of historical data recording modules, the system can continuously learn and optimize task scheduling strategies, forming a virtuous circle and improving long-term performance. By reducing task waiting time and improving execution efficiency, users can get faster responses and higher task success rates, thereby improving the overall user experience and satisfaction.

[0206] Step 6: System evaluation and optimization, including:

[0207] System performance evaluation:

[0208] Regularly evaluate system performance, analyze task completion time, resource utilization, and system response speed, and identify potential bottlenecks;

[0209] Interface and parameter optimization:

[0210] Based on the evaluation results, adjust the weight coefficients k1, k2, and k3 in the priority score calculation formula and the parameters of the load forecasting model to optimize the overall performance.

[0211] Specific steps for system performance evaluation:

[0212] Evaluation indicators are determined:

[0213] Clearly define the key performance indicators (KPIs) to be evaluated, including:

[0214] Task Completion Time: Calculates the total time for each task from start to completion.

[0215] Resource utilization: Monitors the usage of CPU, memory and other resources of each core to assess whether resources are being used effectively.

[0216] System response speed: records the system's response time to new task requests, including delays in task allocation and scheduling.

[0217] Regular Evaluation Plan:

[0218] Establish a schedule for system performance evaluation (e.g. hourly, daily, or weekly) to ensure that the system's performance under different load conditions can be fully evaluated.

[0219] Data Collection and Analysis:

[0220] During the evaluation cycle, data for the above indicators is automatically collected and statistically analyzed. Data can be visualized using charts, trend analysis, and other methods to facilitate the identification of performance issues.

[0221] Identify bottlenecks:

[0222] Through in-depth analysis of the collected data, potential bottlenecks in the system can be identified, such as:

[0223] A core is overloaded, causing tasks to take longer to complete.

[0224] Unbalanced resource utilization causes some cores to be idle while others are overloaded.

[0225] Interface and parameter optimization, specific steps:

[0226] Analysis of evaluation results:

[0227] Based on the performance evaluation results, analyze the main factors affecting system performance and identify specific aspects that need to be optimized.

[0228] Adjust the priority score calculation formula:

[0229] According to the evaluation results, adjust the weight coefficients k1, k2, and k3 in the priority score calculation formula

[0230] For example:

[0231] If you find that the sparsity of a certain type of task has a greater impact on performance, you can increase the weight of K1 to ensure that high-sparseness tasks are given priority.

[0232] If the kernel load is too high and affects task response, the weight of K2 can be appropriately reduced to reduce its impact on priority.

[0233] Load forecast model parameter adjustment:

[0234] Based on the feedback from the system performance evaluation, further optimize the parameters of the load forecasting model. Possible adjustments include:

[0235] Retrain the model to adapt to the new workload characteristics.

[0236] Adjust the model's hyperparameters, such as learning rate and regularization coefficient, to improve the model's prediction accuracy.

[0237] Implement optimization plan:

[0238] Apply the optimized parameters and models to the system and conduct small-scale tests to ensure that they can effectively improve performance.

[0239] Advantages compared with existing technologies

[0240] Continuous performance improvements:

[0241] Through regular performance evaluation and feedback, the system can continuously optimize itself and has better adaptability and flexibility than traditional systems with fixed parameters.

[0242] Identify and resolve issues promptly:

[0243] Timely identification and resolution of system bottlenecks reduces the risk of system failures and performance degradation, and improves system stability and reliability.

[0244] Optimize resource allocation:

[0245] By dynamically adjusting the priority scores and load forecasting models, the system can achieve more reasonable resource allocation and avoid resource waste and shortage.

[0246] Improve user satisfaction:

[0247] By improving task completion time and system response speed, the user experience can be greatly improved and user trust and satisfaction with the system can be enhanced.

[0248] The introduction of the evaluation and optimization mechanism enables the system to automatically adapt to different application scenarios and load changes, forming the ability of self-learning. Through continuous evaluation and optimization, it ensures that the system can maintain its competitiveness in an environment of rapid technological development and improve the overall performance level. In the process of evaluation and optimization, a closed-loop feedback mechanism is formed, allowing the system to continuously improve in self-adjustment and have higher operating efficiency. By identifying and resolving potential problems, the long-term stability of the system is improved, ensuring efficient operation even when processing large-scale tasks and meeting user needs.

[0249] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A multi-task allocation and scheduling method for a multi-core data stream processor, characterized by: The steps include: Step 1: System architecture construction; Step 2: Task collection and sparsity calculation; Step 3: Dynamic priority calculation; Step 4: Implementation of load prediction mechanism; Step 5: Task scheduling and allocation; Step 6: System evaluation and optimization.

2. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 1, characterized in that: The step 1: system architecture construction, specifically includes: Build a scheduling system architecture for a multi-core data stream processor, including the following modules: Task management module: responsible for managing all pending tasks and their status; Core status monitoring module: real-time monitoring of the load, processing speed, and temperature status of each core; Priority calculation module: dynamically calculates priority scores based on kernel status and task characteristics; Task allocation module: realizes task allocation and scheduling based on priority scores; Historical data recording module: records the historical performance data of each core as the basis for subsequent load prediction.

3. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 2, characterized in that: The second step: task collection and sparsity calculation, specifically includes: Collect task information: The system receives information about pending tasks from the task management module, including task size, density, and dependencies; For each task, the system calculates the sparsity based on its density and the actual task information of the kernel. The formula for setting the sparsity is: Pass the sparsity information to the priority calculation module.

4. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 3, characterized in that: The step three: dynamic priority calculation, specifically includes: Dynamic priority score calculation: The formula for calculating the priority score is set as: Among them, k 1、 k2 and k3 are adjustable weight coefficients, which are optimized based on system performance testing; Historical performance feedback: The real-time monitoring module collects the historical performance data of the kernel and feeds it back to the priority calculation module so that the weight coefficient can be dynamically adjusted when calculating the priority score.

5. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 1, characterized in that: Step 4: Implementation of the load prediction mechanism, specifically including: Data Collection and Analysis: Through the historical data recording module, the historical load data of each core is collected and a machine learning model is established for load prediction. Real-time load forecasting: At regular intervals, the system uses the established model to predict the load of each core within a certain period of time in the future. Based on the current status and historical data, it calculates the expected load and generates a report; Prediction result feedback: The load prediction results are passed to the task allocation module to determine the load situation of each core and optimize resource allocation.

6. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 1, characterized in that: The step five: task scheduling and allocation, specifically includes: Task allocation decisions: The task allocation module uses the following strategies to schedule tasks based on the real-time priority scores and load forecast results: Assign tasks to the highest priority and least loaded cores; If the load of a core is higher than the set threshold, its priority is temporarily lowered; Task execution and feedback: The kernel starts executing tasks, monitors the execution status of tasks in real time, and feeds back the execution data to the task management module and the historical data recording module.

7. The multi-task allocation and scheduling method for a multi-core data stream processor according to claim 1, characterized in that: Step 6: System evaluation and optimization, specifically including: System performance evaluation: Regularly evaluate system performance, analyze task completion time, resource utilization, and system response speed, and identify potential bottlenecks; Interface and parameter optimization: According to the evaluation results, adjust the weight coefficient k in the priority score calculation formula 1、 k 2、 k3, and the parameters of the load prediction model to optimize the overall performance.

Images