Task scheduling system of data stream processor

Abstract

A task scheduling system for a data stream processor particularly relates to the technical field of task scheduling, and adopts the scheme that a task priority calculation module calculates the current task dependency closeness according to the data interaction frequency and time urgency of tasks, and a node load calculation module calculates the current task dependency closeness according to the load state of nodes and the task dependency closeness. The data transmission scheduling module calculates a data transmission urgency index according to the data transmission delay, the residual processing capacity of the node and the current load of the node, and the task scheduling decision module outputs a task scheduling instruction for adjusting the task priority, a task execution node and a data transmission path. According to the method, multiple influence factors of tasks, nodes and data transmission are comprehensively considered, efficient scheduling of the data processing tasks of the data flow processor can be achieved, and the overall performance and the resource utilization rate of the system are improved.

Description

Technical Field

[0001] This invention relates to the field of task scheduling technology, specifically a data stream processor task scheduling system. Background Technology

[0002] In modern computing systems, data stream processor task scheduling systems are one of the key technologies for achieving efficient data processing. With the rapid development of technologies such as big data, artificial intelligence, and the Internet of Things, the number and complexity of tasks that data stream processors need to handle are constantly increasing, making task scheduling increasingly difficult.

[0003] Existing task scheduling systems primarily rely on task priority or node load status for scheduling, lacking a comprehensive consideration of multiple factors such as task data dependency strength, time urgency, remaining node processing capacity, and data transmission latency. This single-factor scheduling approach often leads to low system resource utilization, low task execution efficiency, and high data transmission latency. With technological advancements, some task scheduling systems have begun to consider the combined effects of multiple factors; however, these systems often employ complex algorithms and models, resulting in high computational complexity, unsuitability for hardware implementation, and consequently, high scheduling latency, making it difficult to meet the demands of real-time data processing.

[0004] Therefore, a data stream processor task scheduling system is needed that can comprehensively consider multiple influencing factors, achieve global optimization, and has the characteristics of low computational complexity, hardware friendliness, and high real-time performance. Summary of the Invention

[0005] The technical problem solved by this invention is to provide a data stream processor task scheduling system that comprehensively considers multiple factors such as the data dependency intensity of tasks, time urgency, remaining processing capacity of nodes, and data transmission latency, thereby achieving the goal of global optimization.

[0006] To address the above problems, the present invention provides the following technical solution: A data stream processor task scheduling system, the system comprising: The task feature extraction module is used to obtain the current data interaction frequency, historical average data interaction frequency, remaining time of the task, and total execution time of the task. The task priority calculation module is used to calculate the current task dependency tightness based on the current data interaction frequency of the task, the historical average data interaction frequency, the remaining time of the task, and the total execution time of the task. The node status monitoring module is used to obtain the node's idle processing speed, the node's current load processing speed, and the node's current load. The node load calculation module is used to calculate the node load elasticity index based on the node's idle processing speed, the node's current load processing speed, the node's current load, and the task dependency tightness. The data transmission scheduling module is used to calculate the data transmission urgency index based on the data transmission delay, the remaining time of the task, the node load elasticity index, and the current load; The task scheduling decision module is used to provide feedback on the task execution status and, based on the task dependency tightness, the node load elasticity index, and the data transmission urgency index, output task scheduling instructions, including adjusting task priority, task execution node, and data transmission path. The task execution module is used to execute the task scheduling instructions and update the task status in real time.

[0007] Further: The task priority calculation module includes: The data dependency strength calculation module is configured to evaluate the data dependency strength of a task by comparing the current data interaction frequency of the current task with the historical average data interaction frequency of historical task data. The time urgency calculation module is configured to assess the time urgency of a task by comparing the remaining time of the task with the total execution time of the task. The comprehensive urgency calculation module is configured to multiply the data dependency strength by the time urgency to obtain a task dependency tightness that reflects the overall priority of the task.

[0008] Further: The data dependency strength calculation module includes: The current frequency acquisition submodule is configured to monitor the number and time of data interactions in the current task and calculate the current data interaction frequency of the current task; The historical frequency query submodule is configured to query the historical average data interaction frequency of the same type of task from the historical database. The dependency strength calculation submodule is configured to compare the current data interaction frequency with the historical average data interaction frequency to obtain the data dependency strength of the task.

[0009] Further: The time urgency calculation module includes: The remaining time calculation submodule is configured to calculate the remaining time of the task from the task's deadline and the current time; The total execution time acquisition submodule is configured to acquire the total execution time of the task from the task's estimated execution time parameter; The urgency calculation submodule is configured to obtain the time urgency of the task based on the calculation of the remaining time of the task and the total execution time of the task.

[0010] Further: The node load calculation module includes: The performance degradation calculation module is configured to evaluate the degree of performance degradation of a node by comparing the node's idle processing speed with the node's current load processing speed. The urgency correction module is configured to calculate and correct the performance degradation of the node by the task dependency tightness, and obtain the corrected performance degradation coefficient. The load resilience calculation module is configured to add the corrected performance degradation coefficient to the node load baseline value to obtain the node load resilience index, which reflects the remaining processing capacity of the node.

[0011] Further: The performance degradation calculation module includes: The idle speed acquisition submodule is configured to calculate the idle processing speed of the node through a test task when the node is in an idle state. The current speed monitoring submodule is configured to monitor the task processing speed of the node in real time to obtain the current load processing speed of the node; The attenuation calculation submodule is configured to compare the node's idle processing speed with the node's current load processing speed to calculate the degree of performance attenuation of the node.

[0012] Further: The data transmission scheduling module includes: The transmission delay impact calculation module is configured to obtain a value indicating the degree of impact of the data transmission delay on the task deadline by comparing the data transmission delay with the remaining time of the task. The load resilience correction module is configured to calculate and correct the impact value by multiplying the node load resilience index with the impact value to obtain the corrected impact value. The transmission urgency calculation module is configured to add the corrected impact value to the node's current load impact value to obtain the data transmission urgency index, which reflects the overall priority of data transmission.

[0013] Furthermore: the transmission delay impact calculation module includes: The delay monitoring submodule is configured to monitor the data transmission delay in real time; The remaining time acquisition submodule is configured to acquire the remaining time of the task from the task feature extraction module; The impact degree calculation submodule is configured to calculate and compare the data transmission delay with the remaining time of the task to obtain the impact degree value.

[0014] Furthermore: the task scheduling decision module includes: The status feedback module is configured to provide real-time feedback on parameters such as task execution status, node load status, and data transmission latency. The priority adjustment module is configured to adjust the task priority in real time based on the task dependency tightness. The node adjustment module is configured to adjust the task execution nodes in real time based on the node load elasticity index. The path adjustment module is configured to adjust the data transmission path in real time based on the data transmission urgency index.

[0015] The effects of the above solution are as follows: 1. The task priority calculation module of the present invention evaluates the data dependency strength of a task by comparing the data interaction frequency of the current task with the historical average data interaction frequency; evaluates the time urgency of a task by comparing the remaining time of the task with the total execution time of the task; and then combines the data dependency strength and time urgency to obtain the overall urgency of the task, thereby providing a basis for the task priority marking of the task scheduling decision module.

[0016] 2. The node load calculation module of the present invention evaluates the performance degradation of a node by comparing its current load processing speed with its idle processing speed; it corrects the performance degradation of a node based on the overall urgency of the task, and the more urgent the task, the smaller the impact of the node performance degradation; finally, the calculated load elasticity of the node can provide a basis for the task scheduling decision module to select the optimal execution node.

[0017] 3. This invention assesses the impact of data transmission delay on task deadlines by comparing data transmission delay with remaining task time; it corrects the impact of data transmission delay by considering the load elasticity of nodes, with a greater impact from the higher the remaining processing capacity of a node; it also assesses the impact of current node load on data transmission, with a higher node load indicating a higher urgency of data transmission; finally, the corrected impact of data transmission delay is added to the impact of current node load to obtain the urgency of data transmission, thus providing a basis for the task scheduling decision module to select the optimal transmission path.

[0018] Thus, this invention, through a real-time monitoring and feedback mechanism, dynamically adjusts task priority, execution nodes, and transmission paths based on the actual situation during task execution to ensure that tasks are always executed under optimal conditions. Furthermore, the task priority calculation module, node load calculation module, and data transmission scheduling module of this invention have low computational complexity, are suitable for hardware implementation, and have low scheduling latency, which can meet the needs of real-time data processing. Attached Figure Description

[0019] Figure 1 This is a flowchart illustrating the data processing framework for task scheduling in the data stream processor task scheduling system of the present invention. Detailed Implementation

[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings.

[0021] Example 1, please refer to Figure 1 A data stream processor task scheduling system, the system comprising: The task feature extraction module is used to obtain the current data interaction frequency, historical average data interaction frequency, remaining time of the task, and total execution time of the task. The task priority calculation module is used to calculate the current task dependency tightness based on the current data interaction frequency, historical average data interaction frequency, remaining time of the task, and total execution time of the task. The node status monitoring module is used to obtain the node's idle processing speed, the node's current load processing speed, and the node's current load. The node load calculation module is used to calculate the node load elasticity index based on the node's idle processing speed, current load processing speed, current load, and task dependency tightness. The data transmission scheduling module is used to calculate the data transmission urgency index based on data transmission delay, remaining task time, node load elasticity index, and current load. The task scheduling decision module is used to provide feedback on the task execution status and, based on the task dependency tightness, node load elasticity index, and data transmission urgency index, output task scheduling instructions, including adjusting task priority, task execution node, and data transmission path. The task execution module is used to execute task scheduling instructions and update task status in real time; In the above embodiments, the task feature extraction module, as the system input, provides accurate and dynamic raw data for subsequent calculations through real-time monitoring and historical data querying. The task priority calculation module calculates task dependency tightness by multiplying data dependency strength by time urgency, achieving a quantitative evaluation of task priority and providing the primary basis for scheduling decisions. The node load calculation module not only considers the performance degradation of the node itself (the difference between idle speed and current speed) but also introduces task urgency as a correction factor. In urgent task scenarios, the system tends to ignore slight performance degradation of nodes, thus calculating a node load elasticity index that better reflects the real-time scheduling potential. The data transmission scheduling module innovatively combines the impact of data transmission delay on task deadlines with the node's processing elasticity and current load to calculate a data transmission urgency index, ensuring that data transmission scheduling can be optimized in tandem with computation task scheduling. The task scheduling decision module integrates the above three core indices to make global optimization decisions, and its output instructions directly determine resource allocation efficiency. Finally, the task execution module is responsible for implementing decisions and providing feedback on results, driving the system to continuously adapt and optimize.

[0022] It is understandable that this embodiment constructs a complete closed-loop scheduling system from feature extraction and multi-dimensional index calculation to intelligent decision-making and execution. This system no longer relies on a single, static rule, but responds to system state changes in real time and dynamically through a series of low-complexity, hardware-based calculation formulas.

[0023] In this embodiment, the calculation formulas for each module are designed to balance accuracy and real-time performance requirements. The value ranges of all core indices are constrained within a reasonable range (such as around 0-1) to facilitate subsequent unified comparison and decision-making.

[0024] Thus, a data stream processor task scheduling system capable of comprehensively considering task characteristics, node status, and data transmission conditions, and performing real-time, dynamic, and global optimization, has been constructed.

[0025] Example 2: Based on Example 1, the task priority calculation module provided in this embodiment of the invention includes: The data dependency strength calculation module is configured to evaluate the data dependency strength of a task by comparing the current data interaction frequency of the current task with the historical average data interaction frequency of historical task data. The time urgency calculation module is configured to assess the time urgency of a task by comparing the remaining time of the task with the total execution time of the task. The comprehensive urgency calculation module is configured to multiply the data dependency strength by the time urgency to obtain the task dependency tightness that reflects the overall priority of the task.

[0026] The formula for calculating task dependency tightness is as follows: ; In the formula: TDC stands for Task Dependency Tightness; DIF represents the current task data interaction frequency, which is the number of times a task interacts with its upstream task per unit of time. DIFavg represents the historical average data interaction frequency, which is the average number of data interactions between the same type of task during historical execution. The calculation can reflect the strength of the task's current data dependencies; TRT stands for Task Remaining Time, which is the time remaining until the task's deadline from the current time. TET stands for Task Total Execution Time, which is the estimated total time from the start to the completion of the task. The calculation reflects the time urgency of the task; the less time remaining, the closer the value is to 1.

[0027] In the above embodiments, the data dependency strength calculation module effectively identifies "critical" tasks with intensive data interactions and significant impact on upstream and downstream data flows by comparing real-time frequency with historical baselines. The time urgency calculation module ensures that tasks nearing their deadlines are prioritized by the system. The comprehensive urgency calculation module multiplies these two factors rather than simply adding them, meaning that a task will only obtain an extremely high Task Dependency Density (TDC) value when it is both "critical" (strong data dependency) and "urgent" (time-sensitive). This design avoids tasks that are only nearing their deadlines but have low computational load excessively consuming resources, or tasks with frequent data interactions but long cycles occupying high priority for extended periods.

[0028] Understandably, this module achieves dynamic and quantitative evaluation of task priorities through simple calculations. The historical average data interaction frequency DIFavg in the formula can be dynamically obtained from the historical database or updated periodically according to the task type, giving the system a certain degree of learning and adaptability.

[0029] In this embodiment, to prevent the denominator from being zero or to prevent calculation overflow, a minimum value (e.g., 1) can be set for the historical average data interaction frequency DIFavg, and... The ratio is limited to a range (e.g., [0,1]).

[0030] At this point, the system obtains a real-time, quantified priority metric for each task—Task Dependency Density (TDC)—providing core input for subsequent node selection and path scheduling.

[0031] Example 3: Based on Example 2, the node load calculation module provided in this embodiment of the invention includes: The performance degradation calculation module is configured to assess the degree of performance degradation of a node by comparing the node's idle processing speed with the node's current load processing speed. The urgency correction module is configured to calculate and correct the performance degradation of nodes based on task dependency tightness, and obtain the corrected performance degradation coefficient. The load resilience calculation module is configured to add the corrected performance degradation coefficient to the node load baseline value to obtain the node load resilience index, which reflects the remaining processing capacity of the node.

[0032] The formula for calculating the node load elasticity index is as follows: ; In the formula: NLE is the node load elasticity index, NLP is the node idle processing speed (the task processing speed of the node when there is no load), and CLPS is the node current load processing speed (the actual task processing speed of the node under the current load). The calculation can reflect the degree of performance degradation of a node under the current load, and thus directly reflect the performance of the computing node under the current load: a node with a high performance degradation coefficient indicates a heavy load and should avoid being assigned new tasks; a node with a low performance degradation coefficient indicates that there is still remaining processing capacity and more tasks can be assigned. The calculation can correct the node load elasticity based on task urgency: the more urgent the task (the higher the TDC), the lower the weight of the node load elasticity index NLE. Adding 0.5 as a baseline value for node load ensures that the node load elasticity index (NLE) is between 0 and 1, and that 0.5 indicates that the node is under moderate load with moderate performance degradation.

[0033] In the above embodiments, the performance degradation calculation module objectively measures the impact of a node's current load on its maximum performance. The urgency correction module introduces task context (Task Dependency Tightness, TDC), making the node's "availability" assessment no longer static, but related to the characteristics of the task to be executed. The load resilience calculation module adds a baseline value and uses (1-TDC) for inverse weighting, resulting in a clear and intuitive node load resilience index (NLE). The higher the index, the stronger the remaining processing capacity and the better the resilience of the node when executing the current urgent task.

[0034] Understandably, this design leads the system to prioritize nodes with relatively low absolute performance (small difference between current CLPS and NLP) when allocating high-urgency tasks (high task dependency density, TDC), thus avoiding congestion caused by overwhelming a few high-performance nodes with all urgent tasks. For low-urgency tasks (low task dependency density, TDC), the system prioritizes the degree of absolute performance degradation of nodes, tending to allocate them to nodes with lighter loads.

[0035] In this embodiment, the node idle processing speed (NLP) can be obtained and updated through periodic benchmark tests, while the node current load processing speed (CLPS) is provided in real time by the node status monitoring module.

[0036] At this point, the system obtains the node load elasticity index (NLE) for each "task-node", providing a direct basis for the task scheduling decision module to select the optimal execution node.

[0037] Example 4: Based on Example 3, the data transmission scheduling module provided in this embodiment of the invention includes: The transmission delay impact calculation module is configured to obtain the degree of impact of the data transmission delay on the task deadline by comparing the data transmission delay with the remaining task time; The load resilience correction module is configured to calculate and correct the impact value by multiplying the node load resilience index with the impact value, and obtain the corrected impact value. The transmission urgency calculation module is configured to add the corrected impact value to the node's current load impact value to obtain a data transmission urgency index that reflects the overall priority of data transmission.

[0038] The formula for calculating the data transmission urgency index is as follows: ; In the formula: DTC stands for Data Transmission Urgency Index, and DT stands for Data Transmission Delay, which is the time it takes for data to travel from the source node to the destination node. The calculation reflects the impact of data transmission delay on task deadlines; a higher ratio indicates a greater impact of data transmission delay on task deadlines. The calculation of multiplying by the node load resilience index NLE can correct the urgency of data transmission based on the node state: the higher the node load resilience (the higher the node load resilience index NLE), the higher the weight of data transmission urgency; NL represents the current load of the node, which is the current length of the task queue of the data stream node. Plus The calculation can reflect the impact of the node's current load on data transmission.

[0039] In the above embodiments, the transmission delay impact calculation module assesses the urgency of transmission behavior from a time dimension. The load elasticity correction module innovatively incorporates the processing capacity of computing nodes (Node Load Elasticity Index, NLE) into the transmission scheduling considerations, realizing the linkage assessment of computing power and capacity. The node load impact supplementary submodule considers the immediate receiving capability of the target node. Finally, the transmission urgency calculation module integrates the three factors to obtain the Data Transmission Urgency Index (DTC), which comprehensively reflects the urgency of data transmission required to ensure the successful completion of a specific task on the target node within its remaining time.

[0040] Understandably, this module enables the system to intelligently differentiate data transmission requests. For example, in the case of a high-urgency task (small remaining time to response, TRT) and a target node with high elasticity (high node load elasticity index, NLE), even if the absolute value of the transmission delay (data transmission delay, DT) is not large, its data transmission urgency index (DTC) may be very high, thus triggering a high-speed path. Conversely, for a low-urgency task or a target node that is already fully loaded, a normal or low-speed path may be allocated to save resources.

[0041] In this embodiment, the data transmission delay DT can be estimated based on network status, data volume, and historical transmission records.

[0042] At this point, the system obtains a quantitative indicator of the urgency of each data transmission request—the Data Transmission Urgency Index (DTC)—providing a core basis for data transmission path selection in the task scheduling decision module.

[0043] Example 5: Based on Example 4, the task scheduling decision module provided in this embodiment of the invention has the following specific workflow: Status feedback module: Provides real-time feedback on parameters such as task execution status, node load status, and data transmission latency.

[0044] Dynamic task priority marking: The priority adjustment module dynamically classifies tasks based on the real-time calculated task dependency density (TDC). The task priority register is a 3-bit register; different bit combinations represent different priorities. High priority: 100; Medium priority: 010; Normal priority: 001.

[0045] Understandably, when the Task Dependency Density (TDC) is close to 1, it indicates that the task is very urgent and should be marked as high priority. This allows it to directly enter the fast scheduling channel and be immediately allocated computing resources, thus preempting the computing resources of currently executing ordinary priority tasks. This is suitable for latency-sensitive real-time tasks. When the Task Dependency Density (TDC) is between 1 and 0, such as 0.5, it indicates that the task is moderately urgent and should be marked as medium priority. This allows it to enter the ordinary scheduling channel and be allocated computing resources sequentially. It cannot preempt currently executing tasks, but can be inserted into the execution intervals. This is suitable for most scientific computing tasks. When the Task Dependency Density (TDC) is close to 0, it indicates that the task is not urgent and should be marked as an ordinary priority task. This allows it to enter the delayed scheduling channel, waiting for idle computing resources. It will only be executed when there are no high-priority or medium-priority tasks. This is suitable for latency-insensitive background tasks.

[0046] Intelligent node selection: When a task needs to be assigned or migrated to a node, the node adjustment module calculates the Node Load Elasticity Index (NLE) of all candidate nodes. Once a task enters the scheduling queue, it automatically calculates the NLE of all available nodes, sorts the nodes according to their NLE values, prioritizing nodes with higher NLE values, and assigns the task to the node with the highest NLE value. For example: When the node load elasticity index (NLE) is 0.8-1.0, it indicates that the load is relatively light, the performance degradation is small, and tasks can be prioritized and multiple tasks can be assigned. When the node load elasticity index (NLE) is 0.5-0.8, it indicates a medium load with moderate performance degradation, allowing for normal task allocation and preventing overload. When the Node Load Elasticity Index (NLE) is between 0.0 and 0.5, it indicates a heavy load and significant performance degradation. Tasks should be allocated cautiously, and only high-priority tasks should be assigned.

[0047] Data transmission path optimization: The path adjustment module selects a path for each data transmission request to be scheduled based on its Data Transmission Urgency Index (DTC). When a data transmission request is generated, the DTC is automatically calculated, and the corresponding transmission path is selected based on the DTC. For example: DTC>0.7: Select a high-speed transmission path to ensure data reaches the target node quickly; 0.3≤DTC≤0.7: Select a normal transmission path to balance transmission speed and resource consumption; DTC<0.3: Select a low-speed transmission path to save high-speed transmission resources.

[0048] After data transmission is completed, parameters such as data transmission delay are updated in real time.

[0049] Finally, execution begins according to the selected task priority, execution node, and transmission path, and the task parameters are updated in real time until the task is completed.

[0050] In the above embodiments, the task scheduling decision module does not simply apply thresholds, but rather makes decisions collaboratively based on three dimensions: TDC, NLE, and DTC. For example, a task with a high TDC value will trigger high NLE requirements on its execution node, and may also increase the DTC value of its related data transmission, thereby achieving a comprehensive bias in resource allocation. Through a built-in feedback loop, the module can fine-tune the above threshold parameters based on the effects after instruction execution (such as task completion time and node load changes), achieving incremental optimization.

[0051] Understandably, this module is the core of the system's intelligence. It transforms the quantitative indicators calculated by the preceding modules into specific, executable, and interconnected scheduling actions, thus achieving a leap from "perception" to "decision-making."

[0052] In this embodiment, all priority markers, node selection logic, and path selection strategies can be dynamically adjusted through a configuration table to adapt to different application scenarios and data processing requirements for task scheduling.

[0053] At this point, the system can output clear and coordinated task scheduling instructions to guide the entire data stream processing system to operate efficiently and stably.

[0054] Example 6: Based on Example 5, the task execution module provided in this embodiment of the invention is used to execute task scheduling instructions and update task status in real time.

[0055] In the above embodiments, the task execution module is the "executor" of the scheduling system, translating intelligent decisions into actual actions. Its key role lies in realizing a closed loop of "scheduling-execution-feedback". By providing real-time feedback on the actual execution status, it ensures that the upstream feature extraction and status monitoring data remain fresh and accurate, thereby guaranteeing the adaptability and effectiveness of the entire scheduling system.

[0056] Understandably, the reliability and efficiency of this module directly determine whether the scheduling strategy can be implemented. Its design focuses on the atomicity of instruction execution, the safety of state transitions, and the timeliness of feedback information.

[0057] In this embodiment, the task execution module is typically deployed on each computing node in the form of a lightweight agent or daemon process, and interacts with the central scheduler through an efficient communication protocol.

[0058] Thus, the system's scheduling strategy was fully implemented, and feedback was continuously collected during execution, providing a data foundation for continuous optimization, thereby truly realizing efficient and intelligent scheduling of data stream processor tasks.

[0059] While the present invention has been disclosed above, it is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A data stream processor task scheduling system, characterized in that, The system includes: The task feature extraction module is used to obtain the current data interaction frequency, historical average data interaction frequency, remaining time of the task, and total execution time of the task. The task priority calculation module is used to calculate the current task dependency tightness based on the current data interaction frequency of the task, the historical average data interaction frequency, the remaining time of the task, and the total execution time of the task. The node status monitoring module is used to obtain the node's idle processing speed, the node's current load processing speed, and the node's current load. The node load calculation module is used to calculate the node load elasticity index based on the node's idle processing speed, the node's current load processing speed, the node's current load, and the task dependency tightness. The data transmission scheduling module is used to calculate the data transmission urgency index based on the data transmission delay, the remaining time of the task, the node load elasticity index, and the current load; The task scheduling decision module is used to provide feedback on the task execution status and, based on the task dependency tightness, the node load elasticity index, and the data transmission urgency index, output task scheduling instructions, including adjusting task priority, task execution node, and data transmission path. The task execution module is used to execute the task scheduling instructions and update the task status in real time.

2. The data stream processor task scheduling system according to claim 1, characterized in that, The task priority calculation module includes: The data dependency strength calculation module is configured to evaluate the data dependency strength of a task by comparing the current data interaction frequency of the current task with the historical average data interaction frequency of historical task data. The time urgency calculation module is configured to assess the time urgency of a task by comparing the remaining time of the task with the total execution time of the task. The comprehensive urgency calculation module is configured to multiply the data dependency strength by the time urgency to obtain a task dependency tightness that reflects the overall priority of the task.

3. The data stream processor task scheduling system according to claim 2, characterized in that, The data dependency strength calculation module includes: The current frequency acquisition submodule is configured to monitor the number and time of data interactions in the current task and calculate the current data interaction frequency of the current task; The historical frequency query submodule is configured to query the historical average data interaction frequency of the same type of task from the historical database. The dependency strength calculation submodule is configured to compare the current data interaction frequency with the historical average data interaction frequency to obtain the data dependency strength of the task.

4. The data stream processor task scheduling system according to claim 2, characterized in that, The time urgency calculation module includes: The remaining time calculation submodule is configured to calculate the remaining time of the task from the task's deadline and the current time; The total execution time acquisition submodule is configured to acquire the total execution time of the task from the task's estimated execution time parameter; The urgency calculation submodule is configured to obtain the time urgency of the task based on the calculation of the remaining time of the task and the total execution time of the task.

5. A data stream processor task scheduling system according to claim 1, characterized in that, The node load calculation module includes: The performance degradation calculation module is configured to evaluate the degree of performance degradation of a node by comparing the node's idle processing speed with the node's current load processing speed. The urgency correction module is configured to calculate and correct the performance degradation of the node by the task dependency tightness, and obtain the corrected performance degradation coefficient. The load resilience calculation module is configured to add the corrected performance degradation coefficient to the node load baseline value to obtain the node load resilience index, which reflects the remaining processing capacity of the node.

6. A data stream processor task scheduling system according to claim 5, characterized in that, The performance degradation calculation module includes: The idle speed acquisition submodule is configured to calculate the idle processing speed of the node through a test task when the node is in an idle state. The current speed monitoring submodule is configured to monitor the task processing speed of the node in real time to obtain the current load processing speed of the node; The attenuation calculation submodule is configured to compare the node's idle processing speed with the node's current load processing speed to calculate the degree of performance attenuation of the node.

7. A data stream processor task scheduling system according to claim 1, characterized in that, The data transmission scheduling module includes: The transmission delay impact calculation module is configured to obtain a value indicating the degree of impact of the data transmission delay on the task deadline by comparing the data transmission delay with the remaining time of the task. The load resilience correction module is configured to calculate and correct the impact value by multiplying the node load resilience index with the impact value to obtain the corrected impact value. The transmission urgency calculation module is configured to add the corrected impact value to the node's current load impact value to obtain the data transmission urgency index, which reflects the overall priority of data transmission.

8. A data stream processor task scheduling system according to claim 7, characterized in that, The transmission delay impact calculation module includes: The delay monitoring submodule is configured to monitor the data transmission delay in real time; The remaining time acquisition submodule is configured to acquire the remaining time of the task from the task feature extraction module; The impact degree calculation submodule is configured to calculate and compare the data transmission delay with the remaining time of the task to obtain the impact degree value.

9. A data stream processor task scheduling system according to claim 1, characterized in that, The task scheduling decision module includes: The status feedback module is configured to provide real-time feedback on parameters such as task execution status, node load status, and data transmission latency. The priority adjustment module is configured to adjust the task priority in real time based on the task dependency tightness. The node adjustment module is configured to adjust the task execution nodes in real time based on the node load elasticity index. The path adjustment module is configured to adjust the data transmission path in real time based on the data transmission urgency index.

Images