Parallel Task Graph Incremental Scheduling Method Based on OCaml Functional Language

By constructing a type identifier mapping and incremental scheduling mechanism, the shortcomings of OCaml parallel programming methods in task graph representation and incremental scheduling are solved, and efficient and maintainable parallel task scheduling is achieved.

CN121070543BActive Publication Date: 2026-11-10NANJING UNIV OF AERONAUTICS & ASTRONAUTICS
View PDF 1 Cites 0 Cited by

Patent Information

Application Number
CN202511151919.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-08-18
Publication Date
2026-11-10
Estimated Expiration
2045-08-18

AI Technical Summary

Technical Problem

Existing OCaml parallel programming methods, such as Domainslib, lack flexibility in task encapsulation, struggle to express complex task graphs, and lack incremental scheduling support, resulting in difficult program maintenance and low execution efficiency.

Method used

By registering initial values ​​for task types and their type-to-string functions, a type identifier mapping is constructed, dependencies between tasks are established, a structured task graph is built, and initial scheduling is performed on multi-core computing resources. Incremental task scheduling is supported, and only the affected parts are recomputed.

Benefits of technology

It improves the readability and maintainability of the task graph, significantly reduces the overhead of redundant computation, and improves the execution efficiency of large-scale task graphs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121070543B_ABST
    Figure CN121070543B_ABST
Patent Text Reader

Abstract

This invention discloses a parallel task graph incremental scheduling method based on the OCaml functional language, belonging to the field of data processing. This method constructs a type identifier mapping, creates tasks using the type identifier mapping, establishes dependencies between tasks, derives a task graph, performs topological initialization on the task graph, achieves the first complete scheduling execution of the task graph, and after detecting changes in the task graph, identifies and tracks the propagation paths of affected tasks, recompiling only necessary parts to achieve efficient incremental task scheduling. This invention features simplicity, controllability, and flexible scalability in parallel program construction and scheduling. Users can quickly construct task dependencies according to specific parallel scenarios and flexibly access heterogeneous resources or implement personalized optimization strategies, making it suitable for various parallel computing tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a parallel task graph incremental scheduling method based on the OCaml functional language. Background Technology

[0002] In fields such as electronic design automation (EDA), large-scale machine learning, and scientific computing, there is an increasing pressure to improve performance on multi-core and heterogeneous systems, prompting the adoption of advanced concurrent programming frameworks. With the continuous development of hardware, a key challenge in design languages ​​and system development is how to build efficient, maintainable, and type-safe concurrent programs.

[0003] To support this programming paradigm, the functional language OCaml introduced several task scheduling libraries on top of its multi-core runtime, among which Domainslib is a typical example.

[0004] Domainslib is built on the OCaml multi-core runtime system and employs a thread pool and work-stealing scheduling mechanism. Users can create asynchronous tasks using `Task.async` and express dependencies between tasks using `Task.await`. Data transfer between tasks is encapsulated through promise types, thus leveraging OCaml's strong typing system to achieve a type-safe parallel programming interface to some extent. This mechanism performs well in scenarios with low parallelism, simple structures, and clear dependencies, such as image batch processing and static numerical computation, achieving practical results.

[0005] However, as the scale and complexity of parallel computing tasks increase, Domainslib's design has gradually revealed significant shortcomings in the following two aspects:

[0006] 1. Weak structural expressiveness and difficult program maintenance. Domainslib's dependencies are explicitly expressed in each task call through Task.await, but this approach is only suitable for linear, flat dependency scenarios. For complex task graphs with control structures such as branches, loops, and task nesting, users can only achieve this by manually combining multiple async / await calls. The task structure is entirely dependent on the program flow, lacking abstraction and layered representation. This approach is detrimental to program readability and maintainability, and also makes it difficult for the scheduler to analyze the task graph structure, limiting the implementation of advanced features such as static optimization and automatic verification.

[0007] 2. Lacks support for incremental scheduling, resulting in low execution efficiency. In many real-world scenarios, input data changes frequently, but only affects a specific sub-region in the task graph. For example, in EDA time-series analysis, design fine-tuning only requires reanalyzing a portion of the path. Domainslib lacks the ability to manage task states, cannot track the propagation chain of task dependencies, and cannot identify affected nodes. Therefore, even if only a very small number of tasks change, the entire task graph must be executed from scratch. This redundant computation leads to performance waste, especially noticeable in systems with large or frequently updated task graphs. Summary of the Invention

[0008] This invention provides a parallel task graph incremental scheduling method based on the OCaml functional language to solve the problems of insufficient task encapsulation flexibility and lack of incremental execution support in existing OCaml parallel programming methods such as Domainslib.

[0009] This invention provides a parallel task graph incremental scheduling method based on the OCaml functional language, comprising the following steps:

[0010] S1, register the initial value of the task type and its type-to-string function, and construct the type identifier mapping;

[0011] S2, use the type identifier mapping to create various tasks, establish dependencies between tasks, and obtain a task graph;

[0012] S3, perform topology initialization on the task graph, identify schedulable tasks and execute them in parallel on multi-core computing resources, and achieve the first complete scheduling and execution of the task graph;

[0013] S4, after detecting changes in the task graph, identifies and tracks the propagation path of the affected tasks, recomputes only the necessary parts, and performs incremental task scheduling.

[0014] Optionally, in one embodiment of the present invention, step S1 specifically includes:

[0015] By using an initial value for the task type and the corresponding string conversion function as the characteristics of the type, the type characteristics are registered in the type mapping dictionary to obtain a unique identifier for the type. The type identification mapping process is implemented through the function Dict.add.

[0016] Optionally, in one embodiment of the present invention, step S2 specifically includes:

[0017] The task graph object graph is registered and initialized by Graph.registerGraph. The task function of different task types is uniformly encapsulated by the type identifier of the task type. Task nodes are added to the task graph by Graph.registerTask.

[0018] The task graph with structured control flow is constructed by using Graph.registerGroup to achieve unified encapsulation of task groups, Graph.registerBranch to implement a conditional branching structure based on Boolean decision, and Graph.precede to explicitly establish dependencies between tasks.

[0019] Optionally, in one embodiment of the present invention, step S3 specifically includes:

[0020] Perform a topological traversal of the task graph, calculate the predecessor dependency count for each task, and mark all tasks with a dependency count of zero as initially schedulable tasks and add them to the scheduling queue.

[0021] The scheduler is started on the multi-core resources specified by the user. It uses the parallel domain mechanism in the OCaml multi-core runtime system to allocate tasks from the scheduling queue to threads for parallel execution. During the execution, after each task is completed, the dependency count of its successor tasks is decremented. If all the predecessor tasks of a certain successor task have been completed, it is marked as a schedulable task and added to the scheduling queue until all tasks are completed.

[0022] Optionally, in one embodiment of the present invention, step S4 specifically includes:

[0023] When a task in the task graph changes, the initial affected task's state is first reset to an incomplete state, and an incremental scheduling process is initiated. During the scheduling process, if a task is in an incomplete state, it is re-executed. If the task meets the incremental state propagation condition, the state of all its successor tasks will also be set to an incomplete state and enter the subsequent scheduling queue; otherwise, the execution of the task is skipped.

[0024] The parallel task graph incremental scheduling method based on the OCaml functional language in this invention has the following beneficial effects:

[0025] 1. This invention enhances structural expressiveness and system maintainability. By introducing a type registration mechanism and existence encapsulation technology, this invention achieves unified modeling of heterogeneous tasks and natively supports the construction of structured control flow (such as branches, loops, and task groups) in the task graph. The constructed task graph maintains a high degree of structural consistency with the source program (structural isomorphism), significantly enhancing the readability, maintainability, and verifiability of the task graph, and providing strong support for the organization and management of complex parallel logic.

[0026] 2. This invention achieves efficient incremental scheduling execution. It supports rescheduling and recompiling only the affected subgraph range after an input disturbance occurs, avoiding repeated execution of the entire task graph. This mechanism effectively reduces the overhead of repeated computation and can achieve significant performance improvements in applications that rely on local changes. Experiments demonstrate that in large-scale task graphs, the incremental scheduling mechanism of this invention provides a considerable speedup compared to the traditional full scheduling method.

[0027] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0028] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the following description of the embodiments taken in conjunction with the accompanying drawings, wherein:

[0029] Figure 1 This is a flowchart illustrating a parallel task graph incremental scheduling method based on the OCaml functional language according to an embodiment of the present invention. Detailed Implementation

[0030] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0031] Figure 1 This is a flowchart illustrating a parallel task graph incremental scheduling method based on the OCaml functional language according to an embodiment of the present invention.

[0032] like Figure 1 As shown, this parallel task graph incremental scheduling method based on the OCaml functional language includes the following steps:

[0033] S1, register the initial value of the task type and its type-to-string function, and construct the type identifier mapping.

[0034] In an embodiment of the present invention, step S1 is specifically implemented as follows: by passing in an initial value of the target type and the string conversion function corresponding to that type as the characteristic of the type, the type characteristic is registered in the type mapping dictionary to obtain a unique identifier for the type. This process is defined through the auxiliary function Dict.add, and the specific definition is as follows:

[0035] val Dict.add:value:a->tostring:(a->string)->a key

[0036] Here, `value` represents an initial value for the target type, `tostring` represents the string conversion function corresponding to that type, and `key` is the output type identifier. When the `Dict.add` function is called correctly, the system will use the combination of `value` and `tostring` as the characteristic information of the type, register the type in the global type mapping dictionary, and generate a unique type identifier for it. This identifier is not only used to identify the type but also supports subsequent unified encapsulation and scheduling of heterogeneous tasks.

[0037] S2 uses type identifier mapping to create various tasks, establishes dependencies between tasks, and obtains a task graph.

[0038] In an embodiment of the present invention, step S2 is specifically implemented as follows: First, a task graph object `graph` is registered and initialized using `Graph.registerGraph`. Then, using the type identifier obtained in step S1, different types of task functions are uniformly encapsulated, and task nodes are added to the task graph using `Graph.registerTask`, where `a key` represents the task type identifier, `unit->a` represents the task function, and the generated task object `task` is returned.

[0039] Next, we implement a unified encapsulation of task groups using `Graph.registerGroup` and a conditional branching structure based on Boolean decisions using `Graph.registerBranch`, further enhancing the expressive power of the graph. Finally, we explicitly establish dependencies between tasks using `Graph.precede`, thereby constructing a task graph with structured control flow.

[0040] Among them, the encapsulation structure of the task node is based on the principle of existence type encapsulation. It can hide the specific type while retaining the type information, so that heterogeneous tasks can be managed in a unified structure. In the scheduling and execution phases, the original type can be safely restored through the type identifier. It is one of the key mechanisms to achieve type-safe heterogeneous task scheduling.

[0041] Specifically, in step S2, firstly, an empty task graph is created. This process is defined using the function Graph.reigsterGraph, as follows:

[0042] val Graph.registerGraph:unit->graph

[0043] Here, "unit" represents no parameters, and "graph" represents an empty task graph output.

[0044] Next, tasks are created using the type identifier mapping obtained in step S1. The types of tasks that can be defined are: Execution Task (Task), Task Group, and Conditional Task (Branch). They are implemented using registerTask, registerGroup, and registerBranch, as defined below:

[0045] val Graph.registerTask:graph:graph->key:(a key)->f:(unit->a)->task;

[0046] val Graph.registerGroup:graph:graph->tasks:(task list)->task;

[0047] val Graph.registerBranch:graph:graph->yes:(task)->no:(task)->task;

[0048] The `Graph.registerTask` function registers a task of type `a` in the task graph. Here, `key` is a unique identifier for the type, `f` is the function body called during task execution, and `task` is the registered task node. `Graph.registerGroup` and `Graph.registerBranch` provide structured construction methods: `Graph.registerGroup` aggregates multiple tasks into a task group to achieve structured parallel execution; `Graph.registerBranch` provides a way to register conditional branch tasks, selecting between `yes` and `no` task branches based on the boolean return value of the conditional function. These task structure encapsulations are all based on the principle of existential type encapsulation, hiding the specific type while preserving type information, enabling the management of heterogeneous tasks within a unified structure.

[0049] Next, dependencies are established for tasks already registered in the task graph to ensure their execution order in parallel scheduling. Each task has a list storing predecessor and successor tasks, denoted as `deps` and `sucs`, used to record dependencies between tasks. This process is implemented through the helper function `Graph.precedence`, defined as follows:

[0050] val Graph.precede:dep:task->suc:task->unit

[0051] Here, `dep` represents the predecessor task, and `suc` represents the successor task. This method of establishing dependencies between tasks not only solves the problem of coupling in parallel task construction but also allows for more flexible implementation of complex dependencies.

[0052] Finally, a task graph with complex dependencies between different tasks was obtained.

[0053] S3 performs topology initialization on the task graph, identifies schedulable tasks, and executes them in parallel on multi-core computing resources, achieving the first complete scheduling and execution of the task graph.

[0054] In an embodiment of the present invention, step S3 is implemented as follows: First, a topological traversal is performed on the task graph constructed in step S2, and the predecessor dependency count of each task is calculated. All tasks with a dependency count of zero are marked as initially schedulable tasks and added to the scheduling queue. Then, the scheduler is started on the user-specified multi-core resources, and the parallel domain mechanism in the OCaml multi-core runtime system is used to allocate tasks from the scheduling queue to threads for parallel execution. During execution, after each task is completed, the dependency count of its successor tasks is decremented. If all predecessor tasks of a certain successor task have been completed, it is marked as a schedulable task and added to the scheduling queue, until all tasks are completed.

[0055] Specifically, a topology analysis is performed on the task graph constructed in step S2 to count the number of incomplete predecessor tasks (i.e., dependency count) for each task. All tasks with a dependency count of zero are identified as initially schedulable tasks and added to the task scheduling queue to prepare for parallel execution. Then, the scheduler is started on the user-specified multi-core resources, allocating tasks from the scheduling queue to threads. During execution, the dependency count of each task's successor is decremented after each task is completed. If all predecessor tasks of a successor task are completed, it is marked as a schedulable task and added to the scheduling queue, until all tasks are completed. This process is implemented by the function `Pool.run`, specifically defined as follows:

[0056] val Pool.run:pool:pool->graph:graph->unit

[0057] Where pool represents the multi-core resources specified by the user, and graph is the task graph output in step S2.

[0058] S4, after detecting changes in the task graph, identifies and tracks the propagation path of the affected tasks, and only recomputes the necessary parts to achieve efficient incremental task scheduling.

[0059] In an embodiment of the present invention, step S4 is specifically implemented as follows: when a task in the task graph changes, the initial state of the affected task is first reset to Incoming, and an incremental scheduling process is initiated. During the scheduling process, if a task is in the Incoming state, it is re-executed; if the task meets the incremental state propagation condition, the state of all its successor tasks will also be set to Incoming and enter the subsequent scheduling queue; otherwise, the execution of the task is skipped to reduce unnecessary overhead. To further optimize execution efficiency, the present invention introduces a state condition checking mechanism to precisely control the state propagation path, avoid invalid task scheduling, and thus minimize computational redundancy. This mechanism is particularly effective when facing local changes in input, significantly improving overall scheduling performance. The present invention provides the interface Pool.runincr to support the above incremental scheduling process.

[0060] Specifically, when a change in the task graph detected in step S3 is detected, incremental scheduling mode is initiated. Affected tasks are marked as Incoming and executed; otherwise, execution is skipped. This process is implemented by the function Pool.runicr, defined as follows:

[0061] val Pool.runincr:pool:pool->graph:graph->unit

[0062] Here, `pool` represents the user-specified multi-core resources, and `graph` is the task graph completed in step S3. When a change is detected in a task (such as task t1) within the task graph constructed in step S3, the system marks its state as "Incoming," indicating that the task needs to be re-executed. Once t1 is marked as "Incoming," the scheduler will first re-execute t1. If, after execution, t1 meets the incremental state propagation condition, all subsequent tasks dependent on t1 will also be marked as "Incoming" and continue to be scheduled for execution; otherwise, its subsequent tasks will remain unaffected, avoiding redundant computation. Throughout this process, the system introduces a refined state judgment mechanism to ensure that only truly affected task chains are rescheduled, thereby minimizing execution costs and improving scheduling efficiency in scenarios with localized disturbances.

[0063] This invention proposes an incremental scheduling method for parallel task graphs based on the OCaml functional language. The method constructs a type identifier mapping, creates tasks using the type identifier mapping, establishes dependencies between tasks, derives a task graph, performs topological initialization on the task graph, achieves the first complete scheduling execution of the task graph, and, upon detecting changes in the task graph, identifies and tracks the propagation paths of affected tasks, recompiling only necessary parts to achieve efficient incremental task scheduling. This method designs a unified task modeling approach with type safety and structural expressive capabilities, enabling explicit description of complex dependencies between parallel tasks, and builds an incremental scheduling mechanism based on this. This invention offers simplicity, controllability, and flexible scalability in parallel program construction and scheduling. Users can quickly construct task dependencies according to specific parallel scenarios and flexibly access heterogeneous resources or implement personalized optimization strategies, making it suitable for various parallel computing tasks.

[0064] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of different embodiments or examples.

[0065] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "N" means at least two, such as two, three, etc., unless otherwise explicitly specified.

[0066] Any process or method described in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or N executable instructions for implementing custom logic functions or processes, and the scope of the preferred embodiments of this application includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as should be understood by those skilled in the art to which embodiments of this application pertain.

Claims

1. A parallel task graph incremental scheduling method based on the OCaml functional language, characterized in that, Includes the following steps: S1, register the initial value of the task type and its type-to-string function, and construct the type identifier mapping; S2, using the type identifier mapping to uniformly encapsulate task functions of different task types, creating various types of tasks, supporting task group encapsulation and Boolean-based conditional branching structures, explicitly establishing dependencies between tasks, and obtaining a task graph with structured control flow. S3, perform topology initialization on the task graph, identify schedulable tasks, and use the parallel domain mechanism in the OCaml multi-core runtime system to allocate tasks to threads on multi-core computing resources for parallel execution, thereby achieving the first complete scheduling and execution of the task graph. S4. After detecting a change in the task graph, by identifying and tracing the propagation path of the affected tasks, only the necessary parts are recalculated, and incremental task scheduling is performed. Specifically, when a task in the task graph changes, the initial state of the affected task is first reset to an incomplete state, and the incremental scheduling process is started. During the scheduling process, if a task is in an incomplete state, it is re-executed; if the task meets the incremental state propagation condition, the state of all its successor tasks will also be set to an incomplete state and enter the subsequent scheduling queue; otherwise, the execution of the task is skipped.

2. The method according to claim 1, characterized in that, Step S1 specifically includes: By using an initial value for the task type and the corresponding string conversion function as the characteristics of the type, the type characteristics are registered in the type mapping dictionary to obtain a unique identifier for the type. The type identification mapping process is implemented through the function Dict.add.

3. The method according to claim 1, characterized in that, Step S2 specifically includes: The task graph object `graph` is registered and initialized using `Graph.registerGraph`. The task functions of different task types are uniformly encapsulated using the type identifier of the task type. Task nodes are added to the task graph using `Graph.registerTask`. The task graph with structured control flow is constructed by using Graph.registerGroup to achieve unified encapsulation of task groups, Graph.registerBranch to implement a conditional branching structure based on Boolean decision, and Graph.precede to explicitly establish dependencies between tasks.

4. The method according to claim 1, characterized in that, Step S3 specifically includes: Perform a topological traversal of the task graph, calculate the predecessor dependency count for each task, and mark all tasks with a dependency count of zero as initially schedulable tasks and add them to the scheduling queue. The scheduler is started on the multi-core resources specified by the user. It uses the parallel domain mechanism in the OCaml multi-core runtime system to allocate tasks from the scheduling queue to threads for parallel execution. During the execution, after each task is completed, the dependency count of its successor tasks is decremented. If all the predecessor tasks of a successor task have been completed, it is marked as a schedulable task and added to the scheduling queue until all tasks are completed.

Citation Information

Patent Citations

  • Cloud computing parallel task optimization scheduling method based on priority dependency graph

    CN119806776A