Complex document-oriented dynamic DAG task decomposition and scheduling method and system

Abstract

The invention belongs to the field of complex document processing, and provides a complex document-oriented dynamic DAG task decomposition and scheduling method and system in order to solve the problem of low complex document generation efficiency. The complex document-oriented dynamic DAG task decomposition and scheduling method comprises the following steps of: constructing an initial directed acyclic graph to form an initial task chain; receiving a node insertion request, calculating a dependency probability between any two nodes in all the non-ring adjacency lists by using a graph neural network model, and determining whether to add an implicit dependency edge according to the dependency probability so as to form an implicit dependency edge set; calculating task resource requirements of a to-be-generated document according to the video memory prediction model, dynamically allocating hardware resources and executing corresponding tasks in combination with node task types and resource requirement labels of all implicit dependency edges in the implicit dependency edge set, and aggregating execution results of all the tasks to generate a final document. The reliability and efficiency of a large model in a complex document generation task can be effectively improved.

Description

Technical Field

[0001] This invention belongs to the field of complex document processing, and in particular relates to a dynamic DAG task decomposition and scheduling method and system for complex documents. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] For the generation of complex documents containing images, text, and tables, the current approach mainly uses a large model combined with a static DAG (Directed Acyclic Graph) algorithm to decompose and schedule the tasks. However, this approach relies excessively on predefined DAG structures and cannot adapt to the dynamically changing subtask dependencies in complex document generation (such as real-time user additions and deletions). This results in poor reliability and low efficiency of the large model's decomposition results for complex document generation tasks. Furthermore, the computational characteristics of the large model (such as memory bottlenecks) are not considered during the complex document generation process, leading to resource contention and further reducing the efficiency of complex document generation. Summary of the Invention

[0004] To address the aforementioned technical problems, this invention provides a method and system for dynamic DAG task decomposition and scheduling for complex documents. This method enables intelligent decomposition and scheduling of complex document generation tasks, effectively improving the reliability and efficiency of large models in complex report generation tasks.

[0005] To achieve the above objectives, the present invention adopts the following technical solution: The first aspect of the present invention provides a dynamic DAG task decomposition and scheduling method for complex documents.

[0006] In one or more embodiments, a dynamic DAG task decomposition and scheduling method for complex documents is provided, including: Based on the domain of the document to be generated, the document template is matched and searched from the corresponding knowledge base to construct an initial directed acyclic graph and form an initial task chain; each node is a task and is labeled with a resource requirement tag. Upon receiving a node insertion request, when the hardware resource utilization is less than a preset threshold, the corresponding node is inserted at the corresponding position in the current directed acyclic graph. Real-time loop detection is performed on each adjacency list of the directed acyclic graph after the node is inserted, and adjacency lists with loop structures are removed. The dependency probability between any two nodes in all non-cyclic adjacency lists is calculated using a graph neural network model, and implicit dependency edges are added accordingly, thereby forming a set of implicit dependency edges. The task resource requirements of the document to be generated are calculated based on the memory prediction model. Then, the task type and resource requirement tags of each implicit dependency edge in the implicit dependency edge set are combined to dynamically allocate hardware resources and execute the corresponding tasks. The execution results of all tasks are aggregated to generate the final document.

[0007] As one implementation method, the document template is formatted as follows: document template = {nodes:["X1", "X2", ..., "XN"], dependencies:["Xi→Xj", ...]}; where Xi and Xj are nodes, i and j are not equal and their values ​​range from 1 to N; N is a positive integer greater than or equal to 2; → indicates a dependency relationship.

[0008] As one implementation method, the expression for the memory prediction model is: ; in, Resource requirements for the task to generate the document; For large model size parameters; This represents the number of the smallest units that need to be processed. This is the memory correction factor; , It is a constant coefficient.

[0009] As one implementation method, during the dynamic allocation of hardware resources, the priority level of hardware resource allocation is determined based on the node task type of each implicit dependency edge in the implicit dependency edge set; and the hardware allocation rules are determined based on the node resource requirement tags of each implicit dependency edge in the implicit dependency edge set.

[0010] As one implementation method, when the task type of the implicitly dependent edge node is text generation, its hardware resource allocation priority is the highest; when the task type of the implicitly dependent edge node is data cleaning, its hardware resource allocation priority is medium; and when the task type of the implicitly dependent edge node is chart rendering, its hardware resource allocation priority is the lowest.

[0011] As one implementation method, after aggregating the execution results of all tasks, the method also includes: invoking a consistency verification engine to verify the uniformity of terminology and the logical coherence of the documents.

[0012] A second aspect of the present invention provides a dynamic DAG task decomposition and scheduling system for complex documents.

[0013] In one or more embodiments, a dynamic DAG task decomposition and scheduling system for complex documents includes: The initial directed acyclic graph (DAG) construction module is used to match and search for document templates from the corresponding knowledge base based on the domain of the document to be generated, construct an initial DAG, and form an initial task chain; each node is a task and is labeled with a resource requirement tag. The task dependency inference module is used to receive node insertion requests. When the hardware resource utilization is less than a preset threshold, it inserts the corresponding node at the corresponding position in the current directed acyclic graph. It performs real-time loop detection on each adjacency list of the directed acyclic graph after the node is inserted, removes adjacency lists with loop structures, calculates the dependency probability between any two nodes in all non-cyclic adjacency lists using a graph neural network model, and determines whether to add implicit dependency edges accordingly, thereby forming a set of implicit dependency edges. The resource scheduling and task execution module is used to calculate the task resource requirements of the document to be generated based on the memory prediction model. Then, it combines the node task type and resource requirement label of each implicit dependency edge in the implicit dependency edge set to dynamically allocate hardware resources and execute the corresponding tasks. Finally, it aggregates the execution results of all tasks to generate the final document.

[0014] A third aspect of the present invention provides a computer program product.

[0015] A computer program product includes a memory storing a computer program configured to perform the steps of the dynamic DAG task decomposition and scheduling method for complex documents as described above.

[0016] A fourth aspect of the present invention provides a computer storage medium.

[0017] A computer storage medium having a computer program stored thereon, the program being configured to perform the steps of the dynamic DAG task decomposition and scheduling method for complex documents as described above.

[0018] A fifth aspect of the present invention provides an electronic device.

[0019] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the dynamic DAG task decomposition and scheduling method for complex documents as described above.

[0020] Compared with the prior art, the beneficial effects of the present invention are: This invention constructs an initial directed acyclic graph (DAG) to form an initial task chain, dynamically inserts nodes into the DAG, and performs real-time cycle detection on each adjacency list of the DAG after node insertion. It then uses a graph neural network model to calculate the dependency probability between any two nodes in all non-cyclic adjacency lists, determining whether to add implicit dependency edges and constructing an implicit dependency edge set. This effectively improves the reliability of large models in complex report generation tasks. Furthermore, it calculates the task resource requirements of the document to be generated based on a memory prediction model, and then dynamically allocates hardware resources and executes corresponding tasks by combining the node task type and resource requirement labels of each implicit dependency edge in the implicit dependency edge set. This avoids hardware resource contention, improves hardware resource utilization, and increases the efficiency of complex document generation. Attached Figure Description

[0021] The accompanying drawings, which form part of this invention, are used to provide a further understanding of the invention. The illustrative embodiments of the invention and their descriptions are used to explain the invention and do not constitute an improper limitation of the invention.

[0022] Figure 1 This is a flowchart illustrating the dynamic DAG task decomposition and scheduling method for complex documents according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the structure of a dynamic DAG task decomposition and scheduling system for complex documents according to an embodiment of the present invention; Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0024] It should be noted that the following detailed description is illustrative and intended to provide further explanation of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0025] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0026] Figure 1 This is a flowchart illustrating a dynamic DAG task decomposition and scheduling method for complex documents according to an embodiment of the present invention, as shown below. Figure 1As shown, the dynamic DAG task decomposition and scheduling method for complex documents in this embodiment may include the following steps S101 to S103.

[0027] The specific implementation process of steps S101 to S103 is as follows: Step S101: Based on the domain of the document to be generated, match and search for document templates from the corresponding knowledge base, construct an initial directed acyclic graph, and form an initial task chain; each node is a task and is labeled with a resource requirement tag.

[0028] In step S101, the document template is formatted as follows: document template = {nodes:["X1", "X2", ..., "XN"], dependencies:["Xi→Xj", ...]}; where Xi and Xj are nodes, i and j are not equal and their values ​​range from 1 to N; N is a positive integer greater than or equal to 2; → represents a dependency relationship.

[0029] For example, if the document to be generated belongs to the financial field, the corresponding document template in the knowledge base will be formatted as: Financial Document = {Chapter:["Summary","Financial Analysis","Risk"], Dependencies:["Summary → Financial Analysis"]}.

[0030] Based on the document requirement type input by the user, the template is automatically matched to generate an initial DAG subgraph, i.e., an adjacency list: {node:[predecessor node]}.

[0031] The resource requirements listed here include, but are not limited to, GPU (Graphics Processing Unit) intensive and CPU (Central Processing Unit) intensive.

[0032] Step S102: Receive node insertion request. When the hardware resource utilization is less than the preset threshold, insert the corresponding node at the corresponding position in the current directed acyclic graph. Perform real-time loop detection on each adjacency list of the directed acyclic graph after the node is inserted, remove adjacency lists with loop structures, use a graph neural network model to calculate the dependency probability between any two nodes in all non-cyclic adjacency lists, and determine whether to add implicit dependency edges accordingly, thereby forming an implicit dependency edge set.

[0033] For example, when hardware resource utilization is less than 80%, respond to node insertion requests; When the hardware resource utilization rate is greater than or equal to 80%, the node insertion request will not be responded to.

[0034] This can prevent the hardware resources from being used beyond the set load, which could lead to the entire system running slowly or even crashing.

[0035] It should be noted that the preset threshold for hardware resource utilization can be set according to the actual situation, which will not be detailed here.

[0036] Receive user interaction commands or change requests from large models. Format: {Operation: "Insert Node", Parameter: {Node ID: ESG Evaluation, Prerequisite: Data Crawling}}.

[0037] In the real-time cycle detection process of each adjacency list in the directed acyclic graph after node insertion, a stack depth truncation method is used, abandoning the time-consuming recursive detection mechanism and adopting the maximum depth method. The node state is represented by 0, 1, and 2, where 0 indicates unvisited, 1 indicates being visited, and 2 indicates being visited. If a state of 1 is encountered during traversal, it indicates that a cycle has been detected, and the adjacency list is abandoned.

[0038] The process of using a graph neural network model to calculate the dependency probability between any two nodes in all non-cyclic adjacency lists, and then determining whether to add implicit dependency edges based on this probability, thus forming a set of implicit dependency edges, is as follows: Constructing a Graph Neural Network (GNN) model to infer implicit task dependencies; formula for extracting node feature matrices. (Including historical execution duration) Resource requirements), using graph neural networks to process node feature matrices This yields the embedding representation of any node i in each non-cyclic adjacency list. Then, use the Sigmoid function to output the dependency probability between any two nodes in the non-cyclic adjacency list. The formula for calculating the dependency probability between any two nodes in a non-cyclic adjacency list is: ; This represents the probability that there is an implicit dependency between node i and node j, with a probability value ranging from 0 to 1. Embedded representation of node i transpose; Let be the embedding representation of node j.

[0039] like Greater than a set probability threshold (e.g., If so, add a dependency edge. (i.e., the edge from node i to node j); The final output is a set of implicit dependency edges. ,in, It is the final set of implicitly dependent edges, where each element is a directed edge. .

[0040] Add version tags (e.g., V1.0→V1.1) to affected nodes to support state rollback, while saving the graph neural network (GNN) model.

[0041] Step S103: Calculate the task resource requirements of the document to be generated based on the memory prediction model, and then combine the node task type and resource requirement label of each implicit dependency edge in the implicit dependency edge set to dynamically allocate hardware resources and execute the corresponding tasks. Aggregate the execution results of all tasks to generate the final document.

[0042] In step S103, the expression for the memory prediction model is: ; in, Resource requirements for the task to generate the document; For large model size parameters; This represents the number of the smallest units that need to be processed. This is the memory correction factor; , It is a constant coefficient.

[0043] The memory correction factor is a calibration constant term used to compensate for memory consumption that is not covered by parameters of large model size, i.e., the number of smallest units to be processed (such as fixed or random memory consumption such as frame overhead, cache usage, and temporary variable storage during model runtime).

[0044] in, , The specific settings can be customized according to the actual situation, for example, It is 1.2; It is 0.5.

[0045] Specifically, in the process of dynamically allocating hardware resources, the priority of hardware resource allocation is determined based on the task type of each node of the implicit dependency edge in the implicit dependency edge set; and the hardware allocation rules are determined based on the resource requirement tags of each node of the implicit dependency edge in the implicit dependency edge set.

[0046] For example, when the task type of the implicitly dependent edge node is text generation, its hardware resource allocation priority is the highest; when the task type of the implicitly dependent edge node is data cleaning, its hardware resource allocation priority is medium; and when the task type of the implicitly dependent edge node is chart rendering, its hardware resource allocation priority is the lowest.

[0047] During task execution, a three-level fault tolerance strategy is implemented, namely heartbeat detection, node restart, and error log generation of LoRA dataset.

[0048] For example, heartbeat detection: the task state vector is checked every 5 seconds. ; Node restart: If any node's task state vector And restart the counter. If so, then the dynamic task decomposition and scheduling will be re-executed; Error logs generate LoRA dataset: ; These represent incorrect inputs and corrected outputs, respectively.

[0049] The memory prediction model is updated based on hardware resource utilization to optimize the dynamic allocation of hardware resources.

[0050] Ensuring cross-task content consistency through a shared context cache, with each task output... Write to cache pool Downstream task reading ;in, For dependency mask; Calculation method: (The core update formula of the policy gradient method in reinforcement learning); in: objective function For strategy parameters The gradient; Indicates element-wise product; The expectation symbol represents the expectation of the state-action pair enclosed in parentheses. "Take the expected value (because of the strategy)" (It is random, and all possible cases of s and a need to be considered). i.e., strategy The logarithm of the parameter The gradient of (i.e., the score function); That is, the policy function; input: state s; output: "in the parameters "Probability distribution of taking action a in state s" (describing the rules for the agent to "select action"); The state-action value function (Q-function) represents the expected cumulative reward that can be obtained after taking action a in state s. It is used to measure the "goodness or badness" of "choosing action a in state s" and to provide "reward feedback" for policy updates. state: =[Node queue, CPU / GPU utilization, task latency]; action: =Scheduling decisions (such as prioritizing GPU allocation); award: ;in, and is the weighting coefficient, and is a constant coefficient.

[0051] In one or more embodiments, after aggregating the execution results of all tasks, the method further includes: Invoke the consistency check engine to verify the uniformity of terminology and the logical coherence of the document.

[0052] The process of retrieving the consistency verification engine to verify the uniformity of terminology in the document is as follows: Constructing domain knowledge graphs (For example, in the financial sector, "ROE" = "Return on Equity"). The term similarity is checked, and the term similarity is calculated as follows: ; in, For domain terms to be verified (such as "ROE", "Return on Equity", or "MRI" or "Magnetic Resonance Imaging" in the medical field). The final output is the semantic similarity value between the two terms; if this value is less than the similarity threshold (e.g., ... If two terms are deemed to have inconsistent semantics and require consistent expression, they are marked as "term conflict" (e.g., the two terms are considered to have inconsistent semantics and require consistent expression). The transpose symbol is used to transpose... The transformation from "column vector" to "row vector" prepares for the subsequent "vector dot product"; This is the vector dot product operation; The result after semantic embedding of the term t using the BERT model; In some alternative embodiments, existing methods can be used to verify the logical coherence of a document, such as using causal graph (directed graph) algorithms to verify logical coherence, including time constraint verification and causal dependency detection.

[0053] In other embodiments, those skilled in the art can choose specific methods to verify the uniformity of terminology and the logical coherence of documents according to the actual situation, which will not be described in detail here.

[0054] This embodiment constructs an initial directed acyclic graph (DAG) to form an initial task chain. Nodes are dynamically inserted into the initial DAG. Real-time cycle detection is performed on each adjacency list of the DAG after node insertion. A graph neural network model is used to calculate the dependency probability between any two nodes in all non-cyclic adjacency lists to determine whether to add implicit dependency edges. An implicit dependency edge set is constructed, effectively improving the reliability of large models in complex report generation tasks. Furthermore, the task resource requirements of the document to be generated are calculated based on a memory prediction model. Combined with the node task type and resource requirement labels of each implicit dependency edge in the implicit dependency edge set, hardware resources are dynamically allocated and corresponding tasks are executed. This avoids hardware resource contention, improves hardware resource utilization, and increases the efficiency of complex document generation.

[0055] like Figure 2 As shown, the dynamic DAG task decomposition and scheduling system for complex documents provided in this embodiment of the invention can be implemented in software. The dynamic DAG task decomposition and scheduling system for complex documents includes the following software modules: dynamic directed acyclic graph construction and decomposition module 201, task dependency inference module 202, and resource scheduling and task execution module 203.

[0056] The following section introduces the functions of each software module in the dynamic DAG task decomposition and scheduling system for complex documents: The initial directed acyclic graph construction module 201 is used to match and search for document templates from the corresponding knowledge base based on the domain of the document to be generated, construct an initial directed acyclic graph, and form an initial task chain; each node is a task and is labeled with a resource requirement tag; The task dependency inference module 202 is used to receive node insertion requests. When the hardware resource utilization is less than a preset threshold, it inserts the corresponding node at the corresponding position in the current directed acyclic graph. It performs real-time loop detection on each adjacency list of the directed acyclic graph after the node is inserted, removes adjacency lists with loop structures, calculates the dependency probability between any two nodes in all non-cyclic adjacency lists using a graph neural network model, and determines whether to add implicit dependency edges accordingly, thereby forming a set of implicit dependency edges. The resource scheduling and task execution module 203 is used to calculate the task resource requirements of the document to be generated based on the memory prediction model, and then dynamically allocate hardware resources and execute corresponding tasks by combining the node task type and resource requirement label of each implicit dependency edge in the implicit dependency edge set, and aggregate the execution results of all tasks to generate the final document.

[0057] It should be noted that each module in the dynamic DAG task decomposition and scheduling system for complex documents in this embodiment corresponds one-to-one with each step in the dynamic DAG task decomposition and scheduling method for complex documents in the above embodiment, and their specific implementation processes are the same, so they will not be repeated here.

[0058] The structure of the electronic device according to an embodiment of the present invention will be described in detail below. Figure 3 This is a schematic diagram of the composition structure of an electronic device provided in an embodiment of the present invention. It can be understood that... Figure 3 The diagram shows only an exemplary structure of the electronic device, not the entire structure. Some or all of the structures shown may be implemented as needed.

[0059] The electronic device provided in this embodiment of the invention includes: at least one processor 301, a memory 302, a user interface 303, and at least one network interface 304. The various components in the dynamic DAG task decomposition and scheduling system for complex documents are coupled together via a bus system 305. It can be understood that the bus system 305 is used to implement communication between these components. In addition to a data bus, the bus system 305 also includes a power bus, a control bus, and a status signal bus. However, for clarity, in... Figure 3 The general designated all buses as Bus System 305.

[0060] The user interface 303 may include a monitor, keyboard, mouse, trackball, click wheel, buttons, touchpad, or touch screen.

[0061] It is understood that memory 302 can be volatile memory or non-volatile memory, or both. In this embodiment of the invention, memory 302 is capable of storing data to support the operation of the terminal. Examples of this data include any computer programs used to operate on the terminal, such as operating systems and applications. The operating system includes various system programs, such as framework layers, core library layers, driver layers, etc., used to implement various basic services and handle hardware-based tasks. Applications can include various applications.

[0062] In some embodiments, the dynamic DAG task decomposition and scheduling system for complex documents provided in this invention can be implemented using a combination of hardware and software. As an example, the dynamic DAG task decomposition and scheduling system for complex documents provided in this invention can be a processor in the form of a hardware decoding processor, programmed to execute the dynamic DAG task decomposition and scheduling method for complex documents provided in this invention. For example, the processor in the form of a hardware decoding processor can employ one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0063] As an example, processor 301 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc., wherein the general-purpose processor can be a microprocessor or any conventional processor, etc.

[0064] As an example of the hardware implementation of the dynamic DAG task decomposition and scheduling system for complex documents provided in this embodiment of the invention, the device provided in this embodiment of the invention can be directly executed by a processor 301 in the form of a hardware decoding processor. For example, it can be executed by one or more application-specific integrated circuits (ASICs), DSPs, programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components to implement the dynamic DAG task decomposition and scheduling method for complex documents provided in this embodiment of the invention.

[0065] The memory 302 in this embodiment of the invention is used to store various types of data to support the operation of a dynamic DAG task decomposition and scheduling system for complex documents, or to store data for execution. Figure 1The program code for the method shown. Examples of this data include: any executable instructions for operating on a dynamic DAG task decomposition and scheduling system for complex documents, such as executable instructions that can be included in the executable instructions, implementing the dynamic DAG task decomposition and scheduling method for complex documents according to embodiments of the present invention.

[0066] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including functions for executing... Figure 1 The program code for the method shown. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by the central processing unit, it performs the various functions defined in the apparatus of this application.

[0067] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, as well as combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0068] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dynamic DAG task decomposition and scheduling method for complex documents, characterized in that, include: Based on the domain of the document to be generated, the document template is matched and searched from the corresponding knowledge base to construct an initial directed acyclic graph and form an initial task chain; each node is a task and is labeled with a resource requirement tag. Upon receiving a node insertion request, when the hardware resource utilization is less than a preset threshold, the corresponding node is inserted at the corresponding position in the current directed acyclic graph. Real-time loop detection is performed on each adjacency list of the directed acyclic graph after the node is inserted, and adjacency lists with loop structures are removed. The dependency probability between any two nodes in all non-cyclic adjacency lists is calculated using a graph neural network model, and implicit dependency edges are added accordingly, thereby forming a set of implicit dependency edges. The task resource requirements of the document to be generated are calculated based on the memory prediction model. Then, the task type and resource requirement tags of each implicit dependency edge in the implicit dependency edge set are combined to dynamically allocate hardware resources and execute the corresponding tasks. The execution results of all tasks are aggregated to generate the final document.

2. The dynamic DAG task decomposition and scheduling method for complex documents as described in claim 1, characterized in that, The document template format is: document template = {nodes:["X1", "X2", ..., "XN"], dependencies:["Xi→Xj", ...]}; where Xi and Xj are nodes, i and j are not equal and their values ​​are both in the range of 1 to N; N is a positive integer greater than or equal to 2; → indicates a dependency relationship.

3. The dynamic DAG task decomposition and scheduling method for complex documents as described in claim 1, characterized in that, The expression for the memory prediction model is: ； in, Resource requirements for the task to generate the document; For large model size parameters; This represents the number of the smallest units that need to be processed. This is the memory correction factor; , It is a constant coefficient.

4. The dynamic DAG task decomposition and scheduling method for complex documents as described in claim 1, characterized in that, During the dynamic allocation of hardware resources, the priority of hardware resource allocation is determined based on the task type of each node of the implicit dependency edge set; the hardware allocation rules are determined based on the resource requirement tags of each node of the implicit dependency edge set.

5. The dynamic DAG task decomposition and scheduling method for complex documents as described in claim 4, characterized in that, When the task type of the implicitly dependent edge node is text generation, its hardware resource allocation priority is the highest; when the task type of the implicitly dependent edge node is data cleaning, its hardware resource allocation priority is medium; and when the task type of the implicitly dependent edge node is chart rendering, its hardware resource allocation priority is the lowest.

6. The dynamic DAG task decomposition and scheduling method for complex documents as described in claim 1, characterized in that, After aggregating the execution results of all tasks, the process also includes: invoking the consistency verification engine to verify the uniformity of terminology and the logical coherence of the documents.

7. A dynamic DAG task decomposition and scheduling system for complex documents, characterized in that, include: The initial directed acyclic graph (DAG) construction module is used to match and search for document templates from the corresponding knowledge base based on the domain of the document to be generated, construct an initial DAG, and form an initial task chain; each node is a task and is labeled with a resource requirement tag. The task dependency inference module is used to receive node insertion requests. When the hardware resource utilization is less than a preset threshold, it inserts the corresponding node at the corresponding position in the current directed acyclic graph. It performs real-time loop detection on each adjacency list of the directed acyclic graph after the node is inserted, removes adjacency lists with loop structures, calculates the dependency probability between any two nodes in all non-cyclic adjacency lists using a graph neural network model, and determines whether to add implicit dependency edges accordingly, thereby forming a set of implicit dependency edges. The resource scheduling and task execution module is used to calculate the task resource requirements of the document to be generated based on the memory prediction model. Then, it combines the node task type and resource requirement label of each implicit dependency edge in the implicit dependency edge set to dynamically allocate hardware resources and execute the corresponding tasks. Finally, it aggregates the execution results of all tasks to generate the final document.

8. A computer program product, comprising a memory, characterized in that, The memory stores a computer program configured to perform the steps of the dynamic DAG task decomposition and scheduling method for complex documents as described in any one of claims 1-6.

9. A computer storage medium having a computer program stored thereon, characterized in that, The program configuration is as described in any one of the steps of the dynamic DAG task decomposition and scheduling method for complex documents as claimed in any one of claims 1-6.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps in the dynamic DAG task decomposition and scheduling method for complex documents as described in any one of claims 1-6.

Images