A unified integrated scheduling platform and method for large multimodal AIGC models

By constructing a task dependency graph and a predicted semantic solution graph, the model capability requirements are quantified, and the model with the highest fit is selected. This solves the problems of resource waste and delivery quality in the scheduling of large multimodal AIGC models, and achieves efficient resource utilization and quality assurance.

CN123070193APending Publication Date: 2026-11-10BEIJING FIBERHOME TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611552193.0
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-10-09
Publication Date
2026-11-10

AI Technical Summary

Technical Problem

Existing unified integration and scheduling platforms for large multimodal AIGC models struggle to accurately quantify the match between model capabilities and task requirements, leading to wasted computing power and call costs or decreased delivery quality.

Method used

By constructing a task dependency graph, segmenting the output data units, constructing a predicted semantic solution graph, obtaining the model capability requirement degree and weight, and combining the conditional capability boundary value, selecting the model with the highest suitability for invocation.

Benefits of technology

This improves the effectiveness of model scheduling and resource utilization efficiency, ensures task delivery quality, and reduces waste of computing power and costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123070193A_ABST
    Figure CN123070193A_ABST
Patent Text Reader

Abstract

This invention relates to the field of model scheduling technology, specifically to a unified integrated scheduling platform and method for multimodal AIGC large models. The invention uses task nodes in the task dependency graph as target task nodes, and divides the obtained node output fields into multiple output data units. It constructs a predicted semantic solution graph for each output data unit to obtain the capability requirements of each model. Based on the downstream propagation range of the capability-related semantic anchor set corresponding to each model capability, and the number of anchors entering the final output slot after processing by downstream task nodes, it obtains the capability requirement weight of each model capability in the output data unit. Combining the conditional capability boundary values ​​of each model on each model capability, it obtains the overall fit of each model to the target task node, and selects the models to be called to execute the target task node. This invention accurately analyzes the overall fit of each model to the target task node, improving the effectiveness of model integrated scheduling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of model scheduling technology, specifically to a unified integrated scheduling platform and method for a multimodal AIGC large model. Background Technology

[0002] The unified integration and scheduling platform for multimodal AIGC large models centrally registers and manages different types of large models and model services, encapsulates the calling differences between models using standardized interfaces, and organizes models such as text generation, image generation, speech synthesis, video generation, and multimodal understanding into a continuous task processing flow based on specific business tasks. This enables users to call, combine, deploy, and manage multiple types of AIGC capabilities within the same platform.

[0003] Existing unified integrated scheduling platforms for large multimodal AIGC models typically select models to be called based on task type, input / output modality, model comprehensive score, or preset priority. However, because existing scheduling methods cannot accurately quantify the actual model capabilities required for the output of task nodes and the degree of influence of different capabilities on the final delivery results, the supply of model capabilities cannot be precisely matched with the actual capability requirements of the current task. This can easily lead to problems such as simple tasks calling high-capability models, resulting in wasted computing power and calling costs, or complex tasks calling models with insufficient capabilities, affecting the final delivery quality. Summary of the Invention

[0004] To address the technical problem of existing scheduling methods failing to accurately quantify the actual model capabilities required by task nodes and the degree of influence of different capabilities on the final deliverables, resulting in poor model invocation performance, this invention aims to provide a unified integrated scheduling platform and method for multimodal AIGC large models. The specific technical solution adopted is as follows: This invention proposes a unified integrated scheduling method for a large multimodal AIGC model, the method comprising: Obtain the task elements and delivery information of the user's task instructions, and construct a task dependency graph containing multiple task nodes; Each task element is matched with a preset output template library to obtain the final output slot of the matching output template; any task node in the task dependency graph is taken as the target task node, and the obtained node output field is divided into multiple output data units based on task dependency relationship and semantic independence. Construct a predicted semantic solution graph for each output data unit. Based on the topological features and semantic relationship structure of the predicted semantic solution graph, obtain the capability requirement degree of each model capability. Obtain the set of capability-related semantic anchor points corresponding to each model capability in the predicted semantic solution graph of the output data unit. Based on the downstream propagation range of the capability-related semantic anchor point set and the number of anchor points that enter the final output slot after being processed by downstream task nodes, obtain the capability requirement weight of each model capability in the output data unit. Construct a unified model library and obtain the conditional capability boundary values ​​of each model in each model capability. Based on the conditional capability boundary values ​​of each model in each model capability, the capability requirement weights and capability requirement degrees of each model capability in different output data units corresponding to the target task node, obtain the adaptability of each model to the target task node in each model capability, and obtain the overall adaptability of each model to the target task node. Based on the overall adaptability of each model to the target task node, select the model to be invoked to execute the target task node.

[0005] Furthermore, the method for obtaining the final output slot includes: Match the task elements with the task type, target output modality, and target delivery form corresponding to each output template in the preset output template library to determine the matching output template; Read the structured fields and output positions from the matching output template, extract the tag attributes of each structured field, and bind them to their corresponding output positions as the final output slots.

[0006] Furthermore, the method for acquiring the output data unit includes: For any target task node in the task dependency graph, after parsing the node output field of the target task node, the node output field of the target task node is split according to the semantic components that its downstream task nodes can read independently. The semantic components include semantic subjects, attributes, relationships, events, states or constraints. If multiple semantic components in a node's output field are continuously read and passed to downstream task nodes, the entire field is treated as a single output data unit.

[0007] Furthermore, the method for obtaining the level of capability requirement includes: Based on the topological features and semantic relation structure of the predicted semantic solution graph, multiple model capabilities of the output data unit are extracted, including semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency preservation span, and constraint coupling degree. The semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency maintenance span, and constraint coupling degree are normalized respectively to represent the capability requirements of each model in the output data unit.

[0008] Furthermore, the method for obtaining the capability requirement weights includes: Starting from the output field of the node where the output data unit is located, the number of corresponding fields in the path tracing along the task dependency graph to each final output slot is obtained as the number of downstream candidate transmission fields; taking the anchors in the set of capability-related semantic anchors as the propagation source, forward tracing is performed according to the data mapping relationship between adjacent task nodes in the task dependency graph, and the fields traversed by the propagation source in the downstream task nodes are taken as the affected transmission fields. The ratio of the number of affected propagation fields to the number of downstream candidate propagation fields for each model capability in each output data unit is obtained as the downstream propagation coverage of each model capability in each output data unit. The ratio of the number of semantic anchors derived from the set of semantic anchors associated with capabilities into the final output slot to the number of semantic anchors associated with each model capability is obtained as the slot semantic retention ratio of each model capability in each output data unit for each final output slot; the average slot semantic retention ratio of each model capability in each output data unit for all final output slots is obtained as the terminal delivery effectiveness of each model capability in each output data unit. The product of the downstream propagation coverage and terminal delivery effectiveness of each model capability in each output data unit is obtained as the capability requirement weight.

[0009] Furthermore, the method for obtaining the conditional capability boundary value includes: Obtain the capability requirement of each preset model capability calibration task on each model capability; for any model capability, select a preset number of calibration tasks whose capability requirement is located within each preset range and whose other model capabilities are all less than the preset minimum boundary threshold. The model evaluation results are obtained after each model performs the corresponding model capability calibration task. The ratio of the number of calibration tasks with model evaluation results greater than or equal to the preset evaluation threshold to the preset number of calibration tasks is calculated as the capability pass rate of the model within each preset range. According to the preset degree range from low to high, the upper limit of the preset degree range in which the continuous pass rate of the capability is greater than or equal to the preset pass threshold is used as the conditional capability boundary value of the model on the corresponding model capability.

[0010] Furthermore, the method for obtaining the fit includes: For the target task node, select the minimum value between the conditional capability boundary value of each model in each model capability and the capability requirement degree of the corresponding model capability in each output data unit, and obtain the cumulative value of the product between the minimum value and the capability requirement weight of the corresponding model capability in each output data unit, as the effective capability overlap of each model for the target task node in each model capability. The cumulative value of the product between the capability requirement weight and capability requirement degree of each model capability in each output data unit is obtained as the weighted total capability requirement of the target task node for each model capability; the cumulative value of the capability requirement weight of each model capability in each output data unit is obtained, and the product between the cumulative value and the conditional capability boundary value of each model for each model capability is calculated as the weighted total capability supply. Based on the effective capability overlap, total weighted capability demand, and total weighted capability supply of each model for the target task node, the fit is obtained. The effective capability overlap is positively correlated with the fit, while the total weighted capability demand and total weighted capability supply are both negatively correlated with the fit.

[0011] Furthermore, the method for obtaining the overall fit includes: The total weighted capability requirement is normalized, and the fitness is weighted and summed based on the normalized results of the target task node's capabilities across all models, which serves as the overall fitness of each model for the target task node.

[0012] Furthermore, the method for selecting the models to be invoked for executing the target task node includes: The model with the highest overall fit among all models is selected as the model to be called.

[0013] This invention also proposes a unified integrated scheduling platform for a multimodal AIGC large model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the unified integrated scheduling method for the multimodal AIGC large model.

[0014] The present invention has the following beneficial effects: This invention takes any task node in the task dependency graph as the target task node, and divides the obtained node output fields into multiple output data units. It constructs a predicted semantic solution graph for each output data unit to obtain the capability requirements of each model. Based on the downstream propagation range of the capability-related semantic anchor set corresponding to each model capability, and the number of anchors entering the final output slot after processing by downstream task nodes, it obtains the capability requirement weight of each model capability in the output data unit, reflecting the significant impact of model capabilities on final delivery. Combining the conditional capability boundary values ​​of each model on each model capability, it obtains the overall adaptability of each model to the target task node, reflecting the optimal utilization efficiency of system computing resources while ensuring task delivery quality, and selects the models to be called to execute the target task node. This invention accurately analyzes the overall adaptability of each model to the target task node, improving the effectiveness of model integration and scheduling. Attached Figure Description

[0015] Figure 1 The flowchart illustrates a unified integrated scheduling method for a multimodal AIGC large model, as provided in one embodiment of the present invention. Detailed Implementation

[0016] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0017] The following description, in conjunction with the accompanying drawings, details the specific scheme of the unified integrated scheduling platform and method for a multimodal AIGC large model provided by this invention.

[0018] Please see Figure 1 The diagram illustrates a flowchart of a unified integrated scheduling method for a multimodal AIGC large model according to an embodiment of the present invention, specifically including: Step S1: Obtain the task elements and delivery information of the user's task instructions, and construct a task dependency graph containing multiple task nodes.

[0019] In embodiments of the present invention, user task instructions are obtained, and task elements and delivery information are obtained using semantic parsing and information extraction techniques. The task elements include the current task type, target generation object, target input modality, target output modality, and target delivery format. The delivery information includes at least content-type information for defining semantic content and constraint-type information for defining output format or downstream interface specifications. The content-type information includes subject identifiers, attribute parameters, functional characteristics, entity relationships, status information, and timing information related to the target generation object. The constraint-type information includes mandatory content, prohibited content, output format requirements, generation style requirements, output duration, and downstream system interface requirements.

[0020] It should be noted that, in the embodiments of the present invention, constructing a task dependency graph containing multiple task nodes includes: based on existing workflow orchestration and directed acyclic graph construction techniques, recording abstract logical processing steps that can complete an independent input-output processing as task nodes, thus forming the task dependency graph of the current task.

[0021] The specific process is as follows: Based on the parsed task elements and delivery information, the complex current task is broken down into multiple subtasks. Each model call capable of completing an independent input / output processing logic is classified as a task node. Each task node records the node's input fields, output fields, field data types, and the correspondence between input / output fields of adjacent task nodes. Based on the data flow sequence between task nodes, the workflow orchestration engine connects the task nodes to form a directed acyclic graph, where directed edges represent data dependencies, ensuring that task execution does not result in circular deadlocks. The edge attributes of the task dependency graph record... Record the correspondence between input and output fields between adjacent task nodes, precisely indicating which specific output field of an upstream task node is assigned to which input field of a downstream task node. For example, if the upstream task node is a "multimodal understanding node" with output fields including [scene description text, sentiment label], and the downstream task node is an "image generation node" with input fields including [Prompt body, style parameters], then the mapping relationship is recorded in the dependency graph: the upstream [scene description text] maps to the downstream [Prompt body], and the upstream [sentiment label] maps to the downstream [style parameters].

[0022] Step S2: Match each task element with the preset output template library to obtain the final output slot of the matching output template; take any task node in the task dependency graph as the target task node, and divide the obtained node output field into multiple output data units based on task dependency relationship and semantic independence.

[0023] The raw output of multimodal large models is mostly discrete, unstructured data. Matching task elements with a pre-defined output template library allows for the pre-definition of the specific format, data type constraints, and spatial and temporal arrangement rules of the final deliverables, ensuring the foundation for multimodal data alignment and assembly. Each task element is matched with the pre-defined output template library to obtain the final output slots of the matched output templates.

[0024] Preferably, in one embodiment of the present invention, the method for obtaining the final output slot includes: Match the task elements with the task type, target output modality, and target delivery form corresponding to each output template in the preset output template library to determine the matching output template; Read the structured fields and output positions from the matching output template, extract the tag attributes of each structured field, and bind them to their corresponding output positions as the final output slots.

[0025] It should be noted that, in the embodiments of the present invention, the platform pre-establishes an output template library, i.e., a preset output template library, according to different task types, target output modalities, business delivery specifications and downstream system interface requirements; and uses data pattern parsing and semantic slot extraction technology to read and match the structured fields and output positions in the output templates, which are used as the final output slots; the specific means are well known to those skilled in the art and will not be described in detail here.

[0026] A single output field of a large model often contains complex semantic information. Directly evaluating the entire field will mask the real need for complex local semantics. Therefore, any task node in the task dependency graph is taken as the target task node. Based on task dependency and semantic independence, the obtained node output field is divided into multiple output data units.

[0027] Preferably, in one embodiment of the present invention, the method for acquiring the output data unit includes: For any target task node in the task dependency graph, after parsing the node output field of the target task node, the node output field of the target task node is split according to the semantic components that its downstream task nodes can read independently. The semantic components include semantic subjects, attributes, relationships, events, states or constraints. If multiple semantic components in a node's output field are continuously read and passed to downstream task nodes, the entire field is treated as a single output data unit.

[0028] It should be noted that, in the embodiments of the present invention, data pattern parsing and semantic slot extraction techniques are used to parse the node output fields of the target task node and extract the various semantic components contained in the node output fields. If multiple semantic components in the node output fields must be continuously transmitted and jointly read in downstream task nodes, the entire field is treated as an output data unit. The specific criterion for continuous transmission and reading is as follows: when multiple semantic components are mapped to the same composite input object in downstream task nodes, or have a strong subordinate modification relationship, and splitting them would cause ambiguity or breakage of contextual logic in the downstream model, the system determines that they are continuously transmitted and read.

[0029] Step S3: Construct the predicted semantic solution graph for each output data unit. Based on the topological features and semantic relationship structure of the predicted semantic solution graph, obtain the capability requirement degree of each model capability. Obtain the set of capability-related semantic anchor points corresponding to each model capability in the predicted semantic solution graph of the output data unit. Based on the downstream propagation range of the capability-related semantic anchor point set and the number of anchor points that enter the final output slot after being processed by downstream task nodes, obtain the capability requirement weight of each model capability in the output data unit.

[0030] It should be noted that, in the embodiments of the present invention, the process of constructing the predicted semantic solution graph for each output data unit is as follows: A semantic information extraction technique corresponding to the input data modality of the task node is adopted to obtain the set of input semantic facts; for each output data unit, the expected target semantic facts of the output data unit are formed based on the task elements, delivery information, final output slot, semantic subject, semantic type and generation constraints corresponding to the output data unit. Starting with the expected target semantic facts, the system searches the pre-defined semantic processing rule base for semantic processing rules that take the expected target semantic facts as the conclusion semantic facts. Multiple optional semantic processing rules are constructed into an OR branch, and multiple premise semantic facts corresponding to any one semantic processing rule are constructed into an AND branch. The platform pre-establishes a semantic processing rule base, and each semantic processing rule records the premise semantic facts, semantic processing operation type, conclusion semantic facts, and non-negative search cost. The search cost is pre-determined based on the processing complexity of the semantic processing rule. Premise semantic facts refer to the input conditions or pre-existing data that are available in advance when a specific semantic processing rule is triggered. AO * The target backtracking search algorithm recursively expands each premise semantic fact until the premise semantic facts at the end of the branch can be matched with the semantic facts existing in the input semantic fact set, thus obtaining each feasible solution subgraph; the feasible solution subgraph with the minimum sum of search costs of each semantic processing rule in each feasible solution subgraph is determined as the predicted semantic solution graph of the output data unit.

[0031] Preferably, in one embodiment of the present invention, the method for obtaining the level of capability requirement includes: Based on the topological features and semantic relation structure of the predicted semantic solution graph, multiple model capabilities of the output data unit are extracted, including semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency preservation span, and constraint coupling degree. Preferably, in one embodiment of the present invention, the method for obtaining the semantic aggregation breadth is as follows: extract leaf input semantic facts that can directly match the set of input semantic facts and are not generated by other semantic processing rules from the estimated semantic solution graph, and count the number of leaf input semantic facts obtained after deduplication according to semantic subject, semantic type and semantic value, as the semantic aggregation breadth of the output data unit; The method for obtaining the semantic inference depth is as follows: Based on the longest path analysis technique of directed acyclic graph, the solution paths from the input semantic facts in each leaf to the expected target semantic facts in the estimated semantic solution graph are obtained. The number of non-directly retained semantic processing rules in each solution path is counted, and the maximum value of the number in all solution paths is selected as the semantic inference depth of the output data unit. Among them, the semantic processing operation type of the semantic processing rule is read from the preset semantic processing rule library, and the semantic processing rule whose semantic processing operation type does not belong to the direct retention operation is regarded as the non-directly retained semantic processing rule. The method for obtaining relational coupling density is as follows: The ratio of the number of effective undirected coupling edges in the estimated semantic solution graph to the preset maximum number of undirected coupling edges is used as the relational coupling density of the output data unit. Based on semantic fact parsing technology, entity, attribute, event, and state semantic nodes in the input semantic facts, intermediate semantic facts, and target semantic facts in the estimated semantic solution graph are obtained, and entity relationships, attribute subordination relationships, event participation relationships, temporal relationships, and spatial relationships are extracted. After removing prerequisite dependency edges, generation dependency edges, AND branch relationships, OR branch relationships, source relationships, and field mapping relationships between semantic facts and semantic processing rules, the remaining substantive semantic relationships are converted into undirected coupling edges according to the two semantic nodes they connect. After deduplication, effective undirected coupling edges are obtained. Any two different semantic nodes may form a set of coupling relationships, and the preset maximum number of undirected coupling edges is the number of binary combinations of all semantic nodes. If the number of semantic nodes is less than 2, there is no relational coupling between different semantic nodes, and the relational coupling density of the output data unit is recorded as 0. The method for obtaining cross-modal binding degree is as follows: after aligning the source modal semantic facts and target modal semantic facts corresponding to the cross-modal semantic binding rules in the predicted semantic solution graph with cross-modal entities, the cross-modal binding relationship is obtained; the number of cross-modal binding relationships after deduplication according to the source semantic fact identifier, target semantic fact identifier, and binding attribute type is used as the cross-modal binding degree of the output data unit; the cross-modal semantic binding rule is a semantic processing rule whose semantic processing operation type is cross-modal semantic binding. The method for obtaining the consistency maintenance span is as follows: read the semantic subject identifier, semantic type, and corresponding attribute identifier or state identifier in the output data unit, and record the combination of the above identifiers as the semantic anchor point of the output data unit; according to the downstream receiving field and the final output slot in the task dependency graph, use semantic entity linking technology to identify the downstream output position that references the same semantic anchor point; count the number of task graph edges traversed from the target task node to each reference position, and use the difference between the maximum number of edges and the minimum number of edges as the consistency maintenance span of the output data unit; The method for obtaining the degree of constraint coupling is as follows: Based on the pattern matching technology of the preset semantic processing rule base, the various content constraints, format constraints, style constraints, and prohibitive constraints that simultaneously act on the output data unit are obtained; based on the dependency analysis of the constraint object and preconditions, and the conflict detection based on the constraint attributes, value ranges, and prohibitive relationships, the constraint coupling relationships that act on the same semantic subject and have precondition dependencies or value conflicts are obtained; the ratio of the number of constraint coupling relationships to the preset maximum number of constraints is obtained as the degree of constraint coupling of the output data unit; the preset maximum number of constraints is the number of binary combinations of all constraints; if the number of constraints is less than 2, there is no coupling relationship between multiple constraints, and the degree of constraint coupling of the output data unit is recorded as 0.

[0032] Based on this, the semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency maintenance span, and constraint coupling degree are normalized respectively to represent the capability requirements of each corresponding model in the output data unit.

[0033] It should be noted that, in the embodiments of the present invention, the capabilities of each model are analyzed, and the global absolute historical maximum and minimum values ​​corresponding to each model capability are pre-configured in the system platform for maximum and minimum value normalization. The normalization result is used as the capability requirement level, with a value range of 0-1. The specific means are well known to those skilled in the art and will not be described in detail here.

[0034] The system obtains a set of capability-related semantic anchor points corresponding to each model capability in the predicted semantic solution graph of the output data unit. It should be noted that, in one embodiment of the invention, the platform pre-establishes a correspondence between semantic processing rule types and each model capability. For the first four model capabilities, the corresponding semantic processing rules are extracted from the predicted semantic solution graph of the output data unit according to the correspondence, and the intermediate semantic facts, target semantic facts, and substantive semantic relationships generated by these rules are read. For consistency maintenance span, the semantic subjects, attributes, and states that need to be continuously maintained at the downstream output position are read. For constraint coupling degree, the constraint semantic facts that simultaneously act on the output data unit and their defined target semantic facts are read. The above content is assigned semantic anchor point identifiers, and deduplication is performed according to semantic subject, semantic type, and semantic value. The deduplication result is recorded as the set of capability-related semantic anchor points for the corresponding model capability.

[0035] The downstream propagation range reflects the breadth and cascading impact of the semantic results generated by the model's capabilities in the task chain. The larger the propagation range, the more downstream task nodes will be affected by errors caused by insufficient capabilities, and the greater the capability requirement weight. The number of anchor points entering the final output slot reflects the degree to which the semantic results generated by a certain model capability actually participate in the core determination of the final delivery and the terminal perception. Based on the downstream propagation range of the set of capability-related semantic anchor points and the number of anchor points entering the final output slot after processing by downstream task nodes, the capability requirement weight of each model capability in the output data unit is obtained.

[0036] Preferably, in one embodiment of the present invention, the method for obtaining the capability requirement weight includes: The first step is to take the output field of the node where the output data unit is located as the starting point, obtain the number of corresponding fields in the path tracing along the task dependency graph to each final output slot, and take them as the number of downstream candidate transmission fields; take the anchors in the set of capability association semantic anchors as the propagation source, and perform forward tracing according to the data mapping relationship between adjacent task nodes in the task dependency graph, and take the fields that the propagation source passes through in the downstream task nodes as the affected transmission fields. The ratio of the number of affected propagation fields to the number of downstream candidate propagation fields for each model capability in each output data unit is obtained as the downstream propagation coverage of each model capability in each output data unit.

[0037] It should be noted that the extracted capability-related semantic anchor points are set as data propagation sources and assigned taint tags. Based on the correspondence between input and output fields of adjacent task nodes in the task dependency graph, the downstream task nodes are traced forward along the data flow, starting from the output field where the capability-related semantic anchor point is located. During the forward tracing process, the dependency relationship of data generation within the downstream task nodes is analyzed. If the generated content or constraint parameters of the downstream task nodes use the input field with the taint tag as a precondition, the corresponding downstream output field is determined to be the affected propagation field, and the taint tag is passed to it until all related final output slots are traversed.

[0038] It should be noted that the number of affected transit fields reflects the number of nodes that actually participate in the computation to generate semantic anchors in the model's capability; the number of downstream candidate transit fields reflects the number of all potentially traversed node fields in the task dependency graph of the output data unit; by analyzing the ratio of the number of affected transit fields to the number of downstream candidate fields, the extent to which the deviation of the model's capability can spread in the task chain can be quantified; in particular, if the number of downstream candidate transit fields is 0, the corresponding downstream propagation coverage is recorded as 0.

[0039] The second step is to obtain the number of semantic anchors derived from the set of semantic anchors associated with capabilities into the final output slot, and the ratio of the number of semantic anchors associated with each model capability, as the slot semantic retention ratio of each model capability in each output data unit to each final output slot; and to obtain the average slot semantic retention ratio of each model capability in each output data unit to all final output slots, as the terminal delivery effectiveness of each model capability in each output data unit. It should be noted that capability-related semantic anchors only truly affect the deliverables when they are still represented in the final output slot or used as final generation constraints. Therefore, for the final output slot of the output data unit, direct matching is performed according to the semantic anchor identifier. For derived semantic anchors formed through semantic rewriting, semantic entity linking and semantic similarity matching techniques are used to determine the correspondence. For derived semantic anchors that have undergone modal transformation, cross-modal entity alignment techniques are used to determine the correspondence. Derived semantic anchors that are derived from capability-related semantic anchors and enter the final output slot, and that have a direct inheritance relationship, semantic equivalence relationship, or cross-modal correspondence with the original capability-related semantic anchors, are read. If the number of semantic anchors associated with the capabilities of each model is 0, the corresponding slot semantic retention ratio is recorded as 0.

[0040] Based on this, the higher the slot semantic retention ratio, the more completely the semantic results formed by the model capabilities are retained in the final output slot, and the more likely the improvement of model capabilities is to be actually perceived by end users or downstream systems; the terminal delivery effect reflects the degree to which the diffused semantic deviation enters the final output slot.

[0041] The third step is to obtain the product between the downstream propagation coverage and the terminal delivery effectiveness of each model capability in each output data unit, and use it as the capability requirement weight.

[0042] Based on this, the greater the downstream propagation coverage, the wider the influence of the semantic results formed by each model capability on the subsequent task chain of the output data unit, and the more easily the bias caused by the inadequacy of the corresponding model capability will be inherited by multiple downstream processes; the greater the slot semantic retention ratio, the more completely the semantic results formed by each model capability are retained in each final output slot, and the more likely they are to be actually perceived by the end user or downstream system, and the greater the terminal delivery effect; therefore, the greater the downstream propagation coverage, the greater the terminal delivery effect, the more significant the impact of model capabilities on final delivery, and the greater the weight of capability requirements.

[0043] Step S4: Construct a unified model library and obtain the conditional capability boundary values ​​of each model in each model capability; based on the conditional capability boundary values ​​of each model in each model capability, the capability requirement weights and capability requirement degrees of each model capability in different output data units corresponding to the target task node, obtain the adaptability of each model to the target task node in each model capability.

[0044] Register all available models and build a unified model library, including large language models, image generation models, audio generation models, multimodal understanding models, and vertical models. Each model records its model identifier, model version, input and output modalities, supported task types, and model adaptation interface parameters. The input and output interfaces of each candidate model are encapsulated through a unified adaptation layer to enable each model to support standardized node input fields and node output fields.

[0045] Existing large-scale model evaluations mostly adopt general comprehensive scores or blindly worship parameter size, ignoring the model's specialization and difficulty ceiling. A model may be able to perfectly handle extremely simple format generation with a massive number of parameters, but if a few more derivations or multiple constraints are added, its answer quality will collapse precipitously. Therefore, it is necessary to analyze the boundary values ​​of conditional capabilities.

[0046] Preferably, in one embodiment of the present invention, the method for obtaining the conditional capability boundary value includes: The first step is to obtain the capability requirement of each preset model capability calibration task on each model capability; for any model capability, select a preset number of calibration tasks whose capability requirement is located in each preset range and whose other model capabilities are all less than the preset minimum boundary threshold. It should be noted that, in the embodiments of the present invention, the platform pre-establishes multi-information aggregation calibration tasks, multi-step semantic inference calibration tasks, relational structure processing calibration tasks, cross-modal semantic binding calibration tasks, cross-output consistency maintenance calibration tasks, and multi-constraint collaborative compliance calibration tasks; the preset degree range is set to 0-0.2, 0.2-0.4, 0.4-0.6, 0.6-0.8, and 0.8-1; in order to avoid the influence of accidental factors and to truly reflect the capabilities of the model, the preset quantity is set to 50 based on relevant historical experience; wherein, the preset minimum boundary threshold is set to 0.2.

[0047] The second step is to obtain the model evaluation results after each model performs the corresponding model capability calibration task, and calculate the ratio of the number of calibration tasks with model evaluation results greater than or equal to the preset evaluation threshold to the preset number of calibration tasks, which is used as the capability pass rate of the model capability in each preset range. It should be noted that, in the embodiments of the present invention, the method for obtaining the model evaluation results is as follows: the input data and generation constraints of the calibration task are sent to each model to obtain the output data generated by the model; according to the model capability type corresponding to the calibration task, the corresponding data feature parsing technology is used to extract output features from the output data; the output features are compared with the standard validation set bound to the calibration task, and the proportion of output features that meet the requirements of the standard validation set is calculated as the model evaluation result of the calibration task; in order to ensure that the model that passes the test can achieve the plug-and-play engineering-level robustness standard when facing real tasks of the same difficulty, thereby ensuring the safety and stability of platform-level automated scheduling, the preset evaluation threshold is set to 0.9 based on relevant historical experience; in other embodiments of the present invention, the size of the preset evaluation threshold can be set according to specific circumstances, and is not limited or elaborated here.

[0048] It should be noted that, for multi-information aggregation calibration tasks, semantic slot extraction and slot-level precise matching techniques are used to statistically determine the proportion of necessary semantic units correctly included in the model output; for multi-step semantic derivation calibration tasks, entities, relations, and derivation steps are extracted from the model output to construct a model output derivation relation graph; graph matching techniques are used to match nodes and relation edges between the model output derivation relation graph and a preset standard derivation relation graph, and the proportion of matching overlapping nodes and edges is calculated as the quantity ratio; for relation structure processing calibration tasks, entity relation extraction and relation graph matching techniques are used to obtain relation F1 values; for cross-modal semantic binding calibration tasks, target detection, attribute recognition, action recognition, and cross-modal entity alignment techniques are used to obtain the correct binding ratio; for cross-output consistency maintenance calibration tasks, semantic entity linking or target re-identification techniques are used to determine the same semantic subject in different output positions, and the subject identifier, attribute value, and state value of the semantic subject in different output positions are compared for consistency to obtain the consistency maintenance ratio; for multi-constraint collaborative compliance calibration tasks, rule engines, field integrity verification, and JSON Schema verification techniques are used to obtain the constraint satisfaction ratio. The third step is to take the upper limit of the preset range of continuous capability pass rates that are greater than or equal to the preset pass threshold as the conditional capability boundary value of the model in the corresponding model capability, in order of the preset range of capability from low to high.

[0049] It should be noted that, in one embodiment of the present invention, the average pass rate of the industry benchmark model obtained through historical testing under the calibration task of the corresponding preset difficulty range is calibrated as the preset pass threshold; the size of the preset pass threshold is set to 0.85 based on relevant historical experience; in other embodiments of the present invention, the size of the preset pass threshold can be set according to specific circumstances, and is not limited or elaborated here; if the pass rate of the model in all difficulty level ranges of the corresponding model capability is lower than the preset pass threshold, the conditional capability boundary value is recorded as the preset minimum boundary value, such as 0.01.

[0050] When the same model executes a target task node once, its same model capability is applied to multiple output data units within that target task node. The higher capabilities a model possesses to meet the needs of high-demand output data units will be repeatedly identified as capability redundancy in multiple low-demand output data units, failing to reflect the shared usage of the same model capability across the entire target task node. Based on the conditional capability boundary values ​​of each model for each capability, the capability requirement weights and degree of capability requirement for each model capability in different output data units corresponding to the target task node, the adaptability of each model to the various model capabilities within the target task node is obtained.

[0051] Preferably, in one embodiment of the present invention, the method for obtaining the fit includes: The first step is to select the minimum value between the conditional capability boundary value of each model in each model capability and the capability requirement degree of the corresponding model capability in each output data unit for each model, and obtain the cumulative value of the product between the minimum value and the capability requirement weight of the corresponding model capability in each output data unit, which is used as the effective capability overlap of each model for the target task node in each model capability. It should be noted that when the conditional capability boundary value of the model is lower than the capability requirement of the output data unit, the model can only provide capability within its conditional capability boundary value range; when the conditional capability boundary value of the model is higher than the capability requirement of the output data unit, the capability exceeding the actual requirement of the output data unit will not further improve the output data unit. Therefore, the minimum value between the conditional capability boundary value and the capability requirement is selected to analyze the effective capability of the model; the larger the minimum value, the greater the capability requirement weight, reflecting the higher the degree of overlap between the capability provided by the model and the actual capability requirement of the output data unit.

[0052] In one embodiment of the present invention, the formula for the effective capability overlap is expressed as: ;in, Indicates the first Each model targets the task node. The Middle Effective capability overlap in the item model capabilities; Indicates the first The first output data unit The capability requirement weights of the item model; Indicates the first The model in the first Conditional capability boundary values ​​for the capability of the item model; Indicates the first The first output data unit The level of capability required for the item model; This represents a function that takes the minimum value. Indicates the target task node The number of output data units.

[0053] The second step is to obtain the cumulative value of the product between the capability requirement weight and capability requirement degree of each model capability in all output data units, which is used as the weighted total capability requirement of the target task node for each model capability; obtain the cumulative value of the capability requirement weight of each model capability in each output data unit, calculate the product between the cumulative value and the conditional capability boundary value of each model for each model capability, which is used as the weighted total capability supply. In one embodiment of the present invention, the formula for the total weighted capacity requirement is expressed as: ;in, Indicates the target task node In the The total weighted capability requirement of the model; Indicates the first The first output data unit The capability requirement weights of the item model; Indicates the first The first output data unit The level of capability required for the item model; Indicates the target task node The number of output data units.

[0054] The formula for the total weighted capacity supply is expressed as: ;in, Indicates the first Each model targets the task node. The Middle The total weighted capacity supply based on the model's capabilities; Indicates the first The first output data unit The capability requirement weights of the item model; Indicates the first The model in the first Conditional capability boundary values ​​for the capability of the item model; Indicates the target task node The number of output data units.

[0055] The third step is to obtain the fit degree of each model to the target task node in terms of the effective capability overlap, total weighted capability demand, and total weighted capability supply for each model. The effective capability overlap is positively correlated with the fit degree, while the total weighted capability demand and total weighted capability supply are negatively correlated with the fit degree.

[0056] It should be noted that the effective capability overlap reflects the total weighted capability actually used by the target task node in terms of model capabilities. The larger the effective capability overlap, the larger the total weighted capability that can be actually used by the target task node. The total weighted capability demand reflects the overall demand of all output data units in the target task node for each model capability. The larger the total weighted capability demand, the larger it is than the effective capability overlap, and the total weighted capability that can be actually used cannot meet the target task node's demand for the corresponding model capability, resulting in a smaller fit. The total weighted capability supply reflects the capability provided by the model for each model capability. The larger the total weighted capability supply, the larger it is than the effective capability overlap, and the larger the capability provided by the model for each model capability is than the weighted capability that can be actually used, resulting in model capability redundancy and a smaller fit. Therefore, the effective capability overlap is positively correlated with the fit, while the total weighted capability demand and the total weighted capability supply are both negatively correlated with the fit.

[0057] In one embodiment of the present invention, the ratio of effective capability overlap to total weighted capability demand is calculated as capability demand coverage; the ratio of effective capability overlap to total weighted capability supply is calculated as model capability effective absorption; and the product of capability demand coverage and model capability effective absorption is obtained as the fit of each model to the target task node in terms of various model capabilities.

[0058] Based on this, capability requirement coverage reflects the degree to which the weighted actual requirements of the target task node for each model capability are covered by the conditional capability boundary values ​​of the model. The greater the capability requirement coverage, the more model capability requirements the model can meet. Model capability effective absorption reflects the degree to which the capabilities provided by the model for each model capability are actually utilized by the target task node. The greater the capability effective absorption, the smaller the redundancy of the model capabilities.

[0059] Preferably, in one embodiment of the present invention, the method for obtaining the overall fit includes: The total weighted capability requirement is normalized. Based on the normalized result of the target task node in all model capabilities, the fitness of each model to the target task node in the corresponding model capability is weighted and summed to obtain the overall fitness of each model to the target task node.

[0060] It should be noted that, in the embodiments of the present invention, the total weighted capability requirement is normalized by obtaining the ratio between the total weighted capability requirement of the target task node for each model capability and the sum of the total weighted capability requirements of all model capabilities, that is, normalizing the total weighted capability requirement.

[0061] The weighted summation method is as follows: the normalized result is used as the requirement weight of each model capability, and the product of the requirement weights and fitness of all model capabilities is accumulated and summed as the overall fitness of each model to the target task node.

[0062] Based on this, the overall fit reflects the model's optimal utilization efficiency of the system's computing resources while ensuring the quality of task delivery.

[0063] Step S5: Based on the overall adaptability of each model to the target task node, select the model to be called to execute the target task node.

[0064] Overall fit is used to characterize the precise matching between the model's capability supply and the actual needs of the task. It comprehensively reflects the degree to which the model avoids redundant computing power and wasted calling costs while meeting the requirements of downstream nodes and final delivery, and selects the models to be called to execute the target task nodes.

[0065] Preferably, in one embodiment of the present invention, the method for selecting the model to be invoked to execute the target task node includes: The model with the highest overall fit among all models is selected as the model to be called.

[0066] Based on this, the node input data and corresponding output constraints of the target task node are sent to the model to be called to obtain the output data units of the target task node. According to the correspondence of the input and output fields of the task dependency graph, the data is sent to the downstream task node, which helps to reduce insufficient or redundant capabilities and improve the scheduling accuracy and resource utilization of multimodal AIGC tasks.

[0067] In summary, this invention takes any task node in the task dependency graph as the target task node, divides the obtained node output fields into multiple output data units, constructs a predicted semantic solution graph for each output data unit, and obtains the capability requirement degree of each model capability. Based on the downstream propagation range of the capability-related semantic anchor set corresponding to each model capability, and the number of anchors entering the final output slot after processing by downstream task nodes, the capability requirement weight of each model capability in the output data unit is obtained. Combined with the conditional capability boundary values ​​of each model on each model capability, the overall adaptability of each model to the target task node is obtained, and the models to be called for executing the target task node are selected. This invention accurately analyzes the overall adaptability of each model to the target task node, improving the effectiveness of model integration and scheduling.

[0068] This invention also proposes a unified integrated scheduling platform for a multimodal AIGC large model, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of a unified integrated scheduling method for a multimodal AIGC large model.

[0069] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0070] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A unified integrated scheduling method for a multimodal AIGC large model, characterized in that, The method includes: Obtain the task elements and delivery information of the user's task instructions, and construct a task dependency graph containing multiple task nodes; Each task element is matched with a preset output template library to obtain the final output slot of the matching output template; any task node in the task dependency graph is taken as the target task node, and the obtained node output field is divided into multiple output data units based on task dependency relationship and semantic independence. Construct a predicted semantic solution graph for each output data unit. Based on the topological features and semantic relationship structure of the predicted semantic solution graph, obtain the capability requirement degree of each model capability. Obtain the set of capability-related semantic anchor points corresponding to each model capability in the predicted semantic solution graph of the output data unit. Based on the downstream propagation range of the capability-related semantic anchor point set and the number of anchor points that enter the final output slot after being processed by downstream task nodes, obtain the capability requirement weight of each model capability in the output data unit. Construct a unified model library and obtain the conditional capability boundary values ​​of each model in each model capability. Based on the conditional capability boundary values ​​of each model in each model capability, the capability requirement weights and capability requirement degrees of each model capability in different output data units corresponding to the target task node, obtain the adaptability of each model to the target task node in each model capability, and obtain the overall adaptability of each model to the target task node. Based on the overall adaptability of each model to the target task node, select the model to be invoked to execute the target task node.

2. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The method for obtaining the final output slot includes: Match the task elements with the task type, target output modality, and target delivery form corresponding to each output template in the preset output template library to determine the matching output template; Read the structured fields and output positions from the matching output template, extract the tag attributes of each structured field, and bind them to their corresponding output positions as the final output slots.

3. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The method for acquiring the output data unit includes: For any target task node in the task dependency graph, after parsing the node output field of the target task node, the node output field of the target task node is split according to the semantic components that its downstream task nodes can read independently. The semantic components include semantic subjects, attributes, relationships, events, states or constraints. If multiple semantic components in a node's output field are continuously read and passed to downstream task nodes, the entire field is treated as a single output data unit.

4. The unified integrated scheduling method for a multimodal AIGC large model according to claim 3, characterized in that, The methods for obtaining the required level of capability include: Based on the topological features and semantic relation structure of the predicted semantic solution graph, multiple model capabilities of the output data unit are extracted, including semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency preservation span, and constraint coupling degree. The semantic aggregation breadth, semantic derivation depth, relation coupling density, cross-modal binding degree, consistency maintenance span, and constraint coupling degree are normalized respectively to represent the capability requirements of each model in the output data unit.

5. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The methods for obtaining the capability requirement weights include: Starting from the output field of the node where the output data unit is located, the number of corresponding fields in the path tracing along the task dependency graph to each final output slot is obtained as the number of downstream candidate transmission fields; taking the anchors in the set of capability-related semantic anchors as the propagation source, forward tracing is performed according to the data mapping relationship between adjacent task nodes in the task dependency graph, and the fields traversed by the propagation source in the downstream task nodes are taken as the affected transmission fields. The ratio of the number of affected propagation fields to the number of downstream candidate propagation fields for each model capability in each output data unit is obtained as the downstream propagation coverage of each model capability in each output data unit. The ratio of the number of semantic anchors derived from the set of semantic anchors associated with capabilities into the final output slot to the number of semantic anchors associated with each model capability is obtained as the slot semantic retention ratio of each model capability in each output data unit for each final output slot; the average slot semantic retention ratio of each model capability in each output data unit for all final output slots is obtained as the terminal delivery effectiveness of each model capability in each output data unit. The product of the downstream propagation coverage and terminal delivery effectiveness of each model capability in each output data unit is obtained as the capability requirement weight.

6. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The method for obtaining the conditional capability boundary value includes: Obtain the capability requirement of each preset model capability calibration task on each model capability; for any model capability, select a preset number of calibration tasks whose capability requirement is located within each preset range and whose other model capabilities are all less than the preset minimum boundary threshold. The model evaluation results are obtained after each model performs the corresponding model capability calibration task. The ratio of the number of calibration tasks with model evaluation results greater than or equal to the preset evaluation threshold to the preset number of calibration tasks is calculated as the capability pass rate of the model within each preset range. According to the preset degree range from low to high, the upper limit of the preset degree range in which the continuous pass rate of the capability is greater than or equal to the preset pass threshold is used as the conditional capability boundary value of the model on the corresponding model capability.

7. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The method for obtaining the fit includes: For the target task node, select the minimum value between the conditional capability boundary value of each model in each model capability and the capability requirement degree of the corresponding model capability in each output data unit, and obtain the cumulative value of the product between the minimum value and the capability requirement weight of the corresponding model capability in each output data unit, as the effective capability overlap of each model for the target task node in each model capability. The cumulative value of the product between the capability requirement weight and capability requirement degree of each model capability in each output data unit is obtained as the weighted total capability requirement of the target task node for each model capability; the cumulative value of the capability requirement weight of each model capability in each output data unit is obtained, and the product between the cumulative value and the conditional capability boundary value of each model for each model capability is calculated as the weighted total capability supply. Based on the effective capability overlap, total weighted capability demand, and total weighted capability supply of each model for the target task node, the fit is obtained. The effective capability overlap is positively correlated with the fit, while the total weighted capability demand and total weighted capability supply are both negatively correlated with the fit.

8. The unified integrated scheduling method for a multimodal AIGC large model according to claim 7, characterized in that, The method for obtaining the overall fit includes: The total weighted capability requirement is normalized, and the fitness is weighted and summed based on the normalized results of the target task node's capabilities across all models, which serves as the overall fitness of each model for the target task node.

9. The unified integrated scheduling method for a multimodal AIGC large model according to claim 1, characterized in that, The method for selecting the models to be invoked to execute the target task nodes includes: The model with the highest overall fit among all models is selected as the model to be called.

10. A unified integrated scheduling platform for a multimodal AIGC large model, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the unified integrated scheduling method for a multimodal AIGC large model as described in any one of claims 1 to 9.