Multi-mode intelligent agent scheduling method and system

By combining cross-modal feature encoding and dynamic priority scheduling with context token passing and circuit breaking mechanisms, the problems of low service quality and poor reliability in multimodal proxy methods are solved, and efficient and reliable multimodal task execution is achieved.

CN122019102APending Publication Date: 2026-05-12GUANGZHOU TAIDONG TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
GUANGZHOU TAIDONG TECH CO LTD
Filing Date
2026-02-06
Publication Date
2026-05-12

AI Technical Summary

Technical Problem

Existing multimodal proxy methods suffer from low service quality and poor reliability in resource allocation, task execution strategies, and emergency response strategies. In particular, they suffer from severe latency during sudden requests, loss of cross-modal task context association, and cascading crashes caused by single-modal proxy failures.

Method used

A three-level collaborative scheduling framework of perception-decision-execution is constructed by using cross-modal feature encoding to generate semantic complexity vectors, combining resource demand prediction and dynamic priority scheduling, passing context tokens across agent shared memory to maintain semantic coherence, and monitoring modal health in real time to execute circuit breaking or self-healing operations.

Benefits of technology

It improves the service quality and reliability of multimodal tasks, reduces latency and computational resource waste, enhances system resilience and fault recovery capabilities, and improves task execution accuracy and throughput.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN122019102A_ABST
    Figure CN122019102A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of artificial intelligence agency, in particular to a multi-modal intelligent agency scheduling method and system.The method comprises the steps that in response to receiving of a multi-modal task, cross-modal feature coding is conducted on the multi-modal task, and a semantic complexity vector is output; inputting the semantic complexity vector into a preset resource demand prediction algorithm, and calculating to obtain a resource demand prediction result; thirdly, analyzing a service level protocol carried by the multi-modal task, calculating a dynamic priority of the multi-modal task according to the current load state of the system, the service level protocol and a resource demand prediction result, and generating scheduling instructions for different modal agents according to the dynamic priority; and finally, according to the scheduling instruction, triggering a corresponding modal agent to execute a reasoning task. In the execution period of the task, a context token is transmitted through a cross-agent shared memory so as to maintain semantic coherence, and the modal health degree is monitored in real time so as to execute fusing or self-healing operation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence agent technology. More specifically, this invention relates to a multimodal intelligent agent scheduling method and system. Background Technology

[0002] Against the backdrop of the explosive growth of AIGC (Artificial Intelligence Generated Content) and multimodal AI applications, the intelligent agents deployed by enterprises have been upgraded from single-modality to a collaborative framework that integrates vision, hearing, and semantics. This multimodal collaborative framework can support multiple application areas such as video generation, CT image analysis, or customer service graphic interaction.

[0003] Existing multimodal proxy methods mainly adopt the following approaches to achieve applications in the above-mentioned fields: Firstly, regarding resource allocation, existing solutions mainly pre-allocate fixed computing resources (such as GPU slices) for different modal tasks to execute inference tasks. This approach cannot detect dynamic changes in tasks, and when a sudden increase in requests occurs, it can cause a 300% delay in text customer service time. Taking the NVIDIA Triton inference server as an example, when 100 "1-second speech-to-text" tasks and 1 "4K medical image segmentation" task arrive simultaneously, its average system latency will increase from 200ms to 4.7s, which is sufficient to demonstrate that the above solutions have the problem of service quality degradation due to modal resource contention.

[0004] Secondly, regarding the execution strategy of tasks, multimodal tasks are broken down into multiple relatively independent sub-tasks, and the cross-modal contextual association is lost. For example, in the intelligent customer service scenario, if a user uploads a "faulty photo with an invoice", although the image recognition agent has identified "device model A", the text agent cannot obtain its context and will still ask the customer to enter the device model, which greatly reduces the task completion rate and efficiency, resulting in poor service quality for the user.

[0005] Third, regarding emergency strategies for agent timeouts, existing solutions typically build independent processing pipelines for each modality. When a single-modality agent fails, it can easily trigger cascading crashes. For example, in 2024, a bank's customer service system experienced a timeout due to a voice recognition agent, which caused other cross-modal task queues to become blocked, resulting in a 15-minute system downtime and economic losses exceeding 23 million yuan.

[0006] In summary, existing agency service methods suffer from low quality and poor reliability. Summary of the Invention

[0007] To address the aforementioned technical problems of low quality and poor reliability, this invention discloses a multimodal intelligent agent scheduling method and system.

[0008] In a first aspect, the present invention discloses a multimodal intelligent agent scheduling method, comprising: S10: In response to the reception of the multimodal task, perform cross-modal feature encoding on the multimodal task and output a semantic complexity vector; and input the semantic complexity vector into the preset resource demand prediction algorithm to calculate the resource demand prediction result; S20: Parse the service level agreement carried by the multimodal task, and calculate the dynamic priority of the multimodal task based on the current system load status, service level agreement and resource demand prediction results. Then generate scheduling instructions for different modal proxies based on the dynamic priority. S30: According to the scheduling instructions, trigger the corresponding modal agent to execute the inference task; During any of the execution processes of steps S10-S30, the method further includes: Context tokens are passed across agent shared memory to maintain semantic coherence, and modal health is monitored in real time to perform circuit breaking or self-healing operations.

[0009] Preferably, in step S10, cross-modal feature encoding is performed on the multimodal task to output a semantic complexity vector, specifically as follows: A preset modal encoder is used to extract feature vectors from video, audio and text modalities respectively, and the feature vectors are concatenated and fused into a unified fused feature; The fused features are mapped into a multi-dimensional complexity vector using a pre-trained weight matrix.

[0010] Preferably, the current system load status includes at least CPU utilization, GPU utilization, and queue utilization.

[0011] Preferably, step S20 includes: Parse the service level agreement to obtain the business criticality, maximum tolerable latency, and minimum accuracy requirements for multimodal tasks; The global load index is calculated based on CPU utilization, GPU utilization, and queue utilization. The business criticality, maximum tolerable latency, minimum accuracy requirement, and global load index are incorporated into a preset priority algorithm to calculate dynamic priority. When the global load index is higher than the load threshold and the dynamic priority is higher than the priority execution threshold, a preemptive scheduling instruction is generated to prioritize the execution of multimodal tasks.

[0012] The preferred priority algorithm is as follows:

[0013] In the formula, Indicates task priority. Indicates the degree of business criticality. Indicates the maximum tolerable delay. This indicates the waiting time or processing time for the task. This represents the global load index. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and exp() represents the exponential function.

[0014] Preferably, context tokens are passed across agent shared memory, specifically as follows: During the multimodal task reception phase, a corresponding context token is generated. The context token contains a semantic embedding vector, a time-to-live, and a set of dependent tasks. When the modal agent triggered in step S30 starts execution, it reads the context token from the shared memory and calculates the similarity between the current task features and the semantic embedding vector to extract the associated context. After the agent completes the inference, it uses the current output to perform a weighted update on the semantic embedding vector in the context token, and writes the updated token back to shared memory.

[0015] Preferably, the steps for maintaining semantic coherence are as follows: Calculate the semantic drift between the initial semantic embedding vector of the context token and the weighted updated semantic embedding vector; When the semantic drift exceeds a preset safety threshold, a correction mechanism is triggered, which corrects the current semantic embedding vector by introducing a correction factor.

[0016] Preferably, the circuit breaker is triggered by real-time monitoring of modal health, specifically as follows: The modal health of the corresponding modal agent is calculated by weighting and summing the inference success rate, latency bias and inference error rate of different modal agents respectively. If the modal health of any modal agent falls below the circuit breaker threshold, the circuit breaker is triggered to cut off the traffic input to that modal agent.

[0017] The preferred self-healing procedure is as follows: If the business criticality is below the criticality threshold, a preset lightweight model is invoked to take over the traffic input of the circuit breaker modal proxy; otherwise, the manual channel is switched to take over.

[0018] Secondly, the present invention discloses a multimodal intelligent agent scheduling system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal intelligent agent scheduling method described in the first aspect is implemented.

[0019] The beneficial effects of this invention are as follows: This invention constructs a three-level collaborative multimodal scheduling framework of "perception-decision-execution." Through cross-modal feature encoding and resource demand prediction, it transforms abstract tasks into scheduling-supporting complexity vectors and calculates resource demand prediction results to support dynamic priority calculations. This allows for intelligent scheduling based on task urgency during actual task execution, thus solving the service quality degradation problem caused by modal resource contention in existing technologies. Furthermore, this invention maintains semantic coherence in multimodal task execution through semantic vector alignment and context token passing, resolving the context breakage problem in existing technologies. In addition, this invention features fault circuit interruption and automatic recovery capabilities, preventing system-level crashes caused by single points of failure. Compared to existing technologies, tasks executed using this invention offer higher service quality and stronger reliability. Attached Figure Description

[0020] The above and other objects, features, and advantages of exemplary embodiments of the present invention will become readily apparent upon reading the following detailed description with reference to the accompanying drawings. In the drawings, several embodiments of the invention are illustrated by way of example and not limitation, and like or corresponding reference numerals denote like or corresponding parts, wherein: Figure 1 This is a flowchart of the multimodal intelligent agent scheduling method in Embodiment 1 of the present invention; Figure 2 This is a schematic diagram of the structure of the multimodal intelligent agent scheduling system in Embodiment 2 of the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0023] Example 1 like Figure 1 As shown, this embodiment discloses a multimodal intelligent agent scheduling method, including: S10: In response to the reception of the multimodal task, perform cross-modal feature encoding on the multimodal task and output a semantic complexity vector; then input the semantic complexity vector into the preset resource demand prediction algorithm to calculate the resource demand prediction result.

[0024] In this embodiment, the multimodal task mainly includes three modalities: image, audio, and text. Specifically, the encoding process in step S10 is as follows: S11: Use a preset modal encoder to extract feature vectors from video, audio and text modalities respectively, and then concatenate and fuse the feature vectors into a unified fused feature.

[0025] Specifically, upon receiving a multimodal task, the modal encoder first defines the multimodal task as follows:

[0026] In the definition, Vector representation of multimodal tasks. Indicates video modality, Indicates audio modality, Represents the text modality.

[0027] Then, feature encoding is performed using the three built-in models of the modal encoder.

[0028] For visual feature encoding, this embodiment uses the CLIP (Contrastive Language-Image Pre-training) model, and the specific encoding formula is as follows:

[0029] In the formula, Represents visual feature vectors. This represents the feature encoding function of the CLIP model, with dimensions taking [value missing]. .

[0030] For audio feature encoding, this embodiment uses the VGGish (Visual Geometry Group Audio Embedding) model, and the specific encoding formula is as follows:

[0031] In the formula, Represents the audio feature vector. This represents the feature encoding function of the VGGish model, with dimensions taking [value missing]. .

[0032] For text feature encoding, this embodiment uses the BERT (Bidirectional Encoder Representations from Transformers) model, and the specific encoding formula is as follows:

[0033] In the formula, Represents the text feature vector. This represents the feature encoding function of the BERT model, with dimensions taking [value missing]. .

[0034] The three encoding models can be concurrently encoded in the modal encoder, and after all models have completed encoding, the final feature fusion is performed. The specific fusion expression is as follows:

[0035] In the formula, Represents the fused feature vector. This represents the batch normalization function. This represents the concatenation function.

[0036] S12: The fused features are mapped into a multi-dimensional complexity vector using the pre-trained weight matrix.

[0037] Specifically, the expression for the above complexity vector is:

[0038] In the formula, the complexity vector These are used to characterize CPU, GPU, memory, and latency requirements or utilization, respectively; among them, The complexity mapping weight matrix is ​​represented. Represents the bias vector; , This represents the total number of columns in the concatenated and merged vector.

[0039] The above technical solution can compress and map 1920-dimensional abstract features into a weight matrix of four dimensions: CPU, GPU, memory, and latency.

[0040] Furthermore, the resource demand forecasting algorithm in step S10 is as follows:

[0041] In the formula, This indicates the predicted number of CPU cores required. This indicates the percentage of GPU computing power required for the prediction; This represents the memory usage required for the prediction. It should be noted that the above parameter matrix can be calculated through regression analysis of historical tasks, and its resource error must be less than 8.7%.

[0042] S20: Parse the service level agreement carried by the multimodal task, and calculate the dynamic priority of the multimodal task based on the current system load status, service level agreement and resource demand prediction results. Then generate scheduling instructions for different modal agents based on the dynamic priority.

[0043] In this embodiment, the Service Level Agreement (SLA) is metadata attached to each multimodal task, telling the scheduler "what kind of treatment this task must receive." The current system load status includes at least CPU utilization, GPU utilization, and queue utilization.

[0044] Furthermore, step S20 above includes: S21: Parse the service level agreement to obtain the business criticality, maximum tolerable latency, and minimum accuracy requirements for multimodal tasks.

[0045] In this embodiment, when publishing a multimodal task, the user needs to make a requirement declaration, and these specific declarations are encapsulated in the Service Level Agreement (SLA). For ease of quantification, the maximum tolerable latency is defined as... The minimum accuracy requirement mentioned above is defined as Define the above business criticality as .

[0046] S22: The global load index is calculated based on CPU utilization, GPU utilization, and queue utilization.

[0047] Specifically, the supporting hardware system collects the agent pool status in real time:

[0048] In the formula, Indicates the first CPU utilization of each proxy node Indicates the first GPU utilization of each proxy node Indicates the first Queue occupancy rate of each proxy node This indicates the total number of proxy nodes.

[0049] For calculating the global load index, this embodiment provides the following algorithm:

[0050] In the formula, This represents the global load index. This indicates the upper limit of CPU computing resources. This indicates the upper limit of GPU computing resources. This indicates the upper limit of the queue length.

[0051] S23: The business criticality, maximum tolerable latency, minimum accuracy requirement, and global load index are incorporated into a preset priority algorithm to calculate the dynamic priority.

[0052] The aforementioned priority algorithm is as follows:

[0053] In the formula, Indicates task priority. Indicates the degree of business criticality. Indicates the maximum tolerable delay. This indicates the waiting time or processing time for the task. This represents the global load index. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and exp() represents the exponential function.

[0054] Preferably, the first weight coefficient can be 0.4-0.6, the second weight coefficient can be 0.2-0.4, and the third weight coefficient can be 0.1-0.3. The sum of the three must be equal to 1. The three can be determined by model training.

[0055] S24: When the global load index is higher than the load threshold and the dynamic priority is higher than the priority execution threshold, a preemptive scheduling instruction is triggered to prioritize the execution of multimodal tasks.

[0056] Preferably, the load threshold can be set to 0.8-0.9, and the priority execution threshold can be set to 0.65-0.75.

[0057] It should be noted that when a preemptive scheduling instruction is issued, the multimodal task carrying that preemptive scheduling instruction will be executed first.

[0058] S30: According to the scheduling instructions, trigger the corresponding modal agent to execute the inference task.

[0059] For example, when the multimodal task of "generating a general medical review document" is located at the front of the execution queue of the task execution queue of the system supporting the method of this embodiment, while the task of "generating an emergency CT diagnostic image" is located at the back of the execution queue, this method will automatically generate a preemptive scheduling instruction for the emergency task to enable the emergency task to be executed first.

[0060] In any of the above steps S10-S30, the method of this embodiment further includes: S400: Maintains semantic coherence by passing context tokens across agent shared memory and monitors modal health in real time to perform circuit breaking or self-healing operations.

[0061] Specifically, passing context tokens across agent shared memory involves: S401: Generate a corresponding context token during the multimodal task reception phase. The context token contains a semantic embedding vector, a time-to-live, and a set of dependent tasks.

[0062] The definition of a context token is as follows:

[0063] In the formula, Represents a context token; Represents a semantic embedding vector. Indicates survival time. This indicates a set of dependent tasks.

[0064] S402: When the modal agent triggered in step S30 starts to execute, the context token is read from the shared memory and the similarity between the current task features and the semantic embedding vector is calculated to extract the associated context.

[0065] The extraction relation for extracting the associated context is as follows:

[0066] In the above condition determination, This indicates the associated context, and cos represents the cosine similarity algorithm. Indicates the characteristics of the current task. This represents the similarity threshold.

[0067] S403: After the agent completes the inference, the semantic embedding vector in the context token is updated with weights using the current output result, and the updated token is written back to shared memory.

[0068] For example, the semantic embedding vector after state update can be represented as:

[0069] In the formula, This represents the semantic embedding vector after the state update. This indicates the current output result.

[0070] Furthermore, the step of maintaining semantic coherence in step S400 above specifically includes: S404: Calculate the semantic drift between the initial semantic embedding vector of the context token and the weighted updated semantic embedding vector.

[0071] The formula for calculating semantic drift is as follows:

[0072] In the formula, Indicates semantic drift degree, This represents the initial semantic embedding vector. This represents the weighted updated semantic embedding vector.

[0073] S405: When the semantic drift exceeds the preset safety threshold, a correction mechanism is triggered, which corrects the current semantic embedding vector by introducing a correction factor.

[0074] Specifically, the callback correction algorithm is as follows:

[0075] In the formula, This represents the current semantic embedding vector after the callback correction; The value represents the correction coefficient, which is preferably 0.3 in this embodiment.

[0076] Through steps S404-S405 above, the method of this embodiment supports context consistency verification. When the context offset is too large, a rollback correction will be performed to ensure that the final result generation is on topic. Compared with the 76.4% accuracy of traditional technology, the above scheme can improve the execution accuracy of cross-modal tasks to 92.7%.

[0077] Furthermore, the specific steps in step S400 above for real-time monitoring of modal health to execute circuit breaker are as follows: S406: Calculate the modal health of the corresponding modal agent by weighted summation of the inference success rate, latency bias, and inference error rate for different modal agents.

[0078] Specifically, the formula for calculating the above modal health is as follows:

[0079] In the formula, , and This represents the health weighting coefficient. Indicates the success rate of reasoning. Indicates the preset target delay. Indicates delay deviation, This represents the reasoning error rate.

[0080] Preferred, .

[0081] S407: If the modal health of any modal agent is lower than the circuit breaker threshold, trigger the circuit breaker to cut off the traffic input to that modal agent.

[0082] It should be noted that the conditions for triggering the circuit breaker mentioned above can also be added using Boolean operations.

[0083] Furthermore, the specific implementation steps of the self-healing action in step S400 above are as follows: S408: If the business criticality is lower than the criticality threshold, call the preset lightweight model to take over the traffic input of the circuit breaker modal proxy; otherwise, switch to manual channel to take over.

[0084] It should be noted that, after the task takeover is completed, there are three main strategies for achieving automatic recovery in this embodiment: restarting the agent, hot reloading the model, and switching traffic back after a certain period of time.

[0085] Through the above steps S10-S30 and S400, the method of this embodiment has at least the following beneficial effects: First, the service quality satisfaction rate for multimodal tasks increased from 63.3% to 99.9%, and the latency during task execution decreased from 2.3s to 380ms, which is sufficient to show that the service quality has been significantly improved.

[0086] Secondly, the computing resource waste rate decreased from 37.2% to 9.8%, and the throughput of tasks supported by a single cluster increased by 4.7 times. Furthermore, the response time to burst traffic was shortened from 8.2 minutes to 12 seconds, and the cost of elastic scaling down was reduced by 63%. This clearly demonstrates that the method in this embodiment achieves effective optimization of resource utilization.

[0087] Third, by adding a fuse mechanism, the cascading failure rate is reduced from 23% to 0.2%, effectively overcoming the unreliability problem in existing technologies.

[0088] Fourth, it has better cross-domain applicability, demonstrating good multimodal understanding and analysis capabilities in fields such as healthcare, finance, and e-commerce, and producing better output results.

[0089] Example 2 like Figure 2 As shown, the present invention discloses a multimodal intelligent agent scheduling system, including a processor and a memory. The memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal intelligent agent scheduling method described in the first aspect is implemented.

[0090] The system in this embodiment also includes other components well known to those skilled in the art, such as communication interfaces. Their settings and functions are known in the art, and therefore will not be described in detail here.

[0091] In this invention, the aforementioned memory can be any tangible medium containing or storing a program that can be used or combined with an instruction execution system, apparatus, or device. For example, a computer-readable storage medium can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc., or any other medium that can be used to store desired information and can be accessed by an application, module, or both. Any such computer storage medium can be part of a device or accessible to or connected to a device. Any application or module described in this invention can be implemented using computer-readable / executable instructions that can be stored or otherwise maintained by such a computer-readable medium.

[0092] In the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

[0093] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A multimodal intelligent agent scheduling method, characterized in that, include: S10: In response to the reception of the multimodal task, perform cross-modal feature encoding on the multimodal task and output a semantic complexity vector; The semantic complexity vector is then input into a preset resource demand prediction algorithm to calculate the resource demand prediction result. S20: Parse the service level agreement carried by the multimodal task, and calculate the dynamic priority of the multimodal task according to the current system load status, the service level agreement and the resource demand prediction results, and then generate scheduling instructions for different modal proxies according to the dynamic priority. S30: According to the scheduling instruction, trigger the corresponding modal agent to execute the inference task; During any of the execution processes of steps S10-S30, the method further includes: Context tokens are passed across agent shared memory to maintain semantic coherence, and modal health is monitored in real time to perform circuit breaking or self-healing operations.

2. The multimodal intelligent agent scheduling method according to claim 1, characterized in that, In step S10, cross-modal feature encoding is performed on the multimodal task to output a semantic complexity vector, specifically as follows: A preset modal encoder is used to extract feature vectors from video, audio and text modalities respectively, and the feature vectors are concatenated and fused into a unified fused feature; The fused features are mapped into a multi-dimensional complexity vector using a pre-trained weight matrix.

3. The multimodal intelligent agent scheduling method according to claim 1, characterized in that, The current system load status includes at least CPU utilization, GPU utilization, and queue utilization.

4. The multimodal intelligent agent scheduling method according to claim 3, characterized in that, Step S20 includes: The service level agreement is parsed to obtain the business criticality, maximum tolerable latency, and minimum accuracy requirements of the multimodal task; The global load index is calculated based on the CPU utilization rate, the GPU utilization rate, and the queue utilization rate. The business criticality, the maximum tolerable latency, the minimum accuracy requirement, and the global load index are placed into a preset priority algorithm to calculate the dynamic priority. When the global load index is higher than the load threshold and the dynamic priority is higher than the priority execution threshold, a preemptive scheduling instruction is generated to prioritize the execution of the multimodal task.

5. The multimodal intelligent agent scheduling method according to claim 4, characterized in that, The priority algorithm is as follows: In the formula, Indicates task priority. Indicates the degree of business criticality. Indicates the maximum tolerable delay. This indicates the waiting time or processing time for the task. This represents the global load index. This represents the first weighting coefficient. This represents the second weighting coefficient. This represents the third weighting coefficient, and exp() represents the exponential function.

6. The multimodal intelligent agent scheduling method according to claim 1, characterized in that, Passing context tokens across agent shared memory is specifically as follows: During the multimodal task reception phase, a corresponding context token is generated, which includes a semantic embedding vector, a time-to-live, and a set of dependent tasks. When the modal agent triggered in step S30 starts execution, the context token is read from shared memory, and the similarity between the current task features and the semantic embedding vector is calculated to extract the associated context; After the agent completes the inference, it uses the current output to perform a weighted update on the semantic embedding vector in the context token, and writes the updated token back to shared memory.

7. The multimodal intelligent agent scheduling method according to claim 6, characterized in that, The specific steps to maintain semantic coherence are as follows: Calculate the semantic drift between the initial semantic embedding vector and the weighted updated semantic embedding vector of the context token; When the semantic drift exceeds a preset safety threshold, a correction mechanism is triggered, which corrects the current semantic embedding vector by introducing a correction factor.

8. The multimodal intelligent agent scheduling method according to claim 1, characterized in that, Real-time monitoring of modal health to trigger circuit breakers, specifically: The modal health of the corresponding modal agent is calculated by weighting and summing the inference success rate, latency bias and inference error rate of different modal agents respectively. If the modal health of any modal agent falls below the circuit breaker threshold, the circuit breaker is triggered to cut off the traffic input to that modal agent.

9. The multimodal intelligent agent scheduling method according to claim 1, characterized in that, The specific steps of the self-healing process are as follows: If the criticality of the business is lower than the criticality threshold, a preset lightweight model is invoked to take over the traffic input of the circuit-breaking modal proxy; otherwise, a manual channel is switched to take over.

10. A multimodal intelligent agent scheduling system, characterized in that, It includes a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the multimodal intelligent agent scheduling method according to any one of claims 1-9 is implemented.