A scheduling method and system for large multi-modal model inference intention decoupling under a hybrid cloud

Abstract

The application discloses a scheduling method and system for large multi-modal model inference intention decoupling under a hybrid cloud, relates to the technical field of cloud computing and artificial intelligence scheduling, and comprises the following steps: receiving a large multi-modal model inference request, generating a multi-dimensional resource demand vector for each subtask, and acquiring a high-dimensional state space feature snapshot of the hybrid cloud infrastructure in real time; inputting the multi-dimensional resource demand vector and the high-dimensional state space feature snapshot into a deep reinforcement learning strategy network to obtain a continuous time scalar and use the continuous time scalar as the tolerance time of the current task in local waiting; if the current task obtains a scheduling opportunity within the tolerance time, local deployment is performed; if the waiting time of the current task in the local queue exceeds the tolerance time, cross-cloud offloading is performed according to the privacy sovereignty label of the current task. In this way, an efficient, robust, safe and economical scheduling solution is provided for large multi-modal model inference.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of cloud computing and artificial intelligence scheduling technology, and in particular to a scheduling method and system for decoupling inference intent of large-scale multimodal models in hybrid cloud environments. Background Technology

[0002] The explosive growth of large-scale multimodal models (LMMs) is reshaping cloud workloads from simple text processing to complex heterogeneous pipelines encompassing vision, audio, and language. Faced with the massive resource gap created by LMM inference, deploying such workloads across hybrid clouds (combining private on-premises clusters and elastic public cloud instances) has become an industry trend. However, existing scheduling paradigms are prone to underlying abstraction failures in multimodal hybrid cloud scenarios, mainly facing the following challenges:

[0003] First, there's the multidimensional fragmentation of the spatial dimension. Existing cluster schedulers (like Kubernetes) primarily rely on scalar resource abstractions based on GPU counting, failing to capture the orthogonal hardware requirements within the LMM pipeline. For example, text encoders are extremely computationally dependent, while image generators are limited by GPU memory. Scalar binning leads to severe "computing power stalling" within a single node, with up to 70% of high-value computation cycles idle due to GPU memory exhaustion, forcing the system to trigger expensive public cloud bursts despite having a large amount of idle computing power.

[0004] Secondly, there is the issue of cross-modal stagnation in the time dimension. Existing DAG workflow schedulers (such as HEFT) often make overly optimistic network assumptions. When directly applied to hybrid clouds with limited wide area networks (WANs), blind cross-cloud DAG scheduling can cause severe synchronization congestion. For example, after offloading the text encoder to the cloud, the subsequent time cost of transmitting massive amounts of embedding tensors across the WAN is extremely high, causing the high-performance GPUs in the private cloud to be in a long-term idling waiting state. The network penalty severely weakens or even negates the benefits of computation offloading. Summary of the Invention

[0005] The purpose of this invention is to overcome the shortcomings of the prior art and provide a scheduling method and system for decoupling the inference intent of large-scale multimodal models in hybrid cloud environments. Through a multidimensional vectorized binning mechanism, the resource requirements of different modal tasks are accurately matched in the physical resource space, eliminating resource gaps and bringing the utilization rate of single-node hardware close to the physical limit, thereby avoiding premature triggering of expensive public cloud leasing costs due to false resource depletion.

[0006] In a first aspect, embodiments of the present invention provide a scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments, including:

[0007] Step S1: Receive a large multimodal model inference request containing multiple subtasks and parse its directed acyclic graph structure, generate a multidimensional resource requirement vector for each subtask, and attach a privacy sovereignty label to the subtask carrying sensitive information.

[0008] Step S2: Real-time acquisition of the resource fragmentation status of the hybrid cloud infrastructure to form a high-dimensional state space feature snapshot containing the available computing power and memory reserves of each physical node of the private cloud;

[0009] Step S3: Input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue;

[0010] Step S4: If the current task obtains a scheduling opportunity within the tolerance time, then local deployment based on multidimensional resource requirements is executed; if the current task's waiting time in the local queue exceeds the tolerance time, then cross-cloud offloading is executed according to the current task's privacy sovereignty label.

[0011] Step S5: Based on the topology of the directed acyclic graph, use the output features of the completed subtasks as the inputs of the downstream subtasks, and repeat steps S2-S4 until all subtasks are completed.

[0012] In a preferred embodiment, in step S1, a large multimodal model inference request containing multiple subtasks is received and its directed acyclic graph structure is parsed. A multidimensional resource requirement vector is generated for each subtask, and privacy sovereignty tags are attached to subtasks carrying sensitive information, including:

[0013] The large-scale multimodal model inference request is modeled as a directed acyclic graph, where each node represents a subtask and each edge represents the dependency relationship between subtasks and the associated feature transfer amount.

[0014] For each subtask in the directed acyclic graph, the historical resource benchmark of each subtask is obtained in advance through offline performance profiling, and the benchmark is dynamically calibrated at runtime using online memory snapshots to generate a multi-dimensional resource requirement vector for each subtask.

[0015] Context scanning is performed on the large multimodal model inference requests for each subtask, and a Boolean privacy sovereignty label is assigned to each subtask, wherein the label is used to indicate whether the subtask is subject to local execution constraints.

[0016] In a preferred embodiment, the step of obtaining historical resource benchmarks for each subtask through offline performance profiling beforehand, and dynamically calibrating the benchmarks at runtime using online memory snapshots, generates a multi-dimensional resource requirement vector for each subtask, including:

[0017] Based on historical performance analysis data, the prior resource benchmarks for each subtask are extracted, including the expected computing unit utilization and peak memory usage.

[0018] During runtime, the actual peak memory usage of subtasks is periodically collected and weighted and fused with the prior resource benchmark to achieve dynamic calibration.

[0019] Based on the calibration results, a two-dimensional resource requirement vector is generated, which includes the normalized expected computing unit utilization and peak memory usage.

[0020] In a preferred embodiment, in step S2, the resource fragmentation status of the hybrid cloud infrastructure is acquired in real time, forming a high-dimensional state space feature snapshot containing the available computing power and memory reserves of each physical node in the private cloud, including:

[0021] Real-time collection of normalized available computing power idle rate and video memory balance of each physical node in the private cloud;

[0022] The idle computing power and memory availability of all collected nodes are vectorized and combined to form a high-dimensional state space feature snapshot that represents the fragmentation state of cluster resources.

[0023] In a preferred embodiment, in step S3, the multidimensional resource demand vector and the high-dimensional state space feature snapshot are input into the deep reinforcement learning policy network to obtain a continuous time scalar, including:

[0024] The multidimensional resource demand vector and the high-dimensional state space feature snapshot are used together as the state input of the deep reinforcement learning policy network.

[0025] The deep reinforcement learning policy network outputs a one-dimensional continuous time scalar, which is directly mapped to the tolerance time for the current subtask to wait in the local private cloud queue; wherein, the action space dimension of the deep reinforcement learning policy network is always 1 and does not change with the expansion of the hybrid cloud cluster size.

[0026] In a preferred embodiment, in step S4, if the current task obtains a scheduling opportunity within the tolerance time, then a local deployment based on multidimensional resource requirements is performed, including:

[0027] The type of the current subtask is determined based on the multidimensional resource requirement vector of the current subtask. The subtask type includes computationally intensive subtasks and memory-intensive subtasks.

[0028] Scan the remaining resource vectors of each physical node in the local private cloud, and deploy compute-intensive subtasks and memory-intensive subtasks complementaryly on the same physical node to fill the resource gaps caused by the orthogonality of resource requirements.

[0029] As a preferred implementation, the type of the current subtask is determined based on the multidimensional resource requirement vector of the current subtask, including:

[0030] Extract the multidimensional resource requirement vector of the subtask, wherein the multidimensional resource requirement vector includes the normalized expected computing unit utilization and peak memory usage;

[0031] If the normalized expected computation unit utilization of the subtask is greater than its peak memory usage, then the subtask is determined to be a compute-intensive subtask; otherwise, it is determined to be a memory-intensive subtask.

[0032] In a preferred embodiment, in step S4, if the waiting time of the current task in the local queue exceeds the tolerance time, cross-cloud offloading is performed based on the privacy sovereignty tag of the current task, including:

[0033] Verify the Boolean privacy sovereignty label for the current subtask:

[0034] If the label indicates that the subtask is subject to local execution constraints, then the execution domain of the subtask is forcibly locked to the private cloud, and only the de-anonymized intermediate feature tensor output by the subtask is transmitted across the wide area network to the downstream nodes of the public cloud.

[0035] If the label indicates that the subtask allows cross-cloud deployment, then the original input data or intermediate features of the subtask are offloaded to the public cloud for execution.

[0036] In a preferred embodiment, the reward function of the deep reinforcement learning policy network is constructed as a composite penalty term, which simultaneously penalizes the risk of service level agreement default due to extended tolerance time, as well as the rental costs incurred due to premature triggering of public cloud emergencies.

[0037] Secondly, embodiments of the present invention also provide a scheduling system for decoupling inference intent of large-scale multimodal models in hybrid cloud environments, comprising:

[0038] The modality receiving and parsing module is used to receive large multimodal model inference requests containing multiple subtasks and parse their directed acyclic graph structure, generate multidimensional resource requirement vectors for each subtask, and attach privacy sovereignty labels to subtasks carrying sensitive information.

[0039] The status monitoring module is used to acquire the fragmented status of resources in the hybrid cloud infrastructure in real time, forming a high-dimensional state space feature snapshot that includes the available computing power and memory reserves of each physical node in the private cloud.

[0040] The cardinality query agent module is used to input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue.

[0041] The task scheduling module is used to execute local deployment based on multidimensional resource requirements if the current task obtains a scheduling opportunity within the tolerance time; if the current task waits in the local queue for longer than the tolerance time, it performs cross-cloud offloading based on the privacy sovereignty label of the current task.

[0042] The topology execution module is used to take the output features of the completed subtasks as the input of the downstream subtasks according to the topology of the directed acyclic graph, and repeat the above steps until all subtasks are completed.

[0043] Thirdly, embodiments of the present invention also provide an electronic device, the electronic device comprising:

[0044] One or more processors;

[0045] Storage device for storing one or more programs;

[0046] When the one or more programs are executed by the one or more processors, the one or more processors implement the scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments, as described in any embodiment of the present invention.

[0047] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements a scheduling method for decoupling inference intent of a large-scale multimodal model under hybrid cloud as described in any embodiment of the present invention.

[0048] In summary, this invention provides an efficient, robust, secure, and economical scheduling solution for large-scale multimodal model inference in hybrid cloud environments through a two-layer architecture that decouples intent prediction from physical execution, achieving the following beneficial effects:

[0049] (1) This invention generates a two-dimensional resource demand vector containing normalized expected computing unit utilization and peak memory usage through a modality-aware analyzer, and executes a vectorized binning algorithm by a deterministic execution engine to complementaryly deploy computationally intensive and memory-intensive subtasks on the same physical node. Experimental results show that the utilization rate of the private cloud local cluster can stably approach 95%, and the memory utilization rate is maintained at the theoretical limit high level (>95%), effectively filling resource gaps and avoiding premature triggering of expensive public cloud bursts due to false resource depletion.

[0050] (2) This invention collapses the high-dimensional node selection action space into a one-dimensional tolerance time scalar through a cardinality query agent, and achieves deep overlap between local computation and cross-cloud feature transmission based on this tolerance time. Experiments show that under extreme low bandwidth (100Mbps) conditions, the job completion time of this invention is only slightly disturbed (about 3.1 seconds), while the baseline method increases dramatically to more than 17 seconds; the overall execution delay is compressed to the theoretical lower limit of computation + forced transmission, effectively masking the wide area network transmission delay.

[0051] (3) This invention attaches a Boolean privacy sovereignty label to subtasks and enforces it during cross-cloud overflow: privacy-sensitive subtasks are locked to the private cloud and only their de-privacyized high-dimensional intermediate feature tensors are output to the public cloud. Experimental data show that the privacy violation rate of this invention is close to 0%, and the cross-cloud data transmission volume is reduced by about 95% (from more than 7000 GB to about 350 GB), achieving a significant reduction in cross-cloud bandwidth overhead while ensuring strict security and compliance.

[0052] (4) This invention uses a composite penalty term to reinforce the learning reward function, guiding the agent to automatically find the optimal trade-off between the risk of Service Level Agreement (SLO) default and the cost of public cloud leasing. Experimental results show that this invention strictly limits the SLO default rate to within 1.8%, with a normalized cost as low as 0.29. Compared with K8s-Native, it reduces total expenditure by 61%, and compared with HEFT, it eliminates about 69% of sudden overhead. Moreover, it converges to the theoretical Pareto optimal boundary under different loads and operational preferences. Attached Figure Description

[0053] Other features, objects, and advantages of the invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings. The drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings:

[0054] Figure 1 This is a flowchart of a scheduling method for decoupling inference intent of large-scale multimodal models in a hybrid cloud environment, provided by an embodiment of the present invention.

[0055] Figure 2 This is a schematic diagram of the cardinality lookup proxy provided in an embodiment of the present invention;

[0056] Figure 3 This is a comparison chart of overall performance under the Helios trajectory provided in the embodiments of the present invention;

[0057] Figure 4 This is a comparison chart of the ultimate robustness under the Philly trajectory provided in the embodiments of the present invention;

[0058] Figure 5 This is a comparison chart of cost-SLO Pareto optimality and strategy adaptability provided in the embodiments of the present invention;

[0059] Figure 6 This is a cost-performance trade-off comparison chart provided by an embodiment of the present invention;

[0060] Figure 7 This is a robustness comparison diagram under multimodal strength provided in the embodiments of the present invention;

[0061] Figure 8 This is a schematic diagram of the two-dimensional resource saturation distribution of the various strategies provided in the embodiments of the present invention on a private cloud node;

[0062] Figure 9 This is a schematic diagram of dependency-aware pipeline alignment provided in an embodiment of the present invention;

[0063] Figure 10 This is a comparison chart of data transmission volume and privacy compliance provided in an embodiment of the present invention;

[0064] Figure 11 This is a schematic diagram illustrating the impact of different cross-cloud bandwidths on average job completion time, provided by an embodiment of the present invention.

[0065] Figure 12 This is a schematic diagram comparing the scheduling overhead of various strategies provided in the embodiments of the present invention;

[0066] Figure 13 This is a schematic diagram of the structure of a scheduling system for decoupling inference intent of large-scale multimodal models in a hybrid cloud environment, provided by an embodiment of the present invention.

[0067] Figure 14 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0068] The present invention will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, the accompanying drawings show only the parts relevant to the present invention, and not all of the structures.

[0069] Before discussing the exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as being processed sequentially, many of these operations (or steps) may be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations may be rearranged. The process may be terminated when its operation is completed, but may also have additional steps not included in the figures. The process may correspond to a method, function, procedure, subroutine, subroutine, etc.

[0070] Example 1

[0071] This invention proposes a scheduling method for decoupling inference intent in large-scale multimodal model (LMM) scenarios in hybrid cloud environments. It abandons the traditional approach of directly predicting high-dimensional physical nodes through reinforcement learning. By working collaboratively with three core components—a modality-aware analyzer, a cardinality query agent, and a deterministic execution engine—the scheduling decision is reduced to a temporal intent, which is then handed over to the underlying rule engine for physical deployment. This achieves efficient and secure scheduling of inference requests for large-scale multimodal model (LMM) scenarios.

[0072] like Figure 1 The diagram shows a flowchart of a scheduling method 100 for decoupling inference intent of a large-scale multimodal model in a hybrid cloud environment, as provided in Embodiment 1 of the present invention. The method 100 specifically includes the following steps:

[0073] Step S1: Receive a large multimodal model inference request containing multiple subtasks and parse its directed acyclic graph structure, generate a multidimensional resource requirement vector for each subtask, and attach privacy sovereignty labels to subtasks carrying sensitive information.

[0074] As a preferred embodiment, traditional cluster schedulers, such as Kubernetes, rely on user-submitted coarse-grained scalar specifications. This approach of treating multimodal loads as monolithic black boxes is the root cause of multidimensional fragmentation. To break down this information barrier, a modality-aware analyzer is used as the system's semantic gateway to deeply intercept and deconstruct multimodal requests during the admission control phase. Specifically, this includes the following steps:

[0075] First, the analyzer models a complex, large-scale multimodal model inference request as a directed acyclic graph (DAG) by parsing workflow definitions, such as user-submitted YAML or computation graph traces. For each subtask in the pipeline For example, in text encoders and image generators, the analyzer performs the following operations:

[0076] Prior benchmark extraction: Using offline historical performance profiling data, the prior resource benchmarks for this subtask are extracted, including expected computing unit utilization and peak memory usage.

[0077] Dynamic calibration: During runtime, the actual peak memory usage of subtasks is periodically collected and weighted and fused with the prior resource benchmark to compensate for fluctuations in the runtime environment;

[0078] Vector Construction: Based on the calibration results, a fine-grained multi-dimensional resource demand vector is constructed. ,in Capture the core computational requirements of this subtask, and This represents peak video memory usage.

[0079] At the same time, for the dependency edges between tasks The analyzer quantifies the accompanying data transfer volume. This allows for fine-grained characterization of the heterogeneous bandwidth requirements of cross-modal networks.

[0080] More importantly, to address the privacy-gravity deadlock problem in hybrid cloud environments, the analyzer introduces a unique semantic and compliance annotation mechanism. This mechanism performs a context scan on each incoming request, processing each subtask... Add a boolean privacy sovereignty tag ,in, This indicates that it is subject to strict local execution hard constraints, such as medical prompt processing, and its data gravity soft constraints are evaluated based on the size of the output embedding tensor. 0 indicates that cross-cloud deployment is allowed.

[0081] In addition, the analyzer will assess the data gravity soft constraint based on the size of the embedding tensor expected to be output by the subtask, providing a reference for subsequent cross-cloud transmission decisions.

[0082] Through the aforementioned deep feature extraction, previously invisible physical and semantic conflicts are fully revealed, providing a high-dimensional and complete state space input for the efficient dimensionality reduction of the subsequent cardinality query agent (deep reinforcement learning module) and the compliant interception of the deterministic execution engine.

[0083] Step S2: Real-time acquisition of the resource fragmentation status of the hybrid cloud infrastructure to form a high-dimensional state space feature snapshot containing the available computing power and memory reserves of each physical node in the private cloud.

[0084] In a preferred embodiment, the core of this step lies in transforming the heterogeneous resource states of the physical cluster into a high-dimensional, structured representation that can be understood by the reinforcement learning agent. Existing reinforcement learning-based cluster schedulers generally fall into the architectural trap of "end-to-end placement," forcing the agent to directly search for the optimal placement decision in a discrete space containing tens of thousands of nodes, thereby triggering an unacceptable curse of dimensionality. To effectively overcome this architectural limitation, this invention achieves a paradigm-level dimensionality reduction breakthrough through a cardinality query agent—reshaping the agent into a "patient evaluator" focused on temporal decision-making, completely stripping it of its direct control over the underlying physical nodes.

[0085] Specifically, for each physical node within the private cloud, two key resource metrics are collected in real time:

[0086] Normalized available computing power idle rate: defined as the ratio of the node’s current remaining computing power (measured by the available Tensor Core cycles) to its total capacity, with a value range of [0,1].

[0087] Memory availability ratio: Defined as the ratio of the currently free HBM memory of the node to its total memory, with a value range of [0,1].

[0088] The two key resource metrics mentioned above are obtained in real time by calling the underlying hardware monitoring interface or the container runtime interface to ensure high data fidelity. This collection process is carried out continuously at a fixed frequency or in an event-driven manner.

[0089] Next, the resource metrics of all collected physical nodes are fused together to form a high-dimensional state vector with a dimension of 2N. This vector is called a high-dimensional state space feature snapshot. This snapshot fully depicts the fragmented distribution of resources in the spatial dimension of the entire private cluster and explicitly preserves the computing power idle rate and memory reserve of each node.

[0090] Through the high-dimensional state representation constructed in this step, the agent gains global observability for the underlying computing power stalling phenomenon—the agent can not only see how many memory slots are left on each node, but also accurately perceive the matching degree between the remaining computing power cycles of each node and the requirements of the tasks to be scheduled. This fine-grained observation information fundamentally avoids the blind boxing and false resource exhaustion caused by traditional scalar schedulers (such as the default Kubernetes scheduler) that only report boolean values ​​of whether nodes meet resource requests.

[0091] In summary, the state space provided by this invention fully preserves the multidimensional orthogonality information of resources, enabling intelligent agents to make precise decisions such as scheduling computationally intensive tasks to nodes with abundant computing power and scheduling memory-intensive tasks to nodes with abundant memory, thus laying a solid perceptual foundation for eliminating multidimensional fragmentation.

[0092] Step S3: Input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue.

[0093] In a preferred embodiment, this step is performed by the system's Cardinal-Query Agent. Existing reinforcement learning-based cluster schedulers generally fall into the architectural trap of "end-to-end placement," forcing agents to directly search for the optimal placement decision in a discrete action space containing tens of thousands of physical nodes, thus triggering an unacceptable curse of dimensionality. To effectively overcome this architectural limitation, this invention reshapes the scheduling decision from selecting which node to choose to how long to wait before overflow, achieving a paradigm-level dimensionality reduction breakthrough. Specifically, it includes the following sub-steps:

[0094] At any decision point, the cardinality query agent receives the multidimensional resource demand vector and the high-dimensional state space feature snapshot from steps S1 and S2 above, and concatenates the multidimensional resource demand vector and the high-dimensional state space feature snapshot into a high-dimensional state vector, which is used as the input to the deep reinforcement learning policy network. This state vector deeply integrates the resource demand at the task granularity and the fragmented distribution at the cluster granularity, providing the agent with global observability for the underlying "computing power stalling" phenomenon.

[0095] As a preferred embodiment, unlike traditional methods that define the action space as a high-dimensional node mapping, the cardinality query agent of this invention strictly defines an action as a one-dimensional continuous time scalar, i.e., the maximum tolerable time for the current task to wait in the local private cloud queue. The core advantage of this design is that regardless of the scale of the underlying hybrid cloud cluster, such as expanding from 16 nodes to thousands of nodes, the dimension of the neural network output layer remains constant. This enables the model to have good convergence speed and generalization ability across cluster sizes; that is, a strategy trained on a small-scale cluster can be directly deployed to a large-scale cluster without fine-tuning.

[0096] As a preferred embodiment, the deep reinforcement learning policy network adopts an Actor-Critic architecture based on Proximal Policy Optimization (PPO) or Deep Deterministic Policy Gradient (DDPG). The Actor network is responsible for mapping high-dimensional state vectors to tolerance times, while the Critic network is responsible for evaluating the value of the current state. During the online inference phase, only one forward propagation of the Actor network is required to obtain the scheduling decision, meeting the stringent low-latency requirements of production environments.

[0097] Combination Figure 2 As shown, along Figure 2The feedback loop below, with the reward function driving the self-evolution of the cardinality query agent, ensures absolute privacy compliance through a hard constraint imposed by the underlying deterministic execution engine, enabling the policy network to achieve multi-objective optimization between minimizing service default rate and reducing rental costs.

[0098] This reward function simultaneously penalizes two coupled objectives: on the one hand, too short a tolerance time will cause tasks to overflow to the public cloud prematurely, resulting in high rental costs; on the other hand, too long a tolerance time may cause tasks to wait too long in the local queue, thus triggering SLO default. Through this compound penalty, the reinforcement learning agent is guided to automatically find an economic equilibrium point in the multidimensional coupling crisis, that is, to obtain the greatest cost savings with the least delay sacrifice.

[0099] This step fundamentally solves the curse of dimensionality problem faced by traditional reinforcement learning schedulers by reducing scheduling decisions from node selection to waiting time prediction. Simultaneously, a carefully designed composite reward function ensures that the agent can adaptively find the optimal trade-off between performance and cost, providing high-quality temporal intent input for subsequent deterministic physical deployment.

[0100] Step S4: If the current task obtains a scheduling opportunity within the tolerance time, then perform local deployment based on multidimensional resource requirements; if the current task waits in the local queue for longer than the tolerance time, then perform cross-cloud offloading based on the current task's privacy sovereignty label.

[0101] In a preferred implementation, this step is executed by the system's deterministic execution engine. This engine receives the tolerance time output from step S3 above and transforms it into a lossless physical deployment decision. Through hard-coded rule guardrails, the inherent trial-and-error uncertainty of reinforcement learning under complex constraints is eliminated, ensuring the determinism and compliance of scheduling decisions.

[0102] Specifically, it is divided into two scenarios: local deployment and cross-cloud uninstallation.

[0103] If the current subtask's waiting time in the local private cloud queue has not exceeded the tolerance time, thus obtaining a scheduling opportunity, then local deployment is executed. This process uses a vectorized bin-packing algorithm, specifically including:

[0104] The type of the current subtask is determined based on the multidimensional resource demand vector of the current subtask, which includes the normalized expected computing unit utilization and peak video memory usage.

[0105] If the normalized expected computation unit utilization of a subtask is greater than its peak memory usage, the subtask is determined to be a compute-intensive subtask; otherwise, it is determined to be a memory-intensive subtask. This determination rule utilizes the resource orthogonality of multimodal workloads: compute-intensive tasks mainly consume Tensor Core computing power, while memory-intensive tasks mainly consume HBM memory.

[0106] Scan the remaining resource vectors of each physical node in the local private cloud, and deploy compute-intensive subtasks and memory-intensive subtasks complementaryly on the same physical node. For compute-intensive tasks, prioritize nodes with a larger current normalized available computing power idle rate and a relatively smaller proportion of available memory to balance resource utilization. For memory-intensive tasks, prioritize nodes with a larger current proportion of available memory and a relatively smaller current normalized available computing power idle rate to fill resource gaps caused by the orthogonality of resource requirements.

[0107] If there is a compute-intensive task and a memory-intensive task waiting to be scheduled at the same time, we will try to place them on the same physical node to achieve positive resource interaction and fill the computing power idleness caused by a single resource bottleneck.

[0108] Through this vectorized binning mechanism, the execution engine brings the hardware utilization of a single node close to the theoretical limit, effectively avoiding premature triggering of expensive public cloud outbursts due to false resource depletion.

[0109] If the current subtask's waiting time in the local queue exceeds the tolerance time, the engine forcibly triggers the cross-cloud overflow mechanism and calls the built-in semantic-aware offloader, specifically including:

[0110] The uninstaller first verifies the privacy sovereignty label of the current subtask;

[0111] If the privacy sovereignty label is 1, it indicates that the subtask is subject to local execution constraints, forcibly locking the execution domain of the subtask within the private cloud local cluster for secure computation, ensuring that the original sensitive data never leaves the private cloud boundary. After the subtask is completed within the private cloud, the engine does not transmit the original input data, but only extracts its output, de-privacyized high-dimensional intermediate feature tensor, and securely offloads the high-dimensional intermediate feature tensor across the wide area network to the downstream generator on the elastic public cloud.

[0112] If the privacy sovereignty label is 0, it indicates that the subtask is allowed to be deployed across clouds. The original input data or intermediate features of the subtask are then offloaded to the public cloud for execution. After execution, the public cloud returns the results to the downstream task or user.

[0113] In summary, by leveraging this deterministic mechanism, the system not only significantly reduces cross-modal stagnation time in extremely harsh network environments, but also rigorously controls the overall system's privacy violation rate to an extremely low marginal level when dealing with extreme multimodal load spikes, while reducing cross-cloud bandwidth overhead by approximately 95%, thereby achieving the dual goals of performance improvement and security compliance.

[0114] Step S5: Based on the topology of the directed acyclic graph, use the output features of the completed subtasks as the inputs of the downstream subtasks, and repeat steps S2-S4 until all subtasks are completed.

[0115] In a preferred embodiment, this module dynamically drives the scheduling decisions and physical deployment of each subtask based on the topological order of a directed acyclic graph (DAG) until all subtasks are completed. Specifically, it includes the following steps:

[0116] The system maintains an execution state machine for each subtask in the current DAG, with states including: Not Ready (not all dependencies satisfied), Ready (all upstream dependencies completed), Running (scheduled but not yet completed), and Completed. When a subtask completes execution, the system stores its output features or intermediate embedding tensors in a local cache or passes them by reference to all its downstream dependent tasks. For each downstream subtask, it checks whether all its upstream dependencies have been completed and is in the "Completed" state. If so, the system switches the downstream subtask's state from "Not Ready" to "Ready" and adds it to the scheduling queue.

[0117] For multiple subtasks in the "ready" state, repeat steps S2 to S4. Crucially, the system achieves deep overlap between computation and communication through the following mechanism:

[0118] When an upstream subtask is executed on a local private cloud, the scheduling decisions (i.e., wait time tolerance) of its downstream subtasks can be made in parallel.

[0119] When the output embedding of an upstream subtask needs to be transferred across the cloud, this transfer process can be performed simultaneously with the post-processing of the upstream subtask or the local computation of other ready subtasks.

[0120] The tolerance time of the deterministic execution engine output naturally provides a time budget for this overlap, and network latency can be completely masked as long as the transmission time is less than the waiting tolerance of the downstream task.

[0121] The system continues to execute the above loop until all subtasks in the DAG are marked as "completed". At this point, the final result of the entire multimodal inference pipeline has been generated, and the system returns it to the user or upper-layer application, releasing the private and public cloud resources it has occupied.

[0122] During execution, if a subtask fails due to insufficient resources, network failure, or other reasons, the system can retry a limited number of times according to a preset retry policy. If the retry still fails, the subtask and all its downstream components are marked as failed, and an error is reported to the user.

[0123] Based on the above embodiments, the present invention achieves the following beneficial effects:

[0124] (1) This invention generates a two-dimensional resource demand vector containing normalized expected computing unit utilization and peak memory usage through a modality-aware analyzer, and executes a vectorized binning algorithm by a deterministic execution engine to complementaryly deploy computationally intensive and memory-intensive subtasks on the same physical node. Experimental results show that the utilization rate of the private cloud local cluster can stably approach 95%, and the memory utilization rate is maintained at the theoretical limit high level (>95%), effectively filling resource gaps and avoiding premature triggering of expensive public cloud bursts due to false resource depletion.

[0125] (2) This invention collapses the high-dimensional node selection action space into a one-dimensional tolerance time scalar through a cardinality query agent, and achieves deep overlap between local computation and cross-cloud feature transmission based on this tolerance time. Experiments show that under extreme low bandwidth (100Mbps) conditions, the job completion time of this invention is only slightly disturbed (about 3.1 seconds), while the baseline method increases dramatically to more than 17 seconds; the overall execution delay is compressed to the theoretical lower limit of computation + forced transmission, effectively masking the wide area network transmission delay.

[0126] (3) This invention attaches a Boolean privacy sovereignty label to subtasks and enforces it during cross-cloud overflow: privacy-sensitive subtasks are locked to the private cloud and only their de-privacyized high-dimensional intermediate feature tensors are output to the public cloud. Experimental data show that the privacy violation rate of this invention is close to 0%, and the cross-cloud data transmission volume is reduced by about 95% (from more than 7000 GB to about 350 GB), achieving a significant reduction in cross-cloud bandwidth overhead while ensuring strict security and compliance.

[0127] (4) This invention uses a composite penalty term to reinforce the learning reward function, guiding the agent to automatically find the optimal trade-off between the risk of Service Level Agreement (SLO) default and the cost of public cloud leasing. Experimental results show that this invention strictly limits the SLO default rate to within 1.8%, with a normalized cost as low as 0.29. Compared with K8s-Native, it reduces total expenditure by 61%, and compared with HEFT, it eliminates about 69% of sudden overhead. Moreover, it converges to the theoretical Pareto optimal boundary under different loads and operational preferences.

[0128] Example 2

[0129] To further verify the effectiveness and superiority of the scheduling method for decoupling reasoning intent of large-scale multimodal models in hybrid cloud environments proposed in this invention, this embodiment conducted a large-scale comparative experiment in a simulation environment, and systematically evaluated the invention and existing typical scheduling strategies from multiple dimensions such as performance, cost, security compliance and robustness.

[0130] To accurately reflect the complex characteristics of large-scale multimodal model (LMM) inference workloads in production environments, this embodiment constructs a synthetic multimodal workload as shown in Table 1, based on AI job metadata publicly available from Microsoft Azure and referencing benchmarks established by the latest systems such as ModServe. This workload injects real DAG topology, modal composition, and privacy constraints into the original Helios and Philly trajectories through deterministic mapping, achieving a high-fidelity reproduction of the multidimensional coupling crisis in hybrid clouds.

[0131] Table 1

[0132] A high-fidelity discrete event simulator was developed using Python and SimPy to characterize task queuing, resource allocation, and cross-network transmission latency at the microsecond level. The private cloud side was configured with 16 homogeneous nodes, with the physical capacity of each node aligned with an NVIDIA A100 GPU. The public cloud side was modeled as an on-demand instance pool with unlimited throughput elasticity. A 1 Gbps bandwidth limit and a 50 ms round-trip latency were applied to the WAN link to simulate an extremely constrained network environment.

[0133] This embodiment selects seven representative systems and strategies covering three core scheduling dimensions as comparison benchmarks:

[0134] Basic Abstractions and Network Optimistic Paradigms: K8s-Default (Scalar Boxing), HEFT (Heterogeneous Earliest Completion Scheduler);

[0135] Cost-aware hybrid cloud offloading paradigm: Starburst (full version), Starburst-No-TE (removal of time prediction module);

[0136] Heuristic rigid burst paradigms: Cost-Greedy (pure cost-driven), Constant (rigid capacity overflow), Zero (rapid unloading).

[0137] In terms of evaluation system, we have constructed a Pareto matrix covering three dimensions: performance dimension tracks job completion time (JCT) and end-to-end SLO default rate; economic dimension quantifies the total cost of public cloud on-demand leasing; and security and compliance dimension closely monitors the global cross-network data transmission volume and focuses on assessing the absolute privacy violation rate controlled by the underlying safeguards. This rigorous indicator system ensures a comprehensive view of the system's maximum performance.

[0138] To comprehensively evaluate the system's performance under different congestion levels, the following presents macroscopic benchmark results driven by two production datasets, Helios and Philly, across the entire system load range (0.7 to 3.0). The results show that:

[0139] Overall performance under the Helios trajectory (e.g.) Figure 3 As shown): The method of this invention achieves extremely high local cluster utilization (approaching 95%) across all load ranges and maximizes cost savings. Compared to cost-insensitive heuristics (such as HEFT, No-Wait), the average JCT increases only moderately by about 0.5 hours, but significantly reduces public cloud expenditure.

[0140] Overall effectiveness under the Philly trajectory (e.g.) Figure 4 As shown): The Philly dataset exhibits strong burstiness and a complex DAG structure. Traditional strategies (such as HEFT) rapidly collapse under high load congestion (load ≥ 1.5), leading to public cloud budget overruns. The method of this invention maintains stable positive cost savings under extreme congestion by strategically exchanging the average JCT over approximately 1 hour, while preserving the highest local cluster utilization.

[0141] like Figure 5 As shown, the distribution of normalized costs and end-to-end SLO default rates for each comparative system under massive heterogeneous loads indicates that:

[0142] The method of this invention strictly limits the SLO default rate to a marginal level of 1.8%, while the normalized cost is as low as 0.29. Compared to K8s-Native, the method of this invention reduces total expenditure by 61%; compared to HEFT, it eliminates approximately 69% of burst overhead. Even under extreme configuration boundaries ranging from cost-conscious to performance-oriented, the execution trajectory of the method of this invention still closely converges to the theoretical Pareto optimal boundary, while the distribution of traditional heuristic algorithms is scattered. This demonstrates that the dimensionality reduction RL design proposed by the method of this invention possesses excellent robustness and zero-shot generalization ability.

[0143] like Figure 6 As shown, the trade-off between average JCT and cost savings rate indicates that:

[0144] Traditional performance-oriented algorithms (such as HEFT) achieve lower JCT, but the cost savings rate is 0%; baseline cost-aware strategies (such as Constant-Wait) have an economic benefit of less than 15%; the method of this invention radically reduces unnecessary cross-cloud bursts with only minimal and acceptable latency overhead (an average increase in JCT of about 0.15 hours), achieving cost savings of up to about 90%, effectively bridging the gap between performance requirements and budget constraints.

[0145] like Figure 7 As shown, during the process of increasing the proportion of multimodal tasks from 0% to 100%:

[0146] As the proportion of multimodal architectures increases, the JCT and operating costs of K8s-Native, HEFT, and Starburst grow exponentially, exposing the fragility of the scalar abstraction paradigm.

[0147] The performance curve of the method of this invention exhibits smooth scalability. Even under extreme loads of 100% pure multimodal operation, it successfully keeps the average JCT below the Cost-Greedy queuing baseline, while stabilizing on-demand operating costs at a level far lower than that of Kubernetes and HEFT. This strongly demonstrates the high robustness of this scheduling architecture in the face of extreme modal evolution.

[0148] like Figure 8 As shown, the two-dimensional resource saturation (Compute-Memory Saturation) distribution of each strategy on private cloud nodes indicates that:

[0149] Traditional scalar abstractions (K8s-Native and HEFT) often stagnate at suboptimal binning densities (approximately 80% compute, 80% memory), exhibiting about 20% rigid resource voids. In contrast, the distribution matrix of the method presented in this invention converges to the physical limit: HBM utilization remains stably maintained at a theoretically high level of >95%, while compute utilization is densely distributed between 60% and 80%. This demonstrates that the deterministic execution engine effectively realizes vectorized binning design, fundamentally alleviating the multidimensional fragmentation crisis.

[0150] like Figure 9 As shown, the results of extracting the Top-5 tasks with the largest data transmission volume from 51,045 candidate DAG tasks and performing a fine-grained breakdown of their end-to-end latency are as follows:

[0151] Traditional baselines (HEFT, K8s, Cost-Greedy) suffer from severe cross-cloud blocking defects, resulting in extremely high invalid idle wait times. The method of this invention, with its dependency-aware pipeline alignment mechanism, successfully achieves a high degree of overlap between cross-cloud communication and intra-node computation, greatly reducing idle wait times and strictly compressing the overall execution latency to the theoretical lower limit of computation + forced transmission.

[0152] like Figure 10 As shown, the dual-axis comparison of privacy violation rate and total data transmission volume for each strategy indicates that:

[0153] When Kubernetes and HEFT compete for resources, they tend to engage in indiscriminate cross-cloud bursts, resulting in the migration of massive amounts of raw sensitive data (>7,000GB) to the public cloud, leading to privacy violation rates as high as 75.6% and 78.1%, respectively. The method of this invention, through an EmbeddingOffloading mechanism, strictly keeps the raw data local to the private cloud, transmitting only the decoupled intermediate layer features to the public cloud. Experimental results show that the method of this invention achieves a near-0% privacy violation rate while significantly reducing cross-cloud data transfer volume by approximately 95% (to approximately 350GB). This fully demonstrates that the method of this invention possesses the outstanding ability to minimize cross-cloud bandwidth overhead while ensuring extremely high compliance standards.

[0154] like Figure 11 As shown, the impact of different cross-cloud bandwidths (100 Mbps to 10 Gbps) on the average JCT indicates that:

[0155] Baseline methods (K8s-Native and HEFT) exhibit extremely high bandwidth sensitivity in low-bandwidth-constrained scenarios (such as 100 Mbps), with the average JCT increasing dramatically to over 17 seconds.

[0156] The method of this invention exhibits remarkable immunity to network fluctuations: even under extremely low bandwidth conditions, JCT experiences only weak perturbations (lasting for approximately 3.1 seconds) and maintains a near-horizontal performance curve throughout the entire test bandwidth range. This result strongly validates the effectiveness of the pipeline masking mechanism proposed in this paper—by achieving deep overlap between communication and computation, it successfully masks the vast majority of data transmission latency.

[0157] like Figure 12 As shown, the comparison of scheduling overhead (decision delay) for each strategy indicates that:

[0158] Heuristic algorithms such as HEFT offer extremely low latency (approximately 0.002 ms), but are prone to getting stuck in suboptimal solutions; K8s-Native incurs relatively high overhead (approximately 0.034 ms) due to node filtering and scoring processes; the method of this invention has an average decision latency of only approximately 0.013 ms (microseconds). This superior efficiency is attributed to the effective separation of offline training and online inference, offloading the massive computational complexity from the scheduling path. Considering that the typical execution time of multimodal tasks is on the order of seconds, this sub-millisecond scheduling overhead is negligible, proving that the method of this invention can fully meet the stringent requirements of extremely low overhead in high-throughput production environments while providing state-of-the-art scheduling quality.

[0159] Based on the above experiments, the scheduling method and system proposed in this invention for decoupling reasoning intent of large-scale multimodal models in hybrid cloud environments are significantly superior to existing typical scheduling strategies in multiple core dimensions, and have significant industrial application value.

[0160] Example 3

[0161] Figure 13 This is a schematic diagram of the structure of a scheduling system 700 for decoupling inference intent of large-scale multimodal models in a hybrid cloud environment, as provided in Embodiment 3 of the present invention. Figure 13 As shown, the system includes:

[0162] The modal receiving and parsing module 710 is used to receive large multimodal model inference requests containing multiple sub-tasks and parse their directed acyclic graph structure, generate a multidimensional resource requirement vector for each sub-task, and attach privacy sovereignty labels to sub-tasks carrying sensitive information.

[0163] The status monitoring module 720 is used to acquire the fragmented status of resources in the hybrid cloud infrastructure in real time, forming a high-dimensional state space feature snapshot that includes the available computing power and memory reserves of each physical node in the private cloud.

[0164] The cardinality query agent module 730 is used to input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue.

[0165] The task scheduling module 740 is used to perform local deployment based on multidimensional resource requirements if the current task obtains a scheduling opportunity within the tolerance time; and to perform cross-cloud offloading based on the privacy sovereignty label of the current task if the waiting time of the current task in the local queue exceeds the tolerance time.

[0166] The topology execution module 750 is used to take the output features of the completed subtasks as the input of the downstream subtasks according to the topology of the directed acyclic graph, and repeat the above steps until all subtasks are completed.

[0167] The scheduling system for decoupling inference intent of large-scale multimodal models under hybrid cloud provided in this embodiment of the invention can execute the scheduling method for decoupling inference intent of large-scale multimodal models under hybrid cloud provided in any of the embodiments of the invention above. It has the corresponding functions and beneficial effects of executing the scheduling method for decoupling inference intent of large-scale multimodal models under hybrid cloud. For detailed process, please refer to the relevant operations of the scheduling method for decoupling inference intent of large-scale multimodal models under hybrid cloud in the foregoing embodiments.

[0168] Example 4

[0169] Figure 14This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. The electronic device 10 is intended to represent various forms of digital computers, and may also represent various forms of mobile devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the invention described and / or claimed herein.

[0170] like Figure 14 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded into the RAM 13 from storage unit 18. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0171] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0172] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing unit (CPU), graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 executes a scheduling method described above for decoupling the inference intent of large-scale multimodal models in a hybrid cloud environment.

[0173] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0174] The above embodiments are merely illustrative examples and are not intended to limit the implementation. Those skilled in the art will recognize that other variations or modifications can be made based on the above description. It is neither necessary nor possible to exhaustively list all possible implementations. However, obvious variations or modifications derived therefrom are still within the scope of protection of this invention.

Claims

1. A scheduling method for decoupling inference intent in large-scale multimodal models in hybrid cloud environments, characterized in that, include: Step S1: Receive a large multimodal model inference request containing multiple subtasks and parse its directed acyclic graph structure, generate a multidimensional resource requirement vector for each subtask, and attach a privacy sovereignty label to the subtask carrying sensitive information. Step S2: Real-time acquisition of the resource fragmentation status of the hybrid cloud infrastructure to form a high-dimensional state space feature snapshot containing the available computing power and memory reserves of each physical node of the private cloud; Step S3: Input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue; Step S4: If the current task obtains a scheduling opportunity within the tolerance time, then perform local deployment based on multidimensional resource requirements; if the current task waits in the local queue for longer than the tolerance time, then perform cross-cloud offloading based on the privacy sovereignty label of the current task. Step S5: Based on the topology of the directed acyclic graph, use the output features of the completed subtasks as the inputs of the downstream subtasks, and repeat steps S2-S4 until all subtasks are completed.

2. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 1, characterized in that, In step S1, a large multimodal model inference request containing multiple subtasks is received and its directed acyclic graph structure is parsed. A multidimensional resource requirement vector is generated for each subtask, and privacy sovereignty labels are attached to subtasks carrying sensitive information, including: The large-scale multimodal model inference request is modeled as a directed acyclic graph, where each node represents a subtask and each edge represents the dependency relationship between subtasks and the associated feature transfer amount. For each subtask in the directed acyclic graph, the historical resource benchmark of each subtask is obtained in advance through offline performance profiling, and the benchmark is dynamically calibrated at runtime using online memory snapshots to generate a multi-dimensional resource requirement vector for each subtask. Context scanning is performed on the large multimodal model inference requests for each subtask, and a Boolean privacy sovereignty label is assigned to each subtask, wherein the label is used to indicate whether the subtask is subject to local execution constraints.

3. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 2, characterized in that, The historical resource baselines for each subtask are obtained in advance through offline performance profiling, and the baselines are dynamically calibrated at runtime using online memory snapshots to generate a multi-dimensional resource requirement vector for each subtask, including: Based on historical performance analysis data, the prior resource benchmarks for each subtask are extracted, including the expected computing unit utilization and peak memory usage. During runtime, the actual peak memory usage of subtasks is periodically collected and weighted and fused with the prior resource benchmark to achieve dynamic calibration. Based on the calibration results, a two-dimensional resource requirement vector is generated, which includes the normalized expected computing unit utilization and peak memory usage.

4. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 2, characterized in that, In step S2, the resource fragmentation status of the hybrid cloud infrastructure is acquired in real time, forming a high-dimensional state space feature snapshot containing the available computing power and memory reserves of each physical node in the private cloud, including: Real-time collection of normalized available computing power idle rate and video memory balance of each physical node in the private cloud; The idle computing power and memory availability of all collected nodes are vectorized and combined to form a high-dimensional state space feature snapshot that represents the fragmentation state of cluster resources.

5. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 1, characterized in that, In step S3, the multidimensional resource demand vector and the high-dimensional state space feature snapshot are input into the deep reinforcement learning policy network to obtain a continuous time scalar, including: The multidimensional resource demand vector and the high-dimensional state space feature snapshot are used together as the state input of the deep reinforcement learning policy network. The deep reinforcement learning policy network outputs a one-dimensional continuous time scalar, which is directly mapped to the tolerance time for the current subtask to wait in the local private cloud queue; wherein, the action space dimension of the deep reinforcement learning policy network is always 1 and does not change with the expansion of the hybrid cloud cluster size.

6. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 1, characterized in that, In step S4, if the current task obtains a scheduling opportunity within the tolerance time, then a local deployment based on multidimensional resource requirements is executed, including: The type of the current subtask is determined based on the multidimensional resource requirement vector of the current subtask. The subtask type includes computationally intensive subtasks and memory-intensive subtasks. Scan the remaining resource vectors of each physical node in the local private cloud, and deploy compute-intensive subtasks and memory-intensive subtasks complementaryly on the same physical node to fill the resource gaps caused by the orthogonality of resource requirements.

7. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 6, characterized in that, The type of the current subtask is determined based on its multidimensional resource requirement vector, including: Extract the multidimensional resource requirement vector of the subtask, wherein the multidimensional resource requirement vector includes the normalized expected computing unit utilization and peak memory usage; If the normalized expected computation unit utilization of the subtask is greater than its peak memory usage, then the subtask is determined to be a compute-intensive subtask; otherwise, it is determined to be a memory-intensive subtask.

8. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 1, characterized in that, In step S4, if the waiting time of the current task in the local queue exceeds the tolerance time, cross-cloud offloading is performed based on the privacy sovereignty tag of the current task, including: Verify the Boolean privacy sovereignty label for the current subtask: If the label indicates that the subtask is subject to local execution constraints, then the execution domain of the subtask is forcibly locked to the private cloud, and only the de-anonymized intermediate feature tensor output by the subtask is transmitted across the wide area network to the downstream nodes of the public cloud. If the label indicates that the subtask allows cross-cloud deployment, then the original input data or intermediate features of the subtask are offloaded to the public cloud for execution.

9. The scheduling method for decoupling inference intent of large-scale multimodal models in hybrid cloud environments according to claim 1, characterized in that, The reward function of the deep reinforcement learning policy network is constructed as a composite penalty term, which simultaneously penalizes the risk of service level agreement default due to extended tolerance time, as well as the rental costs incurred due to premature triggering of public cloud emergencies.

10. A scheduling system for decoupling inference intent in large-scale multimodal models in hybrid cloud environments, characterized in that, A scheduling method for decoupling inference intent for large-scale multimodal models in a hybrid cloud environment as described in any one of claims 1 to 9, the system comprising: The modality receiving and parsing module is used to receive large multimodal model inference requests containing multiple subtasks and parse their directed acyclic graph structure, generate multidimensional resource requirement vectors for each subtask, and attach privacy sovereignty labels to subtasks carrying sensitive information. The status monitoring module is used to acquire the fragmented status of resources in the hybrid cloud infrastructure in real time, forming a high-dimensional state space feature snapshot that includes the available computing power and memory reserves of each physical node in the private cloud. The cardinality query agent module is used to input the multidimensional resource demand vector and the high-dimensional state space feature snapshot into the deep reinforcement learning policy network to obtain a continuous time scalar, and use the continuous time scalar as the tolerance time for the current task to wait in the local private cloud queue. The task scheduling module is used to execute local deployment based on multidimensional resource requirements if the current task obtains a scheduling opportunity within the tolerance time; if the current task waits in the local queue for longer than the tolerance time, it performs cross-cloud offloading based on the privacy sovereignty label of the current task. The topology execution module is used to take the output features of the completed subtasks as the input of the downstream subtasks according to the topology of the directed acyclic graph, and repeat the above steps until all subtasks are completed.

Images