Cloud edge-end computing power cooperative scheduling solution supporting AI application reasoning
By building an intelligent collaborative scheduling platform, AI inference tasks are dynamically allocated for collaborative execution across the cloud, edge, and terminal, solving the problem of passive resource allocation in the cloud and achieving efficient, secure, and low-cost AI inference resource management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-16
- Publication Date
- 2026-04-07
AI Technical Summary
The existing cloud-based centralized processing model cannot understand the internal working logic and quality requirements of AI applications, resulting in passive resource allocation, long processing cycles, high costs, and security risks.
Build an intelligent, data-driven collaborative scheduling platform. Through AI agent modeling and demand quantification, monitor the status in real time and dynamically allocate tasks for collaborative execution across the cloud, edge, and terminal. Employ a multimodal collaborative scheduling strategy for resource allocation.
It achieves efficient, secure, and low-cost allocation of AI inference tasks, taking into account both efficiency and user latency requirements, and improves the forward-looking and refined adaptation capabilities of resource scheduling.
Smart Images
Figure CN121807477A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of cloud rendering technology, and more specifically to a cloud-edge-device computing power collaborative scheduling solution that supports AI application inference. Background Technology
[0002] With the popularization of artificial intelligence technology, AI applications are penetrating various industries, such as image restoration (e.g., intelligent background removal, content elimination), dynamic generation, and intelligent rendering. Current technologies utilize a centralized cloud processing model, where users upload tasks to cloud servers, where the powerful computing capabilities of the cloud perform AI inference before returning the results to the user's terminal.
[0003] While this model can handle complex tasks, it cannot understand the internal working logic and quality of service (QoS) requirements of AI applications. It can only passively respond to resource requests and cannot perform predictive scheduling. Furthermore, network transmission and cloud queuing may lead to long processing cycles, privacy leaks and data security risks, and increased cloud bandwidth and computing resource requirements resulting in high costs.
[0004] Therefore, a new solution is needed that can intelligently and flexibly allocate AI inference tasks across the cloud, edge, and device, while balancing efficiency, cost, and security. Summary of the Invention
[0005] To address the aforementioned technical problems, the purpose of this invention is to construct an intelligent, data-driven collaborative scheduling platform that dynamically and seamlessly allocates and collaboratively executes tasks across the edge, terminal, and cloud based on the real-time computing power requirements of AI inference tasks, terminal capabilities, network status, and user-tolerable latency thresholds.
[0006] To achieve the above objectives, the technical solution adopted by this invention is: a cloud-edge-device computing power collaborative scheduling solution to support AI application inference, comprising the following steps: S1: AI Application Modeling and Demand Quantification: Real-time analysis of the computational load of AI inference tasks, quantifying the computational resources required; S2: Define a tolerable response latency threshold T for different task types based on application scenarios and user settings; S3: Real-time status awareness and computing power demand prediction: The system continuously monitors the status changes of AI application workflows and predicts the type and amount of computing power required for the next task. S4: Cooperative scheduling strategy and resource allocation: Dynamically select one or more of the following cooperative modes based on computing power requirements and resource status; S5: Closed-loop feedback and optimization.
[0007] In a preferred embodiment of the present invention, S1 specifically includes: S11: Workflow Decomposition and Agent Identification: Decompose the target AI inference task into a workflow; identify the AI agent responsible for execution at each step or state node of the workflow; S12: Quantification of AI agents: Define the scope of tasks that each AI agent can handle; define the quality standards for the output results of each AI agent; each AI agent will emit status signals during its work, and the system will use the status signals to determine the progress of the AI application; S13: Based on the quantization results of S12, establish a table or model for the correspondence between each AI agent and the required computing resources under different working states.
[0008] In a preferred embodiment of the present invention, in S12: The scope of tasks each AI agent handles includes the resolution of the images being processed, the number of frames in the video being analyzed, and the length of the text being generated; The quality standards for the output of each AI agent include detection accuracy, rendering realism, and response latency requirements; Status signals include waiting for input, calculation in progress, calculation complete, and output ready.
[0009] In a preferred embodiment of the present invention, in S13: The function of the "demand-computing power" mapping model is that when the system clearly knows the specific AI agent that needs to perform the task and the two key conditions of the task's specific workload and quality standards, it can immediately map and output the type and specific quantity of computing power resources necessary to achieve the task.
[0010] As a preferred embodiment of the present invention, in step S2: when defining the tolerable response delay threshold T, the system comprehensively determines it based on the specific scenario of the AI application, the task type, and the user's real-time settings; First, different baseline latency requirements are preset for different application scenarios; Secondly, within the system, each identified AI agent is associated with a baseline latency value that matches its computational characteristics. When the AI agent is invoked into a specific workflow for execution, the system will combine the user's current service mode settings with the total response time allowed for the entire workflow to finally determine the latency threshold for the AI agent. This defined threshold T, as one of the key conditions, is input into the "demand-computing power" mapping model along with the "agent identifier" and "task workload" to query and obtain the type and quantity of computing power resources required to meet the baseline latency requirement, thus preparing the basis for resource scheduling.
[0011] In a preferred embodiment of the present invention, S3 specifically includes: S31: Workflow Status Monitoring: The system continuously monitors the status changes of AI application workflows. By capturing the status signals emitted by each AI agent, it perceives the current execution status, including the execution stage, execution subject, execution input, and output. S32: Dynamic Demand Calculation: Combining the "demand-computing power" mapping model, the type and quantity of computing power required to complete the current and next tasks are calculated in real time based on the currently captured AI application status and AI agent task parameters.
[0012] In a preferred embodiment of the present invention, S4 specifically includes: S41: Resource Assessment: Real-time assessment of available computing resources, network status, and load on the edge, cloud, and other sides; S42: Multimodal cooperative scheduling strategy: A. Inner loop: When the computational requirements are simple and the edge resources are sufficient to meet the quality standards, the task is completed entirely on the edge. B. Dual Loop: When the task is complex and the computing power on the edge is insufficient, some computationally intensive tasks are offloaded to edge nodes or the cloud. C. Three-loop: For applications that require support from multiple heterogeneous computing powers, global resource scheduling is initiated. The edge side is responsible for data collection and preliminary processing, the edge side is responsible for high-real-time collaborative inference, and the cloud side is responsible for large-scale model computation and result fusion. S43: Elastic resource provisioning: Based on strategic decisions, instructions are sent from the management platform to the underlying cloud, edge, and terminal infrastructure to dynamically allocate, expand, or release computing resources.
[0013] In a preferred embodiment of the present invention, S5 specifically includes: The system continuously monitors the actual performance of task execution and compares it with the predicted computing power demand. Using this feedback data, it continuously optimizes the "demand-computing power" mapping model in the first stage and the scheduling strategy in the third stage.
[0014] In a preferred embodiment of the present invention, the AI agent includes an object detection agent, an image rendering agent, or a speech synthesis agent, and the AI agent is predefined or customized.
[0015] Compared with the prior art, the present invention has the following technical effects: This invention centers on the AI application itself, with its core scheduling object being the specific AI agent and its workflow, based on the real-time status of the AI application and the quality requirements of the agent. Through application modeling, agent behavior perception, and multimodal collaboration strategies, computing power scheduling is upgraded from passive resource allocation to proactive service with forward-looking and refined scenario adaptation capabilities, thus truly realizing a leap from a "resource perspective" to an "application perspective." Attached Figure Description
[0016] Figure 1 This is a flowchart illustrating the cloud-edge-device computing power collaborative scheduling solution of the present invention. Detailed Implementation
[0017] The technical solution of the present invention will be further described below with reference to the accompanying drawings.
[0018] like Figure 1 As shown, this embodiment discloses a cloud-edge-device computing power collaborative scheduling solution to support AI application inference, including the following steps: S1: AI Application Modeling and Demand Quantification: Real-time analysis of the computational load of AI inference tasks, quantifying the computational resources required; Workflow Decomposition and Agent Identification: The target AI inference task is decomposed into a workflow. At each step or state node of the workflow, the AI agent responsible for execution (such as an object detection agent, an image rendering agent, a speech synthesis agent, etc.) is identified. These agents are either predefined or customized.
[0019] Quantification of the "quality" and "quantity" of intelligent agents: Define the scale or scope of the tasks processed by each intelligent agent, such as the resolution of the image being processed, the number of frames in the video being analyzed, and the length of the generated text; define the quality standards of the output results of each intelligent agent, such as detection accuracy, rendering realism, and response latency requirements; each intelligent agent will emit clear status signals during its work (such as "waiting for input", "computing", "computing completed", "output ready"), and these signals are the "tentacles" for the system to perceive the progress of the application.
[0020] Based on the above "quality" and "quantity", a table or model is established to correspond to the required computing resources for each intelligent agent under different working states.
[0021] The core objective of the "demand-computing power" mapping model is to build a decision-making system similar to a "dictionary" or "lookup table." Its function is to: when the system clearly knows "which AI agent needs to perform the task" (who needs to do the work) and the two key conditions of the task's specific "workload" (such as processing scale and throughput) and "quality requirements" (such as accuracy and latency standards), it can immediately map and output the "types and specific quantities of computing power resources" (what resources are needed) necessary to achieve the task, thereby providing a direct basis for accurate and automated resource scheduling.
[0022] S2: Define a tolerable maximum response latency threshold T for different task types based on application scenarios and user settings.
[0023] When defining the tolerable response latency threshold T, the system comprehensively determines it based on the specific scenario of the AI application, the task type, and the user's real-time settings.
[0024] First, different baseline latency requirements are preset for different application scenarios (such as real-time video analysis, voice interaction, or offline data processing).
[0025] Secondly, within the system, each identified AI agent (such as an object detection or speech synthesis agent) is associated with a baseline latency value that matches its computational characteristics. When the agent is invoked into a specific workflow, the system will combine the user's currently set service mode (such as "fast response" or "energy saving priority") with the total allowable response time of the entire workflow to finally determine the latency threshold for that agent.
[0026] This determined threshold T will be used as one of the key conditions, along with the "agent identifier" and "task workload," and input into the "demand-computing power" mapping model to query and obtain the precise type and quantity of computing power resources required to meet the latency requirement, thus preparing the basis for resource scheduling.
[0027] The baseline latency is the reasonable processing time that an AI agent is expected to or measured to complete a task under "standard working conditions".
[0028] S3: Real-time Status Awareness and Computing Demand Prediction ① Workflow Status Monitoring: The system continuously monitors the status changes of AI application workflows. By capturing status signals emitted by each agent, it accurately perceives which step has been executed, which agent is performing the action, and the status of its input and output.
[0029] ② Dynamic Demand Calculation: Combining the "demand-computing power" mapping model, the type and quantity of computing power required to complete the current and next tasks are calculated in real time based on the currently captured application state and agent task parameters. Simultaneously, user-level SLA requirements (such as task urgency and tolerable latency) are considered.
[0030] Real-time computing refers to the process during system operation where, as the AI workflow progresses, whenever a change in the state signal of an agent is detected (e.g., an agent changes from "waiting for input" to "computing"), the system immediately queries the established "demand-computing power" mapping model based on the current step, the agent's identifier (whether it is currently executing or about to execute), and the specific parameters of the agent's task (e.g., the image resolution or video frame rate to be processed), combined with the user-defined service level agreement (e.g., whether the current task is urgent and the maximum allowed delay time). This allows the system to quickly determine the type of computing power resources (e.g., whether a GPU or CPU is needed) and their specific quantity (e.g., how many computing cores or how much memory are needed) required to complete the current step or the next step.
[0031] This calculation process is continuous and dynamic, capable of updating computing power requirements in real time as the workflow progresses, providing an immediate and accurate basis for subsequent resource scheduling.
[0032] S4: Cooperative scheduling strategy and resource allocation: Dynamically select one or more of the following cooperative modes based on computing power requirements and resource status.
[0033] ①Resource assessment: Real-time assessment of available computing resources, network status, and load on the edge, cloud, and other sides.
[0034] Real-time evaluation refers to the continuous and uninterrupted collection and inspection of the real-time status of resources distributed across three locations—terminal devices, edge nodes, and cloud data centers—during the operation of AI applications.
[0035] Specifically, the system will periodically or through event triggers to proactively acquire three key pieces of information: first, how much idle computing power is currently available at each location, such as the remaining memory size and the number of processor cores that can be allocated; second, the quality of network connectivity, including the data transmission speed, stability, and current latency between the terminal and the edge, and between the edge and the cloud; and third, the current busy level of computing resources at each location, i.e. how much resource is being used by existing tasks and whether the load is too high.
[0036] The system integrates these instantaneously acquired status information to provide a basis for decision-making regarding where to execute tasks and how to allocate resources.
[0037] ② Multimodal cooperative scheduling strategy: A. Inner Loop (Edge-Side Autonomy): When computational needs are simple and edge-side resources are sufficient to meet both the "quality" and "quantity" requirements, the task is completed entirely on the edge. For example, a simple AI filter on a mobile phone.
[0038] B. Dual Loop (Edge-Edge or Edge-Cloud Collaboration): When tasks are complex and edge computing power is insufficient, some computationally intensive tasks (such as AI inference) are offloaded to edge nodes or the cloud. The system needs to decide which parts to offload and how to execute them in parallel or sequentially. For example, mobile games may delegate some rendering tasks to edge servers.
[0039] C. Three-loop (Cloud-Edge-Device Global Collaboration): For extremely complex applications requiring support from various heterogeneous computing power, global resource scheduling is initiated. The edge side is responsible for data collection and preliminary processing, the edge side is responsible for high-real-time collaborative inference, and the cloud side is responsible for large-scale model computation and result fusion. For example, real-time path planning and simulation for large-scale autonomous vehicle swarms.
[0040] ③ Elastic resource supply: Based on strategic decisions, the management platform sends instructions to the underlying cloud, edge, and terminal infrastructure to dynamically allocate, expand, or release computing resources.
[0041] S5: Closed-loop feedback and optimization: The system continuously monitors the actual performance of tasks (such as actual time consumption and output quality) and compares it with the predicted computing power requirements. Using this feedback data, the system continuously optimizes the "demand-computing power" mapping model in the first stage and the scheduling strategy in the third stage, making the system increasingly accurate and efficient.
[0042] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be modified within the scope of the concept described herein by means of the above teachings or the technology or knowledge in related fields.
Claims
1. A cloud-edge-device computing power collaborative scheduling solution to support AI application inference, characterized in that, Includes the following steps: S1: AI Application Modeling and Demand Quantification: Real-time analysis of the computational load of AI inference tasks, quantifying the computational resources required; S2: Define a tolerable response latency threshold T for different task types based on application scenarios and user settings; S3: Real-time status awareness and computing power demand prediction: The system continuously monitors the status changes of AI application workflows and predicts the type and amount of computing power required for the next task. S4: Cooperative scheduling strategy and resource allocation: Dynamically select one or more of the following cooperative modes based on computing power requirements and resource status; S5: Closed-loop feedback and optimization.
2. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 1, characterized in that, S1 specifically includes: S11: Workflow Decomposition and Agent Identification: Decompose the target AI inference task into a workflow; identify the AI agent responsible for execution at each step or state node of the workflow; S12: Quantification of AI agents: Define the scope of tasks that each AI agent can handle; define the quality standards for the output results of each AI agent; each AI agent will emit status signals during its work, and the system will use the status signals to determine the progress of the AI application; S13: Based on the quantization results of S12, establish a table or model for the correspondence between each AI agent and the required computing resources under different working states.
3. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference as described in claim 2, characterized in that, In S12: The scope of tasks each AI agent handles includes the resolution of the images being processed, the number of frames in the video being analyzed, and the length of the text being generated; The quality standards for the output of each AI agent include detection accuracy, rendering realism, and response latency requirements; Status signals include waiting for input, calculation in progress, calculation complete, and output ready.
4. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 2, characterized in that, In S13: The function of the "demand-computing power" mapping model is that when the system clearly knows the specific AI agent that needs to perform the task and the two key conditions of the task's specific workload and quality standards, it can immediately map and output the type and specific quantity of computing power resources necessary to achieve the task.
5. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 2, characterized in that: In S2, when defining the tolerable response delay threshold T, the system comprehensively determines it based on the specific scenario of the AI application, the task type, and the user's real-time settings. First, different baseline latency requirements are preset for different application scenarios; Secondly, within the system, each identified AI agent is associated with a baseline latency value that matches its computational characteristics. When the AI agent is invoked into a specific workflow for execution, the system will combine the user's current service mode settings with the total response time allowed for the entire workflow to finally determine the latency threshold for the AI agent. This defined threshold T, as one of the key conditions, is input into the "demand-computing power" mapping model along with "agent identifier" and "task workload" to query and obtain the type and quantity of computing power resources required to meet the baseline latency requirement, thus preparing the basis for resource scheduling.
6. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 2, characterized in that, Specifically, S3 includes: S31: Workflow status monitoring: The system continuously monitors the status changes of the AI application workflow, and perceives the current execution status by capturing the status signals emitted by each AI agent, including the execution stage, execution subject, execution input and output status; S32: Dynamic Demand Calculation: Combining the "demand-computing power" mapping model, the type and quantity of computing power required to complete the current and next tasks are calculated in real time based on the currently captured AI application status and AI agent task parameters.
7. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 1, characterized in that, S4 specifically includes: S41: Resource Assessment: Real-time assessment of available computing resources, network status, and load on the edge, cloud, and other sides; S42: Multimodal cooperative scheduling strategy: A. Inner loop: When the computational requirements are simple and the edge resources are sufficient to meet the quality standards, the task is completed entirely on the edge. B. Dual Loop: When the task is complex and the computing power on the edge is insufficient, some computationally intensive tasks are offloaded to edge nodes or the cloud. C. Three-loop: For applications that require support from multiple heterogeneous computing powers, global resource scheduling is initiated. The edge side is responsible for data collection and preliminary processing, the edge side is responsible for high-real-time collaborative inference, and the cloud side is responsible for large-scale model computation and result fusion. S43: Elastic resource provisioning: Based on strategic decisions, instructions are sent from the management platform to the underlying cloud, edge, and terminal infrastructure to dynamically allocate, expand, or release computing resources.
8. The cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to claim 1, characterized in that, S5 specifically includes: The system continuously monitors the actual performance of tasks and compares it with the predicted computing power requirements. Using this feedback data, it continuously optimizes the "demand-computing power" mapping model in the first stage and the scheduling strategy in the third stage.
9. A cloud-edge-device computing power collaborative scheduling solution for supporting AI application inference according to any one of claims 1-8, characterized in that: The AI agent includes an object detection agent, an image rendering agent, or a speech synthesis agent, and the AI agent is predefined or customized.