AI agent-oriented end-cloud collaborative reasoning method, system, device and medium

Abstract

This application discloses an edge-cloud collaborative reasoning method for AI agents, applied to edge devices. The method includes: acquiring environmental state information and a pre-set capability map of the edge device, which records the edge-side and cloud-side implementation methods and switching conditions for each AI capability; based on the environmental state and capability map, a dynamic collaborative decision-making model adaptively allocates task subtasks to the edge or cloud side, with the edge performing local lightweight reasoning and the cloud side uploading context and receiving deep reasoning results; continuously monitoring environmental changes, and reassigning unfinished subtasks and switching their execution positions when rescheduling conditions are met. This application achieves optimal matching between low latency on the edge and high intelligence on the cloud by jointly driving task allocation through environmental perception and capability maps, seamlessly degrading in weak network conditions, with edge response latency below 50 ms and weak network availability not less than 95%, overcoming the shortcomings of insufficient intelligence on pure edge devices and high latency and easy interruption of pure cloud devices.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and edge computing technology, specifically to an edge-cloud collaborative reasoning method and system for AI intelligent agents, which can be applied to edge devices such as smart cars and IoT terminals. Background Technology

[0002] The rapid development of software-defined vehicles and the Internet of Things (IoT) has placed higher demands on the real-time performance, intelligence, and reliability of edge AI. Existing solutions mainly suffer from the following shortcomings:

[0003] 1. Limitations of edge-side capabilities: Pure edge-side solutions are limited by computing resources and model size, resulting in low intelligence and poor ability to handle complex tasks; edge-side models cannot continuously learn and evolve, leading to stagnation in the intelligent experience.

[0004] 2. Excessive dependence on cloud: Pure cloud solutions heavily rely on network connectivity, and services are unavailable in weak or no network environments; network round-trip latency affects the real-time interactive experience (especially for real-time applications such as voice and video); uploading private data to the cloud poses security risks.

[0005] 3. Insufficient simple collaboration: Existing edge-cloud collaboration is mostly based on static task division, which cannot be dynamically adjusted according to the scenario; edge-cloud models are trained independently, and inconsistent knowledge leads to a fragmented experience; there is a lack of a unified task scheduling and state management mechanism.

[0006] Therefore, how to design an edge-cloud converged architecture that maximizes the effectiveness of edge-side small models with limited resources, achieves dynamic intelligent allocation and seamless switching of tasks, maintains knowledge consistency between edge and cloud models, and enables collaborative evolution has become an urgent technical problem to be solved. Summary of the Invention

[0007] This application provides an edge-cloud collaborative reasoning method and system for AI agents. Through a three-layer architecture of "lightweight edge perception + deep cloud cognition + intelligent collaborative scheduling," and dynamic decision-making jointly driven by environmental state and capability graphs, it achieves optimal task allocation and seamless migration between the edge and cloud. To achieve the above objectives, this application adopts the following technical solutions.

[0008] In a first aspect, embodiments of this application provide an edge-cloud collaborative reasoning method for AI agents, applied to edge devices, including:

[0009] Receive task requests;

[0010] Obtain the current environmental status information of the terminal device; the environmental status information includes at least one of network status, computing resource status, and privacy requirements.

[0011] Obtain a pre-configured intelligent agent capability map; the capability map contains multiple AI capability entries, each entry defining an edge-side implementation method identifier, a cloud-side implementation method identifier, and switching conditions for a capability;

[0012] Based on the environmental state information and the capability map, a dynamic collaborative decision-making model is used to determine the execution location for at least one subtask in the task request; the execution location is either on the edge or on the cloud.

[0013] When the execution location of the subtask is determined to be on the client side, the local lightweight inference model is invoked to perform inference according to the corresponding client side implementation method identifier, and the response result is obtained.

[0014] When the execution location of a subtask is determined to be the cloud side, the context data of the subtask is encapsulated and sent to the cloud side so that the cloud side can call the corresponding large model for deep inference and receive the inference results returned by the cloud side.

[0015] During task execution, the environmental status information is continuously monitored. In response to changes in the environmental status that meet the preset rescheduling conditions, the execution position of the unfinished subtasks is redefined and the execution position is switched.

[0016] Furthermore, it also includes:

[0017] After the model inference is performed on the device, the inference process data and user feedback data are recorded as local learning samples.

[0018] Under the condition of satisfying privacy protection constraints, gradient information or knowledge representation is extracted from the local learning samples and sent to the cloud side; the gradient information or knowledge representation is used by the cloud side to perform federated aggregation with multiple end-sides and generate incremental update packages of models adapted to the end-sides through knowledge distillation.

[0019] Receive the incremental update package sent from the cloud side, and apply the incremental update package to update the model on the local end side.

[0020] Furthermore, the process of switching the execution location specifically includes:

[0021] Capture the full session state, intermediate inference results, and user history interaction information of the current task, and encapsulate them as a context snapshot;

[0022] Based on the redefined execution location, the context snapshot is transferred along with the unfinished subtask to the corresponding target execution location, so that the inference engine at the target execution location can restore the task state based on the context snapshot and continue processing.

[0023] Furthermore, the dynamic collaborative decision-making model makes allocation decisions by constructing a multi-dimensional decision matrix, specifically including:

[0024] A network condition score is generated based on the network status, a resource availability score is generated based on the computing resource status, a privacy risk score is generated based on privacy requirements, and a real-time score is generated based on the task's latency requirements.

[0025] By combining at least three of the above-mentioned scoring inputs into a pre-defined scoring fusion model, the tendency scores for each subtask corresponding to the edge and cloud sides are obtained.

[0026] The execution location of the subtask is determined based on the relationship between the tendency score and a preset threshold.

[0027] Furthermore, the client-side model consists of a core base model and at least one skill plugin; the process of updating the client-side model includes:

[0028] Receive incremental update packages from the cloud and identify the update type;

[0029] When the update type is core model update, the updated parameter data is loaded in the background, and the system seamlessly switches to the new version of the core model after the current inference task is completed.

[0030] When the update type is a skill plugin update, the new version of the plugin is dynamically loaded and a route is established so that new requests use the new version of the plugin, while requests currently being processed continue to use the old version of the plugin until completion.

[0031] Furthermore, it also includes updating the agent's capability map, specifically including:

[0032] Receive new AI capability entries issued by the cloud side; the capability entries include the terminal-side implementation method identifier, the cloud-side implementation method identifier, and the switching conditions for the new capability;

[0033] The newly added capability entries are added to the local capability graph, and the corresponding end-side implementation components are downloaded from the cloud as needed to complete the dynamic expansion of capabilities.

[0034] Furthermore, the gradient information or knowledge representation is subjected to privacy protection processing before transmission, including applying differential privacy noise to it, or encrypting the gradient using a secure multi-party computation protocol, so that the cloud side can complete the aggregation without being able to decrypt individual gradients.

[0035] The knowledge distillation involves using the large model on the cloud side as the teacher model, distilling its output soft labels or intermediate layer representations to generate student model parameters adapted to the scale of the edge side, which serve as the basis for the incremental update package.

[0036] Furthermore, when the execution location of a subtask is determined to be on the edge, but the dynamic collaborative decision-making model assesses that the complexity of the subtask exceeds the upper limit of the edge's processing capacity, the edge-cloud collaborative reasoning method further includes:

[0037] While performing primary inference on the edge to generate preliminary results, the context data of the subtask and the preliminary results are asynchronously sent to the cloud side.

[0038] Receive the deep results returned by the cloud side after performing enhanced inference based on the context data;

[0039] The deep results are used as annotation signals to perform real-time knowledge feedback and fine-tuning of the model on the edge.

[0040] Furthermore, it also includes:

[0041] Collect historical network state data sequences and use predictive models to predict network state change trends in future time windows;

[0042] When it is predicted that we will enter a weak network area, we can cache the context and intermediate results generated by the tasks that currently depend on the cloud side to the edge side in advance, or trigger a downgrade switch from the cloud side to the edge side for the relevant sub-tasks in advance.

[0043] Secondly, embodiments of this application provide an edge-cloud collaborative reasoning system capable of implementing any of the foregoing edge-cloud collaborative reasoning methods, comprising:

[0044] The edge agent module is used to deploy on edge devices and includes an environment perception unit for collecting network status and computing resource status, a capability graph storage unit for storing the capability graph of the intelligent agent, and an edge inference unit for performing local model inference.

[0045] The cloud-side service module is used to provide cloud-based deep inference services, including large model inference clusters and federated learning aggregation units.

[0046] Collaborative middleware, comprising dynamic task collaboration units and a unified knowledge engine;

[0047] The dynamic task collaboration unit is used to obtain real-time environmental status and capability map from the end-side agent module, decompose the received task request into sub-tasks, and dynamically decide and allocate each sub-task to the execution position of the end-side inference unit or the cloud-side service module, and control the synchronization of context data between the end and the cloud.

[0048] The unified knowledge engine is used to receive privacy-preserving gradient information uploaded from the client side, trigger the federated learning aggregation unit to generate incremental update packages through knowledge distillation, and send them to the client-side agent module to update the local model.

[0049] Thirdly, embodiments of this application provide an electronic device, including: one or more processors;

[0050] A memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are able to implement the steps in the edge-cloud collaborative reasoning method described in any of the preceding claims.

[0051] Fourthly, embodiments of this application provide a computer-readable medium storing a computer program, which, when executed by a processor, can implement the steps in the edge-cloud collaborative reasoning method described in any of the preceding claims.

[0052] This application discloses an edge-cloud collaborative reasoning method for AI agents, applied to edge devices. The method includes: acquiring environmental state information and a pre-set capability map of the edge device, which records the edge-side and cloud-side implementation methods and switching conditions for each AI capability; based on the environmental state and capability map, a dynamic collaborative decision-making model adaptively allocates task subtasks to the edge or cloud side, with the edge performing local lightweight reasoning and the cloud side uploading context and receiving deep reasoning results; continuously monitoring environmental changes, and reassigning unfinished subtasks and switching their execution positions when rescheduling conditions are met. This application achieves optimal matching between low latency on the edge and high intelligence on the cloud by jointly driving task allocation through environmental perception and capability maps, seamlessly degrading in weak network conditions, with edge response latency below 50 ms and weak network availability not less than 95%, overcoming the shortcomings of insufficient intelligence on pure edge devices and high latency and easy interruption of pure cloud devices. Attached Figure Description

[0053] Figure 1 This application provides a core flowchart of an edge-cloud collaborative reasoning method for AI agents.

[0054] Figure 2 A schematic diagram of the module structure of an edge-cloud collaborative reasoning system for AI agents provided in this application embodiment;

[0055] Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0056] To enable those skilled in the art to better understand the technical solutions of this application, exemplary embodiments of this application are described below with reference to the accompanying drawings, including various details of the embodiments of this application to aid understanding. These should be considered merely exemplary. Therefore, those skilled in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this application. Similarly, for clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. Unless otherwise specified, the various embodiments of this application and the features within those embodiments can be combined with each other.

[0057] As used herein, the term "and / or" includes any and all combinations of one or more of the associated enumerated entries. The terminology used herein is for describing particular embodiments only and is not intended to limit the application. As used herein, the singular forms "a" and "the" are also intended to include the plural forms, unless the context clearly indicates otherwise. It should also be understood that when the terms "comprising" and / or "made of" are used herein, the presence of the stated feature, integral, step, operation, element, and / or component is specified, but the presence or addition of one or more other features, integrals, steps, operations, elements, components, and / or groups thereof is not excluded. Terms such as "connected" or "linked" are not limited to physical or mechanical connections but can include electrical connections, whether direct or indirect.

[0058] Unless otherwise specified, all terms used in this application (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art. It should also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and this application, and will not be interpreted as having an idealized or overly formal meaning, unless expressly so defined in this application.

[0059] like Figure 2 As shown, the edge-cloud collaborative reasoning system in one embodiment of this application adopts a three-layer architecture: edge-side AI agent, intelligent collaboration layer, and cloud-side AI brain.

[0060] The edge AI agent (edge ​​agent module) is deployed on an in-vehicle computing platform or smart terminal, and includes an environment perception unit, a capability graph storage unit, and an edge inference unit. The environment perception unit collects network status, computing load, and privacy requirements through NetworkMonitor, ComputeResourceMonitor, and PrivacyGuard; the capability graph storage unit maintains metadata describing AI capabilities and their edge-cloud implementation methods; the edge inference unit loads a lightweight core base model and on-demand skill plugins, and has a local knowledge base.

[0061] The cloud-based AI brain (cloud-based service module) comprises a large-scale model inference cluster and a federated learning aggregation unit. The inference cluster deploys a basic model with hundreds of billions of parameters and a domain-tuned model, responsible for deep inference; the federated learning aggregation unit performs privacy-preserving aggregation and knowledge distillation.

[0062] The intelligent collaboration layer (collaboration middleware) includes the Dynamic Task Collaboration Unit (DTC) and the Unified Knowledge Engine (UKE). The DTC is responsible for obtaining real-time environment and capability graphs from the edge agent, decomposing tasks, and making decisions on task allocation; the UKE is responsible for incremental updates, consistency synchronization, and context management of the edge-cloud model.

[0063] The following section uses an intelligent cockpit scenario as an example to detail the specific implementation steps of the edge-cloud collaborative reasoning method for AI agents proposed in this application, such as... Figure 1 As shown.

[0064] Step S101: Receive task request.

[0065] The device receives task requests from users via voice, touch, or other means, such as "plan a route home and buy coffee along the way." This request is then sent to the device's task queue.

[0066] Step S102: Obtain the current environmental status information of the terminal device; the environmental status information includes at least one of network status, computing resource status, and privacy requirements.

[0067] The environmental perception unit collects real-time data on current network conditions (e.g., 5G, 4G, no network), edge chip computing power utilization, memory usage, battery level, and other computing resource statuses, as well as the privacy requirements of the task. For example, the current network bandwidth is 100 Mbps, computing power utilization is 60%, and voice data privacy is high. This environmental information will serve as the basis for subsequent decision-making.

[0068] Step S103: Obtain a pre-configured intelligent agent capability map; the capability map contains multiple AI capability entries, and each entry defines the edge-side implementation method identifier, cloud-side implementation method identifier, and switching conditions of a capability.

[0069] The pre-defined agent capability map is read from the capability map storage unit. This map defines the edge-side implementation, cloud-side implementation, and switching conditions for each AI capability. An example is the "Intent Recognition" capability, implemented on the edge as "tiny_bert_50M" and on the cloud as "chatglm3_130B." Switching thresholds include switching to the cloud when the edge-side model confidence is below 0.85, the latency requirement is above 200 ms, or the network bandwidth is below 100 Kbps. The "Multi-Step Planning" capability is configured to always be executed on the cloud.

[0070] Through the capability graph, the system has a unified grasp of the dual implementation index and scheduling rules of all its AI capabilities, from the edge to the cloud.

[0071] Step S104: Based on the environmental state information and the capability map, use a dynamic collaborative decision-making model to determine the execution location for at least one subtask in the task request; the execution location is either on the edge or on the cloud.

[0072] The dynamic task collaboration unit receives a task request and first performs semantic understanding and subtask decomposition. For example, "planning the route home and buying coffee" is decomposed into "speech-to-text subtask" and "multi-waypoint route planning subtask".

[0073] Next, based on the environmental state and capability map, a dynamic collaborative decision-making model is used to determine the execution location for each sub-task. This decision-making model specifically includes the following processes:

[0074] A multi-dimensional decision matrix is ​​constructed, including: generating a network condition score based on current network bandwidth and latency; generating a resource availability score based on CPU / memory availability; generating a privacy risk score based on the sensitivity of the data required by the task; and generating a real-time score based on the task's real-time requirements. These four scores are input into a weighted summation score fusion model. The weights of this model are continuously optimized using an online multi-armed learning strategy to derive the bias score for each subtask towards the edge and cloud sides. For example, the "speech-to-text" subtask has a network score of 9, a computing power score of 8, high real-time requirements but high privacy requirements, and an edge bias score of 0.92, exceeding the edge threshold of 0.8, so it is assigned to the edge side. Conversely, the "multi-pathway planning" subtask involves numerous real-time dynamic constraints and has extremely high complexity, resulting in a cloud bias score of 0.85, exceeding the cloud threshold of 0.7, so it is assigned to the cloud side.

[0075] If a subtask is marked "always in the cloud", it will be assigned directly to the cloud side regardless of the score.

[0076] Step S105: When the execution location of the subtask is determined to be the edge, the local lightweight inference model is called to perform inference according to the corresponding edge implementation method identifier, and the response result is obtained.

[0077] For subtasks assigned to the edge, the edge inference unit calls the local lightweight inference model to perform inference based on the corresponding edge implementation identifier in the capability graph. For example, the speech-to-text subtask loads the edge streaming ASR model to convert speech into text "Going home, passing by Starbucks" in real time.

[0078] The edge model (i.e., the model on the device) adopts a "core base model + skill plugins" architecture. The core base model is a simplified but versatile model with billions of parameters. Skill plugins, such as ASR, NLU, and air conditioning control, are dynamically loaded as needed. During inference, the core model first obtains a basic understanding, then matches and calls the corresponding plugins to enhance the output according to the requirements, and finally performs personalized adaptation through the local knowledge base. For example, if the local knowledge base records the user's preference "the Starbucks I often go to is on Zhongshan Road," it can directly correct the recognition result.

[0079] Step S106: When the execution location of the subtask is determined to be the cloud side, the context data of the subtask is encapsulated and sent to the cloud side so that the cloud side can call the corresponding large model for deep inference and receive the inference results returned by the cloud side.

[0080] For subtasks assigned to the cloud, the client encapsulates the currently accumulated contextual data (including recognized text, user preferences, historical interaction records, etc.) into a request and sends it to the cloud service module via the collaboration middleware. The cloud-based intelligent router selects the most suitable large model (such as a route planning-specific fine-tuning model) based on the subtask type, performs deep inference, generates a structured result containing a complete route and purchase suggestions, and returns it to the client. Simultaneously, the cloud extracts knowledge from the results, generating a knowledge increment package that can be used for subsequent updates on the client.

[0081] Step S107: During task execution, continuously monitor the environmental status information, and in response to changes in the environmental status that meet the preset rescheduling conditions, re-determine the execution position of the unfinished sub-tasks and switch the execution position.

[0082] During cloud-based inference or edge-cloud collaboration, if environmental monitoring detects significant changes in network status, computing load, etc., and meets preset rescheduling conditions (such as network interruption or sudden changes in computing power), the system will trigger an execution location switch. This switch is achieved through a context snapshot mechanism: first, the full session state of the current task, the intermediate inference results of each subtask, and the user interaction history are captured and serialized into a context snapshot; then, according to the reassigned target location, the snapshot and unfinished subtasks are transferred to the edge or cloud side. After receiving the snapshot, the inference engine on the target side directly restores the task state and continues execution, without requiring the user to re-enter information, ensuring a seamless experience.

[0083] For example, if a vehicle enters a tunnel and causes a cloud-side subtask to become unresponsive, the device immediately downgrades the incomplete complex planning subtask to be processed by the local rule engine, generates a basic navigation route, and temporarily stores the context. Once the network is restored, the device pushes the cached context and the intermediate results of local processing to the cloud. The cloud then enhances the results and returns the final route, with no redundant interactions throughout the process.

[0084] Furthermore, the system can perform proactive predictions: using an LSTM prediction model deployed on the device side, it can predict 60 seconds in advance that a vehicle is about to enter a tunnel without a signal, based on the navigation path and historical road signal strength data. At this time, it proactively triggers a downgrade switch for subtasks currently being executed on the cloud side, switching the high-precision map service in the cloud to the device-side basic navigation in advance, and caching the real-time data from the cloud to the local machine, achieving a zero-interruption transition.

[0085] Step S108: Collaborative evolution and update of end-side models.

[0086] After inference is performed on the device, the inference process and user feedback are recorded as local learning samples. When privacy constraints are met (such as during overnight charging), model gradient information is extracted from the samples, differential privacy noise is applied, and then uploaded. The cloud-based federated learning aggregation unit collects the noisy gradients uploaded from multiple devices, obtains global gradient updates through secure aggregation, and then performs knowledge distillation using a large model as the teacher and a small model on the device as the student, generating a device-compatible incremental update package.

[0087] After receiving the incremental update packet, the client identifies the update type and applies the update. If it is a skill plugin update, the new version is loaded in the background, new requests are routed to the new plugin, and ongoing requests are still handled by the old version until completion, achieving zero-interruption hot updates. If it is a core base model update, a shadow model mechanism is used, preloading new weights and atomically replacing the inference pointer to achieve hot switching. The entire update process is transparent to the user, and capabilities continue to evolve.

[0088] In some embodiments, asynchronous dual-track inference can be employed to achieve a better balance between response speed and accuracy. When task complexity is close to the capability limit when assigned to edge execution, the edge asynchronously sends the context and the initial result to the cloud for enhanced inference while outputting the initial result. After the cloud returns the deep result, the edge uses this deep result as a supervision signal to fine-tune the sorting or embedding layer parameters of the edge model online, so that each request brings an immediate improvement in model capability.

[0089] Step S109: Dynamic expansion of the agent's capability map.

[0090] To support the evolution of the AI ​​ecosystem, the edge device can dynamically expand its capability graph. When it receives a new capability entry (such as the "Children's Companion Story Generation" capability) from the cloud, including the edge device's implementation as a "Lightweight Story Tree Engine," the cloud device's implementation as a "Large Creation Model," and the switching conditions, the edge device automatically adds the entry to its local graph and downloads the corresponding edge device skill plugin as needed. Afterward, the system can schedule the new capability according to a unified collaborative process, enabling plug-and-play AI skills.

[0091] Table 1 shows a comparison of the measured data of the technical solution of this application with that of the prior art.

[0092] Table 1. Comparison of measured data between the technical solution of this application and existing technologies.

[0093]

[0094] In summary, this application constructs a complete and self-evolving edge-cloud collaborative reasoning closed loop through a series of mechanisms, including dynamic decision-making jointly driven by environment and capability graphs, lossless context transfer, incremental updates via federated distillation, and predictive degradation. This solves the problems of rigidity, fragmentation, and vulnerability in existing solutions, and combines high real-time performance, high intelligence, and high robustness.

[0095] The embodiments of the aforementioned edge-cloud collaborative reasoning method and the embodiments of the aforementioned edge-cloud collaborative reasoning system are identical or related in technical concept. They can be referenced and learned from each other in terms of technical details and technical effectiveness, which will not be repeated here.

[0096] Overall, the advantages of this application compared to the prior art include:

[0097] 1. Lack of edge-side intelligent solidification and evolution capabilities → Achieving continuous evolution of privacy and security through federated distillation

[0098] Existing purely edge-based solutions are limited by computing resources and model size; once deployed, the model becomes static and cannot continuously learn and evolve using new data, resulting in a stagnant intelligent experience over time. While purely cloud-based solutions can update the model in real time, they require the upload of user privacy data, posing a significant risk of leakage. This application addresses this issue by recording inference and feedback samples on the edge, extracting gradients from the samples and applying differential privacy noise when privacy constraints are met, and then uploading them. The cloud uses federated learning to aggregate multi-party noisy gradients, and then uses a large model as a teacher to perform knowledge distillation, generating incremental update packages adapted for the edge and distributing them. This mechanism allows the edge model to continuously evolve without leaking original user data, improving user habit learning accuracy by 35%, and the update process is transparent and imperceptible to the user.

[0099] 2. Strong cloud-side dependency leads to unavailability in weak networks → Ensure service continuity through environment awareness and dynamic rescheduling.

[0100] Existing pure cloud-based solutions heavily rely on network connectivity, leading to complete service interruptions in weak or no-network environments. Even some collaborative solutions only statically divide tasks, failing to adapt to real-time network changes. This application continuously monitors environmental conditions such as network status and computing resources. During task execution, if environmental changes meet preset rescheduling conditions, the execution position of unfinished subtasks is immediately redefined. When the network is interrupted, a context snapshot mechanism seamlessly migrates the cloud-based task state to the edge for degradation processing, and then synchronously enhances it once the network is restored. This achieves over 95% availability of core functions in weak network environments, an 850% improvement compared to pure cloud solutions, ensuring continuous service without user disruption.

[0101] 3. Simple edge-cloud collaboration leads to inconsistent knowledge and fragmented user experience → Adaptive optimal allocation is achieved through capability graphs and multi-dimensional decision-making.

[0102] Existing edge-cloud collaborative solutions mostly involve static task partitioning, with edge and cloud models trained and updated independently. Inconsistent knowledge versions lead to drastically different performances for the same task on both sides, resulting in a fragmented user experience. This application introduces an intelligent agent capability graph to uniformly describe the edge-side and cloud-side implementations and switching conditions of each AI capability. A multi-dimensional decision matrix is ​​constructed, quantifying and scoring based on four dimensions: network conditions, computing resources, privacy requirements, and latency requirements. A scoring fusion model dynamically determines the optimal execution position for each sub-task. This mechanism enables the system to adaptively select inference paths based on real-time scenarios, reducing average response latency by 85% compared to pure cloud solutions, ensuring consistent edge-cloud experience, and improving resource utilization by 40%.

[0103] 4. Task migration leads to context loss and duplicate interactions → Achieve lossless state migration through context snapshots.

[0104] Existing edge-to-cloud switching solutions typically only transmit the original request and discard intermediate inference states during task transfer, forcing the target end to start processing from scratch and compelling users to repeatedly describe their requirements, resulting in a fragmented user experience. This application, when switching execution locations, first captures the full session state of the current task, the intermediate inference results of each subtask, and the user interaction history, encapsulating this into a context snapshot, which is then transferred to the target execution location along with any unfinished subtasks. Upon receiving the snapshot, the target inference engine directly continues processing based on the saved intermediate states, eliminating the need for repeated user input. This ensures zero loss of contextual information during edge-to-cloud migration, making the switching process completely transparent to the user.

[0105] 5. Model updates cause service interruptions → Achieve zero-downtime hot updates through a dual-track approach of shadow models and plugins.

[0106] Existing edge-side model update solutions typically require stopping the inference service, replacing model files, and restarting the process, resulting in service interruptions of several seconds to several minutes, which is unacceptable for scenarios with high real-time requirements, such as automotive applications. This application differentiates update types and implements different online update strategies: for skill plugin updates, after the new version is loaded in the background, new requests are routed to the new plugin, while in-progress requests are still handled by the old version until completion; for core base model updates, a shadow model mechanism is used to preload new weights, and after verification, the inference pointers are atomically replaced to complete the hot switch. The entire update process has zero service interruption time, ensuring 24 / 7 continuous service capability.

[0107] Based on the same inventive concept, embodiments of this application also provide an electronic device. Figure 3 This is a structural block diagram of an electronic device provided in an embodiment of this application. Figure 3As shown in the embodiments of this application, an electronic device includes: one or more processors 101, a memory 102, and one or more I / O interfaces 103. The memory 102 stores one or more programs, which, when executed by the one or more processors, enable the one or more processors to implement any of the end-to-cloud collaborative reasoning methods described in the above embodiments; the one or more I / O interfaces 103 are connected between the processor and the memory, configured to enable information interaction between the processor and the memory.

[0108] The processor 101 is a device with data processing capabilities, including but not limited to a central processing unit (CPU); the memory 102 is a device with data storage capabilities, including but not limited to random access memory (RAM, more specifically SDRAM, DDR, etc.), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), and flash memory (FLASH); the I / O interface (read / write interface) 103 is connected between the processor 101 and the memory 102, and can realize information interaction between the processor 101 and the memory 102, including but not limited to a data bus (Bus).

[0109] In some embodiments, the processor 101, memory 102, and I / O interface 103 are interconnected via bus 104, and thus connected to other components of the computing device.

[0110] In some embodiments, the one or more processors 101 include a field-programmable gate array.

[0111] This application also provides a computer-readable medium. The computer-readable medium stores a computer program, which, when executed by a processor, implements the steps in any of the end-to-cloud collaborative reasoning methods described in the above embodiments. The computer-readable storage medium can be volatile or non-volatile.

[0112] This application also provides a computer program product, including computer-readable code, or a non-volatile computer-readable storage medium carrying computer-readable code. When the computer-readable code is run in the processor of an electronic device, the processor in the electronic device executes the above-described end-to-cloud collaborative reasoning method.

[0113] Those skilled in the art will understand that all or some of the steps, systems, and apparatuses disclosed above, and their functional modules / units, can be implemented as software, firmware, hardware, or suitable combinations thereof. In hardware implementations, the division between functional modules / units mentioned above does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed collaboratively by several physical components. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, digital signal processor, or microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit (ASIC). Such software can be distributed on a computer-readable storage medium, which may include computer storage media (or non-transitory media) and communication media (or transient media).

[0114] As is known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable program instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technologies, portable compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical disc storage, magnetic cartridges, magnetic tape, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer. Furthermore, it is known to those skilled in the art that communication media typically contain computer-readable program instructions, data structures, program modules, or other data in modulated data signals such as carrier waves or other transmission mechanisms, and may include any information delivery medium.

[0115] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0116] The computer program instructions used to perform the operations of this application may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are personalized by utilizing the status information of the computer-readable program instructions. These electronic circuits can execute the computer-readable program instructions to implement various aspects of this application.

[0117] The computer program product described herein can be implemented specifically through hardware, software, or a combination thereof. In one alternative embodiment, the computer program product is specifically embodied in a computer storage medium; in another alternative embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0118] Various aspects of this application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processor of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner; thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0120] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0121] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction containing one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0122] Exemplary embodiments have been disclosed in this application, and while specific terminology has been used, it is used only and should be interpreted in a general illustrative sense and is not intended to be limiting. In some embodiments, it will be apparent to those skilled in the art that features, characteristics, and / or elements described in conjunction with particular embodiments may be used alone, or in combination with features, characteristics, and / or elements described in conjunction with other embodiments, unless otherwise expressly indicated. Therefore, those skilled in the art will understand that various changes in form and detail may be made without departing from the scope of this application as set forth by the appended claims.

Claims

1. A cloud-edge collaborative reasoning method for AI agents, applied to edge devices, characterized in that, include: Receive task requests; Obtain the current environmental status information of the terminal device; the environmental status information includes at least one of network status, computing resource status, and privacy requirements. Obtain a pre-configured intelligent agent capability map; the capability map contains multiple AI capability entries, each entry defining an edge-side implementation method identifier, a cloud-side implementation method identifier, and switching conditions for a capability; Based on the environmental state information and the capability map, a dynamic collaborative decision-making model is used to determine the execution location for at least one subtask in the task request; the execution location is either on the edge or on the cloud. When the execution location of the subtask is determined to be on the client side, the local lightweight inference model is invoked to perform inference according to the corresponding client side implementation method identifier, and the response result is obtained. When the execution location of a subtask is determined to be the cloud side, the context data of the subtask is encapsulated and sent to the cloud side so that the cloud side can call the corresponding large model for deep inference and receive the inference results returned by the cloud side. During task execution, the environmental status information is continuously monitored. In response to changes in the environmental status that meet the preset rescheduling conditions, the execution position of the unfinished subtasks is redefined and the execution position is switched.

2. The end-cloud collaborative reasoning method of claim 1, wherein, Also includes: After the model inference is performed on the device, the inference process data and user feedback data are recorded as local learning samples. Under the condition of satisfying privacy protection constraints, gradient information or knowledge representation is extracted from the local learning samples and sent to the cloud side; the gradient information or knowledge representation is used by the cloud side to perform federated aggregation with multiple end-sides and generate incremental update packages of models adapted to the end-sides through knowledge distillation. Receive the incremental update package sent from the cloud side, and apply the incremental update package to update the model on the local end side.

3. The end-cloud collaborative reasoning method of claim 1, wherein, The process of switching execution locations specifically includes: Capture the full session state, intermediate inference results, and user history interaction information of the current task, and encapsulate them as a context snapshot; Based on the redefined execution location, the context snapshot is transferred along with the unfinished subtask to the corresponding target execution location, so that the inference engine at the target execution location can restore the task state based on the context snapshot and continue processing.

4. The end-cloud collaborative reasoning method of claim 1 or 2, wherein, The dynamic collaborative decision-making model makes allocation decisions by constructing a multi-dimensional decision matrix, specifically including: A network condition score is generated based on the network status, a resource availability score is generated based on the computing resource status, a privacy risk score is generated based on privacy requirements, and a real-time score is generated based on the task's latency requirements. By combining at least three of the above-mentioned scoring inputs into a pre-defined scoring fusion model, the tendency scores for each subtask corresponding to the edge and cloud sides are obtained. The execution location of the subtask is determined based on the relationship between the tendency score and a preset threshold.

5. The edge-cloud collaborative reasoning method according to claim 2, characterized in that, The client-side model consists of a core base model and at least one skill plugin; the process of updating the client-side model includes: Receive incremental update packages from the cloud and identify the update type; When the update type is core model update, the updated parameter data is loaded in the background, and the system seamlessly switches to the new version of the core model after the current inference task is completed. When the update type is a skill plugin update, the new version of the plugin is dynamically loaded and a route is established so that new requests use the new version of the plugin, while requests currently being processed continue to use the old version of the plugin until completion.

6. The end-cloud collaborative reasoning method of claim 1, wherein, It also includes updating the agent's capability map, specifically including: Receive new AI capability entries issued by the cloud side; the capability entries include the terminal-side implementation method identifier, the cloud-side implementation method identifier, and the switching conditions for the new capability; The newly added capability entries are added to the local capability graph, and the corresponding end-side implementation components are downloaded from the cloud as needed to complete the dynamic expansion of capabilities.

7. The end-cloud collaborative reasoning method of claim 2, wherein, The gradient information or knowledge representation is processed for privacy protection before being sent, including applying differential privacy noise to it, or encrypting the gradient using a secure multi-party computation protocol, so that the cloud side can complete the aggregation without being able to decrypt individual gradients. The knowledge distillation involves using the large model on the cloud side as the teacher model, distilling its output soft labels or intermediate layer representations to generate student model parameters adapted to the scale of the edge side, which serve as the basis for the incremental update package.

8. The end-cloud collaborative reasoning method of claim 1, wherein, When the execution location of a subtask is determined to be on the edge, but the dynamic collaborative decision-making model assesses that the complexity of the subtask exceeds the upper limit of the edge's processing capacity, the edge-cloud collaborative reasoning method further includes: While performing primary inference on the edge to generate preliminary results, the context data of the subtask and the preliminary results are asynchronously sent to the cloud side. Receive the deep results returned by the cloud side after performing enhanced inference based on the context data; The deep results are used as annotation signals to perform real-time knowledge feedback and fine-tuning of the model on the edge.

9. The end-cloud collaborative reasoning method of claim 1, wherein, Also includes: Collect historical network state data sequences and use predictive models to predict network state change trends in future time windows; When it is predicted that we will enter a weak network area, we can cache the context and intermediate results generated by the tasks that currently depend on the cloud side to the edge side in advance, or trigger a downgrade switch from the cloud side to the edge side for the relevant sub-tasks in advance.

10. An end-cloud collaborative reasoning system capable of implementing the end-cloud collaborative reasoning method of any one of claims 1-9. include: The edge agent module is used to deploy on edge devices and includes an environment perception unit for collecting network status and computing resource status, a capability graph storage unit for storing the capability graph of the intelligent agent, and an edge inference unit for performing local model inference. The cloud-side service module is used to provide cloud-based deep inference services, including large model inference clusters and federated learning aggregation units. Collaborative middleware, comprising dynamic task collaboration units and a unified knowledge engine; The dynamic task collaboration unit is used to obtain real-time environmental status and capability map from the end-side agent module, decompose the received task request into sub-tasks, and dynamically decide and allocate each sub-task to the execution position of the end-side inference unit or the cloud-side service module, and control the synchronization of context data between the end and the cloud. The unified knowledge engine is used to receive privacy-preserving gradient information uploaded from the client side, trigger the federated learning aggregation unit to generate incremental update packages through knowledge distillation, and send them to the client-side agent module to update the local model.

11. An electronic device, comprising: include: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the steps in the end-to-cloud collaborative reasoning method according to any one of claims 1-9.

12. A computer-readable medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it can implement the steps in the end-to-cloud collaborative reasoning method according to any one of claims 1-9.

Images