Distributed computing power scheduling method for large model and related device

Abstract

The application provides a large model-oriented distributed computing power scheduling method and related device, the method comprises: obtaining an agent task request and task data, the request comprising computing power demand, data sensitivity level and memory calling demand; based on the data sensitivity level and the computing power demand, determining the computing power source as edge computing power or public cloud computing power; if it is edge computing power, issuing a task scheduling instruction to the edge computing power node to control its exclusive allocation and release the computing resources after completion; if it is public cloud computing power, routing the task to a public cloud node; based on the memory calling demand, controlling the execution node to write the task execution memory with a timestamp into a shared knowledge base to trigger the knowledge base to perform time sequence sorting and gradient degradation processing. The application realizes the shunting of heterogeneous computing power, safeguards data privacy while preventing resource preemption of edge multi-task concurrency, and realizes the sharing and storage release of context features through the memory degradation mechanism.

Description

Technical Field

[0001] This application relates to the field of computing power scheduling technology, specifically to a distributed computing power scheduling method and related apparatus for large-scale models. Background Technology

[0002] Currently, the computational power scheduling for Large Language Models (LLMs) and agents mainly adopts centralized cloud deployment or local private deployment. In practical application scenarios, cloud deployment faces data security risks, makes it difficult to meet the compliance requirement of "data not leaving the domain", and the communication latency caused by remote data transmission cannot support high real-time inference tasks.

[0003] Edge computing power carriers in related technologies typically lack on-demand scheduling and coordination capabilities when handling agent tasks. When multiple complex tasks are executed in parallel and share the same physical computing power base, resource contention occurs, leading to decreased inference response speed or even task interruption. Simultaneously, agent workflows often exhibit high concurrency and long-term data dependencies, while the computing and storage resources of edge devices themselves have significant physical limitations. When dealing with continuous, massive amounts of agent tasks, computing power scheduling and underlying data management models struggle to balance the efficiency of multi-task execution with the continuous consumption of hardware resources while ensuring data privacy. Summary of the Invention

[0004] In view of the aforementioned problems, this application is proposed to provide a distributed computing power scheduling method and related apparatus for large-scale models that overcomes or at least partially solves the aforementioned problems, comprising: A distributed computing power scheduling method for large-scale models, applied to a computing power scheduling device, includes: The system obtains intelligent agent task requests and corresponding task data sent by the intelligent agent workflow system. The intelligent agent task requests include at least computing power requirements, data sensitivity levels, and memory retrieval requirements. Based on the data sensitivity level and the computing power requirement, the target computing power source for the intelligent agent task is determined, and the target computing power source includes edge computing power clusters or public cloud computing power access layer. If the source of the target computing power is determined to be the edge computing power cluster, a target edge computing power node is determined from the edge computing power cluster as an execution node, a task scheduling instruction containing an exclusive allocation strategy is generated and sent to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed. If the target computing power source is determined to be the public cloud computing power access layer, the target public cloud computing power node is used as the execution node, and the intelligent agent task and task data are routed to the execution node for processing. Based on the memory retrieval requirements, a memory scheduling instruction is generated for the shared knowledge base. The memory scheduling instruction is configured to: control the execution node corresponding to the agent task to encapsulate the task execution memory generated during the task execution process into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base to trigger the shared knowledge base to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on a preset memory degradation rule.

[0005] According to one embodiment of this application, determining the target computing power source for the agent task includes: Determine whether the data sensitivity level is higher than a preset security compliance threshold; If the data sensitivity level is higher than the security compliance threshold, the source of the target computing power is determined to be the edge computing power cluster, so as to ensure that the task data does not leave the domain; If the data sensitivity level is not higher than the security compliance threshold and the computing power requirement exceeds the total available computing power of the edge computing power cluster, the target computing power source is determined to be the public cloud computing power access layer.

[0006] According to one embodiment of this application, when the source of the target computing power is determined to be the edge computing power cluster, determining the target edge computing power node as the execution node from the edge computing power cluster further includes: The load status parameters of each candidate edge computing node in the edge computing power cluster are obtained in real time. The load status parameters include the CPU utilization, GPU utilization, and task queue length. Candidate edge computing nodes whose load status parameters exceed the preset load threshold are removed from the scheduling candidate list, resulting in an updated set of candidate nodes; From the set of candidate nodes, select a node that meets the computing power requirements and has an idle state as the execution node.

[0007] According to one embodiment of this application, generating a task scheduling instruction containing an exclusive allocation strategy and issuing it to the execution node includes: Simultaneously with issuing the task scheduling instruction, a resource lock identifier corresponding to the execution node is generated on the computing power scheduling device. Intercept and refuse to route other parallel agent tasks to the execution node with the resource locking identifier; The system receives task completion feedback and computing power release signals from the execution node, and simultaneously clears the resource lock flag to restore the execution node to an available state.

[0008] According to one embodiment of this application, the generation of memory scheduling instructions for a shared knowledge base includes: When the memory retrieval request indicates that historical memory needs to be retrieved, a memory query instruction is sent to the shared knowledge base to extract matching target shared memory data from the sorted temporal memory sequence in the shared knowledge base; The target shared memory data is fed back to the execution node so that the execution node can load it as contextual background information during the reasoning process to avoid repeated reasoning.

[0009] According to one embodiment of this application, the step of triggering the shared knowledge base to perform chronological sorting of the recorded memory data according to the timestamp and performing gradient degradation processing based on a preset memory degradation rule specifically includes: triggering the shared knowledge base to perform the following operations: Obtain the generation retention time and access frequency of each historical memory data in the shared knowledge base; If the generation and retention time of target historical memory data exceeds a first threshold and the access frequency is lower than a second threshold, gradient degradation is performed on the target historical memory data. The gradient degradation includes compressing the storage format and / or reducing the query priority of memory retrieval. If the storage space usage rate reaches a preset space threshold, historical memory data that has degraded to the third threshold will be cleaned up.

[0010] According to one embodiment of this application, before issuing a task scheduling instruction to the execution node or routing the agent task to the execution node, the method further includes: The task data is encrypted. Verify the legitimacy of the execution node's access to the current computing power network system; Security checks are performed on the permission level of the intelligent agent task to access the shared knowledge base.

[0011] According to one embodiment of this application, when the target computing power source is the public cloud computing power access layer, the method further includes: The scheduling instructions used to route the agent's tasks are converted into the standard calling protocol format supported by the public cloud computing power node; The converted task data is sent to the execution node via a public computing power interface; The system receives the inference results and the generated shareable memory data returned by the execution node, and generates corresponding memory writing instructions to write them into the shared knowledge base.

[0012] A distributed computing power scheduling device for large-scale models includes: The task parsing module is used to obtain the agent task request and corresponding task data sent by the agent workflow system. The agent task request includes at least computing power requirements, data sensitivity level and memory retrieval requirements. The computing power source determination module is used to determine the target computing power source for the intelligent agent task based on the data sensitivity level and the computing power requirement. The target computing power source includes an edge computing power cluster or a public cloud computing power access layer. The first scheduling module is used to determine the target edge computing node as the execution node from the edge computing cluster when the target computing power source is determined to be the edge computing power cluster, generate a task scheduling instruction containing an exclusive allocation strategy and send it to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed. The second scheduling module is used to, when it is determined that the source of the target computing power is the public cloud computing power access layer, use the target public cloud computing power node as the execution node and route the intelligent agent task and task data to the execution node for processing; The memory retrieval module is used to generate a memory scheduling instruction for the shared knowledge base based on the memory retrieval requirement. The memory scheduling instruction is configured to: control the execution node to encapsulate the task execution memory generated during task execution into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base to trigger the shared knowledge base to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on a preset memory degradation rule.

[0013] A computer device includes a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps of the distributed computing power scheduling method for large models as described above.

[0014] This application has the following advantages: In the embodiments of this application, by dynamically determining the source of computing power based on data sensitivity level and computing power requirements, flexible collaboration between edge clusters and public cloud resources is achieved, satisfying data privacy protection requirements while also taking into account computing demands exceeding edge load. For the edge execution environment, a dedicated local computing resource allocation and release mechanism is adopted to avoid computing power contention issues in multi-task concurrent scenarios, ensuring the execution stability of large model inference tasks. Furthermore, by introducing a timestamped memory encapsulation mechanism and a shared knowledge base, the retention and sharing of task execution states are achieved, reducing the overhead of repeated inference for subsequent similar tasks; simultaneously, dynamic lifecycle management of historical memory data is performed by combining temporal sorting and gradient degradation rules, effectively alleviating storage space pressure on edge devices and improving the continuous operation capability and overall utilization efficiency of distributed system resources. Attached Figure Description

[0015] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0016] Figure 1 This is an architecture diagram of an application environment for a distributed computing power scheduling method for large models provided in an embodiment of this application; Figure 2 This is a flowchart of a distributed computing power scheduling method for large models provided in an embodiment of this application; Figure 3 This is a partial flowchart of step S120 in a distributed computing power scheduling method for large models provided in an embodiment of this application; Figure 4 This is a partial flowchart of step S130 in a distributed computing power scheduling method for large models provided in an embodiment of this application; Figure 5 This is a flowchart of gradient degradation processing performed on the shared knowledge base in a distributed computing power scheduling method for large models provided in an embodiment of this application; Figure 6 This is a structural block diagram of a distributed computing power scheduling device for large models provided in an embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer electronic device provided in an embodiment of this application; 1. Computer electronic device; 2. External device; 3. Processing unit; 4. Bus; 5. Network adapter; 6. I / O interface; 7. Display; 8. Memory; 9. Random access memory; 10. Cache memory; 11. Storage system; 12. Program / utility; 13. Program module. Detailed Implementation

[0017] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0018] To make the objectives, technical solutions, and advantages of this application clearer, before providing a detailed description of the specific embodiments of this application, the following explanations and definitions are provided for some key technical terms involved in this application: Intelligent Agent: In this application, it specifically refers to a digital computing entity built based on a Large Language Model (LLM) or other artificial intelligence model, capable of perceiving the input environment, autonomously performing logical reasoning, planning and executing specific workflow tasks.

[0019] Edge computing node / cluster: refers to physical or virtual computing devices and their collections deployed close to the data generation source or network edge. In the specific scenario of this application, it includes, but is not limited to, industrial control computers, edge computing boxes, edge servers, and IoT gateway devices with a certain AI computing power (such as the ability to run large models with parameters at the tens of byte level).

[0020] Exclusive Allocation Strategy: This application proposes a scheduling mechanism to prevent resource preemption. It refers to an edge computing node entering a resource-locked state while undertaking a task for a specific agent. In this state, the node allocates its full local core computing and storage resources (such as CPU, GPU, and memory) exclusively to executing the current single agent task, and rejects routing access from other concurrent tasks at the system level until the current task is completed and a resource release signal is triggered.

[0021] Task execution memory refers to the collection of data with potential reuse value generated by an agent during the execution of historical tasks and logical reasoning. This includes, but is not limited to, task execution parameters, intermediate reasoning paths, key decision-making criteria, contextual dialogue history, and the final reasoning result.

[0022] Gradient Degradation: This application proposes a memory lifecycle management mechanism for edge storage-constrained environments. It refers to a multi-level dimensionality reduction process implemented on memory data in a shared knowledge base based on dynamic indicators such as the generation time, access frequency, or value weight of the memory data. Specific processing methods include, but are not limited to: converting low-frequency active data into a compressed storage format, reducing the query and retrieval priority of inactive data, and physically erasing long-tail data that has reached a threshold to free up storage space.

[0023] Data Sensitivity Level: This refers to a quantitative security label assigned to task data based on its business domain (e.g., government affairs, industrial control, finance), compliance requirements (e.g., "data not leaving the domain" regulations), and privacy attributes. This level serves as the core decision-making basis for triggering cloud-edge routing in this application.

[0024] Reference Figure 1 As shown in the embodiments of this application, the distributed computing power scheduling method for large models can be applied to, for example... Figure 1 The application environment shown includes an intelligent agent workflow system 101, a computing power scheduling device (central allocation node) 103, an edge computing power cluster 104, a public cloud computing power access layer 105, and a shared knowledge base 106. The intelligent agent workflow system 101 connects to the workflow interaction layer 102 via a network and interacts with the computing power scheduling device 103 throughout the entire process.

[0025] The workflow interaction layer 102 is used to interface with the external intelligent agent workflow system 101, providing interactive interfaces for task input, result output, computing power configuration, and memory query. Specifically, it is responsible for receiving task requests that include task content, computing power requirements, sensitivity level, and memory retrieval requirements, and supports users to manually specify the computing power source (edge ​​or public cloud) and configure memory read and write permissions, providing basic data support for subsequent scheduling decisions.

[0026] The computing power scheduling device 103, as the core control unit, achieves unified management and distribution of network resources through its internal computing power routing module and memory scheduling module. It establishes stable communication links with the edge computing power cluster 104, the public cloud computing power access layer 105, and the shared knowledge base 106, and performs intelligent routing and permission allocation based on task characteristics.

[0027] The edge computing cluster 104 consists of multiple distributed edge computing nodes. Each node uses an industrial control computer-like computing box to support the high computing power (e.g., 200 TOPS) required for large model inference. The cluster is interconnected via a local area network or a dedicated network, supporting hot-swapping of nodes for elastic expansion. Each edge computing node is configured to independently undertake a single intelligent agent task, achieving dedicated use of computing resources and ensuring that sensitive data is inferred and stored locally without leaking out of the node.

[0028] The public cloud computing access layer 105, as a supplement to edge computing power, achieves seamless integration with mainstream public cloud computing platforms through standardized public computing interfaces and protocol conversion modules. It is primarily used to support non-sensitive intelligent agent tasks with high computing power requirements, ensuring smooth operation of these tasks under different computing power environments.

[0029] The shared knowledge base 106 is mounted on the edge computing cluster 104 and adopts a distributed storage architecture to retain necessary memories generated during the execution of various tasks. This knowledge base simulates human memory patterns, sorting all memory data by generation timestamps to form a temporal sequence. Simultaneously, its built-in memory degradation mechanism can compress or clean up early, low-value memories according to preset rules (such as retention time and access frequency), saving edge storage space while achieving cross-task memory sharing and avoiding repetitive reasoning.

[0030] In one embodiment, the intelligent agent workflow system 101, in response to a front-end business trigger event, generates an intelligent agent task request that includes computing power requirements, data sensitivity level, and memory retrieval requirements, and sends the request to the computing power scheduling device 103 through the workflow interaction layer 102. Upon receiving the request, the computing power scheduling device 103 parses the task characteristics and, based on the data sensitivity level and computing power requirements, makes an intelligent routing and distribution decision to determine the target computing power source for the intelligent agent task (edge ​​computing power cluster 104 or public cloud computing power node).

[0031] When the computing power scheduling device 103 determines that the target computing power source is the edge computing power cluster 104, it selects idle target edge computing power nodes from the cluster as execution nodes and issues task scheduling instructions containing exclusive allocation strategies to them. After receiving the instructions, the execution node exclusively allocates local computing resources to the task during the execution of the agent's task and releases them after the task is completed. At the same time, the execution node interacts with the shared knowledge base 106 based on the memory scheduling instructions to read the memory data of the auxiliary execution; and after the inference is completed, it encapsulates the generated task execution memory into a memory data packet with a timestamp and writes it into the shared knowledge base 106, thereby triggering the shared knowledge base 106 to sort the recorded memory data in time sequence according to the timestamp and perform gradient degradation processing based on the preset memory degradation rules.

[0032] The terminal device of the aforementioned intelligent agent workflow system 101 can be, but is not limited to, various personal computers, laptops, smartphones, tablets, industrial cameras, sensors, drones, and IoT devices. The aforementioned edge computing nodes can be, but are not limited to, hardware carriers with certain AI large-scale model inference capabilities, such as industrial control computers, edge computing boxes, micro servers, and local workstations.

[0033] Reference Figure 2 This illustration shows a distributed computing power scheduling method for large-scale models provided by an embodiment of this application. The method can be executed by a computing power scheduling device 103 and includes the following steps: S110. Obtain the agent task request and corresponding task data sent by the agent workflow system 101. The agent task request includes at least computing power requirements, data sensitivity level, and memory retrieval requirements.

[0034] Specifically, the computing power scheduling device 103 establishes a data connection with the intelligent agent workflow system 101 through a network communication interface, thereby receiving raw request data sent by each business node in the intelligent agent workflow system 101. This raw request data, as the initial data stream sent from the upstream business end, arrives at the computing power scheduling device 103 via asynchronous or synchronous communication. After receiving this initial data stream, the computing power scheduling device 103 parses the raw request data and extracts the core features required for large model inference. In the actual data processing, the computing power scheduling device 103 identifies and extracts the core features through field matching or protocol decoding mechanisms. These core features include: task content representing the specific execution logic; computing power requirements representing the computing power scale (e.g., multi-dimensional computing power indicators including the estimated total number of floating-point operations and the minimum GPU memory usage); data sensitivity level representing the data privacy level (e.g., a score used to characterize data compliance requirements); a real-time threshold limiting task processing time; and memory retrieval requirements representing whether historical inference results need to be called.

[0035] After the above analysis and extraction steps, the computing power scheduling device 103 further separates the task data (such as text, images, or structured data to be processed) from the original request data, which serves as the input object for large model inference. The computing power scheduling device 103 structurally binds and stores the extracted core features with the task data, and transforms them into standardized input parameters for subsequent routing determination and task issuance. This step establishes a data mapping relationship from upstream business nodes to the computing power scheduling center, realizes the conversion of unstructured business requests into standardized scheduling instruction parameters, and provides reliable data for routing in the underlying heterogeneous computing power environment.

[0036] S120. Based on the data sensitivity level and the computing power requirement, determine the target computing power source for the intelligent agent task. The target computing power source includes edge computing power cluster 104 or public cloud computing power access layer 105.

[0037] In this step, the computing power scheduling device 103 performs a joint feature evaluation on the incoming data sensitivity level and computing power requirement through the underlying resource addressing module. Specifically, the computing power scheduling device 103 constructs a task feature vector from the parsed sensitivity level parameters and computing power scale parameters, and inputs it into its internally configured routing decision algorithm for data mapping. Based on this routing decision algorithm, the computing power scheduling device 103 calculates the spatial distribution of the task feature vector in a multi-dimensional feature space defined by security compliance thresholds and edge load thresholds, thereby obtaining the adaptability distribution of the agent task to different computing environments and generating a directional identifier with constraint attributes. According to the generated directional identifier, the computing power scheduling device 103 logically defines the data flow direction of the task and maps it to the edge computing power cluster 104 or the public cloud computing power access layer 105.

[0038] S130. If the source of the target computing power is determined to be the edge computing power cluster 104, the target edge computing power node is determined from the edge computing power cluster 104 as the execution node, a task scheduling instruction containing an exclusive allocation strategy is generated and sent to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed.

[0039] In other words, when the target computing power source is determined to be edge computing power, the computing power scheduling device 103 queries the available resource pool topology of the edge computing power cluster 104 and allocates a matching execution node according to the computing power demand. After determining the execution node, the computing power scheduling device 103 constructs the task scheduling instruction and encapsulates the exclusive allocation strategy in the control field of the instruction. The computing power scheduling device 103 issues the task scheduling instruction to instruct the execution node to adjust its resource scheduling mechanism through the hardware abstraction layer. In response to the exclusive allocation strategy, the execution node isolates its local computing cores and storage quotas, allowing it to serve only the thread of the current agent task, thereby shielding resource requests from other concurrent processes. After detecting the execution end signal of the agent task, the execution node performs a resource reclamation operation based on the instruction constraints, releasing the local computing resources from occupation.

[0040] S140. If it is determined that the source of the target computing power is the public cloud computing power access layer 105, the target public cloud computing power node is used as the execution node, and the intelligent agent task and task data are routed to the execution node for processing. In other words, when the target computing power source is determined to be public cloud computing power, the computing power scheduling device 103 activates a communication routing path pointing to an external wide area network. The computing power scheduling device 103 encapsulates the control parameters and task data of the intelligent agent task into a network transmission payload and sends it to the assigned execution node through an external communication link. Using the public cloud computing power node as the execution node, it receives and unpacks the network transmission payload and calls upon its cloud-based cluster computing resources to start the corresponding large model inference process.

[0041] S150. Based on the memory retrieval requirement, a memory scheduling instruction is generated for the shared knowledge base 106. The memory scheduling instruction is configured to: control the execution node corresponding to the intelligent agent task to encapsulate the task execution memory generated during the task execution process into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base 106 to trigger the shared knowledge base 106 to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on the preset memory degradation rules.

[0042] Specifically, during the task data retention phase, the computing power scheduling device 103 responds to the memory retrieval request by generating and issuing a specific memory scheduling instruction. This memory scheduling instruction controls the execution node to capture its state output or feature results during the inference process as the task execution memory, attaches a system-level time-series tag to it, packages it into a memory data packet, and injects it into the shared knowledge base 106. When the shared knowledge base 106 receives a write request, its underlying data management engine is triggered. Based on the timestamp parameter carried by the memory data packet, it performs a rearrangement operation on the memory records in the repository to maintain the temporal consistency of the overall data. Simultaneously, the background maintenance process of the shared knowledge base 106 is awakened, executes a preset lifecycle management script, and performs gradient degradation operations such as dimensionality reduction, compression, or structural simplification on the existing data according to the preset memory degradation rules.

[0043] In the embodiments of this application, by dynamically determining the source of computing power based on data sensitivity level and computing power requirements, flexible collaboration between edge clusters and public cloud resources is achieved, satisfying data privacy protection requirements while also taking into account computing demands exceeding edge load. For the edge execution environment, a dedicated local computing resource allocation and release mechanism is adopted to avoid computing power contention issues in multi-task concurrent scenarios, ensuring the execution stability of large model inference tasks. Furthermore, by introducing a timestamped memory encapsulation mechanism and a shared knowledge base 106, the retention and sharing of task execution states are achieved, reducing the overhead of repeated inference for subsequent similar tasks; simultaneously, dynamic lifecycle management of historical memory data is performed by combining temporal sorting and gradient degradation rules, effectively alleviating storage space pressure on edge devices and improving the continuous operation capability and overall utilization efficiency of distributed system resources.

[0044] Reference Figure 3 As shown, in one embodiment of this application, step S120, determining the target computing power source for the agent task, includes: S210. Determine whether the data sensitivity level is higher than the preset security compliance threshold.

[0045] After receiving a task request from an intelligent agent containing features of a large model inference, the computing power scheduling device 103 compares the extracted data sensitivity level feature value with the security compliance threshold. Specifically, the data sensitivity level can be configured as a one-dimensional scalar value (e.g., a risk index rating of 0-10), or it can be a data security feature vector formed by weighted concatenation of multiple dimensions of feature attributes (e.g., the proportion of personally identifiable information, the frequency of trade secret fields, the trust level of the MAC address of the task initiator, etc.).

[0046] Correspondingly, the preset security compliance threshold can be a static value pre-configured and fixed in the storage node by the system administrator, or it can be an adaptive threshold dynamically updated through periodic analysis of historical security audit logs and network threat profiles. During the judgment process, the routing engine inside the computing power scheduling device 103 constructs comparison logic branches to calculate the numerical relationship or vector Euclidean distance between the feature value of the data sensitivity level and the security compliance threshold, in order to determine the privacy constraint level of the current agent task data.

[0047] S220. If the data sensitivity level is higher than the security compliance threshold, determine that the target computing power source is the edge computing power cluster 104 to ensure that the task data does not leave the domain.

[0048] In response to the determination that the data sensitivity level is higher than the security compliance threshold, the computing power scheduling device 103 triggers the hard constraint logic of local routing. Specifically, when the input data of the intelligent agent task is identified as highly sensitive, the computing power scheduling device 103 limits the destination address pool of the intelligent agent task in the underlying routing table to the edge computing power cluster 104 inside the local area network (LAN) or virtual private cloud (VPC), blocking the transmission path of the task data and request message to the wide area network node at the network architecture protocol level.

[0049] Furthermore, the computing power scheduling device 103 can actively block or ignore availability broadcast messages and resource registration information from public cloud computing power nodes, and force the generation of scheduling instructions that only point to internal edge computing power nodes. Through the aforementioned routing address pool restriction mechanism, it is ensured that the feature extraction, tensor computation, and model inference processes of highly sensitive data are confined within trusted physical or logical boundaries, preventing the risk of data leakage from the perspective of underlying data flow.

[0050] S230. If the data sensitivity level is not higher than the security compliance threshold and the computing power requirement exceeds the total available computing power of the edge computing power cluster 104, the target computing power source is determined to be the public cloud computing power access layer 105.

[0051] When the data sensitivity level is not higher than the security compliance threshold, the computing power scheduling device 103 removes local routing restrictions and further performs a secondary condition assessment of the supply and demand status of computing resources. Specifically, the computing power scheduling device 103 parses the computing power requirements of the intelligent agent task and simultaneously obtains the total available computing power of the edge computing power cluster 104 through a resource registry or probe polling mechanism. When the difference between the computing power requirement and the total available computing power is greater than zero, or the proportion of the computing power requirement to the total available computing power exceeds a preset load safety line, it is determined that the local edge cluster is in a resource bottleneck or saturation state. Based on the dual triggering conditions of data sensitivity compliance and insufficient local edge computing power resources, the computing power scheduling device 103 executes computing power offloading logic, redirects the target computing power source pointer in the internal routing table to the public cloud computing power access layer 105, and generates the corresponding external network call protocol packet. In this process, by introducing public cloud computing power as an elastic buffer pool, memory overflow or computing blockage caused by overload of edge nodes is prevented.

[0052] This embodiment achieves adaptive routing allocation of large-scale intelligent agent tasks in heterogeneous computing networks by constructing a dual determination mechanism based on data sensitivity and computing power supply and demand status. On the one hand, by comparing security compliance thresholds, it avoids the unauthorized outgoing of sensitive data at the underlying network addressing level, meeting strict data compliance and domain-independent requirements. On the other hand, under the premise of confirming data security, by calculating the difference between the remaining available computing power at the edge and the task requirements, it automatically schedules non-sensitive tasks with high computing power overhead to the public cloud computing power access layer 105, effectively alleviating the computing power bottleneck pressure of edge devices and reducing the communication and processing latency of tasks queuing in the local queue.

[0053] Reference Figure 4 As shown, in one embodiment of this application, step S130, which involves determining the target edge computing power node as the execution node from the edge computing power cluster 104 when the target computing power source is determined to be the edge computing power cluster 104, further includes: S310. Real-time acquisition of load status parameters of each candidate edge computing node in the edge computing cluster 104, the load status parameters including CPU utilization, GPU utilization and task queue length.

[0054] After determining that the data flow of the current intelligent agent task is to the edge computing cluster 104, the computing power scheduling device 103 initiates a probe monitoring or status listening mechanism for the local physical cluster resources. Specifically, the computing power scheduling device 103 sends status pull messages to each candidate edge computing node through the cluster intranet at a preset communication period, or receives heartbeat data packets actively reported by each candidate edge computing node at regular intervals through an independent port. The load status parameters are encapsulated into a structured data format (such as a JSON object or an XML load list) at the underlying level, which contains key-value pairs that map the running metrics of each hardware unit and software process. For example, the CPU utilization and GPU utilization are represented as floating-point numbers or percentage scalars to characterize the current instruction processing saturation of the underlying computing cores; the task queue length is represented as an integer number to record the number of pending task threads currently piled up on the node.

[0055] S320. Remove candidate edge computing nodes whose load status parameters exceed the preset load threshold from the scheduling candidate list to obtain the updated candidate node set.

[0056] After obtaining the real-time resource data of each candidate edge computing node, the computing power scheduling device 103 constructs a scheduling candidate list (such as using a hash map table or dynamic array structure) containing the unique network identifier and corresponding status of all nodes in the memory space, and triggers the internal rule filtering engine.

[0057] Specifically, the computing power scheduling device 103 compares the extracted CPU utilization, GPU utilization, and task queue length of each node with corresponding preset load thresholds maintained in system memory. When any dimension's state parameter exceeds its corresponding preset load threshold, or when the comprehensive load score calculated based on a preset weight matrix for multiple parameters exceeds the comprehensive preset threshold, the computing power scheduling device 103 logically masks or physically deletes the identifier data of the candidate edge computing power node from the scheduling candidate list, thereby completing the removal operation for high-load nodes. The aforementioned preset load threshold can be a static scalar fixed in the configuration file, or a dynamic adaptive threshold generated after smoothing the average cluster load within a historical time window. Through the above multi-dimensional parameter comparison and data stripping, the computing power scheduling device 103 logically prevents possible network routing to overloaded nodes and outputs a filtered data set, i.e., the updated candidate node set.

[0058] S330. From the candidate node set, select a node that meets the computing power requirements and has an idle state as the execution node.

[0059] In this step, the computing power scheduling device 103 performs optimal matching and state verification for the agent's task input requirements. Specifically, the computing power scheduling device 103 parses the computing power requirement parameters (e.g., the required memory capacity threshold or the minimum concurrent processing computing power) carried in the agent's task request, and performs a secondary traversal comparison of the remaining available resources of each node in the updated candidate node set. During the traversal calculation, the computing power scheduling device 103 synchronously extracts the status register identifier or availability flag of each node and verifies whether it is in the idle state (e.g., a specific flag is set to a "0" state indicating that it is not occupied, indicating that the current node is not locked by other exclusive instructions). For nodes that simultaneously meet the computing power requirements and pass the status bit verification, the computing power scheduling device 103 establishes its network address as the execution node. When there are multiple nodes in the candidate node set that simultaneously meet the above dual constraints, the computing power scheduling device 103 makes a final confirmation through a preset scheduling strategy, such as using a round-robin scheduling algorithm to allocate nodes in order, or using an optimal matching algorithm to allocate the node with the largest remaining video memory based on the remaining computing power capacity of each node, thereby binding and associating the unique identifier of the node with the routing information of the current intelligent agent task.

[0060] By acquiring multi-dimensional load status in real time and performing secondary filtering and matching, a closed-loop data mapping between the underlying physical status of the edge computing cluster 104 and the upper-layer large model task distribution is achieved. Before the final allocation of execution computing nodes, devices in an overload critical state are removed from the scheduling candidate list through pre-verification of computing hardware saturation and task backlog depth, effectively preventing memory overflow or process crashes caused by edge nodes exhausting computing resources. At the same time, by comparing and verifying the idle status of nodes with the lower limit of computing power requirements, it is ensured that the selected execution nodes have the physical foundation to carry out exclusive large model inference tasks, reducing the risk of task scheduling failure, response timeout, or inference interruption, and improving the throughput and system robustness of the edge computing cluster 104 in complex multi-task concurrent scenarios.

[0061] In one embodiment of this application, generating a task scheduling instruction containing an exclusive allocation strategy and issuing it to the execution node includes: Simultaneously with issuing the task scheduling instruction, a resource lock identifier corresponding to the execution node is generated at the computing power scheduling device 103. For example, the computing power scheduling device 103 generates a specific resource lock identifier for the currently allocated execution node in local memory or a global resource registry. This resource lock identifier can be configured as a key-value pair mapping structure, where the key is the execution node's network address or unique device identifier, and the value is a Boolean value representing the occupancy status, a specific status enumeration bit, or the currently bound agent task identifier. To prevent the execution node from being permanently blocked due to unexpected downtime or network disconnection, a timestamp field or a preset lifecycle parameter can also be embedded in the data structure of the resource lock identifier.

[0062] The routing engine intercepts and refuses to route other parallel agent tasks to the execution node with the resource lock identifier. Specifically, when the agent workflow system 101 issues other parallel agent tasks with high concurrency, the routing distribution engine of the computing power scheduling device 103 forcibly queries the global resource registry before allocating scheduling pointers to any target edge computing power node. In response to detecting that the identifier mapping of the target node has the resource lock identifier in a valid state, the routing distribution engine triggers a data flow interception operation, refusing to forward the scheduling message of the aforementioned parallel agent task to the execution node at the network layer or application layer.

[0063] The system receives task completion feedback and computing power release signals from the execution node and simultaneously clears the resource lock identifier to restore the execution node to an available state. Specifically, after the execution node completes its currently exclusive large model inference task, its underlying computing power monitoring process captures the computation end or abnormal termination status and sends a network packet encapsulating the task execution status to the upstream computing power scheduling device 103. Further, the computing power scheduling device 103 receives the task completion feedback and computing power release signals by listening to a specific port. For example, the computing power scheduling device 103 parses the packet data, and after extracting the node identifier confirming the release and comparing it with the corresponding task characteristics, it sends an asynchronous erase command to the resource state machine to synchronously modify or physically delete the resource lock identifier in the resource registry. Through the above identifier clearing operation, the status indicator bit of the execution node in the global routing topology is flipped from a locked and occupied state to the initial idle and available state, thereby enabling it to be re-addressed by the routing distribution engine and added to the resource allocation pool for the next round of agent tasks.

[0064] This embodiment achieves strict physical isolation and lifecycle management of edge computing devices in complex multi-concurrent task scheduling scenarios through a closed-loop mechanism of resource lock identifier generation, route verification and interception, and synchronous clearing. It effectively prevents memory overflow caused by multi-threaded inference tasks preempting limited edge GPU memory or computational blocking caused by frequent context data exchanges. While ensuring the determinism of inference resources for a single intelligent agent task, it effectively avoids computing deadlock through state reset, improving the overall task flow efficiency and fault tolerance reliability of the edge computing cluster.

[0065] In one embodiment of this application, generating memory scheduling instructions for the shared knowledge base 106 includes: First, when the memory retrieval request indicates that historical memory needs to be retrieved, a memory query instruction is sent to the shared knowledge base 106 to extract matching target shared memory data from the sorted temporal memory sequence of the shared knowledge base 106.

[0066] For example, a memory retrieval request can be configured in the underlying data structure as a boolean status flag or a confidence scalar based on the output of a task intent recognition model. In response to detecting that the status flag is set to true, or that the confidence scalar is greater than a preset trigger threshold, it is determined that the current agent task has a strong dependency on the historical reasoning context. Based on this determination, a structured memory query instruction is generated and sent to the shared knowledge base 106 via an internal remote procedure call or message queue.

[0067] Memory query instructions typically encapsulate a query feature vector extracted based on the current task content and a time window parameter to limit the retrieval space. Upon receiving the memory query instruction, the shared knowledge base 106 invokes the underlying storage engine to perform a retrieval traversal within the sorted temporally sequenced memory sequence. First, it performs an initial screening in a timestamp-based temporal index based on the time window parameter. Then, it calculates the spatial distance between the query feature vector and the feature vectors of each historical memory in the initial screening dataset. Top node data with similarity scores exceeding a preset matching threshold are selected and aggregated into the target shared memory data.

[0068] Then, the target shared memory data is fed back to the execution node so that the execution node can load it as contextual background information during the reasoning process to avoid repeated reasoning.

[0069] During the loading process of the execution node, the execution node receives the target shared memory data in local video memory or main memory and deserializes it into structured context text fields or feature tensor matrices, which are required for transmission from the network. The execution node can load the background information using dynamic template concatenation or tensor fusion. For example, the execution node dynamically injects the target shared memory data as an additional pre-input condition into a specific slot (e.g., a pre-defined large model Prompt template) into a pre-defined slot.<History_Context> Within the text range enclosed by the label, a complete input sequence is formed after concatenation; or, the execution node directly maps the target shared memory data into the initial tensor parameters of the key-value cache, and in the self-attention mechanism calculation layer of the large model, it directly participates in the feature interaction and weight aggregation calculation of the current input task lexical.

[0070] This embodiment utilizes a data processing approach based on temporal memory retrieval and dynamic context loading to achieve cross-task reuse of historical experience features during the large model inference stage. This effectively avoids redundant extraction operations and multi-level matrix multiplications of repetitive or highly similar semantic features by edge execution nodes, reducing the time complexity and computational overhead of a single inference attempt. Simultaneously, it reduces memory usage during invalid feature generation, improving the robustness and response efficiency of task inference in resource-constrained edge computing architectures.

[0071] Reference Figure 5 As shown, in one embodiment of this application, the step of triggering the shared knowledge base 106 to sort the recorded memory data chronologically according to the timestamp and perform gradient degradation processing based on a preset memory degradation rule specifically includes: triggering the shared knowledge base 106 to perform the following operations: S410. Obtain the generation and retention duration and access frequency of each historical memory data in the shared knowledge base 106.

[0072] After the shared knowledge base 106 receives the newly written memory data packet and completes the time-series sorting operation, the background maintenance process running inside the shared knowledge base 106 will periodically or event-drivenly trigger memory state scans. Specifically, the background maintenance process traverses the underlying storage structure (such as a distributed file system or database index table) and reads the metadata bound to each historical memory data node. It parses the initial creation timestamp from the metadata and calculates the generation retention duration (e.g., a floating-point or long integer value in hours or days) by combining it with the current system time. Simultaneously, it reads the historical call log or built-in access counter register for that data node, extracts the number of times the node query was hit within a specified time window, and converts it into a quantified access frequency.

[0073] S420. If the generation and retention time of the target historical memory data exceeds a first threshold and the access frequency is lower than a second threshold, gradient degradation is performed on the target historical memory data. The gradient degradation includes compressing the storage format and / or reducing the query priority of memory retrieval.

[0074] Specifically, the retention duration and access frequency of any target historical memory data encountered in the traversal are extracted and compared with the first threshold and the second threshold maintained in the system memory, respectively. The first and second thresholds are configured as pre-fixed static empirical parameters, or as sliding feature thresholds dynamically and adaptively updated based on the average access distribution heat of the entire cluster. When the generation and retention duration of the target historical memory data exceeds the first threshold, and its access frequency is less than the second threshold, the data is determined to have entered the cold data lifecycle stage, and the gradient degradation processing for that data is then initiated. If the gradient degradation is configured to use a compressed storage format, the underlying data compression algorithm will be scheduled to convert the target historical memory data from a dense feature tensor or plaintext string structure that occupies a lot of video memory into a low-compression binary format, and its physical storage location will be moved from high-speed static random access memory to low-speed disk media. If the gradient degradation is configured to reduce query priority, the retrieval weight label of the target historical memory data in the time-series index tree will be modified, so that when a memory query request for large model inference is received in the future, the recall matching threshold requirement of the data will be increased or its ranking position will be moved backward, thereby reducing invalid feature matching calculations.

[0075] S430. If the storage space usage rate is detected to have reached a preset space threshold, clean up historical memory data that has reached the third threshold of degradation.

[0076] During the continuous asynchronous operation of the aforementioned gradient degradation mechanism, the storage monitoring module of the shared knowledge base 106 performs capacity monitoring of physical disks or allocated quotas in parallel. Specifically, when the storage space utilization rate of the current storage mount point or database tablespace reaches the preset space threshold (e.g., triggering 85% capacity, indicating that storage resources are nearing exhaustion), a forced resource reclamation interruption routine is automatically triggered to avoid data write blocking of new inference tasks due to storage overflow. In response to this triggering event, the set of cold data that has undergone gradient degradation is traversed again to extract the degradation status markers of each data. The degradation status markers are represented by an enumeration value of the level of continuous degradation or a decay score based on the accumulation of unaccessed time. When it is detected that the degradation status markers of some historical memory data reach or exceed the third threshold representing the extreme useless state, the disk clusters or memory data blocks occupied by these discarded historical memory data are released and erased.

[0077] By dynamically compressing and prioritizing low-value long-tail memories, the system reduces the ineffective occupation of cache quotas and query computing power by inactive feature data, thereby reducing the spatial complexity of the system. At the same time, combined with a targeted deep cleanup strategy when the capacity reaches its limit, the system autonomously curbs the risk of storage exhaustion caused by the accumulation of historical data while ensuring the integrity of the context of high-frequency value memories. This improves the resource operation efficiency and system throughput stability of edge nodes when performing long-cycle, high-concurrency, large-model inference tasks.

[0078] In some embodiments of this application, before issuing task scheduling instructions to the execution node or routing the agent task to the execution node, the method further includes: The task data is encrypted. For example, the encryption algorithm is dynamically configured based on the data sensitivity level. For a normal sensitivity level, a symmetric encryption algorithm is used to output ciphertext data packets; for an extremely high sensitivity level, a fully homomorphic encryption protocol is invoked, allowing the data to directly participate in tensor operations of edge nodes in ciphertext form during subsequent inference.

[0079] The legitimacy of the execution node's access to the current computing power network system is verified. Specifically, the computing power scheduling device 103 initiates identity authentication with the execution node and receives the legitimacy credential, configured as a digital certificate or signature token, returned by the node. The digital signature in the legitimacy credential is decrypted and compared for consistency using a locally cached public key or distributed ledger. If the comparison is successful and the credential is within its valid lifespan, the verification is deemed successful, and a secure transmission tunnel is established at the network protocol layer. If the verification fails, a security circuit breaker is triggered, the scheduling instruction message for that node is directly discarded, and its network identifier is removed from the available routing table.

[0080] The access permission level of the intelligent agent task to the shared knowledge base 106 is security-verified. For example, after ensuring the physical identity of the execution node is legitimate, the computing power scheduling device 103 extracts the identity identifier and read / write operation intent of the current intelligent agent task and inputs them into the access control engine. This engine queries the access control matrix in memory, comparing the access level declared by the current task entity with the minimum security permission threshold required by the target memory data partition in the shared knowledge base 106. If the permission verification passes, a time-sensitive access token is generated and attached to the task scheduling instruction for issuance; if an unauthorized access intent is detected, the memory call request is intercepted at the sandbox layer, blocking its cross-boundary call interface to a specific memory address space.

[0081] By encrypting data, authenticating node identities, and verifying access permissions, a proactive security interception is achieved before data is distributed and routes are established. This prevents the risk of plaintext data eavesdropping and node forgery and deception, and also achieves logical isolation between tasks of different sensitivity levels, thereby improving the overall data flow security of the distributed computing power scheduling system.

[0082] In some embodiments of this application, when the target computing power source is the public cloud computing power access layer 105, the method further includes: The scheduling instructions used to route the agent's tasks are converted into a standard calling protocol format supported by the public cloud computing power node. Specifically, the computing power scheduling device 103 extracts the internally generated scheduling instructions used to route the agent's tasks, parses the control fields and task metadata, and calls the underlying protocol conversion module to reconstruct them into a standard calling protocol format supported by the public cloud computing power node. Through the above protocol conversion, the local instruction data is serialized into a general network transmission format, and the request header and authentication information are rewritten according to the public cloud vendor's specifications, ensuring that the heterogeneous cloud platform can seamlessly parse the underlying scheduling requests.

[0083] The transformed task data is sent to the execution node via a public computing power interface. That is, the computing power scheduling device 103 calls the pre-configured public computing power interface (such as a standard SDK interface or API gateway provided by a cloud vendor) to send the transformed task data as the core request payload to the public cloud computing power node designated as the execution node. Preferably, the task data can be sent based on an asynchronous non-blocking communication mode, and a corresponding callback function can be registered to avoid the main thread of the computing power scheduling device 103 being blocked by uncontrollable external network latency.

[0084] The system receives the inference results and generated shareable memory data returned by the execution node, and generates a corresponding memory write instruction to write them into the shared knowledge base 106. For example, the computing power scheduling device 103 receives the response data packet returned by the execution node via a wide area network through a network callback mechanism or a long polling interface based on a connection pool. The response data packet is deserialized and parsed to separate and extract the inference results output by the large model network, as well as the shareable memory data (such as intermediate layer feature tensors, dialogue context summaries, or core logic decision paths) synchronously captured during cloud inference. After obtaining the above data, the computing power scheduling device 103 generates the corresponding memory write instruction according to local data disk persistence rules, and structurally encapsulates the shareable memory data and the extracted system timestamp in the instruction. Subsequently, the computing power scheduling device 103 routes the instruction to the shared knowledge base 106 to trigger the shared knowledge base 106 to perform underlying index update and data insertion operations, completing the persistent write-back of the external public cloud execution context to the local edge storage environment.

[0085] Through the aforementioned protocol conversion, asynchronous data communication, and memory write-back linkage mechanism for the public cloud computing power access layer 105, smooth data flow between the edge scheduling center and the external heterogeneous cloud computing power environment is achieved. Standardized protocol adaptation and serialization processing remove system interface barriers during cross-cloud scheduling. Simultaneously, the shared memory data generated by cloud nodes is automatically transmitted back and synchronized to the local shared knowledge base 106, establishing a closed-loop memory flow in the edge-cloud collaborative computing architecture. This effectively fills the context gap between external public cloud offline tasks and local edge workflows, improving the global feature reuse rate and collaborative reasoning efficiency of the distributed heterogeneous computing power network.

[0086] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0087] Reference Figure 6 This application illustrates a distributed computing power scheduling device for large-scale models, specifically including the following modules: The task parsing module 601 is used to obtain the agent task request and corresponding task data sent by the agent workflow system 101. The agent task request includes at least computing power requirements, data sensitivity level and memory retrieval requirements.

[0088] The computing power source determination module 602 is used to determine the target computing power source for the intelligent agent task based on the data sensitivity level and the computing power requirement. The target computing power source includes the edge computing power cluster 104 or the public cloud computing power access layer 105.

[0089] The first scheduling module 603 is used to determine the target edge computing node as the execution node from the edge computing cluster 104 when the target computing power source is determined to be the edge computing cluster 104, generate a task scheduling instruction containing an exclusive allocation strategy and send it to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed.

[0090] The second scheduling module 604 is used to, when it is determined that the source of the target computing power is the public cloud computing power access layer 105, use the target public cloud computing power node as the execution node and route the intelligent agent task and task data to the execution node for processing. The memory retrieval module 605 is used to generate a memory scheduling instruction for the shared knowledge base 106 based on the memory retrieval requirement. The memory scheduling instruction is configured to: control the execution node to encapsulate the task execution memory generated during task execution into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base 106 to trigger the shared knowledge base 106 to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on a preset memory degradation rule.

[0091] In one embodiment of this application, the computing power source determination module includes: The judgment module is used to determine whether the data sensitivity level is higher than a preset security compliance threshold.

[0092] The first determining module is used to determine that the source of the target computing power is the edge computing power cluster 104 when the data sensitivity level is higher than the security compliance threshold, so as to ensure that the task data does not leave the domain.

[0093] The second determining module is used to determine that the source of the target computing power is the public cloud computing power access layer 105 when the data sensitivity level is not higher than the security compliance threshold and the computing power requirement exceeds the total available computing power of the edge computing power cluster 104.

[0094] In one embodiment of this application, the first scheduling module includes: The acquisition module is used to acquire the load status parameters of each candidate edge computing node in the edge computing cluster 104 in real time. The load status parameters include the CPU utilization, GPU utilization, and task queue length.

[0095] The elimination module is used to remove candidate edge computing nodes whose load status parameters exceed a preset load threshold from the scheduling candidate list, thereby obtaining an updated set of candidate nodes.

[0096] The selection module is used to select, from the candidate node set, a node that meets the computing power requirements and has an idle state as the execution node.

[0097] In one embodiment of this application, the first scheduling module further includes: The generation module is used to generate a resource lock identifier corresponding to the execution node at the computing power scheduling device 103 at the same time as issuing the task scheduling instruction.

[0098] An interception module is used to intercept and refuse to route other parallel agent tasks to the execution node with the resource lock identifier.

[0099] The clearing module is used to receive task completion feedback and computing power release signals sent by the execution node, and simultaneously clear the resource lock flag so that the execution node can be restored to an available state.

[0100] In one embodiment of this application, the memory retrieval module includes: The extraction module is used to send a memory query instruction to the shared knowledge base 106 when the memory retrieval request indicates that historical memory needs to be retrieved, so as to extract matching target shared memory data from the sorted temporal memory sequence of the shared knowledge base 106.

[0101] The feedback module is used to feed back the target shared memory data to the execution node so that the execution node can load it as contextual background information during the inference process to avoid repeated inference.

[0102] In one embodiment of this application, the memory scheduling instruction is specifically used to trigger the shared knowledge base 106 to perform the following operations: The generation and retention duration and access frequency of each historical memory data in the shared knowledge base 106 are obtained.

[0103] If the generation and retention time of the target historical memory data exceeds a first threshold and the access frequency is lower than a second threshold, gradient degradation is performed on the target historical memory data. The gradient degradation includes compressing the storage format and / or reducing the query priority of memory retrieval.

[0104] If the storage space usage rate reaches a preset space threshold, historical memory data that has degraded to the third threshold will be cleaned up.

[0105] In some embodiments of this application, the apparatus further includes: An encryption module is used to encrypt the task data.

[0106] The credential verification module is used to verify the legitimacy of the execution node's access to the current computing power network system.

[0107] The permission verification module is used to perform security verification on the permission level of the intelligent agent task to access the shared knowledge base 106.

[0108] In some embodiments of this application, the apparatus further includes: The protocol conversion module is used to convert the scheduling instructions used to route the intelligent agent's tasks into the standard calling protocol format supported by the public cloud computing power node.

[0109] The task sending module is used to send the converted task data to the execution node through the public computing power interface.

[0110] The memory writing module is used to receive the reasoning results and the generated shareable memory data returned by the execution node, and generate corresponding memory writing instructions to write them into the shared knowledge base 106.

[0111] Reference Figure 7 The illustration shows a computer electronic device for implementing the distributed computing power scheduling method for large models according to this application, which may specifically include the following: The aforementioned computer electronic device 1 is manifested in the form of a general-purpose computing device. The components of the computer electronic device 1 may include, but are not limited to: one or more processors or processing units 3, memory 8, and a bus 4 connecting different system components (including memory 8 and processing unit 3).

[0112] Bus 4 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Audio / Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0113] Computer electronic device 1 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer electronic device 1, including volatile and non-volatile media, removable and non-removable media.

[0114] Memory 8 may include computer system readable media in the form of volatile memory, such as random access memory 9 and / or cache memory 10. Computer electronic device 1 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 7As not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 4 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 13 configured to perform the functions of the embodiments of this application.

[0115] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in memory. Such program modules 13 include—but are not limited to—an operating system, one or more application programs, other program modules 13, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of this application.

[0116] The computer electronic device 1 can also communicate with one or more external devices 2 (e.g., keyboard, pointing device, display 7, camera, etc.), and with one or more devices that enable an operator to interact with the computer electronic device 1, and / or with any device that enables the computer electronic device 1 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through the I / O interface 6. Furthermore, the computer electronic device 1 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) through the network adapter 5. Figure 7 As shown, network adapter 5 communicates with other modules of computer electronic device 1 via bus 4. It should be understood that, although... Figure 7 As not shown, it may be used in conjunction with computer electronic device 1 with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0117] The processing unit 3 executes various functional applications and data processing by running programs stored in memory 8, such as implementing the distributed computing power scheduling method for large models provided in the embodiments of this application.

[0118] In this application embodiment, the application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the distributed computing power scheduling method for large models as provided in all embodiments of the application.

[0119] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0120] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0121] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as a standalone software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the operator's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0122] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0123] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0124] The above provides a detailed description of a distributed computing power scheduling method and related apparatus for large-scale models provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A distributed computing power scheduling method for large-scale models, applied to computing power scheduling equipment, characterized in that, include: The system obtains intelligent agent task requests and corresponding task data sent by the intelligent agent workflow system. The intelligent agent task requests include at least computing power requirements, data sensitivity levels, and memory retrieval requirements. Based on the data sensitivity level and the computing power requirement, the target computing power source for the intelligent agent task is determined, and the target computing power source includes edge computing power clusters or public cloud computing power access layer. If the source of the target computing power is determined to be the edge computing power cluster, a target edge computing power node is determined from the edge computing power cluster as an execution node, a task scheduling instruction containing an exclusive allocation strategy is generated and sent to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed. If the target computing power source is determined to be the public cloud computing power access layer, the target public cloud computing power node is used as the execution node, and the intelligent agent task and task data are routed to the execution node for processing. Based on the memory retrieval requirements, a memory scheduling instruction is generated for the shared knowledge base. The memory scheduling instruction is configured to: control the execution node corresponding to the agent task to encapsulate the task execution memory generated during the task execution process into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base to trigger the shared knowledge base to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on a preset memory degradation rule.

2. The method according to claim 1, characterized in that, Determining the target computing power source for the agent's task includes: Determine whether the data sensitivity level is higher than a preset security compliance threshold; If the data sensitivity level is higher than the security compliance threshold, the source of the target computing power is determined to be the edge computing power cluster, so as to ensure that the task data does not leave the domain; If the data sensitivity level is not higher than the security compliance threshold and the computing power requirement exceeds the total available computing power of the edge computing power cluster, the target computing power source is determined to be the public cloud computing power access layer.

3. The method according to claim 1, characterized in that, The step of determining the target edge computing power node as the execution node from the edge computing power cluster when the target computing power source is determined to be the edge computing power cluster further includes: The load status parameters of each candidate edge computing node in the edge computing power cluster are obtained in real time. The load status parameters include the CPU utilization, GPU utilization, and task queue length. Candidate edge computing nodes whose load status parameters exceed the preset load threshold are removed from the scheduling candidate list, resulting in an updated set of candidate nodes; From the set of candidate nodes, select a node that meets the computing power requirements and has an idle state as the execution node.

4. The method according to claim 1, characterized in that, The process of generating a task scheduling instruction containing an exclusive allocation strategy and issuing it to the execution node includes: Simultaneously with issuing the task scheduling instruction, a resource lock identifier corresponding to the execution node is generated on the computing power scheduling device. Intercept and refuse to route other parallel agent tasks to the execution node with the resource locking identifier; The system receives task completion feedback and computing power release signals from the execution node, and simultaneously clears the resource lock flag to restore the execution node to an available state.

5. The method according to claim 1, characterized in that, The generation of memory scheduling instructions for the shared knowledge base includes: When the memory retrieval request indicates that historical memory needs to be retrieved, a memory query instruction is sent to the shared knowledge base to extract matching target shared memory data from the sorted temporal memory sequence in the shared knowledge base; The target shared memory data is fed back to the execution node so that the execution node can load it as contextual background information during the reasoning process to avoid repeated reasoning.

6. The method according to claim 1, characterized in that, The step of triggering the shared knowledge base to sort the recorded memory data chronologically according to the timestamp and perform gradient degradation processing based on preset memory degradation rules specifically includes: triggering the shared knowledge base to perform the following operations: Obtain the generation retention time and access frequency of each historical memory data in the shared knowledge base; If the generation and retention time of target historical memory data exceeds a first threshold and the access frequency is lower than a second threshold, gradient degradation is performed on the target historical memory data. The gradient degradation includes compressing the storage format and / or reducing the query priority of memory retrieval. If the storage space usage rate is detected to have reached a preset space threshold, historical memory data that has reached the third threshold of degradation will be cleaned up.

7. The method according to claim 1, characterized in that, Before issuing task scheduling instructions to the execution node or routing the agent's task to the execution node, the method further includes: The task data is encrypted. Verify the legitimacy of the execution node's access to the current computing power network system; Security checks are performed on the permission level of the intelligent agent task to access the shared knowledge base.

8. The method according to claim 1, characterized in that, When the target computing power source is the public cloud computing power access layer, the method further includes: The scheduling instructions used to route the agent's tasks are converted into the standard calling protocol format supported by the public cloud computing power node; The converted task data is sent to the execution node via a public computing power interface; The system receives the inference results and the generated shareable memory data returned by the execution node, and generates corresponding memory writing instructions to write them into the shared knowledge base.

9. A distributed computing power scheduling device for large-scale models, characterized in that, include: The task parsing module is used to obtain the agent task request and corresponding task data sent by the agent workflow system. The agent task request includes at least computing power requirements, data sensitivity level and memory retrieval requirements. The computing power source determination module is used to determine the target computing power source for the intelligent agent task based on the data sensitivity level and the computing power requirement. The target computing power source includes an edge computing power cluster or a public cloud computing power access layer. The first scheduling module is used to determine the target edge computing node as the execution node from the edge computing cluster when the target computing power source is determined to be the edge computing power cluster, generate a task scheduling instruction containing an exclusive allocation strategy and send it to the execution node, so as to control the execution node to exclusively allocate local computing resources to the intelligent agent task during the execution of the corresponding intelligent agent task and release them after the task is completed. The second scheduling module is used to, when it is determined that the source of the target computing power is the public cloud computing power access layer, use the target public cloud computing power node as the execution node and route the intelligent agent task and task data to the execution node for processing; The memory retrieval module is used to generate a memory scheduling instruction for the shared knowledge base based on the memory retrieval requirement. The memory scheduling instruction is configured to: control the execution node to encapsulate the task execution memory generated during task execution into a memory data packet with a timestamp, and write the memory data packet into the shared knowledge base to trigger the shared knowledge base to sort the recorded memory data in chronological order according to the timestamp, and perform gradient degradation processing based on a preset memory degradation rule.

10. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the distributed computing power scheduling method for large models as described in any one of claims 1 to 8.

Images