An end-cloud cooperative AI agent capability unified registration and scheduling system

Abstract

The application relates to the technical field of distributed computing, in particular to an end-cloud cooperative AI intelligent agent capability unified registration and scheduling system, which comprises a dynamic behavior mapping module, a semantic feature compression module, an asynchronous state alignment module, an inference flow scheduling module and a thinking chain flow control module.

Description

Technical Field

[0001] This invention relates to the field of distributed computing technology, and in particular to a unified registration and scheduling system for AI intelligent agent capabilities in edge-cloud collaboration. Background Technology

[0002] The evolution of large-scale models and intelligent agent technologies has made edge-cloud collaborative complex control and automated scheduling systems a core direction for current industrial control and system integration. In existing technologies, it is typically necessary to expose standard application programming interfaces (APIs) on client systems or terminal devices. The cloud control center or large-scale model then issues scheduling commands to these standard APIs to drive corresponding interactive behaviors on the edge. To ensure the accuracy of the interaction, the system often needs to manually write corresponding interface protocols and behavior registration mapping tables during the development phase for specific client software, interfaces, or hardware configurations, thereby establishing a mapping link from the large-scale model to edge actions.

[0003] Existing systems heavily rely on the target client's open standard APIs. When faced with non-standard front-end systems without APIs, or when the client experiences rendering delays, resolution changes causing interface control coordinate drift, or when the system's underlying layer displays sudden abnormal prompts, the system requires frequent manual modification or rewriting of the interface code. This results in excessively high integration and adaptation costs for intelligent scheduling systems, and the control flow is prone to severe blockage when node conflicts occur. Summary of the Invention

[0004] To overcome the above shortcomings, this invention provides a unified registration and scheduling system for AI intelligent agent capabilities in edge-cloud collaboration, aiming to improve the problems of excessively high cost of docking and adaptation of intelligent scheduling systems and the serious blockage of control flow when node conflicts occur.

[0005] This invention provides the following technical solution: a unified registration and scheduling system for AI intelligent agent capabilities in an edge-cloud collaborative manner, comprising:

[0006] The dynamic behavior mapping module is used to capture the interaction behavior characteristics of the client and map the interaction behavior characteristics to the behavior control topology. When the behavior control topology deviates, the topology path is adjusted and the adjusted control behavior is registered to the unified MCP capability center.

[0007] The semantic feature compression module is used to perform dimensionality reduction and compression processing on the original multimodal data stream to generate low-dimensional semantic feature stubs;

[0008] The asynchronous state alignment module is used to convert the low-dimensional semantic feature stub into an asynchronous stateless message stream for transmission and to reconstruct the corresponding context image data on the cloud side.

[0009] The inference stream scheduling module is used to obtain the edge running status, cloud inference status and network transmission status when a large model performs inference based on context mirror data, and to generate an inference stream scheduling strategy based on the registered capabilities in the unified MCP capability center, the edge running status, the cloud inference status and the network transmission status.

[0010] The thought flow control module is used to dynamically truncate the thought flow output process of the large model according to the inference flow scheduling strategy, and extract the core control segments and corresponding confidence information.

[0011] The residual collaborative execution module is used to send the core control segment and confidence information to the edge, and call the lightweight decision network at the edge to perform residual logic completion based on the core control segment in combination with the on-site time-series status information, so as to generate control commands and drive the behavior control topology to execute the corresponding control behavior.

[0012] Preferably, in the dynamic behavior mapping module, the capture of client interaction behavior features includes:

[0013] Extract current system memory handle features, accessibility underlying tree topology features, and local semantic features of the current interactive interface image from the edge companion process;

[0014] The current system memory handle features, accessibility underlying tree topology features, and local semantic features of the image are bound together in a multi-dimensional logical space to construct a dynamic multimodal feature matrix. The dynamic multimodal feature matrix is ​​then output as an interactive behavior feature to the corresponding behavior control topology.

[0015] Preferably, in the dynamic behavior mapping module, the topology path adjustment includes:

[0016] The mapping-generated behavior control topology is transformed into the corresponding adjacency matrix, and topology node conflicts caused by interface drift or abnormal pop-ups on the client are monitored in real time as deviations from the behavior control topology.

[0017] When a deviation is detected, the local geometric similarity between the subgraph of the current conflict region and the preset baseline topology template is calculated to identify the target control node that has deviated.

[0018] By using local geometric similarity to perform control flow remapping on the adjacency matrix, local self-correction compensation of the topology control path is completed, which serves as topology path adjustment. The adjusted control behavior is then registered as a registered capability to the unified MCP capability center.

[0019] Preferably, in the semantic feature compression module, the dimensionality reduction compression process includes:

[0020] The system intercepts time-series data streams, video streams, and audio streams generated by the client in real time, and assembles them by aligning and assembling the timestamps to construct the original multimodal data stream.

[0021] The feature thinning network is invoked to extract low-dimensional high-order feature vectors from the original multimodal data stream, and sensitive physical feature elements in the low-dimensional high-order feature vectors are identified through a pre-set database of classified parameter elements.

[0022] A dynamic feature mask is used to strip sensitive physical feature elements in real time and desensitize them by setting them to zero locally, so as to output a low-dimensional semantic feature stub.

[0023] Preferably, in the asynchronous state alignment module, converting the low-dimensional semantic feature stub into an asynchronous stateless message stream for transmission includes:

[0024] A one-way asynchronous finite state machine protocol is adopted to slice and encapsulate the generated low-dimensional semantic feature stubs into an asynchronous stateless message stream composed of multiple sets of single-wave pulse messages.

[0025] On an asymmetric unidirectional link, asynchronous stateless message streams are asynchronously delivered to the cloud side at a preset pulse frequency, and the bidirectional handshake interaction logic is discarded during transmission.

[0026] Preferably, in the asynchronous state alignment module, the reconstruction of the corresponding context image data on the cloud side includes:

[0027] An independent shadow context buffer is created in the cloud-side memory, and the asynchronous stateless message stream is received in real time;

[0028] The asynchronous stateless message stream is time-stacked and the data stream is reverse-reconstructed to extract the corresponding low-dimensional semantic feature stubs. Without obtaining the original multimodal data stream, a virtual state space aligned with the end side is reverse-reconstructed in the shadow context buffer as context mirror data.

[0029] Preferably, in the inference stream scheduling module, obtaining the endpoint running status, cloud-side inference status, and network transmission status includes:

[0030] Real-time monitoring of the communication channel carrying asynchronous stateless message streams, and extraction of the current network channel throughput as the network transmission status;

[0031] Real-time monitoring of computation time when large models perform inference based on context mirror data, and extraction of the current large model's first character output time and token generation rate as cloud-side inference status;

[0032] The remaining tolerance time of the current agent task is monitored in real time, and the reciprocal of the remaining tolerance time is converted into a time urgency index as the edge running status.

[0033] Preferably, in the inference stream scheduling module, the inference stream generation scheduling strategy includes:

[0034] The acquired network transmission status, cloud-side inference status, and edge-side operating status are transformed into a three-dimensional dynamic parameter vector.

[0035] The three-dimensional dynamic parameter vector is input into a preset multi-objective nonlinear optimization function for real-time optimization calculation. The probability of penetrating the red line of strong real-time response is predicted. Based on the comparison between the penetration probability and the preset safety water level threshold, a regular scheduling instruction or a truncated scheduling instruction is generated as the inference flow scheduling strategy.

[0036] Preferably, in the thought flow control module, the dynamic truncation processing includes:

[0037] When the resolved inference stream scheduling policy is a truncated scheduling instruction to initiate backbone residual inference prediction, the streaming forced termination protocol is activated.

[0038] Based on the streaming forced termination protocol, streaming truncation is implemented when the backbone control logic token stream of the large model has completed a preset proportion. The token sequence that has been output before truncation is extracted as the core control segment, and the probability vector output at the truncation position is extracted as the corresponding confidence information.

[0039] Preferably, in the residual collaborative execution module, the residual logic completion includes:

[0040] The received core control segments and confidence information are converted into a local logic cue matrix;

[0041] The local logic prompt matrix and the real-time on-site time-series status information collected at the edge are input into the preset lightweight decision network for residual inference to complete the incomplete control logic, generate the original control sequence, and perform physical red-line filtering and compliance interception on the original control sequence through safety constraint rules to output control commands.

[0042] The present invention has the following beneficial effects:

[0043] 1. In this invention, features are extracted and mapped to control topology through edge companion processes, and local correction compensation is performed using adjacency matrix and local geometric similarity when the topology deviates. This realizes the automatic registration of non-standard behaviors to the standard model capability center, and solves the problems of traditional intelligent agent scheduling being extremely dependent on standard APIs, having high system integration costs and easy control flow blockage when facing interface drift or abnormal pop-ups.

[0044] 2. In this invention, by using feature masking for desensitization and employing a one-way protocol and asymmetric link to discard bidirectional handshakes, combined with cloud-side shadow caching and time-series reassembly algorithms, high-fidelity reconstruction of cloud context images is achieved without obtaining the original multimodal data. This solves the problems of traditional protocols easily causing channel blocking, packets being traced back in reverse, and data easily being out of order and lost under one-way transmission in network isolation environments.

[0045] 3. In this invention, by predicting the timeout probability in real time and performing streaming truncation on the cloud token stream based on semantic entropy, and then using the edge network to perform residual completion in combination with the on-site time sequence status, and finally implementing red-line filtering through security constraint rules, hard convergence of the risk of long-term inference delay in the cloud and high real-time security interception on the edge are achieved. This solves the problem of task timeout caused by uncontrollable time consumption of large model thinking chain inference, and the security risks of the unfinished control logic after cloud truncation being executed in the physical world. Attached Figure Description

[0046] Figure 1 This is an architecture diagram of a unified registration and scheduling system for AI intelligent agent capabilities that is collaborative between the edge and cloud, as proposed in this invention.

[0047] Figure 2 This is a flowchart of the edge feature mapping and de-identification transmission process for an edge-cloud collaborative AI agent capability unified registration and scheduling system proposed in this invention.

[0048] Figure 3 This is a flowchart of the cloud-side mirror reconstruction and dynamic scheduling residual closed-loop process of an edge-cloud collaborative AI agent capability unified registration and scheduling system proposed in this invention. Detailed Implementation

[0049] The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1:

[0051] In the first embodiment of the present invention, the present invention provides a unified registration and scheduling system for AI agent capabilities in an edge-cloud collaborative manner, such as... Figures 1-3 As shown, it includes:

[0052] The dynamic behavior mapping module is used to capture the interaction behavior characteristics of the client and map the interaction behavior characteristics to the behavior control topology. When the behavior control topology deviates, the topology path is adjusted and the adjusted control behavior is registered to the unified MCP capability center.

[0053] Furthermore, in the dynamic behavior mapping module, the characteristics of client interaction behavior are captured, including:

[0054] Extract current system memory handle features, accessibility underlying tree topology features, and local semantic features of the current interactive interface image from the edge companion process;

[0055] The current system memory handle features, accessibility underlying tree topology features, and local semantic features of the image are bound together in a multi-dimensional logical space to construct a dynamic multimodal feature matrix. The dynamic multimodal feature matrix is ​​then output as an interactive behavior feature to the corresponding behavior control topology.

[0056] Furthermore, in the dynamic behavior mapping module, topology path adjustment includes:

[0057] The mapping-generated behavior control topology is transformed into the corresponding adjacency matrix, and topology node conflicts caused by interface drift or abnormal pop-ups on the client are monitored in real time as deviations from the behavior control topology.

[0058] When a deviation is detected, the local geometric similarity between the subgraph of the current conflict region and the preset baseline topology template is calculated to identify the target control node that has deviated.

[0059] By using local geometric similarity to perform control flow remapping on the adjacency matrix, local self-correction compensation of the topology control path is completed, which serves as topology path adjustment. The adjusted control behavior is then registered as a registered capability to the unified MCP capability center.

[0060] Specifically, the client establishes a collaborative connection with the cloud. The dynamic behavior mapping module first perceives the client's real-time state to capture the client's original multimodal interaction behavior characteristics.

[0061] The capture of interactive behavior features is performed independently by an edge companion process running at the client's underlying layer. In an environment without API support, this process obtains the current system memory handle features through low-level hook functions and system callback mechanisms to reflect the process identifier and memory boundaries of the active window. Simultaneously, the process calls the accessibility interface to parse the underlying tree topology features of the client interface, obtaining the hierarchical relationships of controls organized in a tree structure and text labels. Furthermore, the process obtains the video memory data cached in the current interactive interface frame buffer by calling the direct memory access mechanism, and uses an object detection network to extract the two-dimensional pixel coordinates of visual elements in this video memory data as local semantic features of the image. After obtaining these heterogeneous features, the process projects them onto a unified spatiotemporal metric space for multi-dimensional logical space binding. The specific process is as follows: The current system memory handle features, accessibility underlying tree topology features, and local image semantic features are mapped to orthogonal subspace feature vectors of the same dimension using a preset hash function. Subsequently, a tensor Kronecker product operation is performed on each orthogonal subspace feature vector. By extracting the second-order covariance distribution between different modal feature vectors, nonlinear cross-fusion of heterogeneous features at the memory level is achieved, thereby constructing a dynamic multimodal feature matrix to characterize the real-time operation state of the client. This matrix serves as a unified original multimodal interaction behavior feature and is output to the subsequent behavior control topology.

[0062] The dynamic behavior mapping module transforms the received dynamic multimodal feature matrix into a behavior control topology composed of nodes and directed edges using a graph transformation algorithm. The specific mapping rules of the graph transformation algorithm are as follows: The dynamic multimodal feature matrix is ​​parsed, and the memory base addresses of target control instances with independent memory lifetimes and valid pixel boundaries are extracted. These target control instances are defined as entity topology nodes in the behavior control topology. Simultaneously, the system's underlying message loop queue is parsed, and the system-level pointer call relationships between entity topology nodes and their parent-child inheritance states in the accessibility tree are extracted. These are quantified into state transition probability vectors, which are then defined as directed edges in the behavior control topology. After construction, the system transforms this into the corresponding adjacency matrix. During operation, the system monitors changes in the adjacency matrix structure. When the client experiences interface drift or abnormal pop-ups causing changes in control coordinates or displaying blocking prompts, the current topology node connection relationships will deviate from expectations, manifesting as abnormal increases or decreases of non-zero elements in the adjacency matrix. The system determines this state as a deviation in the behavior control topology.

[0063] When a deviation is detected, the system extracts a subgraph of the current conflict region and calculates the local geometric similarity between the subgraph and a preset baseline topology template using an improved graph edit distance algorithm. The specific calculation formula is as follows:

[0064] ;

[0065] In the above formula, S represents the calculated local geometric similarity, and the closer the value is to 1, the more similar the similarity. This represents the current conflict area subgraph; This represents a preset baseline topology template; Indicates will Convert to The minimum required graph edit distance is the sum of the minimum cost weights for inserting, deleting, and replacing vertices and edges; This represents the total number of vertices in the current conflict region subgraph; This represents the total number of directed edges in the current conflict region subgraph. Similarity is calculated. The system accurately identifies the specific target control node that has deviated from its intended path.

[0066] After identifying the target control node, the system utilizes local geometric similarity. Perform control flow remapping on the current adjacency matrix. The specific execution process of remapping is as follows: extract the original local adjacency matrix corresponding to the current conflict region subgraph that has deviated, and calculate the Laplacian graph of the original local adjacency matrix; then, calculate the local geometric similarity... The perturbation factor is converted into a topology perturbation factor and injected into the eigenvectors of the Laplacian graph for feature space alignment, thus obtaining the drift compensation matrix of the target control node. Subsequently, by performing a Hadamard product operation on the original local adjacency matrix and the drift compensation matrix, the state transition probabilities of the broken row and column vectors in the original local adjacency matrix are updated using the drift compensation matrix, thereby correcting the abnormal control flow direction and completing the local self-correction compensation of the topology control path. The corrected control behavior is structured and encapsulated according to standard protocols and registered as a registered capability to the unified MCP capability center for direct recognition and invocation by large models.

[0067] The semantic feature compression module is used to perform dimensionality reduction and compression processing on the original multimodal data stream to generate low-dimensional semantic feature stubs;

[0068] Furthermore, in the semantic feature compression module, the dimensionality reduction compression process includes:

[0069] The system intercepts time-series data streams, video streams, and audio streams generated by the client in real time, and assembles them by aligning and assembling the timestamps to construct the original multimodal data stream.

[0070] The feature thinning network is invoked to extract low-dimensional high-order feature vectors from the original multimodal data stream, and sensitive physical feature elements in the low-dimensional high-order feature vectors are identified through a pre-set database of classified parameter elements.

[0071] A dynamic feature mask is used to strip sensitive physical feature elements in real time and desensitize them by setting them to zero locally, so as to output a low-dimensional semantic feature stub.

[0072] Specifically, the semantic feature compression module undertakes the core functions of data stream dimensionality reduction and compliance-compliant de-identification. To achieve efficient and secure transmission between the client and the cloud, the system first needs to build a unified data foundation. The semantic feature compression module intercepts time-series data streams, video streams, and audio streams generated during client operation in real time through a kernel-cascaded socket mechanism. These raw stream data have extremely high spatiotemporal heterogeneity. The system uses a multiplexed timestamp alignment algorithm to perform cross-modal frame alignment and dynamic window sliding stitching based on microsecond-level timestamps, completing timestamp alignment and assembly, thereby constructing a unified raw multimodal data stream.

[0073] After constructing the original multimodal data stream, the system invokes a pre-deployed feature thinning network on the client side for spatial dimensionality reduction and feature extraction. The feature thinning network is a lightweight multimodal deep autoencoder algorithm. Its network topology sequentially includes: a 3D spatiotemporal residual convolutional layer for extracting local spatial features, a temporal self-attention manifold layer for cross-modal feature alignment, and a low-rank projection bottleneck layer for dimensionality compression. Before formal system deployment, this feature thinning network is trained offline using an unsupervised reconstruction mechanism: a massive amount of unlabeled client-side original multimodal data stream is used as training samples. The pixel-level difference between the decoded output reconstructed data stream and the input original multimodal data stream is calculated using mean squared error, and a feature thinning network based on... The sparsity penalty term of the norm is used to construct the total loss function; then, the network parameters are iteratively updated using the backpropagation algorithm until the total loss function converges. The trained network can filter out high-frequency redundant pixels and environmental noise in multimodal data, extracting low-dimensional high-order feature vectors from the original multimodal data stream. To ensure the security of core assets and privacy data on the edge, the system inputs the extracted low-dimensional high-order feature vectors into the classified information detection engine. Using a pre-set classified parameter feature library, the feature vectors to be identified are compared for similarity, thereby identifying sensitive physical features in the low-dimensional high-order feature vectors; the spatial projection matrix corresponding to the pre-set classified parameter feature library... It is an orthogonal basis matrix constructed by performing principal component analysis on a pre-collected set of historical sensitive physical operation instructions to extract orthogonal feature bases that represent the sensitive information dimension.

[0074] To preserve the integrity of general semantics while removing classified information, the system employs dynamic feature masks to perform real-time stripping and local zeroing of sensitive physical features. The dynamic feature masking process is based on sensitivity mapping matrix calculations. By calculating the sensitivity distribution of each higher-order feature dimension, corresponding flexible zeroing coefficients are generated. The specific feature desensitization formula is as follows:

[0075] ;

[0076] In the above formula, This represents the low-dimensional semantic feature stub vector of the final output; This represents a low-dimensional high-order feature vector extracted by a feature thinning network. This represents a unit vector of all ones that has the exact same dimension as the low-dimensional high-order feature vector. This represents a non-linear activation function. Here, the logistic activation function is used to limit the sensitivity mapping range to between 0 and 1. This indicates that the spatial projection matrix constructed by extracting orthogonal feature basis is used to map higher-order features to the sensitivity evaluation space; This represents the preset sensitivity evaluation bias vector; This represents the Hadamard product, which is the element-wise multiplication of two matrices of the same dimension.

[0077] Through matrix mapping and element-wise logical multiplication operations using the above formulas, the system can perform precise hardware-level nonlinear suppression and local zeroing desensitization on the memory channels corresponding to the identified sensitive physical features. This process completely removes the classified physical elements while preserving the non-sensitive backbone low-dimensional semantic information to the greatest extent, ultimately outputting a very small low-dimensional semantic feature stub. This low-dimensional semantic feature stub serves as a standardized middleware for subsequent transmission, not only eliminating the risk of privacy leakage but also significantly reducing the transmission load on the network channel.

[0078] The asynchronous state alignment module is used to convert low-dimensional semantic feature stubs into asynchronous stateless message streams for transmission and to reconstruct the corresponding context image data on the cloud side.

[0079] Furthermore, in the asynchronous state alignment module, converting the low-dimensional semantic feature stub into an asynchronous stateless message stream for transmission includes:

[0080] A one-way asynchronous finite state machine protocol is adopted to slice and encapsulate the generated low-dimensional semantic feature stubs into an asynchronous stateless message stream composed of multiple sets of single-wave pulse messages.

[0081] On an asymmetric unidirectional link, asynchronous stateless message streams are asynchronously delivered to the cloud side at a preset pulse frequency, and the bidirectional handshake interaction logic is discarded during transmission.

[0082] Furthermore, in the asynchronous state alignment module, reconstructing the corresponding context image data on the cloud side includes:

[0083] An independent shadow context buffer is created in the cloud-side memory, and asynchronous stateless message streams are received in real time;

[0084] The asynchronous stateless message stream is time-stacked and the data stream is reverse-reconstructed to extract the corresponding low-dimensional semantic feature stubs. Without obtaining the original multimodal data stream, a virtual state space aligned with the end side is reverse-reconstructed in the shadow context buffer as context mirror data.

[0085] Specifically, the asynchronous state alignment module is responsible for securely and with low latency cross-network transmission of the low-dimensional semantic feature stubs generated on the endpoint, and for high-fidelity virtual reconstruction of the context on the cloud side. In order to penetrate complex network boundaries and sever possible data traceability chains, the system first initiates an asymmetric transmission process on the endpoint.

[0086] The asynchronous state alignment module invokes standard network layer components and employs a one-way asynchronous finite state machine protocol to deeply modify the received low-dimensional semantic feature stubs. The one-way asynchronous finite state machine protocol, through pre-defined stateless transition logic, performs message encapsulation and transition unidirectionally, driving the state machine without state handshakes or bidirectional send / receive confirmations with the receiving end. Specifically, the generated low-dimensional semantic feature stubs are divided into fixed-length binary data blocks. A direct memory access (DMA) channel is invoked at the network card driver layer. An independent timestamp stub generated by a high-precision hardware clock source and a cyclic redundancy check pseudocode for integrity verification are appended to the beginning and end of each binary data block, thus completing data slicing and stateless encapsulation. By eliminating the contextual dependencies between messages, they are converted into an asynchronous stateless message stream composed of multiple sets of single-pulse messages, each set of pulse messages having completely independent unpacking dependencies.

[0087] After assembly, the asynchronous state alignment module directs the asynchronous stateless message stream to the asymmetric unidirectional link. The asymmetric unidirectional link is transformed into a physically or logically isolated channel that only allows unidirectional data flow by stripping the bidirectional handshake interaction logic from the standard network protocol stack at the transmission control layer (including removing the SYN and ACK mechanisms from the standard protocol and disabling the sliding window and timeout retransmission timer). The system asynchronously delivers the asynchronous stateless message stream to the cloud side using a preset pulse frequency as the transmission step size.

[0088] By discarding traditional cyclic redundancy check retransmission and sliding window acknowledgment, and to address potential packet out-of-order and packet loss issues arising from unidirectional link transmission, the system introduces a time-series self-healing and reassembly mechanism during the cloud-side reconstruction phase. Upon arrival of a single-pulse packet at the cloud side, the cloud-side system receives the asynchronous stateless packet stream in real time. To avoid interfering with the main logic computation of the cloud-side large model, the system utilizes zero-copy technology and calls the page-locking memory allocation operator in the cloud-side server to create an independent shadow context buffer in physical memory, specifically for isolating and streaming the received stateless data stream.

[0089] The cloud-side system utilizes a pre-defined time-series self-healing reassembly algorithm to perform time-series stacking and reverse data stream reassembly on asynchronous stateless message streams. This time-series self-healing reassembly algorithm performs out-of-order rearrangement based on sequence correlation within a dynamic time window. The specific formula for calculating message association weights is as follows:

[0090] ;

[0091] In the above formula, Indicates the first Group of single-wave pulse messages and the first Timing correlation weights between groups of single-pulse messages; and They represent the first Group and No. The characteristic location representation vector carried by a group of single-wave pulse messages in a stateless encapsulation; This represents the L2 norm operation, used to calculate spatial geometric distance; This indicates the preset distance attenuation coefficient; and These represent the timestamps assigned to the two sets of single-pulse messages during slicing; This represents the time window constraint factor. Its value is 1 when the absolute time difference between the two is within the preset sliding window threshold, and 0 otherwise.

[0092] By performing pairwise calculations on all single-pulse messages received within the current dynamic time window using the aforementioned correlation weight formula, a temporal topological correlation matrix is ​​constructed. Subsequently, through graph decomposition and path normalization of this temporal topological correlation matrix, complete low-dimensional semantic feature stubs are extracted from the stateless message sequence. Then, without acquiring the original multimodal data stream, the cloud-side system, based solely on the reconstructed low-dimensional semantic feature stubs, reconstructs a virtual state space perfectly aligned with the edge side in the shadow context buffer, using this as the context mirror data required for large model inference. This approach both prevents the leakage of original multimodal privacy data and achieves deterministic alignment of the cloud-side context mirror.

[0093] The inference stream scheduling module is used to obtain the edge running status, cloud inference status and network transmission status when a large model performs inference based on context mirror data, and to generate an inference stream scheduling strategy based on the registered capabilities in the unified MCP capability center, the edge running status, the cloud inference status and the network transmission status.

[0094] Furthermore, in the inference stream scheduling module, obtaining the endpoint running status, cloud-side inference status, and network transmission status includes:

[0095] Real-time monitoring of the communication channel carrying asynchronous stateless message streams, and extraction of the current network channel throughput as the network transmission status;

[0096] Real-time monitoring of computation time when large models perform inference based on context mirror data, and extraction of the current large model's first character output time and token generation rate as cloud-side inference status;

[0097] The remaining tolerance time of the current agent task is monitored in real time, and the reciprocal of the remaining tolerance time is converted into a time urgency index as the edge running status.

[0098] Furthermore, in the inference stream scheduling module, the generation of inference stream scheduling strategies includes:

[0099] The acquired network transmission status, cloud-side inference status, and edge-side operating status are transformed into a three-dimensional dynamic parameter vector;

[0100] The three-dimensional dynamic parameter vector is input into a preset multi-objective nonlinear optimization function for real-time optimization calculation. The probability of penetrating the red line of strong real-time response is predicted. Based on the comparison between the penetration probability and the preset safety water level threshold, a regular scheduling instruction or a truncated scheduling instruction is generated as the inference flow scheduling strategy.

[0101] Specifically, the inference flow scheduling module, acting as the core decision-making hub, is triggered in real time when the large cloud model performs deep inference based on the generated context image data. This module dynamically allocates the computing flow between the large cloud model and edge devices by comprehensively monitoring various operational metrics of the distributed system.

[0102] To build the data foundation for decision-making, the inference flow scheduling module first executes a multi-dimensional state acquisition process. The system's internal distributed monitoring component first monitors network traffic on the underlying network architecture. By calling the system's underlying network interface card performance counters, it monitors the communication channels carrying asynchronous stateless message streams in real time, extracting the current network channel throughput as the network transmission status. Simultaneously, the system calls performance analysis operators in the cloud-side inference cluster to monitor the computation time of the large model performing inference based on context mirror data in real time. From this, it accurately extracts the time of the first character output of the current large model and the token generation rate per unit time, as the cloud-side inference status. Furthermore, in the task management queue on the edge side, the system monitors the remaining tolerance time of the current agent task in real time, i.e., the absolute remaining time before the current task's deadline. It then uses mathematical transformation to convert the reciprocal of the remaining tolerance time into a timeliness urgency indicator, as the edge-side running status.

[0103] After acquiring the above three states, the inference flow scheduling module enters the strategy generation stage. The system concatenates the network transmission state, cloud-side inference state, and edge-side operation state, transforming them into a three-dimensional dynamic parameter vector that characterizes the current distributed environment. To achieve adaptive optimization in complex time-varying channels and variable computing power environments, the system inputs this three-dimensional dynamic parameter vector into a preset multi-objective nonlinear optimization function for real-time optimization calculation. The multi-objective nonlinear optimization function is constructed as a Lagrange dual equation containing two mutually exclusive objectives: the first objective function is to maximize the throughput of concurrent tokens in the cloud of the large model, and the second objective function is to minimize the total physical latency of end-to-end communication; simultaneously, the remaining available video memory (VRAM) capacity of the current cloud server and the instantaneous maximum physical bandwidth of the asymmetric unidirectional link are used as nonlinear inequality constraint boundaries.

[0104] The system employs the interior-point penalty function method, transforming the aforementioned multi-objective optimization into an unconstrained single-objective extremum solution within the nonlinear inequality constraint boundary, thereby obtaining a physically executable Pareto optimal solution. Based on the limiting time delay parameter extracted from this Pareto optimal solution, the system further predicts the penetration probability of the red line representing the strong real-time response while the inference time consumption of the complete thought chain of the large model penetrates it. The specific formula for calculating the penetration probability is as follows:

[0105] ;

[0106] In the above formula, The value represents the probability of the red line representing the time consumption of the complete thinking chain reasoning of the predicted large model, penetrating the strong real-time response, and its value is distributed between 0 and 1. This indicates the maximum expected total number of tokens required for the current large model to execute a complete thought chain reasoning; This indicates the token generation rate extracted from the cloud-side inference state of the current large model; This indicates the output time of the first character of the current large model; This represents the remaining tolerance time for the current agent's task, as detected by monitoring. This represents the nonlinear adjustment coefficient, used to scale the time difference.

[0107] By solving the above formula in real time, the inference flow scheduling module can quantify the current system timeout risk. After obtaining the penetration probability, the system automatically compares this probability with a preset security waterline threshold in real time. When the penetration probability does not exceed the preset security waterline threshold, it indicates that there is still spare capacity in cloud computing power and network bandwidth, and the system generates a regular scheduling instruction to maintain complete cloud inference. When the penetration probability exceeds the preset security waterline threshold, it indicates that complete cloud-based thought chain inference is likely to trigger a response timeout, and the system immediately generates a truncated scheduling instruction to initiate backbone residual inference prediction. The system outputs the regular scheduling instruction and the truncated scheduling instruction as the inference flow scheduling strategy, and, in conjunction with the registered capabilities in the unified MCP capability center, distributes them to the next level module for execution, thereby ensuring core throughput while realizing intelligent distribution and collaborative scheduling of dynamic computing power flow.

[0108] The thought flow control module is used to dynamically truncate the thought flow output process of the large model according to the inference flow scheduling strategy, and extract the core control segments and corresponding confidence information.

[0109] Furthermore, in the thought flow control module, dynamic truncation processing includes:

[0110] When the resolved inference stream scheduling policy is a truncated scheduling instruction to initiate backbone residual inference prediction, the streaming forced termination protocol is activated.

[0111] Based on the streaming forced termination protocol, streaming truncation is implemented when the backbone control logic token stream of the large model has completed a preset proportion. The token sequence that has been output before truncation is extracted as the core control segment, and the probability vector output at the truncation position is extracted as the corresponding confidence information.

[0112] Specifically, the thought flow control module, as the core entity controlling the streaming output rhythm of the large cloud model, is activated upon receiving the inference flow scheduling strategy generated by the superior. After the inference flow scheduling strategy is generated and sent to the thought flow control module, the module performs real-time deserialization parsing of the strategy content through its internal strategy parsing unit.

[0113] When the parsing unit identifies the current inference stream scheduling policy as a truncated scheduling instruction to initiate backbone residual inference prediction, it indicates that the system is facing a serious timeout risk. At this point, the thought chain flow control module immediately activates the streaming forced termination protocol. The streaming forced termination protocol is a proactive intervention mechanism operating at the decoding layer of the large model inference engine. This mechanism does not rely on conventional software layer application interface calls, but instead injects the highest priority hardware-level interrupt signal directly into the command execution queue of the graphics processing unit or tensor processing unit of the cloud computing cluster through the inter-process communication mechanism at the operating system level, thereby achieving real-time interception and precise truncation of streaming video memory data.

[0114] After the protocol is activated, the large model's thought chain streaming output process is monitored and intercepted. When the large model decodes and generates text, its thought chain stream is continuously output in the form of a token stream. The system uses a streaming forced termination protocol to truncate the stream when the model decoder has outputted a preset proportion of the backbone control logic token stream. This preset proportion is dynamically calculated based on the task type and time urgency. To implement streaming truncation and halt the global inference process at the optimal time, the system introduces interruption timing determination logic based on semantic entropy increase rate. This logic evaluates the integrity of the thought chain backbone control logic by calculating the conditional probability distribution of the continuous streaming token sequence. The specific interruption timing evaluation formula is as follows:

[0115] ;

[0116] In the above formula, Indicates the first generation of the streaming output of the large model The instantaneous semantic entropy value calculated for each token; This represents the known token history sequence that was output in chronological order before the truncation occurred; This indicates that the large model is currently decoding the generated first... One candidate token; This indicates that the current sequence is predicted based on the previously known sequence. The conditional probability distribution of candidate tokens; N represents the total number of candidate tokens in the large model vocabulary.

[0117] The instantaneous semantic entropy value during the streaming output process When a mutation occurs and exceeds a preset semantic convergence threshold, it indicates that the large model has output a deterministic backbone control intent, and subsequent outputs proceed into long-text thought deduction. At this point, the streaming forced termination protocol immediately sends a termination signal to the large model decoder, forcibly terminating the large model's global deduction process, instantly clearing uncommitted GPU memory output buffers, and forcibly releasing and reclaiming the underlying computing resources allocated to this inference task. After the truncation is implemented, the thought chain flow control module extracts the complete token sequence that was already streamed before the truncation and uses it as the core control fragment. This core control fragment contains the backbone intent and action framework required to complete the task. Simultaneously, the module extracts the complete probability vector of the last layer output of the decoder at the specific output position where the truncation is implemented and uses it as the corresponding confidence information. The core control fragment and confidence information are then packaged and distributed to guide subsequent residual completion and collaborative execution.

[0118] The residual collaborative execution module is used to send the core control segment and confidence information to the edge, and call the lightweight decision network at the edge to perform residual logic completion based on the core control segment in combination with the on-site time-series status information, so as to generate control commands and drive the behavior control topology to execute the corresponding control behavior.

[0119] Furthermore, in the residual collaborative execution module, residual logic completion includes:

[0120] The received core control segments and confidence information are converted into a local logic cue matrix;

[0121] The local logic prompt matrix and the real-time on-site time-series status information collected at the edge are input into the preset lightweight decision network for residual inference to complete the incomplete control logic, generate the original control sequence, and perform physical red-line filtering and compliance interception on the original control sequence through safety constraint rules to output control commands.

[0122] Specifically, the residual collaborative execution module serves as the final landing entity in the entire control closed loop. After the thought chain output of the large cloud model is truncated and intermediate data is generated, the residual collaborative execution module initiates the downlink data interaction process, sending the core control segments and confidence information to the edge device. Upon receiving this data, the edge device begins to perform high-real-time residual logic completion locally to eliminate the control logic incompleteness caused by the premature truncation on the cloud side.

[0123] The residual logic completion process first involves the matrix reconstruction of heterogeneous input sources. Edge devices embed the received core control segments into text vectors and combine them with the corresponding confidence information probability vectors through tensor concatenation and linear mapping, converting them into a local logic cue matrix. This local logic cue matrix fully carries the incomplete, uncertain backbone control intent of the large cloud model. Simultaneously, edge devices collect real-time on-site temporal status information at the edge via local sensor arrays and system log components. This on-site temporal status information reflects the latest temporal abrupt changes in the current physical equipment or operating environment. Subsequently, the edge devices input the local logic cue matrix and the on-site temporal status information into a pre-defined lightweight decision network.

[0124] The lightweight decision network is a low-latency residual prediction neural network deployed on the edge to perform residual inference. Physically, it is instantiated as a feedforward deep separable convolutional architecture deployed on an edge processor, cascaded with a multi-head cross-attention mechanism. Before system deployment, the network undergoes supervised training on a historical truncated dataset: using historical human-interventional interruption control sequence vectors and corresponding on-site sensor time-series data as input features, and the final successfully executed complete machine control command sequence as the ground truth. The network parameters are iteratively optimized end-to-end using a cross-entropy loss function, enabling it to possess residual compensation logic that maps incomplete cloud-based prompts to complete local physical execution commands. During inference, the network cross-fuses cloud-based prompts and local context through the aforementioned multi-head cross-attention mechanism to complete the incomplete control logic at the truncated points of the large model and output the original control sequence. To ensure the generated original control sequence is safe and controllable in the physical world or the host system, the system introduces safety constraint rules at the output, performing physical red-line filtering and compliance interception. The safety constraint rules are based on a multi-dimensional linear programming mapping of a preset physical boundary safety domain. Mathematically, dangerous actions that exceed physical limit thresholds are forcibly converged into the safety domain. The specific compliance interception and filtering formula is as follows:

[0125] ;

[0126] In the above formula, This represents the final output and the deterministic control command vector that is issued and executed. This represents the original control sequence vector generated by residual inference from a lightweight decision network. This represents the candidate vector of the target control command to be optimized; The square of the L2 norm is used to constrain the output instructions to be as close as possible to the original reasoning intent; This represents the projection matrix of the physical redline boundary defined in the preset safety constraint rules; This represents the preset system security threshold boundary vector; This represents the nonlinear penalty factor. Through nonlinear truncation and filtering optimization of this formula, the system can locally correct any dangerous control actions that exceed the physical red line boundary in real time and converge them to the safe domain, thereby outputting the final control command.

[0127] After generating control commands, the residual collaborative execution module converts these commands into machine code that can be directly recognized by the underlying operating system or hardware actuator, and immediately drives the behavior control topology in the dynamic behavior mapping module to execute the corresponding control behavior. Through this mechanism, the incomplete control logic that was originally forcibly stream-truncated in the cloud due to system timeout risks is effectively completed at the edge using a lightweight decision network and on-site timing status information. Under the interception and protection of the safety red line, the behavior control topology is precisely driven to complete the final behavior loop.

[0128] Example 2:

[0129] In the cross-platform operation and maintenance upgrade of industrial automation control systems in large enterprises, the numerous legacy client systems, many of which are non-standard and closed front-end systems without API support, prevent cloud-based intelligent agents from achieving automated scheduling through standard interfaces. Furthermore, during multi-device adaptation, the interactive interfaces of these legacy systems frequently experience frequent control coordinate drift due to sudden abnormal error pop-ups or interface rendering delays, leading to conflicts in the behavior control topology nodes and severely disrupting the control flow of the intelligent agent system. This results in extremely high system integration and manual maintenance costs. To address these issues, this invention provides a unified registration and scheduling system for edge-cloud collaborative AI intelligent agent capabilities, the structure of which is as follows: Figure 1 As shown. The specific implementation process of this system is as follows:

[0130] An edge companion process running on the underlying layer of the old industrial automation control client is started. In an environment without API support, the edge companion process obtains the memory handle characteristics of the old client's active window through low-level hook functions and system callback mechanisms, reflecting the process identifier and memory boundaries of the active window. Simultaneously, the process calls the auxiliary function interface to parse the accessibility tree topology characteristics of the client interface to obtain the control hierarchy. Furthermore, the process extracts local semantic features of the current interface image by periodically taking screenshots and calling a lightweight network to locate the two-dimensional pixel coordinates of each control.

[0131] After acquiring the aforementioned heterogeneous features, the edge-company process projects them locally into a unified spatiotemporal metric space for spatial binding and cascading, thereby constructing a dynamic multimodal feature matrix to characterize the real-time operational state of the client. This matrix is ​​output as a unified original multimodal interaction behavior feature.

[0132] The dynamic behavior mapping module transforms the received dynamic multimodal feature matrix into a corresponding behavior control topology using a graph transformation algorithm, and further converts it into an adjacency matrix for easy quantification. During system operation, the system continuously monitors changes in the adjacency matrix to perceive its operational status. When the old system experiences control coordinate drift due to resolution incompatibility, or when an error message pops up from the underlying layer, the current topology node connection relationships will conflict with expectations, manifesting as abnormal increases or decreases of non-zero elements in the adjacency matrix. The system then determines that the behavior control topology has deviated.

[0133] When a deviation is detected, the system extracts a subgraph of the current conflict area and calculates the local geometric similarity between the conflict subgraph and the preset baseline topology template using the improved graph editing distance algorithm of the aforementioned embodiment, thereby accurately identifying the target control node that has deviated in the complex conflict interface.

[0134] After identifying the target control node, the system uses the calculated local geometric similarity as weighting coefficients to perform control flow remapping on the current adjacency matrix. The remapping process adjusts the row and column vectors corresponding to conflicting nodes in the adjacency matrix through matrix transformations, completing local self-correction compensation of the topology control path on the old client.

[0135] After local calibration, the control flow that was previously blocked due to deviation is restored to normal, generating adjusted control behavior. Finally, the system structurally encapsulates the adjusted control behavior according to standard protocols and registers it as a registered capability to the unified MCP capability center. After registration, the non-standard behavior that was originally caused by the heterogeneity or anomalies of the old system is transformed into a standardized capability that can be directly invoked and recognized by the cloud-based large model, realizing the autonomy and unified scheduling of the edge-side control flow.

[0136] Finally, it should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A unified registration and scheduling system for AI intelligent agent capabilities in an edge-cloud collaborative manner, characterized in that, include: The dynamic behavior mapping module is used to capture the interaction behavior characteristics of the client and map the interaction behavior characteristics to the behavior control topology. When the behavior control topology deviates, the topology path is adjusted and the adjusted control behavior is registered to the unified MCP capability center. In the dynamic behavior mapping module, the topology path adjustment includes: The mapping-generated behavior control topology is transformed into the corresponding adjacency matrix, and topology node conflicts caused by interface drift or abnormal pop-ups on the client are monitored in real time as deviations from the behavior control topology. When a deviation is detected, the local geometric similarity between the subgraph of the current conflict region and the preset baseline topology template is calculated to identify the target control node that has deviated. The local geometric similarity is used to remap the adjacency matrix for control flow, and the local self-correction compensation of the topology control path is completed as the topology path adjustment. The adjusted control behavior is then registered as a registered capability to the unified MCP capability center. The semantic feature compression module is used to perform dimensionality reduction and compression processing on the original multimodal data stream to generate low-dimensional semantic feature stubs; The asynchronous state alignment module is used to convert the low-dimensional semantic feature stub into an asynchronous stateless message stream for transmission and to reconstruct the corresponding context image data on the cloud side. The inference stream scheduling module is used to obtain the edge running status, cloud inference status and network transmission status when a large model performs inference based on context mirror data, and to generate an inference stream scheduling strategy based on the registered capabilities in the unified MCP capability center, the edge running status, the cloud inference status and the network transmission status. The thought flow control module is used to dynamically truncate the thought flow output process of the large model according to the inference flow scheduling strategy, and extract the core control segments and corresponding confidence information. The residual collaborative execution module is used to send the core control segment and confidence information to the edge, and call the lightweight decision network at the edge to perform residual logic completion based on the core control segment in combination with the on-site time-series status information, so as to generate control commands and drive the behavior control topology to execute the corresponding control behavior.

2. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the dynamic behavior mapping module, the captured client interaction behavior characteristics include: Extract current system memory handle features, accessibility underlying tree topology features, and local semantic features of the current interactive interface image from the edge companion process; The current system memory handle features, accessibility underlying tree topology features, and local semantic features of the image are bound together in a multi-dimensional logical space to construct a dynamic multimodal feature matrix. The dynamic multimodal feature matrix is ​​then output as an interactive behavior feature to the corresponding behavior control topology.

3. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the semantic feature compression module, the dimensionality reduction compression process includes: The system intercepts time-series data streams, video streams, and audio streams generated by the client in real time, and assembles them by aligning and assembling the timestamps to construct the original multimodal data stream. The feature thinning network is invoked to extract low-dimensional high-order feature vectors from the original multimodal data stream, and sensitive physical feature elements in the low-dimensional high-order feature vectors are identified through a pre-set database of classified parameter elements. A dynamic feature mask is used to strip sensitive physical feature elements in real time and desensitize them by setting them to zero locally, so as to output a low-dimensional semantic feature stub.

4. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the asynchronous state alignment module, converting the low-dimensional semantic feature stub into an asynchronous stateless message stream for transmission includes: A one-way asynchronous finite state machine protocol is adopted to slice and encapsulate the generated low-dimensional semantic feature stubs into an asynchronous stateless message stream composed of multiple sets of single-wave pulse messages. On an asymmetric unidirectional link, asynchronous stateless message streams are asynchronously delivered to the cloud side at a preset pulse frequency, and bidirectional handshake interaction logic is discarded during transmission.

5. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the asynchronous state alignment module, the reconstruction of the corresponding context image data on the cloud side includes: An independent shadow context buffer is created in the cloud-side memory, and the asynchronous stateless message stream is received in real time; The asynchronous stateless message stream is time-stacked and the data stream is reverse-reconstructed to extract the corresponding low-dimensional semantic feature stubs. Without obtaining the original multimodal data stream, a virtual state space aligned with the end side is reverse-reconstructed in the shadow context buffer as context mirror data.

6. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the inference stream scheduling module, obtaining the endpoint running status, cloud-side inference status, and network transmission status includes: Real-time monitoring of the communication channel carrying asynchronous stateless message streams, and extraction of the current network channel throughput as the network transmission status; Real-time monitoring of computation time when large models perform inference based on context mirror data, and extraction of the current large model's first character output time and token generation rate as cloud-side inference status; The remaining tolerance time of the current agent task is monitored in real time, and the reciprocal of the remaining tolerance time is converted into a time urgency index as the edge running status.

7. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the inference stream scheduling module, the inference stream scheduling strategy includes: The acquired network transmission status, cloud-side inference status, and edge-side operating status are transformed into a three-dimensional dynamic parameter vector. The three-dimensional dynamic parameter vector is input into a preset multi-objective nonlinear optimization function for real-time optimization calculation. The probability of penetrating the red line of strong real-time response is predicted. Based on the comparison between the penetration probability and the preset safety water level threshold, a regular scheduling instruction or a truncated scheduling instruction is generated as the inference flow scheduling strategy.

8. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the thought flow control module, the dynamic truncation process includes: When the resolved inference stream scheduling strategy is a truncated scheduling instruction to initiate backbone residual inference prediction, the streaming forced termination protocol is activated. Based on the streaming forced termination protocol, streaming truncation is implemented when the backbone control logic token stream of the large model has completed a preset proportion. The token sequence that has been output before truncation is extracted as the core control segment, and the probability vector output at the truncation position is extracted as the corresponding confidence information.

9. The edge-cloud collaborative AI agent capability unified registration and scheduling system according to claim 1, characterized in that, In the residual collaborative execution module, the residual logic completion includes: The received core control segments and confidence information are converted into a local logic cue matrix; The local logic prompt matrix and the real-time on-site time-series status information collected at the edge are input into the preset lightweight decision network for residual inference to complete the incomplete control logic, generate the original control sequence, and perform physical red-line filtering and compliance interception on the original control sequence through safety constraint rules to output control commands.

Images