A mobile phone terminal side-based AI computing power sharing method, system, device and medium

Abstract

The application discloses a kind of AI computing power sharing method, system, equipment and medium based on mobile phone end side, it is related to mobile computing and distributed artificial intelligence technical field.The method includes: receiving the large language model inference task of initiator;Task is extracted as constraint set by task analysis agent, and converted into node calculation, KV cache and bandwidth constraint by using computing-communication mapping model and generates subtask;Call multi-agent collaborative architecture to scan surrounding available mobile phone node dynamically and score, and map subtask to target mobile phone node using bipartite graph matching;Initial scheme is deployed simulation on end side, and obtain each node simulation report;Through simulation closed loop agent, identify performance bottleneck, and call optimization agent to perform KV cache unloading and other directional adjustment;Loop simulation iteration until no performance violation, output final deployment scheme.The application realizes the collaborative inference of idle computing power of mobile phone in networkless environment, effectively avoids end side memory overflow and privacy leakage risk.

Description

Technical Field

[0001] This invention relates to the fields of mobile computing and distributed artificial intelligence technology, specifically to a method, system, device, and medium for sharing AI computing power on a mobile device. Background Technology

[0002] The statements in this section are provided only as background information in connection with this disclosure and may not constitute prior art.

[0003] In recent years, Large Language Models (LLMs) based on the Transformer architecture have demonstrated outstanding capabilities in natural language understanding and generation tasks. However, the sheer number of parameters in large language models, often exceeding billions, and the token-by-token generation computational pattern during autoregressive decoding, result in extremely high demands on computing power and memory bandwidth for their inference process. When deploying large language models on mobile devices, even quantized and compressed models still require enormous memory space and continuous floating-point computing power, far exceeding the independent capacity of current mainstream single mobile NPU / GPU, severely restricting the application of large models in real-time interactive scenarios on mobile devices.

[0004] Existing solutions primarily rely on cloud-based GPU clusters for large language model inference, but this has significant drawbacks: First, high network round-trip time (RTT) leads to sluggish user experience; second, user-inputted suggestions involve personal privacy, and uploading them to the cloud poses a data leakage risk; third, it heavily depends on internet connectivity, rendering it completely unusable in environments without or with weak network connections. On the other hand, while distributed inference solutions for large language models exist (such as pipelined parallelism and tensor parallelism), these solutions were originally designed for data center-level servers, assuming high-bandwidth, low-latency dedicated interconnects between nodes (such as NVLink), making them unsuitable for the limited bandwidth and fluctuating latency of mobile devices in wireless communication environments. More critically, the dynamically growing key-value cache (KV cache) during large language model inference consumes memory linearly with the context length. Existing scheduling strategies lack a joint optimization mechanism for cross-node allocation and dynamic management of the mobile KV cache, easily leading to edge memory overflow (OOM) and inference failure. Summary of the Invention

[0005] The purpose of this invention is to address the technical challenges of single mobile devices being unable to independently support large language model inference, the high latency and privacy risks associated with cloud reliance, and the inability of existing data center distributed solutions to adapt to mobile wireless communication environments and dynamic management of key-value (KV) caches. This invention provides a mobile device-based AI computing power sharing method, system, device, and medium. Based on a multi-agent collaborative architecture and a computation-communication mapping model, combined with on-device deployment simulation and closed-loop optimization mechanisms, it achieves efficient, adaptive, collaborative, distributed inference and secure allocation of KV caches across multiple idle mobile devices, completely eliminating cloud dependency and on-device memory overflow risks.

[0006] The technical solution of the present invention is as follows: A method for sharing AI computing power on a mobile device, comprising: The system receives a large language model inference task from the initiating mobile phone. The large language model inference task includes model type, number of model parameters, length of input prompt words, maximum generation length, inference accuracy requirements, temperature parameters, maximum end-to-end latency constraints, and maximum allowable power consumption. The task parsing agent extracts the large language model inference task into a set of structured constraints, and uses a computation-communication mapping model to convert the performance indicators of the large language model autoregressive inference into the required node computation, KV cache memory usage and communication bandwidth physical constraints. At the same time, it generates a set of subtasks and corresponding resource requirement vectors. The performance indicators include first token latency, inter-token latency and generation throughput. Based on a multi-agent collaborative architecture, an initial computing power sharing scheme is generated by invoking a discovery agent, an evaluation agent, and a scheduling agent. The discovery agent dynamically scans surrounding available mobile phone nodes, collecting hardware computing power information, current load status, battery level, and network signal strength for each available mobile phone node. The evaluation agent uses a multi-attribute decision-making algorithm to comprehensively score the capabilities of each available mobile phone node and generates a priority queue of candidate nodes based on the resource demand vector. The scheduling agent, according to the priority queue of candidate nodes and the data dependencies between subtasks in the subtask set, uses a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node, and ensures load balancing through backtracking correction. The initial computing power sharing scheme is deployed and simulated on the edge to obtain simulation reports of computing latency, KV cache memory usage, power consumption and communication transmission for each target mobile phone node. By simulating a closed-loop intelligent agent and employing a graph neural network-based performance predictor, the system automatically identifies performance bottleneck information in the computation latency simulation report, the KV cache memory usage simulation report, the power consumption simulation report, and the communication transmission simulation report. Based on the type of performance bottleneck information, the system calls an optimization agent to perform targeted adjustment operations. The types of performance bottleneck information include computational bottlenecks, storage bottlenecks, and communication bottlenecks. The adjusted computing power sharing scheme is deployed and simulated again on the edge and iterated until there are no performance violations in all simulation reports and the generation quality meets the task requirements. The deployable computing power sharing configuration file and the communication topology file between the target mobile phone nodes are then output.

[0007] Furthermore, a computation-communication mapping model is used to convert the performance metrics of autoregressive inference in large language models into physical constraints such as required node computation, key-value cache memory usage, and communication bandwidth. Simultaneously, a set of subtasks and corresponding resource requirement vectors are generated, specifically including: Based on the aforementioned set of structured constraints, the resource requirement weights for each inference stage of the large language model are calculated using normalization operations:

[0008] in, For the first Resource requirement weights for each inference stage For the first Initial computational complexity score for each inference stage. The total number of reasoning stages. The temperature coefficient is used to control the smoothness of demand distribution; the resource demand weights are used to construct the resource demand vector. The data dependency matrix between the Transformer layers contained in the large language model is calculated using a multi-head self-attention mechanism:

[0009] in, The data dependency matrix is... To represent the query matrix of features computed by the current Transformer layer, The key matrix consists of the parameters of each layer and the number of floating-point operations. The transpose of the key matrix. For the dimension of the key, This represents the bias term representing the mandatory constraint on data dependency between layers; Based on the data dependency matrix between each Transformer layer, a set of candidate segmentation templates is obtained by feature extraction using a multi-layer graph attention network. An initial task segmentation strategy is selected from the set of candidate segmentation templates based on the overall computational graph structure of the model to divide and generate the set of subtasks.

[0010] Furthermore, a set of candidate segmentation templates is obtained by feature extraction using a multi-layer graph attention network, specifically including: Through the first layer graph attention network, local computation path features are extracted based on the data dependency matrix between each Transformer layer. The local computation path features include the computational ratio between the self-attention module and the feedforward network module. The second-layer graph attention network extracts global communication pattern features based on the local computation path features. These global communication pattern features are used to characterize the tensor parallel communication overhead during layer combination. Using the third-layer graph attention network, based on the global communication pattern features, historical large language model distributed inference cases, and mobile phone hardware performance database, the overall computation graph of the large language model and the high-level resource feature representations of each layer are extracted. Calculate the matching score between each layer in the advanced resource feature representation and the segmentation scheme in the standard distributed inference template library, extract the segmentation schemes with matching scores greater than a set threshold, and obtain the candidate segmentation template set.

[0011] Furthermore, the evaluation agent uses a multi-attribute decision-making algorithm to score the comprehensive capabilities of each available mobile phone node. The specific calculation formula is as follows:

[0012] in, The first of the available mobile phone nodes A comprehensive capability score for each available mobile node. For the first The benchmark score corresponding to the hardware computing power information of each available mobile node. For the first The current load status of each available mobile node represents the resource utilization rate. For the first The normalized value corresponding to the battery level of each available mobile node. For the first The quality index corresponding to the network signal strength of each available mobile node. to The weights are dynamic and satisfy the following conditions: ; The dynamic weights are adjusted in real time based on the real-time requirements and power consumption constraints of large language model inference tasks. When the maximum end-to-end delay constraint is less than the preset delay threshold, increase and The value of is adjusted, and the values ​​of the remaining weights are reduced accordingly to maintain the constraint that the sum of the weights is 1; when the maximum allowable power consumption is lower than a preset proportion of the initiating mobile phone battery capacity, the value of is increased. The value of is determined, and the values ​​of the remaining weights are reduced accordingly to maintain the constraint that the sum of the weights is 1; or, the values ​​of each dynamic weight are controlled to tend to be evenly distributed.

[0013] Furthermore, based on the type of performance bottleneck information, the optimization agent is invoked to perform targeted adjustment operations, specifically including: If the type of the performance bottleneck information is a computational bottleneck, then adjust the subtask splitting granularity, improve the quantization accuracy of the computation process, or skip a preset proportion of secondary attention points; If the performance bottleneck information is a storage bottleneck, then enable the KV cache offloading strategy to compress some historical token caches and migrate them to the initiating mobile phone for storage, or enable context window pruning. If the performance bottleneck information is a communication bottleneck, then adjust the communication topology between the target mobile phone nodes to a hierarchical aggregation structure, or increase the transmission batch to reduce the number of communication handshakes between the target mobile phone nodes.

[0014] Furthermore, all simulation reports showed no performance violations and the generated quality met the task requirements, specifically meaning that the following conditions were met simultaneously: The simulated computational latency values ​​of all target mobile phone nodes are less than the latency limit of their respective assigned subtasks; The simulated KV cache memory usage of each target mobile node is less than the corresponding preset memory allocation limit safety ratio. The simulated total communication latency between the target mobile phone nodes is less than the preset communication latency threshold.

[0015] This invention also proposes a mobile-side AI computing power sharing system to implement the mobile-side AI computing power sharing method described above. The system includes: The task receiving module is used to receive large language model inference tasks from the initiating mobile phone. The task parsing and mapping module is used to extract the large language model inference task into a set of structured constraints through the task parsing agent, and to use a computation-communication mapping model to convert the performance indicators of the large language model autoregressive inference into the required node computation amount, KV cache memory usage and communication bandwidth physical constraints, while generating a set of subtasks and corresponding resource requirement vectors. The multi-agent collaborative scheduling module is used to call the discovery agent to dynamically scan the surrounding available mobile phone nodes and collect hardware status, call the evaluation agent to perform a comprehensive capability score on each available mobile phone node and generate a candidate node priority queue, and call the scheduling agent to use a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node based on the candidate node priority queue and the data dependency relationship between each subtask in the subtask set, so as to generate an initial computing power sharing scheme. The edge simulation module is used to perform edge deployment simulation of the initial computing power sharing scheme and obtain simulation reports for each target mobile phone node; The closed-loop optimization module is used to identify performance bottleneck information in the simulation report by using a performance predictor through a simulation closed-loop agent. Based on the type of performance bottleneck information, it calls the optimization agent to perform targeted adjustment operations and cyclically sends the adjusted computing power sharing scheme into the edge deployment simulation until there are no performance violations, and outputs a deployable computing power sharing configuration file and the communication topology file between the target mobile phone nodes.

[0016] Furthermore, the directional adjustment operations performed in the closed-loop optimization module include: To address computational bottlenecks, adjust the segmentation granularity, improve quantization accuracy, or skip secondary attention heads; to address storage bottlenecks, perform KV cache unloading or context window pruning; to address communication bottlenecks, adjust the communication topology between the target mobile phone nodes or increase the transmission batch size.

[0017] The present invention also proposes an electronic device, comprising: At least one processor; and a memory communicatively connected to said at least one processor; The memory stores instructions that can be executed by the at least one processor, and the at least one processor executes the instructions stored in the memory to perform the method described above.

[0018] The present invention also proposes a computer-readable storage medium for storing instructions that, when executed, cause the method described above to be implemented.

[0019] Compared with existing technologies, the advantages of this invention are: 1. Completely eliminate cloud dependency and privacy risks: Through a multi-agent collaborative mechanism (discovery, evaluation, and scheduling of agents), idle mobile phone nodes in the vicinity are organized to perform distributed collaborative inference. It eliminates the need for cloud servers, enabling smooth operation of large language models even in offline environments, and physically preventing the leakage of user privacy data.

[0020] 2. Segmentation and Scheduling Highly Adaptable to Mobile Wireless Environments: An innovative multi-layer graph attention network is introduced to extract data dependencies and communication pattern features between Transformer layers. This scheme perfectly aligns the segmentation strategy with the serial autoregressive characteristics of large language models, and uses a bipartite graph matching algorithm combined with real-time node states for mapping and allocation, minimizing cross-node communication overhead and latency.

[0021] 3. A unique simulation closed-loop mechanism ensures absolute reliability: Addressing the pain point of memory overflow easily caused by KV cache, this invention introduces "edge-side deployment simulation" before formal deployment and identifies computing, storage, and communication bottlenecks in advance through a performance predictor based on graph neural networks. Combined with dynamic adjustment strategies (such as dynamically unloading KV cache to the initiating storage and targeted rescheduling), iterative loops ensure that the final output computing power sharing scheme has 100% security and resilience. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments recorded in the embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings.

[0023] Figure 1 A flowchart illustrating an AI computing power sharing method based on a mobile device side, provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0024] It should be noted that relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0025] The features and performance of the present invention will be further described in detail below with reference to embodiments.

[0026] Example 1 In recent years, Large Language Models (LLMs) based on the Transformer architecture have demonstrated outstanding performance in natural language processing. However, their billions of parameters and the token-by-token generation during autoregressive decoding result in extremely high demands on computing power and memory bandwidth, far exceeding the independent capacity of current mainstream single mobile NPU / GPU. Existing technologies typically rely on cloud servers for inference, but this not only suffers from severe network round-trip time (RTT) causing interactive stuttering, but also easily leads to user privacy leaks and is completely unusable in weak network environments. Furthermore, traditional data center-level distributed inference solutions (such as NVLink direct connection) cannot be directly adapted to the wireless communication environment with limited bandwidth and fluctuating latency on the mobile device side.

[0027] To solve the above problems, such as Figure 1 As shown, this embodiment provides a method for sharing AI computing power on a mobile device. Through a multi-agent collaborative mechanism, it organizes idle mobile computing power in the vicinity for distributed inference, fundamentally eliminating cloud latency and privacy risks. The method specifically includes the following steps: Step S1: Receive the large language model inference task from the initiating mobile phone. The large language model inference task includes model type, number of model parameters, length of input prompt words, maximum generation length, inference accuracy requirements, temperature parameters, maximum end-to-end latency constraints, and maximum allowable power consumption.

[0028] Specifically, the initiating mobile device (i.e., the user's local primary device) can be a smartphone, tablet, or other mobile terminal. When the user submits a natural language request through a local AI assistant, real-time translation, or intelligent customer service application, the system acquires the inference task. The multi-dimensional parameters set here (such as model type, input prompt word length, temperature parameters, and kernel sampling parameters such as Top-p) are the cornerstone for subsequent accurate resource prediction. It should be noted that those skilled in the art will understand that the above parameter list is only a preferred specific implementation. In practical applications, relevant parameters can be flexibly added or deleted according to the specific model architecture, which does not depart from the spirit of the present invention.

[0029] Step S2: The task parsing agent extracts the large language model inference task into a set of structured constraints, and uses a computation-communication mapping model to convert the performance indicators of the large language model autoregressive inference into the required node computation, KV cache memory usage and communication bandwidth physical constraints. At the same time, a set of subtasks and corresponding resource requirement vectors are generated. The performance indicators include first token latency, inter-token latency and generation throughput.

[0030] In this step, specifically, the task parsing agent first extracts the aforementioned task parameters using a local lightweight semantic understanding module and converts them into a structured key-value pair format; then, it estimates the total computational cost of the pre-filling and decoding stages based on the locally cached large language model metadata, as well as the estimated total KV cache memory requirements.

[0031] In this embodiment, specifically, when estimating the total computational load, the theoretical computational complexity of the pre-filling stage is:

[0032] The computational complexity of token-by-token computation during the decoding phase is:

[0033] in: For the great Big O notation is used to characterize the asymptotic computational complexity of an algorithm in terms of magnitude. The length of the input sequence; The number of Transformer layers; For the hidden layer dimension; Number of tokens generated; When estimating the total KV cache memory requirement, the formula for the size of each KV cache level is:

[0034] in: Size of each KV cache layer; Total length of the context; Feature dimensions for each attention head; This represents the total number of attention heads in the model; constant term This means that each token needs to cache both the key and value matrices during reasoning.

[0035] To address the unique serial autoregressive characteristics of large language models, a computation-communication mapping model is employed to convert the performance metrics of autoregressive inference in large language models into physical constraints such as required node computation, key-value cache memory usage, and communication bandwidth. Simultaneously, a set of subtasks and corresponding resource requirement vectors are generated, specifically including: First, based on the set of structured constraints, the resource requirement weights for each inference stage of the large language model are calculated using normalization operations:

[0036] in: Let be the resource requirement weight for the i-th inference stage; Score the initial computational complexity of the i-th inference stage (the complexity of the pre-filling stage is typically O(n).) The decoding stage is ); The total number of reasoning stages; This is a temperature coefficient used to control the smoothness of demand distribution.

[0037] This step effectively balances the huge differences in computing power requirements at different stages by introducing a temperature coefficient, preventing extreme resource allocation. After calculating the resource requirement weights, they are weighted and combined with the basic computational characteristics of each inference stage to construct the resource requirement vector that can accurately reflect the real physical overhead of different generation stages, and used as the basis for subsequent evaluation of the agent to select nodes.

[0038] Secondly, a multi-head self-attention mechanism is used to calculate the data dependency matrix between the various Transformer layers contained in the large language model:

[0039] in: This is the data dependency matrix; This represents the query matrix that represents the features computed by the current Transformer layer; The key matrix consists of the parameters of each layer and the number of floating-point operations. This is the transpose of the key matrix; The dimension of the key; This represents the bias term that represents the mandatory constraint of data dependency between layers (such as the pre- and post-residual relationship of residual connections).

[0040] Specifically, the dependency values ​​between adjacent layers in the matrix are close to 1, indicating a forced order dependency during forward propagation; cross-layer residual connections serve as bias terms. Attention is being calculated. This dependency matrix will strictly ensure that subsequent task splitting operations will not disrupt the residual connection paths of the large language model, thereby avoiding distributed inference errors.

[0041] Finally, based on the data dependency matrix between each Transformer layer, a candidate segmentation template set is obtained by feature extraction using a multi-layer graph attention network. An initial task segmentation strategy is selected from this candidate segmentation template set based on the overall computational graph structure of the model to divide and generate the subtask set. The specific segmentation methods of the task segmentation strategy include, but are not limited to: layer-based segmentation, attention head-based segmentation (e.g., tensor parallel segmentation), and pipeline stage-based segmentation, thereby fully adapting to the fragmented computing power of different mobile nodes.

[0042] Furthermore, to make the segmentation scheme more consistent with the autoregressive characteristics of large language models and minimize cross-node communication overhead, the step of using a multi-layer graph attention network for feature extraction to obtain a candidate segmentation template set specifically includes: Through the first layer graph attention network, local computation path features are extracted based on the data dependency matrix between each Transformer layer. The local computation path features include the computational ratio between the self-attention module and the feedforward network module. The second-layer graph attention network extracts global communication pattern features based on the local computation path features. These global communication pattern features are used to characterize the tensor parallel communication overhead during layer combination. Using the third-layer graph attention network, based on the global communication pattern features, historical large language model distributed inference cases, and mobile phone hardware performance database, the overall computation graph of the large language model and the high-level resource feature representations of each layer are extracted. Calculate the matching score between each layer in the advanced resource feature representation and the segmentation scheme in the standard distributed inference template library, extract the segmentation schemes with matching scores greater than a set threshold, and obtain the candidate segmentation template set.

[0043] As an optional implementation, the underlying calculation process of the graph attention network and matching score described above is as follows: The structured model parameter matrix Perform a linear transformation to obtain the input embedding of the first layer:

[0044] in: The structured model parameter matrix, the Specifically, this includes the number of parameters for each layer, the number of floating-point operations, the number of attention heads, and the dimensions of the hidden layers; All are trainable weight matrices; These are the query vector, key vector, and value vector of the graph attention network, respectively.

[0045] Features of global communication mode in the second layer The expression is:

[0046] in: This is a characteristic of global communication mode; For local computation path characteristics; This is a multi-head graph attention operation used to aggregate information from adjacent layers; This is a layer normalization operation used to stabilize the training process; It is a feedforward neural network consisting of two linear layers and a ReLU activation function, used to enhance the non-linear expressive power of features; Residual connection in the formula ( This is used to preserve the original feature information and prevent gradient vanishing.

[0047] Third-level intermediate and advanced resource feature representation The expression is:

[0048] in: It represents high-level resource characteristics, including computational intensity, communication sensitivity, and key-value cache usage characteristics at each layer; This is a characteristic of global communication mode; Encoding embedding vectors for distributed reasoning cases in a historical large language model; For connection operations, and Concatenate them into a joint feature vector; This is a global average pooling operation used to aggregate sequence features into a fixed-length vector. It is a multilayer perceptron, consisting of two fully connected layers, used to map aggregated features into high-level resource feature representations.

[0049] And combine the overall computational graph structure matrix of the model Jointly constructed, among which, For activation function, The structure mapping matrix, It is the bias vector; The final matching score is calculated using cosine similarity:

[0050] in: The matching score is the sum of the m-th high-level resource feature representation and the n-th segmentation template embedding representation. This represents the high-level feature vector of the m-th layer; Let be the embedding vector of the nth segmentation template, which is a standardized feature representation pre-stored in the template library; L2 norm (Euclidean norm) of a vector is used to normalize the vector so that the matching score is not affected by the vector's magnitude. The closer the matching score is to 1, the higher the matching degree.

[0051] In this embodiment, it should be noted that the graph attention network and matching algorithm used in this invention are essentially a physical measurement and objective mapping of the physical hardware resources (computational load, memory usage, and data transmission volume) of the mobile phone. The specific number of network layers and neuron activation functions can be conventionally replaced according to the actual computing power, and are not abstract mathematical derivations.

[0052] Step S3: Based on a multi-agent collaborative architecture, the discovery agent, evaluation agent, and scheduling agent are invoked to generate an initial computing power sharing scheme. Specifically, the discovery agent dynamically scans surrounding available mobile phone nodes, collecting hardware computing power information, current load status, battery level, and network signal strength for each available mobile phone node. The evaluation agent uses a multi-attribute decision-making algorithm to comprehensively score the capabilities of each available mobile phone node and generates a priority queue of candidate nodes based on the resource demand vector. The scheduling agent, according to the priority queue of candidate nodes and the data dependencies between subtasks in the subtask set, uses a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node, and ensures load balancing through backtracking correction.

[0053] In this collaborative architecture, the discovery agent can dynamically scan via short-range wireless communication protocols on the mobile device side (e.g., but not limited to Wi-Fi Direct, Bluetooth 5.0, or 5G D2D). To ensure the stability and reliability of the selected nodes, the evaluation agent uses a multi-attribute decision algorithm to comprehensively score the capabilities of each available mobile device node. The specific calculation formula is as follows:

[0054] in: The overall capability score for the j-th available mobile node; A benchmark score for hardware computing power information; This represents the current resource utilization rate. Normalized battery capacity; This refers to the network signal quality index. to The weights are dynamic and satisfy the following conditions: .

[0055] In actual operation, the dynamic weights are adjusted in real time based on the real-time requirements and power consumption constraints of the large language model inference task: when the maximum end-to-end latency constraint is less than a preset latency threshold, the weights are increased. and The value of is adjusted, and the remaining weights are reduced accordingly to maintain a total weight of 1; when the maximum allowable power consumption is lower than a preset proportion of the initiating mobile phone's battery capacity, the value is increased. The value of is determined, and the remaining weights are reduced accordingly to maintain the constraint that the sum is 1; or, the values ​​of each dynamic weight are controlled to tend to be evenly distributed.

[0056] When scheduling agents to perform bipartite graph matching mappings, as an optional scheme, subtasks can be used. At the node The estimated completion time is used as the edge weight. Taking full account of the activation value transmission volume, the calculation formula is as follows:

[0057] in: The computational cost of subtasks; As a benchmark for computing power; For the amount of data transmitted; This represents the measured communication bandwidth.

[0058] The Kuhn-Munkres algorithm is then used to find the optimal mapping scheme that minimizes the total completion delay. The objective function is: .

[0059] Furthermore, the objective function must satisfy the following constraints: ; Among them, decision variables Subtasks Should it be assigned to a node? , Number of subtasks This represents the number of nodes.

[0060] This constraint strictly ensures that each subtask is assigned, and each target mobile node undertakes at most one subtask to prevent physical overload of a single node. As a further optimization, after obtaining the optimal mapping scheme through the Kuhn-Munkres algorithm, the scheduling agent will perform a backtracking correction operation. Specifically, the system will evaluate the variance of the allocation computation of each node. If the predicted load rate of a node exceeds the system average level (such as a preset threshold of 20%), a backtracking mechanism is triggered: the last assigned subtask in the node's allocation list is withdrawn, and the node is re-added to the matching queue by updating the edge weight constraints. This process is iterated until the overall load balancing variance is less than the safety threshold.

[0061] Step S4: Perform edge deployment simulation on the initial computing power sharing scheme to obtain the computing latency simulation report, KV cache memory usage simulation report, power consumption simulation report and communication transmission simulation report for each target mobile phone node.

[0062] Step S5: Using a graph neural network-based performance predictor, the simulation closed-loop agent automatically identifies performance bottleneck information in the computation latency simulation report, the KV cache memory usage simulation report, the power consumption simulation report, and the communication transmission simulation report. Based on the type of performance bottleneck information, the optimization agent is invoked to perform targeted adjustment operations. The types of performance bottleneck information include computation bottleneck, storage bottleneck, and communication bottleneck.

[0063] Specifically, the step of invoking the optimization agent to perform targeted adjustment operations based on the type of the performance bottleneck information includes: If the type of the performance bottleneck information is a computational bottleneck, then adjust the subtask splitting granularity, improve the quantization accuracy of the computation process, or skip a preset proportion of secondary attention points; If the performance bottleneck information is a storage bottleneck, a KV cache offloading strategy is enabled to compress and migrate some historical tokens to the initiating mobile phone for storage, or context window pruning is enabled. This offloading target is not fixedly discarded, but rather triggered in conjunction with dynamic memory availability to prevent memory overflow. This offloading strategy differs from existing KVCache Eviction schemes in that the offloading target node in this invention is dynamically selected, not subjected to destructive fixed discarding, and its offloading timing is jointly triggered by real-time load and memory availability, maximizing the preservation of context integrity. It should be noted that when context window pruning is enabled, the system does not randomly truncate, but prioritizes retaining the system prompt and the core context of the most recent rounds of dialogue, and removes redundant historical tokens with an attention score below a preset lower limit, thereby ensuring the logical coherence of the generated text while releasing memory.

[0064] If the performance bottleneck information is a communication bottleneck, then adjust the communication topology between the target mobile phone nodes to a hierarchical aggregation structure, or increase the transmission batch to reduce the number of communication handshakes between the target mobile phone nodes.

[0065] Step S6: Perform the edge deployment simulation again on the adjusted computing power sharing scheme and iterate until there are no performance violations in all simulation reports and the generation quality meets the task requirements. Output the deployable computing power sharing configuration file and the communication topology file between each target mobile phone node.

[0066] Specifically, all simulation reports must meet the following conditions simultaneously: No performance violations and the generated quality must satisfy the task requirements. (1) The simulated computational latency values ​​of all target mobile phone nodes are less than the latency limit of their respective assigned subtasks; (2) The simulated KV cache memory usage of each target mobile node is less than the corresponding preset memory allocation limit safety ratio; preferably, the KV cache limit allocation formula for target mobile node j is:

[0067] in: Allocate an upper limit to the KV cache of the target mobile node j; The total remaining physical memory of the target mobile node j; The total key-value cache requirement estimated for large language model inference tasks; The total number of target mobile phone nodes participating in collaborative reasoning; The constant 0.85 is a safety memory factor, used to ensure that nodes reserve 15% of physical memory space to prevent the mobile phone system from lagging. (3) The simulated total communication delay between each target mobile phone node is less than the preset communication delay threshold.

[0068] To prevent the algorithm from getting stuck in a deadlock, this embodiment sets a safety bottom line mechanism for terminating the loop iteration: if the number of loop iterations exceeds the preset maximum number of iterations (e.g., 20 times) and the above three conditions are still not fully met, the system will forcibly stop the simulation, output the optimal feasible solution in the current iteration, and send a performance degradation warning signal to the initiator through the application interface.

[0069] In this embodiment, it should be noted that after the above steps are completed to output the computing power sharing configuration file and officially start the edge-side inference, the initiating mobile phone will distribute the model weights of each stage and the input prompt words to each target mobile phone node.

[0070] Subsequently, each target mobile node loads the model weights according to its configuration, starts inference in a pipelined or tensor parallel order, and transmits the activation values ​​layer by layer and synchronizes its own stored KV cache fragments. After completing the calculation of its assigned layer, each node finally returns logits to the initiating mobile device to generate the first token.

[0071] During the autoregressive decoding phase, the KV cache increment of the target mobile node j when generating the t-th token is:

[0072] The total cache usage accumulated from step 1 to step t is:

[0073] in: This represents the amount of KV cache space added by the target mobile node j when generating the t-th token; The number of Transformer layers assigned to the target mobile node j; Feature dimensions for each attention head; This represents the total number of attention heads in the large language model. constant term This means that each token needs to cache both the key and value matrices simultaneously. This represents the total KV cache usage accumulated by the target mobile node j up to step t.

[0074] The caching agent monitors the cache occupancy rate of each node in real time based on this cumulative formula, and triggers a management strategy when it approaches the upper limit. At this time, the initiating mobile phone receives and aggregates the generation tokens returned by each node through the result fusion agent.

[0075] To ensure the continuity of long text generation tasks in a mobile environment, the system also introduces an abnormal dynamic rescheduling mechanism during the formal operation phase.

[0076] Specifically, if a target mobile node is detected to have a sudden drop in computing power or a network outage due to user operations (such as answering a phone call or other preemptive actions), triggering a performance violation, the system will immediately package the subtasks currently handled by the faulty node and its corresponding KV cache from the nearest checkpoint and migrate them to the most idle mobile node (such as the initiator that has already completed its own stage of computation or other idle nodes) via a short-distance network, thereby achieving seamless recovery of the inference task.

[0077] During task migration, the KV cache allocation of the newly taken-over idle node must undergo strict security verification, specifically satisfying the formula:

[0078] in: The total KV cache size that needs to be handled after the newly taken-over idle nodes have completed their migration; This represents the total amount of KV cache that the faulty node has accumulated at step t when the anomaly occurred; This represents the amount of key-value cache currently stored in the newly taken-over idle node. The total amount of remaining physical memory in the newly acquired free node; A constant of 0.85 is used as a safe memory threshold to ensure that the takeover node retains sufficient physical memory after taking on new tasks, preventing the process from being forcibly terminated by the mobile phone system.

[0079] Furthermore, after all text is generated, the system records the first token latency, average latency between tokens, energy consumption of each node, and BLEU score of the generated text for this distributed inference, storing these records in a local historical performance database. Simultaneously, the system updates each node's long-term reputation score based on its performance in this task (adding points for timely completion and deducting points for slowdowns or disconnections), and uses this experience to adjust the initial weight preferences for subsequent multi-attribute decisions, thereby preventing the recurrence of similar anomalies and forming a complete closed loop for shared computing power evolution.

[0080] The method provided in this embodiment automatically completes the adaptive segmentation, scheduling, and key-value cache pre-allocation of large language models through a multi-agent collaborative mechanism, replacing inefficient manual strategies. The introduction of graph attention networks and a closed-loop simulation rescheduling mechanism not only ensures that the segmentation scheme perfectly matches the characteristics of large models and minimizes communication overhead, but also specifically addresses the core technical pain points of cross-node management of key-value caches, significantly expanding the application boundaries of large AI models in mobile environments without network connectivity.

[0081] Example 2 Embodiment 2 of this invention provides a specific application scenario and test embodiment of an AI computing power sharing method based on a mobile device. To further illustrate the actual technical effects of this invention, this embodiment uses the Qwen-7B large language model under collaborative inference across multiple mobile devices as an example for detailed explanation. This process specifically includes the following steps: Step 1: Acquisition and structured parsing of large language model inference tasks.

[0082] The initiating mobile phone receives user input for a large language model inference task. For example, the user opens the AI ​​assistant application on the initiating mobile phone, inputs the prompt "Please write a short introduction about the development history of artificial intelligence in Chinese, about 300 words", and sets the inference parameters: temperature 0.7, kernel sampling parameter Top-p 0.9, maximum generation length of 512 tokens, inference accuracy requirement of INT8, and maximum end-to-end latency constraint of no more than 8 seconds. The task parsing agent first uses the local lightweight semantic understanding module to extract the above task parameters and transforms them into a set of structured constraints. Then, based on the large language model metadata cached locally on the initiating mobile phone (in this example, the Qwen-7B model, 32 Transformer layers, 4096 hidden layer dimensions, and 32 attention heads), the total computational cost is estimated. The theoretical computational complexity of the pre-filling stage (processing the input prompt, about 50 tokens) is approximately:

[0083] in, Given the length of the input sequence, For the number of Transformer layers, Substituting the dimension of the hidden layer, the pre-padding computation is approximately .

[0084] The token-by-token computational complexity of the decoding phase (generating approximately 300 tokens) is approximately:

[0085] in, To generate the number of tokens, substituting the values, the computational complexity of the decoding stage is approximately... The total number of FLOPs exceeds 10 billion.

[0086] Simultaneously, the memory requirements for the key-value (KV) cache are estimated. The formula for the size of each KV cache level is:

[0087] in, Total length of the context For each dimension of attention head, For the number of attention heads.

[0088] The total KV cache memory requirement is:

[0089]

[0090] Subsequently, the resource requirement weights for the pre-filling and decoding stages are calculated using normalization operations, and the resource requirement vector corresponding to the task is constructed accordingly.

[0091] The data dependency matrix A between each Transformer layer is calculated using a multi-head self-attention mechanism:

[0092] in, This is the query matrix, representing the computational features of the current layer; The key matrix consists of the number of parameters and the number of floating-point operations in each layer; The dimension of the key; The bias term represents the mandatory constraints on data dependencies between layers (including the order of residual connections). In the matrix, the dependency values ​​of adjacent layers (e.g., layer i and layer i+1) are close to 1, indicating a mandatory order dependency during forward propagation. Cross-layer residual connections are also injected as bias term B into the attention computation. This dependency matrix ensures that subsequent splits will not disrupt the residual connection paths, avoiding inference errors.

[0093] Step 2: Dynamic device discovery and node evaluation.

[0094] The initiating mobile phone broadcasts a large language model inference task request via short-range wireless communication protocols such as 5G D2D (Device-to-Device). It discovers that the agent has scanned three available mobile phone nodes within a 50-meter range: available mobile phone node A (test mobile phone 1, NPU computing power 45 TOPS, battery 85%, load 10%), available mobile phone node B (test mobile phone 2, NPU computing power 40 TOPS, battery 60%, load 25%), and available mobile phone node C (test mobile phone 3, NPU computing power 26 TOPS, battery 40%, load 50%).

[0095] Based on the hardware status information reported by the nodes, evaluate the overall computing capability score of the agent:

[0096] in, A benchmark score for the NPU computing power of available mobile node j. This represents the current load resource utilization rate. To normalize the battery capacity, This is the network signal quality index. In this embodiment, because large language model inference has extremely high computational requirements, [the following is used]: (Computing power weight) is set to 0.5. (Load weight) is set to 0.2. (Battery weight) is set to 0.15. The signal weight was set to 0.15. The calculated overall capability score for available mobile node A was the highest, while the score for available mobile node C was the lowest due to low battery power and high load.

[0097] Step 3: Adaptive segmentation and optimal allocation of the large language model.

[0098] The scheduling agent divides the 32-layer Transformer of Qwen-7B into four stages (each stage has eight layers) in a pipelined parallel manner. Simultaneously, it further splits the attention heads of some layers (such as layers 16-24) into subtask sets using tensor parallelism. A bipartite graph matching algorithm is used for mapping, with edge weights... Indicates the estimated completion time of subtask i on target mobile node j:

[0099] in, The computational cost of subtasks (FLOPs). The benchmark for computing power (TOPS) The amount of data transferred between subtasks (intermediate activation value). The measured communication bandwidth fully considers the huge amount of intermediate activation value transmission between Transformer layers and significantly includes communication time.

[0100] The Kuhn-Munkres algorithm is used to find the optimal mapping scheme that minimizes the total completion delay. The objective function is: Under the constraints, the optimal mapping scheme is as follows: target mobile node A is assigned layers 1-8 and 25-32 (the first and last stages have large computational loads), target mobile node B is assigned layers 9-16, and target mobile node C is assigned layers 17-24.

[0101] Simultaneously, based on the remaining memory capacity of each target mobile node, the scheduling agent allocates a corresponding KV cache limit for each node:

[0102] in, 0.85 represents the remaining physical memory of target mobile node j, with a safety memory factor of 0.85. In this embodiment, target mobile node A is allocated 600MB, target mobile node B is allocated 450MB, and target mobile node C is allocated 290MB, ensuring that the memory usage of each node does not exceed 85% of the physical memory.

[0103] Before officially distributing the configuration and executing inference in parallel, the system performed edge deployment simulation and closed-loop optimization on the initial scheme. Specifically, the simulation closed-loop agent predicted that if the initial scheme was executed to the maximum generation length (512 tokens), the target mobile node C, with only 300MB of physical memory remaining, would likely exceed the 290MB allocation limit for its KV cache, triggering a storage bottleneck. Based on this predicted bottleneck, the optimization agent intervened early in the simulation phase, adjusting the KV cache offloading strategy of node C (setting it to trigger a compressed migration of some historical cache to the initiating mobile device when it reaches 260MB). After the adjusted scheme was re-simulated and confirmed to have no performance violations, the final computing power sharing configuration file was output.

[0104] Step 4: Distributed KV cache management and parallel inference.

[0105] After allocation, the initiating mobile phone distributes the model weights for each stage and input prompts to each target mobile phone node. After loading the model weights, each target mobile phone node starts inference in pipeline order: Target mobile phone node A completes the forward computation of layers 1-8 and sends the intermediate activation values ​​to target mobile phone node B; Target mobile phone node B computes layers 9-16 and sends them to target mobile phone node C; Target mobile phone node C completes the computation of the remaining layers and returns logits to the initiating mobile phone to generate the first token.

[0106] During the decoding phase, each target mobile node not only transmits the activation values ​​layer by layer, but also needs to synchronize its stored key-value cache fragments. The key-value cache increment of target mobile node j at step t is:

[0107] The cumulative cache usage is:

[0108] The initiating mobile phone's cache agent monitors the KV cache occupancy rate of each node in real time. When the cache of the target mobile phone node C approaches the 290MB limit, a cache unloading operation is triggered. A portion of the cache of historical tokens is compressed and migrated to the initiating mobile phone's local storage, freeing up space for new tokens. The target node for unloading is dynamically selected (in this embodiment, the initiating mobile phone is selected), rather than being fixed and discarded. The unloading timing is jointly triggered by real-time load and memory availability.

[0109] Step 5: Dynamic rescheduling and anomaly recovery.

[0110] When the inference process reached the generation of the 150th token, the target mobile node C experienced a user action (such as answering a phone call), causing its NPU resources to be preempted and its processing speed to plummet. This resulted in the inter-token latency increasing from the expected 180ms to 450ms, severely exceeding the limit. Upon detecting the performance violation, the system's abnormal dynamic rescheduling mechanism immediately triggered a recovery operation: the subtasks at layers 17-24 currently handled by the target mobile node C, along with their corresponding KV caches, were packaged from the checkpoint and migrated via networks such as Wi-FiDirect to the currently idle target mobile node A (which was idle at this point due to having completed its own phase ahead of schedule). The migration process took 85ms.

[0111] After the migration is complete, the target mobile node A will take over the computation of layers 17-24, and its newly added KV cache allocation must meet security verification requirements:

[0112] in, This represents the total amount of KV cache accumulated by the target mobile node C at step 150. This represents the current KV cache size of target mobile node A. A coefficient of 0.85 ensures that target mobile node A retains 15% of its memory after accepting new tasks, preventing the process from being terminated by the system. After successful verification, the inference pipeline continues running, ultimately controlling the total latency to complete the generation of all 300 tokens within 7.2 seconds.

[0113] Step 6: Results return and history update.

[0114] Once all tokens are generated, the initiating phone combines the 300 tokens into the final answer text and returns it to the AI ​​assistant application interface. Simultaneously, the system records the first token latency (1.2 seconds), average latency between tokens (198ms), power consumption of each target phone node (node ​​A consumes 2.1%, node B 1.8%, node C 1.2%), and the BLEU score of the generated text. This data is stored in a local historical performance database, updating the long-term reputation scores of each node (target phone node A gains reputation points for undertaking additional tasks, while target phone node C suffers a slight deduction due to a mid-process slowdown). Based on this experience, the system adjusts the weight preferences for subsequent scheduling, such as reducing the initial score of high-load nodes to prevent similar anomalies from recurring.

[0115] In actual testing, the Qwen-7B model achieved a generation throughput of 5.05 tokens / s with collaborative inference across three mobile phones, compared to only about 2.1 tokens / s with independent inference on a single test phone, representing a speedup of 2.4 times. Simultaneously, during the generation process, simulated node disconnections were resolved with an average rescheduling recovery time of 92ms, virtually imperceptible to the user. Compared to cloud-based inference solutions (with an average end-to-end latency of approximately 3.2 seconds, requiring a network connection and raising privacy concerns), this invention can run smoothly even without a network connection, significantly expanding the mobile application boundaries of large language models.

[0116] Example 3 Based on the same inventive concept as the above embodiments, Embodiment 3 of the present invention provides an AI computing power sharing system based on a mobile device. This system strictly corresponds to the method in Embodiment 1, and aims to complete the distributed inference preparation and closed-loop optimization of a large language model across multiple mobile device nodes through the collaborative use of virtualization modules.

[0117] In this embodiment, specifically, the system includes: The task receiving module is used to receive large language model inference tasks from the initiating mobile phone. These tasks include model type, number of model parameters, input prompt word length, maximum generation length, inference accuracy requirements, temperature parameters, maximum end-to-end latency constraints, and maximum allowable power consumption. This module serves as the entry point for system interaction with users and upper-layer applications.

[0118] The task parsing and mapping module, connected to the task receiving module, extracts the large language model inference task into a set of structured constraints through a task parsing agent. It then uses a computation-communication mapping model to convert the performance metrics of the large language model's autoregressive inference into physical constraints such as required node computation, KV cache memory usage, and communication bandwidth. Simultaneously, it generates a set of subtasks and corresponding resource requirement vectors. Specifically, this module integrates a semantic understanding unit, a resource weight normalization operation unit, and a multi-layer graph attention network feature extraction unit. It is responsible for accurately mapping abstract natural language inference requirements to underlying hardware resource consumption metrics and outputting an initial task segmentation strategy.

[0119] The multi-agent collaborative scheduling module, connected to the task parsing and mapping module, is used to: invoke the discovery agent to dynamically scan surrounding available mobile phone nodes and collect hardware status (including hardware computing power information, current load status, battery level, and network signal strength); invoke the evaluation agent to use a multi-attribute decision algorithm to comprehensively score the capabilities of each available mobile phone node and generate a candidate node priority queue; and invoke the scheduling agent to use a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node based on the candidate node priority queue and the data dependencies between subtasks in the subtask set, thereby generating an initial computing power sharing scheme. This module is the core hub for establishing communication between physical nodes and realizing the overall allocation of resources.

[0120] The edge simulation module, connected to the multi-agent collaborative scheduling module, is used to perform edge deployment simulation of the initial computing power sharing scheme and obtain simulation reports for each target mobile phone node. These simulation reports specifically include a computation latency simulation report, a KV cache memory usage simulation report, a power consumption simulation report, and a communication transmission simulation report. This module performs a sandbox pre-simulation before actual physical memory usage, effectively preventing physical devices from crashing due to memory overflow.

[0121] The closed-loop optimization module is connected to both the edge simulation module and the multi-agent collaborative scheduling module. It is used to identify performance bottleneck information in the simulation report through a performance predictor based on a graph neural network using the simulation closed-loop agent. Based on the type of performance bottleneck information, it calls the optimization agent to perform targeted adjustment operations and cyclically sends the adjusted computing power sharing scheme into the edge deployment simulation until there are no performance violations, and outputs a deployable computing power sharing configuration file and the communication topology file between the target mobile phone nodes.

[0122] As a further preferred embodiment, the directional adjustment operation performed in the closed-loop optimization module specifically includes: (1) To address computational bottlenecks, adjust the segmentation granularity, improve quantization accuracy, or skip secondary attention heads; (2) To address storage bottlenecks, dynamically unload the KV cache to the initiating phone's storage, or perform context window pruning. (3) To address the communication bottleneck, adjust the communication topology between the target mobile phone nodes to a hierarchical aggregation structure or increase the transmission batch size.

[0123] As a further extension of the solution in this embodiment, the system may also include a distributed execution and monitoring module, which, after receiving the computing power sharing configuration file, controls each target mobile node to load the corresponding model weights and start pipeline or tensor parallel inference; generates tokens through result fusion intelligent agents; and triggers an abnormal dynamic rescheduling mechanism when a target mobile node is detected to have computing power preemption or disconnection or other performance violations, to migrate the faulty node's subtasks and KV caches to idle nodes for task recovery.

[0124] It should be specifically noted to those skilled in the art that the virtual functional modules mentioned in this embodiment, such as the "task parsing agent," "discovery agent," "evaluation agent," "scheduling agent," and "simulation closed-loop agent," are all, at their underlying physical implementation, logical processing units composed of physical processing units such as the central processing unit (CPU) and neural network processor (NPU) in the initiating mobile phone (or computer device), which execute corresponding computer program code stored in memory. Essentially, it is not a purely intellectual activity rule or computer program detached from hardware, but rather a technical solution that physically reconstructs and optimizes the internal execution logic of a computer system based on objective physical states such as physical network communication signals, node power, and physical memory occupancy, possessing an objective physical execution process and technical effect.

[0125] Based on the same technical concept, embodiments of the present invention also provide an electronic device that can implement the AI ​​computing power sharing method based on a mobile device provided in the above embodiments of the present invention. In one embodiment, the electronic device can be a server, a terminal device, or other electronic devices. Figure 2 As shown, the electronic device may include: At least one processor and a memory connected to the at least one processor. In this embodiment of the invention, the specific connection medium between the processor and the memory is not limited. Figure 2 The example used is the connection between the processor and memory via a bus. The bus... Figure 2 The connections between other components are indicated by thick lines and are for illustrative purposes only, not as limiting information. Buses can be divided into address buses, data buses, control buses, etc., but for ease of representation, [the specific bus type is not shown here]. Figure 2 The processor is represented by a single thick line, but this does not imply that there is only one bus or one type of bus. Alternatively, a processor can also be called a controller; there are no restrictions on the name.

[0126] In this embodiment of the invention, the memory stores instructions executable by at least one processor. By executing the instructions stored in the memory, the at least one processor can execute the AI ​​computing power sharing method based on the mobile device side described above. The processor can implement... Figure 2The functions of each module in the device shown.

[0127] The processor is the control center of the device. It can connect to various parts of the control device through various interfaces and lines. By running or executing instructions stored in memory and calling data stored in memory, it can monitor the device's various functions and process data, thereby enabling overall monitoring of the device.

[0128] In an alternative design, the processor may include one or more processing units. The processor may integrate an application processor and a modem processor, wherein the application processor primarily handles the operating system, user interface, and applications, while the modem processor primarily handles wireless communication. It is understood that the modem processor may also not be integrated into the processor. In some embodiments, the processor and memory may be implemented on the same chip; in some embodiments, they may also be implemented separately on separate chips.

[0129] The processor can be a general-purpose processor, such as a CPU, digital signal processor, application-specific integrated circuit, field-programmable gate array or other programmable logic device, discrete gate or transistor logic device, or discrete hardware component, capable of implementing or executing the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the AI ​​computing power sharing method based on the mobile terminal disclosed in the embodiments of this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0130] Memory, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules. Memory can include at least one type of storage medium, such as flash memory, hard disk, multimedia cards, card-type memory, random access memory (RAM), static random access memory (SRAM), programmable read-only memory (PROM), read-only memory (ROM), and electrically erasable programmable read-only memory (EPROM). Only memory (EEPROM), magnetic storage, magnetic disks, optical disks, etc. A memory is any other medium capable of carrying or storing desired program code in the form of instructions or data structures, and accessible by a computer, but is not limited thereto. The memory in embodiments of this invention can also be a circuit or any other device capable of performing storage functions for storing program instructions and / or data.

[0131] By designing and programming the processor, the code corresponding to the AI ​​computing power sharing method based on the mobile device described in the foregoing embodiments can be embedded into the chip, enabling the chip to execute the steps of the method described in the foregoing embodiments during runtime. How to design and program the processor is a technique well-known to those skilled in the art and will not be elaborated upon here.

[0132] Based on the same inventive concept, embodiments of the present invention also provide a storage medium storing computer instructions, which, when executed on a computer, cause the computer to perform an AI computing power sharing method based on a mobile device as described above.

[0133] In some alternative embodiments, the present invention also provides a method for sharing AI computing power based on a mobile device, which can also be implemented as a program product including program code. When the program product is run on a device, the program code is used to cause the control device to perform the steps in the method for sharing AI computing power based on a mobile device according to various exemplary embodiments of the present invention as described above.

[0134] It should be noted that although several units or sub-units of the apparatus have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the invention, the features and functions of two or more units described above can be embodied in one unit. Conversely, the features and functions of one unit described above can be further divided and embodied by multiple units. Furthermore, although the operation of the method of the invention is described in a specific order in the drawings, this does not require or imply that these operations must be performed in that specific order, or that all the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0135] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can be implemented in one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs) containing computer-usable program code. The form of a computer program product implemented on ROM, optical memory, etc.

[0136] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a server, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0137] Program code for performing the operations of this invention can be written using any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0138] In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0139] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0140] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1The steps of the function specified in one or more boxes.

[0141] The embodiments described above merely illustrate specific implementation methods of this application, and while the descriptions are detailed and specific, they should not be construed as limiting the scope of protection of this application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the technical solution of this application, and these modifications and improvements all fall within the scope of protection of this application.

[0142] This background section is provided to generally present the context of the invention. The work of the currently named inventors, the work to the extent described in this background section, and aspects of this section that did not constitute prior art at the time of application are neither expressly nor impliedly acknowledged as prior art to the invention.

Claims

1. A method for sharing AI computing power on a mobile device, characterized in that, include: The system receives a large language model inference task from the initiating mobile phone. The large language model inference task includes model type, number of model parameters, length of input prompt words, maximum generation length, inference accuracy requirements, temperature parameters, maximum end-to-end latency constraints, and maximum allowable power consumption. The task parsing agent extracts the large language model inference task into a set of structured constraints, and uses a computation-communication mapping model to convert the performance indicators of the large language model autoregressive inference into the required node computation, KV cache memory usage and communication bandwidth physical constraints. At the same time, it generates a set of subtasks and corresponding resource requirement vectors. The performance indicators include first token latency, inter-token latency and generation throughput. Based on a multi-agent collaborative architecture, an initial computing power sharing scheme is generated by invoking a discovery agent, an evaluation agent, and a scheduling agent. The discovery agent dynamically scans surrounding available mobile phone nodes, collecting hardware computing power information, current load status, battery level, and network signal strength for each available mobile phone node. The evaluation agent uses a multi-attribute decision-making algorithm to comprehensively score the capabilities of each available mobile phone node and generates a priority queue of candidate nodes based on the resource demand vector. The scheduling agent, according to the priority queue of candidate nodes and the data dependencies between subtasks in the subtask set, uses a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node, and ensures load balancing through backtracking correction. The initial computing power sharing scheme is deployed and simulated on the edge to obtain simulation reports of computing latency, KV cache memory usage, power consumption and communication transmission for each target mobile phone node. By simulating a closed-loop intelligent agent and employing a graph neural network-based performance predictor, the system automatically identifies performance bottleneck information in the computation latency simulation report, the KV cache memory usage simulation report, the power consumption simulation report, and the communication transmission simulation report. Based on the type of performance bottleneck information, the system calls an optimization agent to perform targeted adjustment operations. The types of performance bottleneck information include computational bottlenecks, storage bottlenecks, and communication bottlenecks. The adjusted computing power sharing scheme is deployed and simulated again on the edge and iterated until there are no performance violations in all simulation reports and the generation quality meets the task requirements. The deployable computing power sharing configuration file and the communication topology file between the target mobile phone nodes are then output.

2. The AI ​​computing power sharing method based on the mobile terminal side according to claim 1, characterized in that, The computation-communication mapping model is used to convert the performance metrics of autoregressive inference in large language models into physical constraints such as required node computation, key-value cache memory usage, and communication bandwidth. Simultaneously, it generates a set of subtasks and corresponding resource requirement vectors, specifically including: Based on the aforementioned set of structured constraints, the resource requirement weights for each inference stage of the large language model are calculated using normalization operations: in, For the first Resource requirement weights for each inference stage For the first Initial computational complexity score for each inference stage. The total number of reasoning stages. The temperature coefficient is used to control the smoothness of demand distribution; the resource demand weights are used to construct the resource demand vector. The data dependency matrix between the Transformer layers contained in the large language model is calculated using a multi-head self-attention mechanism: in, The data dependency matrix is... To represent the query matrix of features computed by the current Transformer layer, The key matrix consists of the parameters of each layer and the number of floating-point operations. The transpose of the key matrix. For the dimension of the key, This represents the bias term representing the mandatory constraint on data dependency between layers; Based on the data dependency matrix between each Transformer layer, a set of candidate segmentation templates is obtained by feature extraction using a multi-layer graph attention network. An initial task segmentation strategy is selected from the set of candidate segmentation templates based on the overall computational graph structure of the model to divide and generate the set of subtasks.

3. The AI ​​computing power sharing method based on the mobile terminal side according to claim 2, characterized in that, A set of candidate segmentation templates is obtained by feature extraction using a multi-layer graph attention network, specifically including: Through the first layer graph attention network, local computation path features are extracted based on the data dependency matrix between each Transformer layer. The local computation path features include the computational ratio between the self-attention module and the feedforward network module. The second-layer graph attention network extracts global communication pattern features based on the local computation path features. These global communication pattern features are used to characterize the tensor parallel communication overhead during layer combination. Using the third-layer graph attention network, based on the global communication pattern features, historical large language model distributed inference cases, and mobile phone hardware performance database, the overall computation graph of the large language model and the high-level resource feature representations of each layer are extracted. Calculate the matching score between each layer in the advanced resource feature representation and the segmentation scheme in the standard distributed inference template library, extract the segmentation schemes with matching scores greater than a set threshold, and obtain the candidate segmentation template set.

4. The AI ​​computing power sharing method based on the mobile terminal side according to claim 1, characterized in that, The evaluation agent uses a multi-attribute decision-making algorithm to score the comprehensive capabilities of each available mobile phone node. The specific calculation formula is as follows: in, The first of the available mobile phone nodes A comprehensive capability score for each available mobile node. For the first The benchmark score corresponding to the hardware computing power information of each available mobile node. For the first The current load status of each available mobile node represents the resource utilization rate. For the first The normalized value corresponding to the battery level of each available mobile node. For the first The quality index corresponding to the network signal strength of each available mobile node. to The weights are dynamic and satisfy the following conditions: ; The dynamic weights are adjusted in real time based on the real-time requirements and power consumption constraints of large language model inference tasks. When the maximum end-to-end delay constraint is less than the preset delay threshold, increase and The value of is adjusted, and the values ​​of the remaining weights are reduced accordingly to maintain the constraint that the sum of the weights is 1; when the maximum allowable power consumption is lower than a preset proportion of the initiating mobile phone battery capacity, the value of is increased. The value of is determined, and the values ​​of the remaining weights are reduced accordingly to maintain the constraint that the sum of the weights is 1; or, the values ​​of each dynamic weight are controlled to tend to be evenly distributed.

5. The AI ​​computing power sharing method based on the mobile terminal side according to claim 1, characterized in that, Based on the type of performance bottleneck information, the optimization agent is invoked to perform targeted adjustment operations, specifically including: If the type of the performance bottleneck information is a computational bottleneck, then adjust the subtask splitting granularity, improve the quantization accuracy of the computation process, or skip a preset proportion of secondary attention points; If the performance bottleneck information is a storage bottleneck, then enable the KV cache offloading strategy to compress some historical token caches and migrate them to the initiating mobile phone for storage, or enable context window pruning. If the performance bottleneck information is a communication bottleneck, then adjust the communication topology between the target mobile phone nodes to a hierarchical aggregation structure, or increase the transmission batch to reduce the number of communication handshakes between the target mobile phone nodes.

6. The AI ​​computing power sharing method based on the mobile terminal side according to claim 1, characterized in that, All simulation reports contain no performance violations and the generated quality meets the task requirements, specifically meaning that the following conditions are met simultaneously: The simulated computational latency values ​​of all target mobile phone nodes are less than the latency limit of their respective assigned subtasks; The simulated KV cache memory usage of each target mobile node is less than the corresponding preset memory allocation limit safety ratio. The simulated total communication latency between the target mobile phone nodes is less than the preset communication latency threshold.

7. An AI computing power sharing system based on a mobile terminal, characterized in that, The system is used to implement the AI ​​computing power sharing method based on the mobile terminal as described in any one of claims 1-6, the system comprising: The task receiving module is used to receive large language model inference tasks from the initiating mobile phone. The task parsing and mapping module is used to extract the large language model inference task into a set of structured constraints through the task parsing agent, and to use a computation-communication mapping model to convert the performance indicators of the large language model autoregressive inference into the required node computation amount, KV cache memory usage and communication bandwidth physical constraints, while generating a set of subtasks and corresponding resource requirement vectors. The multi-agent collaborative scheduling module is used to call the discovery agent to dynamically scan the surrounding available mobile phone nodes and collect hardware status, call the evaluation agent to perform a comprehensive capability score on each available mobile phone node and generate a candidate node priority queue, and call the scheduling agent to use a bipartite graph matching algorithm to map each subtask in the subtask set to the target mobile phone node based on the candidate node priority queue and the data dependency relationship between each subtask in the subtask set, so as to generate an initial computing power sharing scheme. The edge simulation module is used to perform edge deployment simulation of the initial computing power sharing scheme and obtain simulation reports for each target mobile phone node; The closed-loop optimization module is used to identify performance bottleneck information in the simulation report by using a performance predictor through a simulation closed-loop agent. Based on the type of performance bottleneck information, it calls the optimization agent to perform targeted adjustment operations and cyclically sends the adjusted computing power sharing scheme into the edge deployment simulation until there are no performance violations, and outputs a deployable computing power sharing configuration file and the communication topology file between the target mobile phone nodes.

8. The AI ​​computing power sharing system based on the mobile terminal side according to claim 7, characterized in that, The targeted adjustment operations performed in the closed-loop optimization module include: To address computational bottlenecks, adjust the segmentation granularity, improve quantization accuracy, or skip secondary attention heads; to address storage bottlenecks, perform KV cache unloading or context window pruning; to address communication bottlenecks, adjust the communication topology between the target mobile phone nodes or increase the transmission batch size.

9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to the at least one processor; The memory stores instructions executable by the at least one processor, which executes the instructions stored in the memory to perform the method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium is used to store instructions that, when executed, cause the method as described in any one of claims 1-6 to be implemented.

Images