Task processing method, system and electronic device based on large language model

Abstract

The application discloses a task processing method and system based on a large language model and electronic equipment. The method comprises the following steps: obtaining target task data corresponding to the large language model; determining an input embedding vector corresponding to the target task data; determining a shared key-value vector corresponding to the input embedding vector and broadcasting the shared key-value vector; splitting the input embedding vector to obtain a plurality of sub-task vectors; determining target wireless terminals corresponding to the plurality of sub-task vectors from a plurality of candidate wireless terminals according to the input embedding vector, the shared key-value vector and a resource state vector; sending corresponding sub-task data to the corresponding target wireless terminals; receiving intermediate result vectors corresponding to the plurality of target wireless terminals respectively, and determining a task output result corresponding to the target task data according to the plurality of intermediate result vectors. The application solves the technical problem of large communication transmission overhead and high time delay in the related art when performing task processing.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence and wireless communication integration technology, and more specifically, to a task processing method, system, and electronic device based on a large language model. Background Technology

[0002] In related technologies, large network models driven by large language models (LLMs) have made significant progress in network management, task planning, service provisioning, interactive dialogue, and multimodal content generation. However, mainstream LLM-based large network models are typically deployed in cloud data centers (e.g., GPT-based large network models), relying on ultra-large-scale computing clusters for inference and fine-tuning. With the intelligent development of edge vertical scenarios such as mobile edge computing, connected vehicles, industrial internet, and VR / AR, and the increasing demand for wireless environment intelligence capabilities of large network models from related businesses, this centralized architecture is gradually exhibiting the following common characteristics and challenges in wireless edge scenarios, including high communication transmission overhead and high latency.

[0003] There is currently no effective solution to the above problems. Summary of the Invention

[0004] This invention provides a task processing method, system, and electronic device based on a large language model, to at least solve the technical problems of high communication transmission overhead and high latency in task processing in related technologies.

[0005] According to one aspect of the present invention, a task processing method based on a large language model is provided, comprising: acquiring target task data corresponding to the large language model; determining an input embedding vector corresponding to the target task data; determining a shared key-value vector corresponding to the input embedding vector and broadcasting the shared key-value vector; splitting the input embedding vector to obtain multiple sub-task vectors; determining target wireless terminals corresponding to the multiple sub-task vectors from multiple candidate wireless terminals based on the input embedding vector, the shared key-value vector, and a resource state vector, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals; sending corresponding sub-task data to the corresponding target wireless terminal, wherein the corresponding sub-task data includes a corresponding sub-task vector, so that the corresponding target wireless terminal obtains an intermediate result vector based on the corresponding sub-task vector and the shared key-value vector; receiving the intermediate result vectors corresponding to the multiple target wireless terminals respectively, and determining a task output result corresponding to the target task data based on the multiple intermediate result vectors.

[0006] Optionally, determining the target wireless terminal corresponding to each of the multiple subtask vectors from multiple candidate wireless terminals based on the input embedding vector, the shared key value vector, and the resource state vector includes: determining the target latency corresponding to each candidate wireless terminal based on the input embedding vector, the shared key value vector, and the resource state vector, wherein the target latency includes computation latency and communication latency, the computation latency includes a first latency for the target base station to perform a first operation, a second latency for the corresponding candidate wireless terminal to perform an intermediate result computation operation, and a third latency for the target base station to perform a task output result computation operation, the first operation including at least one of the following: an input embedding vector determination operation, a shared key value vector computation operation, and a subtask scheduling and allocation operation, the communication latency includes a fourth latency for the target base station to send data to the corresponding candidate wireless terminal, and a fifth latency for the corresponding candidate wireless terminal to send data back to the target base station; and determining the target wireless terminal corresponding to each of the multiple subtask vectors from multiple candidate wireless terminals based on the target latency.

[0007] Optionally, determining the target wireless terminal corresponding to each of the multiple sub-task vectors from multiple candidate wireless terminals based on the target latency includes: acquiring transmission data corresponding to the target base station and the multiple candidate wireless terminals, wherein the transmission data includes bandwidth data, transmission power, and channel fading power; determining the transmission rate corresponding to the target base station and the multiple candidate wireless terminals based on the noise power and the transmission data; and determining the target wireless terminal corresponding to each of the multiple sub-task vectors from multiple candidate wireless terminals based on the target latency and the transmission rate.

[0008] Optionally, receiving intermediate result vectors corresponding to the plurality of target wireless terminals respectively, and determining the task output result corresponding to the target task data based on the plurality of intermediate result vectors, includes: determining the task timeout time corresponding to the plurality of target wireless terminals respectively based on the link status between the target base station and the plurality of target wireless terminals respectively; identifying the timeout wireless terminals that have not received the corresponding intermediate result vectors after the corresponding task timeout time; performing corresponding policy processing according to the target result synchronization policy, and obtaining the task output result corresponding to the target task data based on the plurality of intermediate result vectors obtained after policy processing.

[0009] Optionally, when the target result synchronization strategy includes a global complete aggregation mode and a missing result tolerance compensation mode, corresponding strategy processing is performed according to the target result synchronization strategy, and a task output result corresponding to the target task data is obtained based on multiple intermediate result vectors obtained after strategy processing. This includes: in the global complete aggregation mode, sending retry task data to the timeout wireless terminal to wait for receiving the intermediate result vectors fed back by the timeout wireless terminal, and then obtaining the task output result based on multiple intermediate result vectors; and / or sending replacement task data to the backup wireless terminal to wait for receiving the intermediate result vectors fed back by the backup wireless terminal, and then obtaining the task output result based on multiple intermediate result vectors; in the missing result tolerance compensation mode, adjusting the weights of the received intermediate result vectors to obtain an updated result vector, and then determining the task output result based on the updated result vector.

[0010] Optionally, determining the task output result corresponding to the target task data based on multiple intermediate result vectors includes: performing weighted fusion on the intermediate result vectors to obtain a hybrid expert output vector; performing a nonlinear transformation on the hybrid expert output vector to obtain a target task vector; and obtaining the task output result based on the target task vector.

[0011] According to one aspect of the present invention, a task processing method based on a large language model is provided, comprising: receiving target subtask data, wherein the target subtask data includes a target subtask vector, the target subtask vector being one of a plurality of subtask vectors, the plurality of subtask vectors being obtained by splitting an input embedding vector, the input embedding vector being determined based on target task data corresponding to a large language model; invoking a shared key-value vector, wherein the shared key-value vector is a vector corresponding to the input embedding vector, obtained by a target wireless terminal receiving and caching a shared key-value vector broadcast by a target base station; determining a target intermediate result vector based on the shared key-value vector and the target subtask vector; and sending the target intermediate result vector to a target base station, such that the target base station determines a task output result corresponding to the target task data based on the plurality of intermediate result vectors, wherein the plurality of intermediate result vectors includes the target intermediate result vector.

[0012] Optionally, determining the intermediate result vector based on the shared key-value vector and the target subtask vector includes: if the target subtask data also includes a target query projection matrix, performing local query vector calculation based on the target query projection matrix and the target subtask vector to obtain a target local query vector; and performing multi-head attention calculation based on the target local query vector and the shared key-value vector to obtain a target intermediate result vector.

[0013] According to one aspect of the present invention, a task processing system based on a large language model is provided, comprising: a target base station and a plurality of target wireless terminals, wherein the target base station is configured to: acquire target task data corresponding to the large language model; determine an input embedding vector corresponding to the target task data; determine a shared key-value vector corresponding to the input embedding vector and broadcast the shared key-value vector; split the input embedding vector to obtain a plurality of subtask vectors; determine, from a plurality of candidate wireless terminals, target wireless terminals corresponding to the plurality of subtask vectors respectively, based on the input embedding vector, the shared key-value vector, and a resource state vector, wherein the resource state vector represents the resource state corresponding to the plurality of candidate wireless terminals; and send the corresponding subtask data to the corresponding target wireless terminals. The terminal, wherein the corresponding subtask data includes a corresponding subtask vector; receives intermediate result vectors corresponding to the plurality of target wireless terminals respectively, and determines, based on the plurality of intermediate result vectors; any one of the plurality of target wireless terminals is used to receive target subtask data, wherein the target subtask data includes a target subtask vector, and the target subtask vector is one of the plurality of subtask vectors; invokes a shared key-value vector, wherein the shared key-value vector is obtained by the target wireless terminal receiving a shared key-value vector broadcast by the target base station and caching it; determines a target intermediate result vector based on the shared key-value vector and the target subtask vector; and sends the target intermediate result vector to the target base station, wherein the plurality of intermediate result vectors include the target intermediate result vector.

[0014] According to one aspect of the present invention, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the task processing method based on a large language model as described above.

[0015] According to one aspect of the present invention, a computer-readable storage medium is provided, wherein when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the task processing method based on a large language model as described above.

[0016] In this embodiment of the invention, target task data corresponding to a large language model is obtained; an input embedding vector corresponding to the target task data is determined; a shared key-value vector corresponding to the input embedding vector is determined and broadcast; the input embedding vector is split to obtain multiple subtask vectors; based on the input embedding vector, the shared key-value vector, and the resource state vector, target wireless terminals corresponding to the multiple subtask vectors are determined from multiple candidate wireless terminals, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals; corresponding subtask data is sent to the corresponding target wireless terminal, wherein the corresponding subtask data includes the corresponding subtask vector, so that the corresponding target wireless terminal obtains an intermediate result vector based on the corresponding subtask vector and the shared key-value vector; the intermediate result vectors corresponding to the multiple target wireless terminals are received, and the task output result corresponding to the target task data is determined based on the multiple intermediate result vectors. By employing a combination of base station and multiple wireless terminals collaborative distributed processing, shared key-value vector broadcast multiplexing, and precise unicast of subtask data, the target task is split into multiple subtasks and assigned to suitable target wireless terminals for parallel execution. The shared key-value vector is reused to reduce redundant transmission overhead, and the target wireless terminals and communication resources are dynamically scheduled in conjunction with resource state vectors. This achieves the goals of reducing communication transmission overhead, shortening task processing latency, and improving wireless resource utilization. Thus, it realizes the technical effect of efficient and low-latency distributed task processing in wireless edge scenarios, thereby solving the technical problems of high communication transmission overhead and high latency in related technologies when processing tasks. Attached Figure Description

[0017] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0018] Figure 1 This is a flowchart of a task processing method based on a large language model according to an embodiment of the present invention;

[0019] Figure 2 This is a flowchart of a second task processing method based on a large language model according to Embodiment 1 of the present invention;

[0020] Figure 3 This is a schematic diagram of the MoE architecture in related technologies;

[0021] Figure 4 This is a schematic diagram of a distributed inference system for a large-scale wireless network with collaborative communication computing provided by an optional embodiment of the present invention;

[0022] Figure 5This is a functional module mechanism diagram of a large-scale distributed inference system for communication and computing collaboration in wireless networks, provided by an optional embodiment of the present invention. Detailed Implementation

[0023] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0024] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0025] Example 1

[0026] According to an embodiment of the present invention, an embodiment of a task processing method based on a large language model is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0027] Figure 1 This is a flowchart of a task processing method based on a large language model according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0028] Step S102: Obtain the target task data corresponding to the large language model;

[0029] Large language models refer to language models trained using deep learning techniques with a large number of parameters, typically employing architectures such as Transformer, with parameter scales reaching billions to hundreds of billions. Optionally, the large language model in this application can refer to a Transformer architecture model employing a hybrid expert (MoE) structure and a multi-head attention (MHA) mechanism, where the inference task is executed collaboratively and in a distributed manner by a base station and multiple wireless terminals. The base station is responsible for task scheduling, key-value vector pre-computation, and result aggregation, while the wireless terminals deploy MHA expert modules to perform attention computation in parallel.

[0030] The target task data refers to the original task-related data to be processed. It is the initial processing object in the task processing flow and can provide basic data support for subsequent task processing, such as text reasoning data and image recognition data. The target task data corresponding to the large language model is the input data for the distributed reasoning request initiated against the large language model.

[0031] In this step, the target task data to be processed is obtained as the starting input for the entire task processing flow.

[0032] This step completes the raw data collection for task processing, providing a data foundation for subsequent operations such as vector generation, task splitting, and terminal scheduling, and ensuring the orderly start of the task processing process.

[0033] Step S104: Determine the input embedding vector corresponding to the target task data;

[0034] The input embedding vector refers to the feature vector obtained after vector mapping of the target task data. It can transform unstructured raw task data into computable and transmissible structured feature data, adapting to the needs of subsequent model calculation and data processing, such as a fixed-dimensional feature vector after text data is transformed.

[0035] In this step, based on the acquired target task data, the corresponding input embedding vector is generated through feature mapping processing.

[0036] This step transforms the original task data into structured features, converting discrete, non-standardized task data into vector data in a unified format, reducing the complexity of subsequent data processing and transmission, and improving processing efficiency.

[0037] Step S106: Determine the shared key-value vector corresponding to the input embedding vector, and broadcast the shared key-value vector;

[0038] Among them, the shared key-value vector refers to the feature vector calculated based on the input embedding vector that can be reused by multiple wireless terminals. It can be sent once and reused by multiple terminals through broadcasting, reducing repeated data transmission and communication overhead. For example, the shared key vector and value vector fusion feature in the attention mechanism.

[0039] In this step, a shared key-value vector is calculated based on the input embedding vector and then broadcast to the wireless terminal.

[0040] This step utilizes a broadcast method to achieve unified distribution of shared key-value vectors, avoiding the repeated transmission of the same data to various wireless terminals, reducing communication transmission volume from the source, and effectively reducing communication overhead.

[0041] Step S108: Split the input embedding vector to obtain multiple subtask vectors;

[0042] Among them, the subtask vector refers to the feature vector corresponding to the subdivided task obtained after splitting the input embedding vector. It can split the overall task into multiple lightweight subtasks, which is suitable for multi-terminal parallel processing scenarios, such as splitting long sequence feature vectors into multiple short sequence sub-feature vectors.

[0043] In this step, the input embedding vector is split to generate multiple subtask vectors corresponding to the sub-tasks.

[0044] This step breaks down the overall task into multiple lightweight subtasks, providing a foundation for subsequent parallel processing by multiple wireless terminals and reducing the overall task processing latency through parallel execution.

[0045] Step S110: Based on the input embedding vector, the shared key value vector, and the resource state vector, determine the target wireless terminal corresponding to each of the multiple subtask vectors from multiple candidate wireless terminals, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals.

[0046] The resource state vector refers to the vector representing the state of the resources required for each candidate wireless terminal to complete the task assigned on that terminal. It can include local terminal and computing-related resource information, such as the wireless terminal's computing power, bandwidth, and link usage. It can also include the resource status of the wireless communication link between the terminal and the base station, such as channel gain, available bandwidth, and signal interference level, providing a quantitative basis for terminal selection.

[0047] Among them, candidate wireless terminals refer to wireless terminals that have the ability to execute subtasks and can be selected. They are the alternative execution subjects for distributed task processing, such as wireless terminals with computing capabilities, such as mobile phones and edge devices.

[0048] Among them, the target wireless terminal refers to the wireless terminal selected after screening to execute the corresponding sub-task. It is the actual execution subject of the sub-task and can match the sub-task processing requirements.

[0049] In this step, the target wireless terminal corresponding to each subtask vector is selected from the candidate wireless terminals based on the input embedding vector, the shared key value vector, and the resource state vector.

[0050] This step combines task characteristics with terminal resource status to achieve precise matching, ensuring that subtasks are assigned to suitable wireless terminals, avoiding processing delays or failures due to terminal resource mismatch, and improving the stability and efficiency of task execution.

[0051] Step S112: Send the corresponding subtask data to the corresponding target wireless terminal. The corresponding subtask data includes the corresponding subtask vector, so that the corresponding target wireless terminal can obtain the intermediate result vector based on the corresponding subtask vector and the shared key value vector.

[0052] Subtask data refers to task-related data containing subtask vectors, which are used to send data to the target wireless terminal. This data provides core data support for the target wireless terminal to execute subtasks, such as a set of subtask vectors and associated execution parameters.

[0053] The intermediate result vector refers to the phased result vector calculated by the target wireless terminal based on the subtask vector and the shared key value vector. It is the output result of distributed subtask processing and provides intermediate data for the generation of the final task result.

[0054] In this step, the subtask data containing the subtask vector is sent to the corresponding target wireless terminal, so that the target wireless terminal can combine the shared key-value vector to calculate and generate the intermediate result vector.

[0055] This step involves sending dedicated subtask data only to the target wireless terminal. Combined with the broadcast multiplexing mechanism of shared key-value vectors, this further reduces the amount of unicast data while enabling distributed parallel computing of subtasks.

[0056] Step S114: Receive intermediate result vectors corresponding to multiple target wireless terminals respectively, and determine the task output result corresponding to the target task data based on the multiple intermediate result vectors.

[0057] The task output result refers to the final task processing result obtained after integrating and processing multiple intermediate result vectors. It is the final output of the entire task processing process and can meet the processing requirements of the original target task, such as text reasoning results, image classification results, etc.

[0058] In this step, the intermediate result vectors fed back by each target wireless terminal are received, and after integration and processing, the task output results corresponding to the target task data are generated.

[0059] This step aggregates the intermediate results of parallel processing across multiple terminals, completing the final integrated output of the distributed task. While ensuring processing accuracy, it leverages the advantages of parallel processing to further reduce the overall task processing latency, solving the problems of high communication overhead and high latency in traditional methods.

[0060] Through the above steps S102-S114, the target task data corresponding to the large language model is obtained; the input embedding vector corresponding to the target task data is determined; the shared key-value vector corresponding to the input embedding vector is determined and broadcast; the input embedding vector is split to obtain multiple sub-task vectors; based on the input embedding vector, the shared key-value vector, and the resource state vector, the target wireless terminals corresponding to the multiple sub-task vectors are determined from multiple candidate wireless terminals, where the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals; the corresponding sub-task data is sent to the corresponding target wireless terminal, where the corresponding sub-task data includes the corresponding sub-task vector, so that the corresponding target wireless terminal obtains an intermediate result vector based on the corresponding sub-task vector and the shared key-value vector; the intermediate result vectors corresponding to the multiple target wireless terminals are received, and the task output result corresponding to the target task data is determined based on the multiple intermediate result vectors. By employing a combination of base station and multiple wireless terminals collaborative distributed processing, shared key-value vector broadcast multiplexing, and precise unicast of subtask data, the target task is split into multiple subtasks and assigned to suitable target wireless terminals for parallel execution. The shared key-value vector is reused to reduce redundant transmission overhead, and the target wireless terminals and communication resources are dynamically scheduled in conjunction with resource state vectors. This achieves the goals of reducing communication transmission overhead, shortening task processing latency, and improving wireless resource utilization. Thus, it realizes the technical effect of efficient and low-latency distributed task processing in wireless edge scenarios, thereby solving the technical problems of high communication transmission overhead and high latency in related technologies when processing tasks.

[0061] As an optional embodiment, determining the target wireless terminal corresponding to each of the multiple subtask vectors from multiple candidate wireless terminals based on the input embedding vector, shared key value vector, and resource state vector includes: determining the target latency corresponding to each candidate wireless terminal based on the input embedding vector, shared key value vector, and resource state vector, wherein the target latency includes computation latency and communication latency, the computation latency includes a first latency for the target base station to perform a first operation, a second latency for the corresponding candidate wireless terminal to perform an intermediate result computation operation, and a third latency for the target base station to perform a task output result computation operation, the first operation includes at least one of the following: an input embedding vector determination operation, a shared key value vector computation operation, and a subtask scheduling and allocation operation, the communication latency includes a fourth latency for the target base station to send data to the corresponding candidate wireless terminal, and a fifth latency for the corresponding candidate wireless terminal to send data back to the target base station; and determining the target wireless terminal corresponding to each of the multiple subtask vectors from multiple candidate wireless terminals based on the target latency.

[0062] Among them, target latency refers to the expected latency or latency constraint required to complete the entire target task processing, which is determined based on the input embedding vector, shared key value vector and resource state vector. It can provide a clear latency standard for the selection of target wireless terminals and ensure that the task processing meets the real-time requirements.

[0063] Among them, computational latency refers to the total latency required to complete all computational operations in the task processing process. It is the core component of the target latency, which can quantify the time spent in the task computation process and provide a basis for optimizing computational efficiency and selecting suitable terminals.

[0064] Among them, communication latency refers to the total latency required to complete all data transmission operations in the process of task processing. It is an important component of target latency, which can quantify the time spent in the data transmission process and provide a basis for optimizing the allocation of communication resources and shortening the transmission time.

[0065] The first operation refers to the core basic operation performed by the target base station in the early stage of task processing. Specifically, it includes at least one of the following operations: input embedding vector determination operation, shared key value vector calculation operation, and subtask scheduling and allocation operation. It is a key operation for the base station to complete task initialization and provide support for the execution of subsequent subtasks.

[0066] The first latency refers to the time required for the target base station to perform the first operation. It can quantify the efficiency of the base station's initial basic operations and provide a basis for optimizing the base station's processing flow and shortening the overall computation latency.

[0067] The second latency refers to the time required for the corresponding candidate wireless terminal to perform intermediate result calculation operations. It varies depending on the resource capabilities of each candidate wireless terminal and can intuitively reflect the calculation efficiency of each candidate wireless terminal, providing a core reference for the selection of target wireless terminals.

[0068] The third latency refers to the time required for the target base station to perform the task output result calculation operation (i.e., the operation of integrating intermediate result vectors to generate task output results). It can quantify the efficiency of the base station's later result integration operation and provide a basis for optimizing the result integration process and shortening the overall calculation latency.

[0069] The fourth latency refers to the transmission time required for the target base station to send subtask data to the corresponding candidate wireless terminal. It is an important component of communication latency, which can quantify the data transmission efficiency from the base station to the terminal and provide a basis for optimizing the allocation of transmission resources.

[0070] The fifth latency refers to the transmission time required for the corresponding candidate wireless terminal to send the intermediate result vector back to the target base station. It is an important component of communication latency, which can quantify the data transmission efficiency from the terminal to the base station and provide a basis for optimizing the allocation of transmission resources.

[0071] In this embodiment, the target latency of the entire task processing is first calculated quantitatively using task-related vectors and terminal resource status. Then, the target latency is used as a constraint to select target wireless terminals that can meet the latency requirements and are compatible with the corresponding sub-task vectors.

[0072] This approach breaks down the target latency into computation latency and communication latency, further refining the specific steps and latency parameters corresponding to each. This makes latency quantification more accurate and targeted, avoiding ambiguity in latency assessment. It clarifies the latency contributions of the base station and the wireless terminal, enabling precise identification of latency bottlenecks. For example, if the first latency is too high, the base station's initial operations are optimized; if the second latency difference is large, terminals with high computational efficiency are prioritized, providing a clear direction for subsequent latency optimization. Using the target latency as a constraint to select target wireless terminals ensures that the selected terminals can complete sub-task processing within the specified latency, guaranteeing the real-time performance of the entire task and avoiding timeouts or failures to meet application requirements due to excessive terminal processing or transmission time. Furthermore, it improves the terminal selection logic, deeply integrating latency factors with resource status and task characteristics, further enhancing the rationality and adaptability of target wireless terminal selection and laying the foundation for efficient execution of subsequent tasks.

[0073] As an optional embodiment, determining the target wireless terminal corresponding to each of the multiple sub-task vectors from multiple candidate wireless terminals based on the target latency includes: acquiring transmission data corresponding to the target base station and the multiple candidate wireless terminals, wherein the transmission data includes bandwidth data, transmission power, and channel fading power; determining the transmission rate corresponding to the target base station and the multiple candidate wireless terminals based on noise power and transmission data; and determining the target wireless terminal corresponding to each of the multiple sub-task vectors from multiple candidate wireless terminals based on the target latency and transmission rate.

[0074] Among them, transmission data refers to the set of various parameters related to transmission efficiency when data is transmitted between the target base station and multiple candidate wireless terminals. It can provide basic data support for calculating the transmission rate and accurately reflect the transmission capability between the base station and the terminal.

[0075] Among them, bandwidth data refers to the bandwidth parameters of the communication link between the target base station and the candidate wireless terminal, that is, the amount of data that can be transmitted per unit time. It is one of the core factors affecting the transmission rate and can intuitively reflect the transmission capacity of the link, such as the maximum available bandwidth of the link and the current remaining bandwidth.

[0076] Transmission power refers to the signal transmission power when the target base station sends data to the candidate wireless terminal, and the signal transmission power when the candidate wireless terminal sends data back to the target base station. It can affect the stability and transmission rate of signal transmission, and power adaptation can improve transmission efficiency.

[0077] Among them, channel fading power refers to the signal power attenuation caused by channel loss when the signal is transmitted in the communication channel between the base station and the candidate wireless terminal. It can reflect the channel transmission quality and is an important parameter for calculating the transmission rate and evaluating the communication delay.

[0078] Among them, noise power refers to the power value of various interference signals in the communication link between the base station and the candidate wireless terminal. It affects the clarity and accuracy of signal transmission and is an important reference parameter for calculating the transmission rate.

[0079] Among them, the transmission rate refers to the speed at which data is transmitted between the target base station and the candidate wireless terminal, that is, the amount of data transmitted per unit time. It is determined by both the transmitted data and the noise power, and is the core factor affecting communication latency, such as the fourth latency and the fifth latency. It can intuitively reflect the transmission efficiency of the link.

[0080] In this embodiment, the transmission-related parameters between the base station and each candidate wireless terminal are first obtained, and the transmission rate of each terminal is calculated by combining the noise power. Then, the target wireless terminal that has a suitable transmission rate and can meet the latency requirements is selected by combining the target latency constraints.

[0081] This approach deeply integrates terminal selection with actual transmission parameters, moving beyond solely relying on terminal resource status to consider the transmission capabilities between the base station and the terminal. This results in a more comprehensive and accurate selection of target wireless terminals. By calculating transmission rate using transmitted data and noise power, the transmission efficiency between the base station and the terminal can be quantified, leading to a precise assessment of communication latency. This ensures that selected terminals match target latency requirements, preventing excessive latency and overall task timeouts due to insufficient transmission rate. The clear correlation between transmission parameters, noise power, and transmission rate provides a quantitative basis for subsequent optimization of communication resource allocation, such as allocating more bandwidth to links with low transmission rates and adjusting transmission power, further reducing communication latency and improving transmission efficiency. Furthermore, it refines the logical chain of terminal selection, forming a complete process of acquiring transmission parameters, calculating transmission rate, and selecting terminals based on latency. This enhances the scientific rigor and rationality of terminal selection, ensuring the efficiency and real-time performance of subsequent task processing.

[0082] As an optional embodiment, the method involves receiving intermediate result vectors corresponding to multiple target wireless terminals, and determining the task output result corresponding to the target task data based on the multiple intermediate result vectors. This includes: determining the task timeout time corresponding to each of the multiple target wireless terminals based on the link status between the target base station and the multiple target wireless terminals; identifying the timeout wireless terminals that have not received the corresponding intermediate result vectors after the corresponding task timeout time; performing corresponding policy processing based on the target result synchronization policy; and obtaining the task output result corresponding to the target task data based on the multiple intermediate result vectors obtained after policy processing.

[0083] Among them, link status refers to the real-time transmission status of the wireless communication links between the target base station and multiple target wireless terminals. It can comprehensively reflect the transmission quality and stability of the link and provide core basis for determining the task timeout time and optimizing the transmission strategy. It can include status parameters such as link bandwidth, signal strength, signal-to-noise ratio, channel fading, and packet loss rate.

[0084] The task timeout time refers to the longest waiting time set for each target wireless terminal from the start of sending subtask data to the time when the base station receives the intermediate result vector fed back by the terminal. This can prevent the base station from experiencing task lag and increased overall latency due to waiting indefinitely for the intermediate result of the timeout terminal.

[0085] Among them, a timeout wireless terminal refers to a target wireless terminal that fails to send an intermediate result vector back to the target base station within the corresponding set task timeout period. This may be due to link failure, insufficient resources, or other reasons that cause a delay in feedback or an inability to send feedback.

[0086] Among them, the target result synchronization strategy refers to the processing strategy formulated by the base station in the process of receiving intermediate result vectors for situations such as timeout wireless terminals and missing intermediate results. It can flexibly deal with various abnormal situations and ensure that the task output results can be generated smoothly.

[0087] Among them, strategy processing refers to the targeted processing operations performed by the base station on timed-out wireless terminals and missing intermediate result vectors according to the target result synchronization strategy. This can solve the problem of abnormal intermediate result reception and ensure the smooth progress of the task processing flow.

[0088] In this embodiment, based on the link status between the base station and each target wireless terminal, a differentiated task timeout time is set for each terminal, the timeout wireless terminal is identified, and then targeted processing is performed according to the preset target result synchronization strategy. Finally, the task output result is generated based on the processed intermediate result vector.

[0089] This approach combines link status with differentiated task timeout settings. For terminals with good link status, such as stable transmission and high speed, the timeout can be appropriately shortened to improve overall efficiency. For terminals with poor link status, such as unstable transmission and low speed, the timeout can be appropriately extended to avoid misjudging timeouts. This makes the timeout setting more closely reflect actual link conditions and more reasonable. By setting task timeouts and identifying timeout wireless terminals, abnormal intermediate result feedback can be detected in a timely manner, preventing base stations from waiting indefinitely, effectively controlling overall task latency, and preventing the entire task from stalling due to timeouts of individual terminals. The introduction of a target result synchronization strategy allows for targeted handling of timeout situations, avoiding the limitation of having to wait for all terminal feedback before generating the final output, thus improving the flexibility and fault tolerance of task processing. Combining link status, timeout control, and result synchronization strategies effectively addresses issues such as unstable wireless communication links and terminal feedback delays, ensuring the smooth generation of task output results and further improving the reliability and efficiency of task processing.

[0090] As an optional embodiment, when the target result synchronization strategy includes a global complete aggregation mode and a missing result tolerance compensation mode, the corresponding strategy processing is performed according to the target result synchronization strategy, and the task output result corresponding to the target task data is obtained based on the multiple intermediate result vectors obtained after strategy processing. This includes: in the target result synchronization strategy of global complete aggregation mode, sending retry task data to the timed-out wireless terminal, waiting to receive the intermediate result vector fed back by the timed-out wireless terminal, and obtaining the task output result based on the multiple intermediate result vectors; and / or sending replacement task data to the backup wireless terminal, waiting to receive the intermediate result vector fed back by the backup wireless terminal, and obtaining the task output result based on the multiple intermediate result vectors; in the target result synchronization strategy of missing result tolerance compensation mode, adjusting the weights of the received intermediate result vectors to obtain an updated result vector, and determining the task output result based on the updated result vector.

[0091] Among them, the global complete aggregation mode refers to a specific mode of the target result synchronization strategy. Its core requirement is to wait for all target wireless terminals, including timed-out wireless terminals, to return intermediate result vectors before integrating and processing all intermediate result vectors to generate task output results, which can ensure the integrity and accuracy of task output results.

[0092] The missing result tolerance compensation mode refers to another specific mode of the target result synchronization strategy. Its core principle is that when a timeout wireless terminal fails to receive some intermediate result vectors, there is no need to wait or retry. Instead, the weights of the received intermediate result vectors are adjusted to compensate for the impact of the missing results before generating the task output. This improves task processing efficiency and avoids increased overall latency due to timeouts of individual terminals. Optionally, it can be configured to first assess the impact of the missing intermediate result vectors when a timeout wireless terminal fails to receive some intermediate result vectors. If the preset tolerance conditions are met, there is no need to wait or retry; the weights of the received intermediate result vectors are adjusted to compensate for the impact of the missing results before generating the task output. If the preset tolerance conditions are not met, a reacquisition mechanism is triggered, sending retry task data to the timeout wireless terminal or replacement task data to a backup wireless terminal to obtain the missing intermediate result vectors. That is, the missing result tolerance compensation mode is set with preset tolerance conditions, which include at least one of the following: the number of missing intermediate result vectors is lower than a preset number threshold; the gating weight corresponding to the missing intermediate result vector is lower than a preset weight threshold; the sum of the gating weights corresponding to the missing intermediate result vector is lower than a preset cumulative weight threshold. Through the above mechanism, task processing efficiency can be improved while ensuring the quality of task output results, avoiding an increase in overall latency due to timeouts of individual low-weight terminals, and preventing the output quality from being affected by missing results from high-weight terminals.

[0093] Among them, retry task data refers to the sub-task data that the base station resends to the timed-out wireless terminal in the global complete aggregation mode. It is consistent with the initially sent sub-task data and can prompt the timed-out wireless terminal to re-execute the sub-task and return the intermediate result vector, thus ensuring the integrity of the result.

[0094] In this context, "replacement task data" refers to the subtask data sent by the base station to the backup wireless terminal when a timeout wireless terminal exists in the global full aggregation mode. Its content is identical to the original subtask data sent to the timeout wireless terminal, but the recipient is changed from the original timeout wireless terminal to the backup wireless terminal. By sending replacement task data, the subtasks originally executed by the timeout wireless terminal are transferred to the backup wireless terminal for execution. This ensures that in the global full aggregation mode, all subtasks are completed by a corresponding terminal and that intermediate result vectors are fed back, thereby guaranteeing the integrity of the task output results.

[0095] In the global full aggregation mode, a backup wireless terminal refers to a wireless terminal selected by the base station to take over the corresponding sub-task from a timed-out wireless terminal. The backup wireless terminal is one or more of multiple candidate wireless terminals; it can be a pre-designated redundant terminal or a terminal dynamically selected based on real-time resource status and latency assessment. Its function is to act as a replacement node for the timed-out wireless terminal, receiving replacement task data sent by the base station, executing the corresponding sub-task, and feeding back intermediate result vectors to the base station, thereby ensuring the integrity of results and the system's fault tolerance in the global full aggregation mode.

[0096] Among them, updating the result vector refers to the new set of intermediate result vectors obtained by the base station after adjusting the weights of the received intermediate result vectors in the missing result tolerance compensation mode. This new set of intermediate result vectors can make up for the impact of missing results and ensure that the output results of the subsequently generated tasks meet the accuracy requirements.

[0097] In this embodiment, two specific modes of the target result synchronization strategy are defined: the global complete aggregation mode and the missing tolerance compensation mode. The processing methods for timeout wireless terminals under the two modes are explained, as well as the process of finally generating the task output results.

[0098] This approach provides two differentiated result synchronization modes to adapt to different task requirements: For tasks with high accuracy requirements and no tolerance for missing results, such as precision inference tasks, the global complete aggregation mode can be used to ensure complete results and guarantee output accuracy through retry timeout terminals; for tasks with high efficiency requirements and tolerance for minor result missing results, such as ordinary recognition tasks, the missing result tolerance compensation mode can be used to quickly generate results through weight adjustment, improving processing efficiency and achieving a flexible balance between accuracy and efficiency. The specific processing operations for each mode are clearly defined, making strategy execution more targeted and operable, and avoiding ambiguity in strategy processing; the global complete aggregation mode, through retry timeout terminals, can effectively solve the feedback timeout problem caused by temporary link failures and temporary terminal resource shortages, further ensuring the integrity of results; the missing result tolerance compensation mode, through weight adjustment, compensates for missing results without waiting or retrying, significantly shortening task processing latency, while avoiding the impact of individual terminal timeouts on the overall task, improving the flexibility and fault tolerance of task processing, and further adapting to the diverse task requirements of wireless edge scenarios.

[0099] As an optional embodiment, the task output result corresponding to the target task data is determined based on multiple intermediate result vectors, including: weighted fusion of the intermediate result vectors to obtain a hybrid expert output vector; nonlinear transformation of the hybrid expert output vector to obtain the target task vector; and obtaining the task output result based on the target task vector.

[0100] Weighted fusion refers to the operation of weighted summation and integration of intermediate result vectors fed back by multiple target wireless terminals based on preset weight coefficients, such as gating weights. This can highlight intermediate result vectors with high feature contribution, suppress vectors with low contribution, and improve the accuracy of the fused result.

[0101] Among them, the hybrid expert output vector refers to the feature vector obtained by weighted fusion of multiple intermediate result vectors. It is the core output after integrating the intermediate result vectors and can integrate the calculation results of all sub-tasks to provide high-quality feature support for subsequent nonlinear transformations and generation task output results.

[0102] Nonlinear transformation refers to the processing operation of using nonlinear functions (such as Gaussian error linear units) to perform feature mapping and purification on the hybrid expert output vector. It can introduce nonlinear feature expression capabilities, break through the limitations of linear fusion, explore higher-order correlations between features, and improve the representation ability of feature vectors.

[0103] Among them, the target task vector refers to the final feature vector obtained after performing a nonlinear transformation on the hybrid expert output vector. Its features are more compact and more closely match the task objective, and can be directly used to generate task output results. It is the key carrier connecting intermediate results and final output.

[0104] In this embodiment, all intermediate result vectors are integrated by weighted fusion to obtain a hybrid expert output vector, which is then subjected to a nonlinear transformation to obtain the target task vector. Finally, the task output result is generated based on the target task vector.

[0105] This approach employs weighted fusion to integrate intermediate result vectors, assigning different weights based on the feature contribution of each vector, such as the importance of subtasks and the execution reliability of the terminal. This avoids feature redundancy and accuracy degradation caused by simple average fusion, improving the accuracy and relevance of the fusion results. The introduction of nonlinear transformation overcomes the limitations of simple linear fusion, enabling the discovery of complex high-order correlations between intermediate result vectors. It also purifies the features of the mixed expert output vectors, generating target task vectors that better fit the task objectives, thus enhancing the representational power of the feature vectors. By progressively optimizing feature quality, the final generated task output results are ensured to be more accurate and better meet the original task requirements. Combining weighted fusion with nonlinear transformation balances the comprehensiveness of multi-subtask results while increasing the complexity of feature representation, adapting to the processing needs of complex tasks and further enhancing the practicality and reliability of the task processing method based on a large language model.

[0106] Figure 2 This is a flowchart of the second task processing method based on a large language model according to Embodiment 1 of the present invention, as follows: Figure 2 As shown, the method includes the following steps:

[0107] Step S202: Receive target subtask data, wherein the target subtask data includes a target subtask vector, which is one of multiple subtask vectors. The multiple subtask vectors are obtained by splitting the input embedding vector, and the input embedding vector is determined based on the target task data corresponding to the large language model.

[0108] Step S204: Invoke the shared key value vector, where the shared key value vector is the vector corresponding to the input embedding vector, which is obtained by the target wireless terminal receiving the shared key value vector broadcast by the target base station and caching it;

[0109] Step S206: Determine the target intermediate result vector based on the shared key value vector and the target subtask vector;

[0110] Step S208: Send the target intermediate result vector to the target base station so that the target base station can determine the task output result corresponding to the target task data based on the multiple intermediate result vectors, wherein the multiple intermediate result vectors include the target intermediate result vector.

[0111] Through the above steps S202-S208, target subtask data is received, wherein the target subtask data includes a target subtask vector, which is one of multiple subtask vectors obtained by splitting the input embedding vector, and the input embedding vector is determined based on the target task data corresponding to the large language model; a shared key-value vector is invoked, wherein the shared key-value vector is the vector corresponding to the input embedding vector, which is received and cached by the target wireless terminal from the broadcast shared key-value vector; a target intermediate result vector is determined based on the shared key-value vector and the target subtask vector; the target intermediate result vector is sent to the target base station, so that the target base station determines the task output result corresponding to the target task data based on the multiple intermediate result vectors, wherein the multiple intermediate result vectors include the target intermediate result vector. This invention employs a collaborative distributed processing approach between a base station and multiple wireless terminals, combining shared key-value vector broadcast multiplexing with precise unicast of subtask data. On the base station side, the target task is broken down into multiple subtasks and distributed to suitable target wireless terminals for parallel execution. Reusing shared key-value vectors reduces redundant transmission overhead. Combined with dynamic scheduling of target wireless terminals and communication resources using resource state vectors, this approach reduces communication transmission overhead, shortens task processing latency, and improves wireless resource utilization. On the wireless terminal side, it only receives its own corresponding subtask data, eliminating the need to receive all task-related data. Simultaneously, it uses locally cached shared key-value vectors for calculations, avoiding repeated requests for or reception of the vector from the base station. This allows for efficient local calculations of the target subtask vector and shared key-value vector, rapidly generating intermediate result vectors and promptly feeding them back to the target base station. This further reduces communication interaction frequency and data transmission volume, improving subtask execution efficiency. Combined with the base station-side scheduling strategy, this creates a synergistic effect, achieving efficient, low-latency distributed task processing in wireless edge scenarios. This solves the technical problems of high communication transmission overhead and high latency in related technologies when processing tasks.

[0112] As an optional embodiment, determining the intermediate result vector based on the shared key-value vector and the target subtask vector includes: if the target subtask data also includes a target query projection matrix, performing local query vector calculation based on the target query projection matrix and the target subtask vector to obtain the target local query vector; and performing multi-head attention calculation based on the target local query vector and the shared key-value vector to obtain the target intermediate result vector.

[0113] The target query projection matrix is ​​a matrix that corresponds one-to-one with the target subtask vector and is used to map the target subtask vector to the query vector. It is initially stored in the target base station and can be sent to the target wireless terminal along with the target subtask data. It can achieve accurate conversion from target subtask vector to query vector and adapt to the input requirements of subsequent multi-head attention calculation, such as a dedicated query projection matrix for text subtask vectors.

[0114] Among them, local query vector calculation refers to the calculation operation performed by the target wireless terminal on its own local machine to generate a query vector based on the target query projection matrix and the target subtask vector. It can complete data processing locally on the terminal without additional communication interaction with the base station, thus reducing communication overhead.

[0115] The target local query vector refers to the query vector corresponding to the target subtask, which is calculated by the target wireless terminal through the local query vector. It can cooperate with the shared key vector to perform multi-head attention calculation and provide core computational input for the generation of the target intermediate result vector, such as the query feature vector calculated by the image subtask vector and the corresponding query projection matrix.

[0116] Multi-head attention computation refers to the operation of splitting multiple feature subspaces into parallel computations based on the target local query vector and shared key-value vector, and then fusing the computation results. It can fully explore the complex relationship between the target sub-task features and shared features, and improve the feature representation ability of the target intermediate result vector, such as the operation of obtaining feature vectors by weighted fusion after parallel computation of multiple attention heads.

[0117] Among them, the target intermediate result vector refers to the phased result vector corresponding to the target subtask, which is calculated by the target wireless terminal through multi-head attention. It is the core output of the distributed processing of the target subtask and can provide accurate intermediate data support for the target base station to integrate and generate the final task output result. For example, the phased feature vector is calculated from the query vector and shared key value vector of the text subtask.

[0118] In this embodiment, the target local query vector is first calculated by using the target query projection matrix and the target subtask vector. Then, the target local query vector is combined with the cached shared key-value vector and multi-head attention is performed to finally generate the target intermediate result vector.

[0119] This approach clarifies the role and computational logic of the target query projection matrix, achieving precise mapping from the target subtask vector to the target local query vector. It provides a suitable input carrier for multi-head attention computation, ensuring that multi-head attention computation accurately uncovers the correlation between target subtask features and shared features, effectively improving the feature accuracy and representational capability of the target intermediate result vector. All computational operations are completed locally on the target wireless terminal, eliminating the need for additional communication with the base station. This avoids communication latency and transmission overhead caused by off-site computation, aligning with the core requirements of low latency and low overhead in wireless edge scenarios. The generation steps of the intermediate result vector are further refined, making the computation process more standardized and feasible, reducing ambiguity in the computation process, and improving the stability and reliability of the target intermediate result vector generation. By fully exploring feature correlations through multi-head attention computation, the target intermediate result vector can more comprehensively and accurately reflect the core features of the target subtask, laying a solid foundation for the subsequent integration of multiple intermediate result vectors by the target base station to generate high-quality final task output results. This indirectly helps improve the efficiency and accuracy of the entire task processing flow, further alleviating the problems of high communication transmission overhead and high latency in related technologies.

[0120] Based on the above embodiments and optional embodiments, an optional implementation method is provided, which is described in detail below.

[0121] In related technologies, large network models driven by large language models (LLMs) have made significant progress in network management, task planning, service provisioning, interactive dialogue, and multimodal content generation. However, mainstream LLM-based large network models are typically deployed in cloud data centers (e.g., GPT-based large network models), relying on ultra-large-scale computing clusters for inference and fine-tuning. With the intelligent development of edge vertical scenarios such as mobile edge computing, connected vehicles, industrial internet, and VR / AR, and the increasing demand from related businesses for the wireless environment intelligence capabilities of large network models, this centralized architecture is gradually exhibiting the following common characteristics and challenges in wireless edge scenarios:

[0122] (1) The model parameters are huge (billions to hundreds of billions of parameters), the inference and intermediate results data are huge, the wireless transmission overhead is high and the latency is large;

[0123] (2) Terminal devices (such as vehicle terminals, industrial gateways, drones, mobile phones, etc.) have limited computing power and are difficult to run a complete large-scale network model based on LLM locally;

[0124] (3) For privacy-sensitive or time-sensitive businesses (such as vehicle network security, industrial control, etc.), uploading raw data to the cloud may result in privacy leaks and time-delay uncertainties.

[0125] To reduce computational complexity and improve inference efficiency, the industry has introduced a sparse Mixture of Experts (MoE) structure into Transformer-based LLMs to support the development of large-scale network models (Note: The large-scale network model mentioned in this patent specifically refers to the most common Transformer architecture in the industry, and does not involve other architectures). Specifically, a Transformer network model with a MoE structure introduces multiple sub-models called "experts," which are structurally copies of a key module of the Transformer network model. Furthermore, the MoE architecture introduces a gating network (GN) to select and activate only some of the expert sub-models during inference, thereby reducing the computational load and memory usage per inference while maintaining model accuracy. Unlike dense models, MoE expands a layer of the model into multiple expert networks with the same structure, and the gating network determines which are activated for computation, thus enabling the training of ultra-large-scale sparse models. It is important to note that apart from the modules expanded into multiple experts, other modules within the Transformer remain unchanged. Furthermore, traditional solutions are deployed in centralized devices and do not involve collaboration between physical devices, meaning they lack the ability to collaborate between edge devices in a wireless environment.

[0126] Figure 3 This is a schematic diagram of the MoE architecture in related technologies, such as... Figure 3 As shown, most mainstream MoE research is currently based on GPU cluster environments within data centers and commonly uses the Multi-Layer Perceptron (MLP) module structure in Transformer as the expert sub-model, primarily optimizing for parameter redundancy and computational acceleration. This type of MLP-based expert design has the following characteristics:

[0127] (1) The expert team mainly uses fully connected layers, with a large input and output dimension;

[0128] (2) High-dimensional matrix multiplication needs to be performed on the complete eigenvector;

[0129] (3) The parallelism between experts mainly relies on the internal bus of a single machine or a high-bandwidth cluster, without directly considering the communication cost across wireless nodes.

[0130] In wireless edge computing scenarios, large-scale network models need to be executed in a distributed and collaborative manner between base stations and multiple terminals, where wireless link bandwidth is limited and channels are dynamically changing. If a traditional MLP is directly used as the MoE expert and distributed to wireless terminals, problems such as excessive communication load, insufficient parallelism, and difficulty in controlling latency will be encountered.

[0131] Meanwhile, the Multi-Head Attention (MHA) mechanism in the Transformer architecture generates Q / K / V vectors by performing parallel attention calculations on the Query / Key / Value projection matrices of different subspaces. This inherently possesses the structural characteristic of "head-parallel" execution, with the potential for fine-grained decomposition and distributed execution across nodes. If the MoE expert can be reconstructed from the traditional MLP structure into an expert centered on MHA, and combined with wireless communication characteristics for overall design, it is expected to achieve superior distributed inference performance for large network models in both computational and communication dimensions.

[0132] In summary, existing technologies still lack a distributed inference system for large-scale wireless network models that is oriented towards wireless edge environments, uses multi-head attention as the expert unit, and achieves collaborative optimization of communication and computing.

[0133] In view of this, an optional embodiment of the present invention provides a task processing method based on a large language model, specifically involving a wireless network large model distributed inference system and method with communication and computing collaboration, which is applicable to wireless intelligent edge network service scenarios with high requirements for latency, bandwidth and computing power, such as mobile edge computing, vehicle networking, industrial internet, virtual reality (VR) / augmented reality (AR).

[0134] The optional embodiments of the present invention have identified and addressed at least the following problems existing in the prior art, and have resolved them one by one:

[0135] (1) The communication transmission overhead is large and the end-to-end inference latency of the large network model is high;

[0136] Traditional LLM-based large-scale network inference uses centralized or simple "cloud-edge splitting" schemes. In these schemes, the terminal needs to send the input sequence, feature vectors, and even intermediate layer representations to the cloud or base station, and then obtain the inference output from the cloud. In large-scale network models based on Transformer, the length of the input sequence and the embedding dimension are often large, and the data volume of a single layer's intermediate features can reach the MB level. If complete feature tensors are frequently transmitted between multiple layers, it will put enormous pressure on the wireless link bandwidth, easily leading to congestion and queuing delays; end-to-end inference latency will increase significantly, making it difficult to meet the stringent requirements of low-latency services such as vehicle networking and industrial control; and the system scalability will be poor in multi-user concurrent scenarios.

[0137] (2) The MLP expert structure is not compatible with wireless distributed scenarios;

[0138] In existing MoE architectures, most use MLPs as expert modules. These experts have the following characteristics: they perform high-dimensional feedforward transformations on the complete input vector, making it difficult to naturally decompose them into multiple "sub-experts"; in distributed deployments, it is often necessary to send the complete intermediate feature tensor to a certain expert node, resulting in a heavy communication burden on a single node; they cannot utilize the fine-grained distributed capabilities in multi-terminal environments and lack a natural cross-node parallel mechanism.

[0139] In a wireless environment, directly migrating MLP-type experts to the terminal means that each activated expert node needs to receive the entire block of feature data and complete the entire MLP computation. This increases the link communication load and makes it difficult to flexibly adjust the task granularity according to the computing power of different terminals. Therefore, there is a structural mismatch between the MLP expert structure and the distributed inference scenario of large models in wireless networks.

[0140] (3) MoE expert selection and resource scheduling lack communication / computing power awareness;

[0141] Existing gated networks in the MoE architecture primarily select experts based on input semantic features, assuming that each expert operates in a relatively uniform environment. This ignores key factors in the wireless edge environment: differences in channel state (channel gain, available bandwidth, interference level, etc.) and differences in computing resources (CPU / GPU capabilities, current load, queuing time, etc.) among terminal nodes. In this scenario, the gated network may select experts located in weak channels, with low computing power, or with high load, leading to: specific nodes becoming bottlenecks, slowing down overall latency; and some nodes remaining under low load for extended periods, resulting in low system resource utilization and uneven load distribution.

[0142] (4) The computation and communication processes are highly coupled and lack a collaborative optimization mechanism;

[0143] Due to the constraints of traditional MoE architecture, many existing solutions treat the computation process of large network models as the main optimization object, and regard communication as a fixed constraint or a black box. Common problems include: task partitioning is based solely on computational load, without considering transmission costs; expert routing strategies are based solely on semantic features, without considering end-to-end latency; and there is a lack of a unified communication-computation joint optimization framework, making it difficult to achieve optimal overall performance.

[0144] (5) The data transmission structure is not optimized for the wireless environment;

[0145] In traditional LLM-based large-scale network model implementations, the model internally transmits Query / Key / Value projection matrices, Q / K / V vectors, and intermediate features via a bus or high-bandwidth network, making it insensitive to communication overhead. However, migrating to a wireless edge environment presents the following challenges: directly transmitting the complete token embedding sequence or intermediate tensor will result in extremely high wireless data volume; and the lack of a splitting or sharing design for the intermediate representation leads to significant redundant transmissions.

[0146] (6) Lack of end-to-end closed-loop feedback and adaptive mechanism;

[0147] Existing solutions often employ offline strategies and fixed parameters, lacking utilization of information such as real-time communication latency and packet loss rate; actual task completion time and timeout status; and long-term trends in node operating status. Therefore, the system cannot adaptively adjust to network conditions and workload, making it difficult to maintain high efficiency over long-term operation.

[0148] To address the above problems, the main technical issues to be solved by this invention are: (1) how to significantly reduce end-to-end inference latency and wireless transmission overhead while ensuring model accuracy in a wireless edge environment; (2) how to design a MoE expert structure adapted to wireless distributed scenarios, enabling it to be fine-grained, decomposable, parallelizable, and distributed across nodes; (3) how to construct a communication-computation collaborative expert selection and task allocation mechanism, considering channel, computing power, and model structure to achieve system-level optimization; and (4) how to improve the overall network large model system resource utilization and scalability through lightweight transmission structure, phased execution process, and closed-loop feedback mechanism. The following is a detailed introduction:

[0149] Figure 4 This is a schematic diagram of a large-scale distributed inference system for collaborative wireless networks provided in an optional embodiment of the present invention, such as... Figure 4 As shown, the distributed inference system for a large wireless network model based on communication computing collaboration of the present invention consists of a base station (BS), multiple wireless devices (WDs) with computing power, and a wireless communication subsystem. A BS is represented by n, and its computing resources are represented by c_n; all WDs connected to BS n are represented by the set M = {1,2,…,m,…,M}, and the computing resources of WD m are represented by c_m. The system achieves efficient, low-latency distributed inference of a large network model in a wireless scenario by distributing multiple MHA expert modules on different wireless devices, combined with unified K / V vector pre-computation at the base station, a communication computing collaboration gated network decision-making mechanism, and an adaptive scheduling mechanism. The system provided in this application is described below:

[0150] (1) Base station BS node

[0151] As the central node of the system, the base station node possesses relatively sufficient computing power and stable network connectivity, primarily undertaking core functions such as input preprocessing, centralized generation of key-value vectors, expert selection, task scheduling, bandwidth allocation, result aggregation, and policy updating. A brief overview of the specific modules is as follows:

[0152] 1) Input preprocessing module;

[0153] It is responsible for loading input data, performing word segmentation and embedding operations, and generating the required input sequences for subsequent attention layers and expert modules.

[0154] 2) Key-value vector pre-computation and caching module;

[0155] Key / Value vectors are pre-computed using the Key / Value projection matrix and reused throughout the distributed inference process, reducing the burden of repeated computation and transmission at the terminal.

[0156] 3) Communication computing gating network decision module;

[0157] By combining input features (semantics, etc.), real-time communication status of each terminal (channel, bandwidth, etc.) and computing status (available computing power, load, etc.), it determines which MHA experts to activate to participate in reasoning and implements granular allocation of tokens or subtasks.

[0158] 4) Task distribution and synchronization control module;

[0159] Responsible for tasks such as distributing network large model tasks, bandwidth allocation, setting scheduling priorities, and MHA expert execution synchronization strategies.

[0160] 5) Results aggregation and MLP post-processing module;

[0161] The calculation results of the activation experts on each WD will be fused according to the gating weights and the MLP layer nonlinear transformation will be performed.

[0162] 6) Performance monitoring and adaptive strategy update module.

[0163] It is used to record communication and computation latency, and continuously optimize future expert selection and scheduling strategies to achieve closed-loop adaptive reasoning.

[0164] (2) Wireless terminal WD node

[0165] Wireless terminals deploy distributed MHA expert modules, based on BS The assigned task performs the complete attention computation. Because the terminal is closer to the data source (e.g., vehicle-mounted units, industrial equipment, mobile terminals), this method effectively reduces path latency and improves parallelism for large network models when processing tasks. A brief overview of the specific modules is as follows:

[0166] 1) Local resource monitoring module;

[0167] Continuously monitor local CPU / GPU utilization, video memory usage, task queue length, device temperature, and other resource metrics.

[0168] 2) MHA Expert Module;

[0169] Deploy one or more complete MHA experts, use the Query projection matrix to perform local Q-vector computation, and use shared K / V vectors to perform MHA computation.

[0170] 3) Lightweight communication interface module;

[0171] It is used for efficient data transmission and reception, and supports rate adaptation and low-overhead protocol formats.

[0172] 4) Buffer and retransmission control module.

[0173] Ensure reliable uploading of task results in weak network environments, including mechanisms such as timeout retransmission, data packet queue caching, and out-of-order reassembly.

[0174] (3) Wireless communication subsystem:

[0175] Used to provide BS The bidirectional communication links with each WD support essential functions such as wireless channel status feedback, rate adjustment, and dynamic bandwidth allocation, forming a crucial foundation for building collaborative scheduling of communication and computing. It provides a BS (Browser / Server) interface. The uplink and downlink connections with each WD; and the specific implementation of strategies such as bandwidth allocation, rate control, and channel state feedback.

[0176] Figure 5 This is a functional module mechanism diagram of a large-scale distributed inference system for communication and computing collaboration in wireless networks, provided by an optional embodiment of the present invention, as shown below. Figure 5 As shown, its functions are described below:

[0177] (1) Base station side module functions:

[0178] 1) Input preprocessing module:

[0179] Receive input instructions (text or multimodal instructions) from users or upper-layer services.

[0180] Perform token segmentation, encoding, and embedding mapping to obtain the input embedding vector sequence. This input token sequence can be represented as... Its length is Embedding dimension is .

[0181] 2) Key / Value Pre-calculation and Caching Module:

[0182] This module employs the Key / Value projection matrix from the Transformer architecture to perform K / V vector computation on the input embedding vector locally at the base station and caches the results. Its core technologies are as follows:

[0183] All MHA experts share a unified Key / Value projection matrix, which performs a linear transformation on the input embedding at the base station node to generate a K / V vector. The calculation formula is as follows:

[0184]

[0185]

[0186] in, and It is a key / value projection matrix.

[0187] The key-value vector is cached locally at the base station and shared during execution by multiple MHA experts (i.e., it is transmitted to the corresponding expert, and when multiple experts execute in parallel, the key-value vector only needs to be broadcast once). This achieves one-time key-value vector generation and multiple reuses, significantly reducing redundant calculations.

[0188] Each WD Using their respective Query projection matrices and control information, WD can perform local Q-vector pre-computation, greatly reducing the amount of data transmitted downlink.

[0189] 3) Communication computing gated network decision module:

[0190] This module enables MHA expert selection and task partitioning for collaborative communication and computing based on input features, shared K / V vector representation, and real-time feedback from wireless terminals.

[0191] The main functions and embedded innovations of the gating network include:

[0192] 1. Expert Candidate Set Management:

[0193] Responsible for monitoring and collecting the following information from the experts installed on WD:

[0194] Experts have different initializations, different feature preferences, and different parameter sizes;

[0195] Since each expert is deployed on a different terminal, the base station maintains an expert node mapping table.

[0196] 2. Multi-dimensional joint perception of resource status:

[0197] Input the following information into the gated network of the MoE network large model: input semantic features; statistical features of shared K / V vectors (such as token length); wireless terminal status: channel gain, available bandwidth, CPU / GPU utilization, task waiting queue, and real-time computing power (floating-point operations per second, FLOPS).

[0198] 3. Construction of a unified end-to-end latency model:

[0199] For each Transformer layer, the total latency is calculated as: computation latency + transmission latency.

[0200] The computational latency includes: (a) the latency of the preprocessing procedure performed by the BS. , (a) The computational cost incurred by the BS in calculating the K / V vector; (b) The latency of the MHA process executed on each WD. , (c) The computational cost of executing the MHA procedure on the WD; and (d) the latency of executing the MLP procedure on the BS. , This refers to the computational cost of executing the FFN procedure on the BS.

[0201] Communication latency includes: (a) the latency of the BS sending the input token sequence and K / V vector to each WD. , This is the transmission size of the input token sequence. It is the transmission size of the K / V vector. It is BS To WD (b) The delay at which the WD feeds back the output of the MHA processing to the BS. Generally, the output sequence and the input sequence are of the same size. It's WD To BS The transmission rate.

[0202] The transmission rate can be calculated based on monitored communication parameters. For example:

[0203]

[0204] in, It is BS With WD Unlimited bandwidth between It is BS To WD Transmission power, It is the channel fading power. It is Gaussian noise power.

[0205] Ultimately, for this Transformer layer, the total latency is... It can be calculated as follows:

[0206]

[0207] The actual meaning of this formula is: the total inference latency of the Transformer layer = the latency of the BS preprocessing the K / V vector + the latency of the slowest WD returning the processing result to the BS + the latency of the BS executing the MLP process.

[0208] 4. Expert selection strategy:

[0209] The method of selecting experts by considering the latency of handling distributed inference tasks of large network models in a wireless environment completely breaks the shortcomings of traditional centralized methods that do not consider the interaction between distributed physical devices, thus making it fully compatible with wireless distributed services.

[0210] Select multiple experts based on the minimum end-to-end delay;

[0211] Task feasibility can be constrained; for example, if certain experts require additional resources, they can be automatically downgraded or replaced.

[0212] Therefore, the inputs and outputs of the gating network can be summarized as follows:

[0213] Input: Input feature representation, key-value vector, resource status of each terminal (channel, bandwidth, computing power, load);

[0214] Output: The set of MHA experts activated in this round of inference, the task subset (e.g., the assigned token segment) for each expert, and a comprehensive evaluation of the expected computational and communication latency for this Transformer layer.

[0215] 4) Task distribution and synchronization control module:

[0216] This module is responsible for distributing tasks to different terminals and managing execution synchronization.

[0217] Send expert indexes, control commands, tokens, and other information and data to WD;

[0218] Different bandwidths, packet transmission rates, or priorities are allocated to different WDs based on link status.

[0219] Set the task timeout period;

[0220] Define a synchronization strategy: aggregate only after all experts have returned, or tolerate the absence of some experts, with compensation adjusted by gating weights;

[0221] Dynamically reduce the weight of terminals with long latency.

[0222] 5) Results aggregation and MLP post-processing module:

[0223] Receive all intermediate results returned by activated MHA experts and perform weighted fusion according to the weights set by the gating network;

[0224] The fusion result is input into the MLP layer for nonlinear transformation, achieving the output of the current Transformer layer or the final output. If using... Representing the output of the preceding MHA process, the MLP processing performed on the BS can be expressed as:

[0225]

[0226] in, , , , These are all relevant parameters, which have been defined in the original Transformer mechanism. The GELU function is the Gaussian Error Linear Unit, a commonly used activation function in deep learning.

[0227] If the network model has a multi-layer structure, the output will be used as the input for the next layer to continue execution.

[0228] 6) Performance monitoring and adaptive strategy update module:

[0229] Continuously record the communication latency, computation time, and task success rate of each terminal;

[0230] Update expert priorities, task allocation strategies, and gating weights regularly or in task batches.

[0231] A learning mechanism can be used to optimize overall performance over the long term.

[0232] (3) Terminal-side module functions:

[0233] 1) Local resource monitoring module:

[0234] Monitor local CPU / GPU utilization, number of queued tasks, available memory, etc.

[0235] Detect indicators that can affect available computing power, such as temperature, power consumption, and battery level.

[0236] It reports to the base station at a certain period or when the status changes significantly. The reporting frequency can be adjusted adaptively to reduce overhead.

[0237] 2) MHA Expert Module:

[0238] Deploy a fully-structured MHA module as a MoE expert on each WD;

[0239] Upon receiving the task, load the shared key-value vector, the query projection matrix, and the input sequence. ;

[0240] After receiving the task, MHA is calculated using the Query projection matrix. First, WD of The formula for calculating a vector is:

[0241]

[0242] in, This is the Query projection matrix. The MHA process then continues as follows:

[0243]

[0244] in, It is the dimension of a single self-attention head. This is the parameter matrix. The softmax function is an activation function that transforms any real vector into a probability distribution, often used in multi-class classification problems in neural networks. It maps each input element to a value between 0 and 1, and the sum of all output values ​​is 1.

[0245] Output Uploaded to BS.

[0246] 3) Lightweight communication interface module:

[0247] Responsible for receiving input data and control commands from the base station;

[0248] Upload local calculation results and resource status information to the base station;

[0249] Adaptive transmission is performed based on the bandwidth / rate allocated by the base station.

[0250] 4) Buffering and retransmission control module:

[0251] Buffer packets that have not yet been sent when the link is temporarily interrupted or the quality degrades;

[0252] It supports a retransmission mechanism based on base station requests to ensure reliable delivery of results.

[0253] The following uses a single-round reasoning example to illustrate the typical processing flow of this invention.

[0254] S1, Input and Preprocessing (BS Execution);

[0255] Users initiate requests to the network big data model from terminal applications or business systems, inputting natural language commands. The input data is passed from the upper-layer system to the input preprocessing module on the BS side, where it undergoes operations such as word segmentation, encoding, and embedding to generate input embedding vectors. (Same as above for obtaining the target task data corresponding to the large language model; determine the input embedding vector corresponding to the target task data).

[0256] S2, K / V pre-computation and caching (BS execution);

[0257] Subsequently, the key-value pre-computation and caching module on the BS uses the centrally deployed key-value projection matrix to perform a linear transformation on the input embedding vector to obtain the key-value vector, which is then cached. Since all experts share the key-value vector, this computation only needs to be performed once (similar to determining the shared key-value vector corresponding to the input embedding vector and broadcasting the shared key-value vector).

[0258] S3, Communication and Computation Information Collection (BS and WD Collaboration);

[0259] Each WD reports its local computing power and current load information to the BS periodically or as needed through a lightweight communication interface module via its local resource monitoring module; the wireless communication subsystem provides information such as current channel gain and available bandwidth estimation. After collecting the above information, the BS forms the current system resource state vector.

[0260] S4, Gated Network Decision Making and Expert Selection (BS Execution);

[0261] The BS's communication computing gating network decision module takes input features, pre-computed key-value vectors, and resource state vectors as joint inputs to calculate the comprehensive latency evaluation of each expert node. Then, based on low latency, it selects the set of experts to be activated for this inference, and the task distribution and synchronization control module determines the token fragment or subtask responsible for each expert (similarly, based on the input embedding vector, shared key-value vector, and resource state vector, it determines the target wireless terminal corresponding to each subtask vector from multiple candidate wireless terminals).

[0262] S5, task distribution and bandwidth allocation (BS execution);

[0263] Subsequently, based on the expert pool and bandwidth resources, the BS sends the corresponding input token sequence, key-value vector, and necessary intermediate information to the corresponding WD (via the lightweight communication interface module), and allocates appropriate transmission resources to each WD through a bandwidth scheduling strategy. At this time, the BS's task distribution and synchronization control module also sets parameters such as task timeout for the WD processing (similar to sending the corresponding subtask data to the corresponding target wireless terminal).

[0264] S6, expert parallel computing (WD execution).

[0265] After receiving the task, each WD performs MHA operations in its local MHA expert calculation module, using the shared key-value vector and related information from the query projection matrix to calculate the output vector. Multiple WDs execute in parallel, thereby improving the overall computational throughput (similar to receiving target subtask data; calling the shared key-value vector; and determining the target intermediate result vector based on the shared key-value vector and the target subtask vector).

[0266] S7, Result Return and Error Handling (WD and BS Collaboration);

[0267] Through a lightweight communication interface module, each WD transmits its intermediate result vector (the output of the MHA expert) back to the BS's result aggregation and MLP post-processing module via a wireless link. If no results are received from certain WDs within a preset time, the BS can trigger a retransmission request or activate a backup expert node. The buffering and retransmission control module on the WD responds to the BS's decision to retransmit or perform supplementary calculations to ensure the stability of the large-scale wireless network system (similar to sending the target intermediate result vector to the target base station).

[0268] S8, Results aggregation and post-processing (BS execution);

[0269] After the BS results are aggregated and the MLP post-processing module collects all valid expert results, they are weighted and fused according to the gate weights to form the MoE output vector of this layer. Then, the MLP processing is performed to obtain the next stage input or final output of LLM (similar to receiving the intermediate result vectors corresponding to multiple target wireless terminals respectively, and determining the task output result corresponding to the target task data based on the multiple intermediate result vectors).

[0270] S9, output generation and return to the user (BS execution);

[0271] For multi-layered structures, repeat steps S2 to S8 until the network large model inference is complete; the final result can be a text response, control command, or other multimodal forms supported by existing large models. The BS returns the inference result to the user terminal or business system.

[0272] S10, Performance Feedback and Adaptive Updates (BS Execution).

[0273] After this round of inference is completed, the BS performance monitoring and adaptive strategy update module will record the transmission latency, computation latency and abnormal situations of each WD in this task, and update the statistical information in the performance monitoring and adaptive strategy module for the optimization of gating strategy and bandwidth allocation strategy in subsequent inference.

[0274] The above optional implementation methods can achieve at least the following beneficial effects:

[0275] (1) Innovation in MoE structure in wireless environment: This invention no longer uses the traditional MLP as the MoE expert, but uses the structurally complete multi-head attention (MHA) module as the expert unit, and distributes multiple MHA experts on wireless terminals with computing power. This design directly targets the most computationally intensive attention operator in the large network model based on LLM to sparsify and sink it, making the large network model more suitable for distributed inference in wireless edge environment while maintaining the original Transformer structure and model accuracy;

[0276] (2) Lightweight transmission mechanism driven by shared K / V vector: This invention uses a unified Key / Value projection matrix on the base station side to pre-compute the input embedding, generate a shared K / V vector and cache it, and only relies on the Query projection matrix distributed to each WD to generate the Q vector locally to complete the MHA calculation. By centrally pre-compiling the K / V vector + one broadcast, multiple reuses + distributing lightweight token embedding sequence and control information, the amount of uplink and downlink transmission data and the overhead of repeated calculations are significantly reduced, and the bandwidth pressure of the wireless link is reduced;

[0277] (3) Communication computing collaborative gated network decision-making and task allocation mechanism: This invention designs a communication computing gated network decision-making module on the base station side, which jointly models the resource status such as input semantic features, channel gain of each WD, available bandwidth, CPU / GPU utilization, and task queue length to construct an end-to-end latency estimation model. With the goal of minimizing the total latency of the network large model task processing, the MHA expert set activated in this round and its corresponding token processing sub-task are selected to realize the communication computing collaborative expert selection and task allocation that simultaneously considers model structure, channel conditions and terminal computing power;

[0278] (4) For long-term operation, the closed-loop feedback and adaptive scheduling mechanism of this invention continuously records the communication latency, computation time, task success rate and abnormal conditions of each terminal through the performance monitoring and adaptive policy update module, and dynamically adjusts the expert priority, bandwidth allocation strategy and task granularity accordingly. This closed-loop feedback mechanism enables the system to automatically optimize the scheduling strategy according to changes in network status and service load, maintain low end-to-end latency and high resource utilization in wireless edge scenarios for a long time, and make the implemented large-scale distributed network model have good engineering availability and scalability.

[0279] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the present invention is not limited to the described order of actions, because according to the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0280] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0281] Example 2

[0282] According to embodiments of the present invention, a system for implementing the above-described task processing method based on a large language model is also provided. The system includes a target base station and multiple target wireless terminals. The target base station is used to: acquire target task data corresponding to the large language model; determine an input embedding vector corresponding to the target task data; determine a shared key-value vector corresponding to the input embedding vector and broadcast the shared key-value vector; split the input embedding vector to obtain multiple sub-task vectors; determine target wireless terminals corresponding to the multiple sub-task vectors from multiple candidate wireless terminals based on the input embedding vector, the shared key-value vector, and the resource state vector, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals; send corresponding sub-task data to the corresponding target wireless terminal, wherein the corresponding sub-task data includes the corresponding sub-task vector; receive intermediate result vectors corresponding to the multiple target wireless terminals respectively, and determine based on the multiple intermediate result vectors;

[0283] Any one of the multiple target wireless terminals is used to receive target subtask data, wherein the target subtask data includes a target subtask vector, which is one of the multiple subtask vectors; invoke a shared key value vector, wherein the shared key value vector is obtained by the target wireless terminal receiving and caching the shared key value vector broadcast by the target base station; determine a target intermediate result vector based on the shared key value vector and the target subtask vector; and send the target intermediate result vector to the target base station, wherein the multiple intermediate result vectors include the target intermediate result vector.

[0284] It should be noted that the examples and application scenarios implemented with the corresponding steps are the same, but not limited to the content disclosed in Embodiment 1 above.

[0285] Example 3

[0286] According to another aspect of the present invention, an electronic device is also provided, comprising: a processor; and a memory for storing processor-executable instructions, wherein the processor is configured to execute instructions to implement the task processing method based on a large language model as described above.

[0287] Example 4

[0288] According to another aspect of the present invention, a computer-readable storage medium is also provided, which, when the instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform any of the above-described task processing methods based on a large language model.

[0289] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0290] In the above embodiments of the present invention, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0291] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. The device embodiments described above are merely illustrative; for example, the division of units can be a logical functional division, and in actual implementation, there may be other division methods. For instance, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the displayed or discussed mutual coupling, direct coupling, or communication connection may be through some interfaces; the indirect coupling or communication connection between units or modules may be electrical or other forms.

[0292] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0293] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0294] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, read-only memory (ROM), random access memory (RAM), portable hard drives, magnetic disks, or optical disks.

[0295] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A task processing method based on a large language model, characterized in that, include: Obtain the target task data corresponding to the large language model; Determine the input embedding vector corresponding to the target task data; Determine the shared key-value vector corresponding to the input embedding vector, and broadcast the shared key-value vector; The input embedding vector is split to obtain multiple subtask vectors; Based on the input embedding vector, the shared key vector, and the resource state vector, target wireless terminals corresponding to the multiple subtask vectors are determined from multiple candidate wireless terminals, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals. Send the corresponding subtask data to the corresponding target wireless terminal, wherein the corresponding subtask data includes the corresponding subtask vector, so that the corresponding target wireless terminal obtains the intermediate result vector based on the corresponding subtask vector and the shared key value vector; Receive intermediate result vectors corresponding to multiple target wireless terminals respectively, and determine the task output result corresponding to the target task data based on the multiple intermediate result vectors.

2. The method according to claim 1, characterized in that, Based on the input embedding vector, the shared key vector, and the resource state vector, the target wireless terminal corresponding to each of the multiple subtask vectors is determined from multiple candidate wireless terminals, including: Based on the input embedding vector, the shared key-value vector, and the resource state vector, a target latency corresponding to each of the plurality of candidate wireless terminals is determined. The target latency includes computation latency and communication latency. The computation latency includes a first latency for the target base station to perform a first operation, a second latency for the corresponding candidate wireless terminal to perform an intermediate result computation operation, and a third latency for the target base station to perform a task output result computation operation. The first operation includes at least one of the following: an input embedding vector determination operation, a shared key-value vector computation operation, and a subtask scheduling and allocation operation. The communication latency includes a fourth latency for the target base station to send data to the corresponding candidate wireless terminal and a fifth latency for the corresponding candidate wireless terminal to send data back to the target base station. Based on the target latency, target wireless terminals corresponding to the multiple subtask vectors are determined from multiple candidate wireless terminals.

3. The method according to claim 2, characterized in that, Based on the target latency, the target wireless terminal corresponding to each of the multiple sub-task vectors is determined from multiple candidate wireless terminals, including: Obtain the transmission data corresponding to the target base station and the plurality of candidate wireless terminals, wherein the transmission data includes bandwidth data, transmission power, and channel fading power; Based on the noise power and the transmitted data, the transmission rates corresponding to the target base station and the plurality of candidate wireless terminals are determined; Based on the target latency and the transmission rate, target wireless terminals corresponding to the multiple subtask vectors are determined from multiple candidate wireless terminals.

4. The method according to claim 1, characterized in that, Receiving intermediate result vectors corresponding to the plurality of target wireless terminals respectively, and determining the task output result corresponding to the target task data based on the plurality of intermediate result vectors, including: Based on the link status between the target base station and the plurality of target wireless terminals, the task timeout time corresponding to each of the plurality of target wireless terminals is determined; Identify the timed-out wireless terminals that have not received the corresponding intermediate result vector beyond the corresponding task timeout period; Based on the target result synchronization strategy, corresponding strategy processing is performed, and based on the multiple intermediate result vectors obtained after strategy processing, the task output result corresponding to the target task data is obtained.

5. The method according to claim 4, characterized in that, When the target result synchronization strategy includes a global complete aggregation mode and a missing tolerance compensation mode, the corresponding strategy processing is performed according to the target result synchronization strategy, and based on the multiple intermediate result vectors obtained after strategy processing, the task output result corresponding to the target task data is obtained, including: In the target result synchronization strategy of global complete aggregation mode, retry task data is sent to the timeout wireless terminal to wait for the intermediate result vector fed back by the timeout wireless terminal, and the task output result is obtained based on multiple intermediate result vectors; and / or, replacement task data is sent to the backup wireless terminal to wait for the intermediate result vector fed back by the backup wireless terminal, and the task output result is obtained based on multiple intermediate result vectors. When the target result synchronization strategy is in the missing tolerance compensation mode, the weights of the received intermediate result vectors are adjusted to obtain an updated result vector, and the task output result is determined based on the updated result vector.

6. The method according to any one of claims 1 to 5, characterized in that, Based on multiple intermediate result vectors, the task output result corresponding to the target task data is determined, including: The intermediate result vector is weighted and fused to obtain a hybrid expert output vector; The target task vector is obtained by performing a nonlinear transformation on the hybrid expert output vector. Based on the target task vector, the task output result is obtained.

7. A task processing method based on a large language model, characterized in that, include: Receive target subtask data, wherein the target subtask data includes a target subtask vector, the target subtask vector is one of multiple subtask vectors, the multiple subtask vectors are obtained by splitting the input embedding vector, and the input embedding vector is determined based on the target task data corresponding to the large language model; The shared key-value vector is invoked, wherein the shared key-value vector is a vector corresponding to the input embedding vector, which is obtained by the target wireless terminal receiving the shared key-value vector broadcast by the target base station and caching it; Based on the shared key-value vector and the target subtask vector, determine the target intermediate result vector; The target intermediate result vector is sent to the target base station so that the target base station determines the task output result corresponding to the target task data based on the multiple intermediate result vectors, wherein the multiple intermediate result vectors include the target intermediate result vector.

8. The method according to claim 7, characterized in that, Based on the shared key-value vector and the target subtask vector, an intermediate result vector is determined, including: If the target subtask data also includes a target query projection matrix, a local query vector is calculated based on the target query projection matrix and the target subtask vector to obtain the target local query vector; Based on the target local query vector and the shared key-value vector, multi-head attention calculation is performed to obtain the target intermediate result vector.

9. A task processing system based on a large language model, characterized in that, include: Target base station, multiple target wireless terminals, among which, The target base station is used to acquire target task data corresponding to a large language model; determine an input embedding vector corresponding to the target task data; determine a shared key-value vector corresponding to the input embedding vector and broadcast the shared key-value vector; split the input embedding vector to obtain multiple sub-task vectors; based on the input embedding vector, the shared key-value vector, and the resource state vector, determine the target wireless terminal corresponding to each of the multiple sub-task vectors from multiple candidate wireless terminals, wherein the resource state vector represents the resource state corresponding to the multiple candidate wireless terminals; send the corresponding sub-task data to the corresponding target wireless terminal, wherein the corresponding sub-task data includes the corresponding sub-task vector; receive intermediate result vectors corresponding to the multiple target wireless terminals respectively, and determine based on the multiple intermediate result vectors; Any one of the plurality of target wireless terminals is used to receive target subtask data, wherein the target subtask data includes a target subtask vector, which is one of the plurality of subtask vectors; invoke a shared key-value vector, wherein the shared key-value vector is obtained by the target wireless terminal receiving a shared key-value vector broadcast by the target base station and caching it; determine a target intermediate result vector based on the shared key-value vector and the target subtask vector; and send the target intermediate result vector to the target base station, wherein the plurality of intermediate result vectors include the target intermediate result vector.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the task processing method based on a large language model as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the task processing method based on a large language model as described in any one of claims 1 to 8.

Images