Large language model reasoning scheduling method and device based on semantic communication

Abstract

The invention discloses a big language model reasoning scheduling method and device based on semantic communication. The method comprises the steps that edge equipment extracts semantic information of a reasoning request through a combined lightweight model, predicts the length of an output token and uploads the length to a big language model server; the large language model server sorts the waiting requests according to a semantic scheduling scoring function, the scoring function performs dynamic weighted calculation based on token length, channel quality and historical retransmission times, and a request group with the highest priority is selected and a corresponding edge device is indicated to upload semantic tensors of reasoning requests extracted by the joint lightweight model; and the large language model server receives the semantic tensor and then reconstructs information for batch reasoning. According to the method, the throughput capacity of an inference system in a high-concurrency environment can be effectively improved, the task response time delay is reduced, the stability and the service quality of an edge semantic communication system are enhanced, and the method has wide applicability and popularization value for deployment of a large language model in an edge calculation and multi-user semantic communication scene.

Description

Technical Field

[0001] The present invention belongs to the technical field of intersection of wireless communication and artificial intelligence, and specifically relates to a large language model reasoning scheduling method and device based on semantic communication. Background Art

[0002] In recent years, with the rapid development of semantic communication technology, the traditional communication model based on bit transmission is shifting towards an intelligent communication paradigm based on semantic understanding. By extracting and transmitting semantic features rather than raw data, semantic communication provides a new technical path for efficient information exchange in edge computing scenarios. These advantages rely on powerful semantic understanding and reasoning capabilities, which are the core advantages of large language models.

[0003] Semantic communication systems encompass infrastructure such as terminal devices, edge servers, and cloud computing nodes, supporting a variety of semantically driven application services such as intelligent question answering, anomaly detection, and content generation. Therefore, modern semantic communication systems must simultaneously meet multiple requirements, including low latency, high energy efficiency, and strong semantic understanding. They are characterized by diverse service types, high quality of service sensitivity, and dynamically changing resource requirements. To address the complex and ever-changing demands of semantic communication, a heterogeneous computing architecture with "cloud-edge-end" collaboration is needed to achieve efficient distribution and scheduling of semantic processing capabilities. To optimize overall system performance, intelligent task scheduling mechanisms, dynamic resource allocation strategies, and efficient semantic encoding schemes must be established. Furthermore, the deployment environments of edge devices exhibit significant heterogeneity: some are located in urban areas with good network conditions, while others are distributed in remote areas with poor network coverage and scarce computing resources. Furthermore, in semantic communication scenarios, with the widespread adoption of smart terminal devices, systems often face a large number of concurrent semantic requests, posing significant challenges to the resource management and task scheduling mechanisms of the communication network.

[0004] To address these issues, semantic-aware scheduling technology based on large language models, with its powerful semantic understanding capabilities and dynamic resource adaptation features, can provide intelligent solutions for high-concurrency, low-latency semantic communication scenarios. In a semantic communication system for large language models, a high-performance large language model inference engine is deployed on cloud servers while a lightweight semantic extraction module is deployed on edge devices, forming a cloud-edge collaborative intelligent communication architecture. Cloud and edge devices interact through semantic information rather than raw data, which can significantly reduce communication load and improve semantic understanding capabilities. After user requests are semantically encoded, inference processing can be performed on the edge or cloud based on computational complexity. Traditional task scheduling methods use a simple first-come, first-served strategy. In large language model inference scenarios, this ignores the semantic differences and resource requirements between requests, which can easily lead to low GPU utilization and reduced system throughput, and cannot meet the needs of real-time semantic communication.

[0005] Chinese patent application publication number CN115629865A provides a method for scheduling deep learning inference tasks based on edge computing. This method achieves load balancing by predicting task execution time, but fails to fully consider the impact of the autoregressive nature of LLM inference on scheduling. Chinese patent application publication number CN117135131A provides a method for sensing task resource requirements for cloud-edge collaboration scenarios. This method optimizes task allocation by monitoring GPU utilization in real time, but lacks consideration for the coordinated optimization of transmission and computing in semantic communication scenarios.

[0006] Therefore, there is an urgent need to develop an intelligent scheduling system that integrates large language model reasoning and semantic communication to improve the semantic service quality and resource utilization efficiency in edge computing environments. Summary of the Invention

[0007] In view of the above, the purpose of the present invention is to provide a large language model inference scheduling method and device based on semantic communication, which predicts the model inference complexity based on the semantic understanding of the inference request on the edge side, and combines a multi-factor dynamic scheduling mechanism to realize request sorting and batch inference optimization, thereby improving communication efficiency and model inference performance.

[0008] To achieve the above-mentioned purpose, the present invention provides the following technical solutions: In a first aspect, an embodiment of the present invention provides a large language model reasoning scheduling method based on semantic communication, which is applied to a semantic communication reasoning system including a communication base station equipped with a large language model server and multiple distributed edge devices, including the following steps: The edge device extracts the semantic information of the large language model inference request through the joint lightweight model and predicts the output token length. The output token length is uploaded to the large language model server via the communication base station. The large language model server sorts waiting requests according to a semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality, and historical retransmission count. Based on the sorting, the request group with the highest priority is selected and the corresponding edge device is instructed to upload the semantic tensor of the inference request extracted by the joint lightweight model through the communication base station. The large language model server receives the semantic tensor and reconstructs the information for batch inference.

[0009] Preferably, the joint lightweight model includes: The general encoder module maps the inference request into a semantic vector through the encoder; The semantic processing module converts the semantic vector extracted by the general encoder module into a semantic tensor suitable for channel transmission through the encoder; The classification / regression prediction module converts the semantic vector extracted by the universal encoder module into a token length classification or outputs a token length prediction value through the residual module and the fully connected layer.

[0010] Preferably, the semantic scheduling scoring function is used to measure the scheduling priority of each request under the current system state, and its form satisfies the following function expression: , in, Indicates the The output token length of each request, Indicates transmission The corresponding link channel quality, Indicates the number of retransmissions. 、 and are the set scheduling weight factors respectively. The scheduling goal is to minimize the average waiting time and inference delay of the terminal. After scheduling, the large language model server selects priority requests according to the scores for centralized inference.

[0011] Preferably, the optimization strategy for batch inference is used to dynamically determine the batch size for each round of inference. The optimization goal is to improve throughput efficiency while avoiding memory overflow. The optimization strategy expression is as follows: , in, Indicates the actual batch size, Indicates the hard batch limit set by the server. Indicates the maximum allowed safe batch size calculated in real time by the memory monitoring module based on the current available memory. It is a dynamically changing value. Represents the current set of pending requests The actual batch size cannot exceed the total number of pending requests.

[0012] Preferably, the difference in output token length for inference requests in each batch is controlled within a preset threshold. To avoid prolonging the overall batch inference time due to a single long task, it is recorded as: , in, and Respectively represent batches The output token length of any two inference requests in , Indicates the requested index.

[0013] Preferably, the joint lightweight model is trained in the following manner: Collect historical reasoning requests, the corresponding complete large language model reasoning output token sequence and token sequence length label; input the reasoning request into the universal encoder module to extract the semantic vector, and then obtain the original semantic tensor and reconstructed semantic tensor of the semantic information through the semantic processing module to calculate the semantic reconstruction loss. ; Input the semantic vector output by the universal encoder module into the classification / regression prediction module to obtain the token length prediction value, and calculate the length prediction loss based on the token length prediction value and the token sequence length label Based on the joint training strategy, the shared BERT encoder includes semantic reconstruction loss and length prediction loss Total loss To optimize the model parameters, Expressed as: , in, and Represent the weight coefficients respectively.

[0014] Preferably, the semantic reconstruction loss The length prediction loss is calculated based on the similarity index of semantic reconstruction, namely BLEU score. L1 norm or cross entropy is used to measure the difference between the predicted output and the true label.

[0015] Preferably, the dataset used for semantic reconstruction is much smaller than the dataset for length prediction, and the loss parameter is set to satisfy .

[0016] In a second aspect, an embodiment of the present invention further provides a large language model reasoning scheduling device based on semantic communication, which is implemented using the above-mentioned large language model reasoning scheduling method based on semantic communication and is applied to a semantic communication reasoning system including a communication base station equipped with a large language model server and multiple distributed edge devices, including: an edge semantic extraction module, a request scheduling upload module, and a dynamic batch reasoning module; The edge semantic extraction module is used by the edge device to extract the semantic information of the reasoning request through the joint lightweight model, predict the output token length, and upload the output token length to the large language model server via the communication base station; The request scheduling upload module is used by the large language model server to sort the waiting requests according to the semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality, and historical retransmission count. The request group with the highest priority is selected according to the sorting and the corresponding edge device is instructed to upload the semantic tensor of the inference request extracted by the joint lightweight model through the communication base station. The dynamic batch inference module is used by the large language model server to reconstruct information and perform batch inference after receiving semantic tensors.

[0017] Compared with the prior art, the present invention has the following beneficial effects: This method schedules inference requests based on semantic perception. Edge devices extract semantic information and predict the output token length for upload. A large language model server dynamically weights the semantic scheduling scoring function to accurately sort pending requests and select the highest-priority request group. This model effectively coordinates resource allocation between the edge and the center, improving communication efficiency, reducing unnecessary data transmission, and optimizing task response performance. It can quickly process important requests, offers a flexible solution, and has good scalability and engineering application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0019] Figure 1 1 is a flow chart of a large language model reasoning and scheduling method based on semantic communication provided by an embodiment of the present invention; Figure 2 Schematic diagram of the deployment and application of the large language model reasoning and scheduling method based on semantic communication provided by an embodiment of the present invention; Figure 3 It is a schematic diagram of the structure and process of the joint lightweight model provided by an embodiment of the present invention; Figure 4 It is a structural diagram of a large language model reasoning and scheduling device based on semantic communication provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and do not limit the scope of protection of the present invention.

[0021] The inventive concept of the present invention is: in response to the problems of resource waste, uneven scheduling and response delay in the large language model (LLM) inference process in existing semantic communication systems, the embodiments of the present invention provide a large language model inference scheduling method and device based on semantic communication. By deploying a joint lightweight model on the edge device, the semantic information of the inference request is first extracted and the output token length is predicted and uploaded, providing a scheduling basis for the large language model server; the server uses a semantic scheduling scoring function to dynamically weight and sort the output token length, channel quality and historical retransmission times, selects a high-priority request group, and then instructs the corresponding edge device to upload the semantic tensor. Finally, the server reconstructs the information for batch inference, ultimately achieving reasonable resource allocation and efficient scheduling.

[0022] like Figure 1 and Figure 2 As shown, the embodiment provides a large language model reasoning scheduling method based on semantic communication, including the following steps: S1: The edge device extracts the semantic information of the large language model inference request through a joint lightweight model and predicts the output token length, and uploads the output token length to the large language model server via the communication base station.

[0023] In the embodiment, a joint lightweight model is used to implement output length prediction in a semantic communication environment, aiming to provide key prior information for subsequent large language model inference scheduling, thereby effectively improving inference scheduling efficiency and resource utilization.

[0024] Specifically, the lightweight model is deeply coupled with the semantic extraction process, and the shared pre-trained language model (BERT) is used to perform semantic modeling on the inference request, such as Figure 3 Shown, including: (1) General encoder module: First, after the edge device initiates an inference request (such as question and answer, instruction, etc.), it is first converted into a structured text input in the format of [CLS] sentence 1 [SEP] sentence 2 [SEP], and sent to the lightweight BERT encoder for context modeling. Then, feature extraction and fusion are performed, and the output of the encoder is sent to the nonlinear feature enhancement module to extract more predictive and discriminative semantic features. The nonlinear feature enhancement module includes the following structures: a fully connected layer and activation function for dimensional conversion and feature nonlinear enhancement; a residual connection structure for weighted fusion of features before and after enhancement; layer normalization (LayerNorm) to keep the feature values ​​stable; and a Dropout mechanism to prevent overfitting of the model. The semantic vector obtained after processing As input features for classification / regression prediction modules.

[0025] (2) Semantic processing module: The semantic vector extracted by the universal encoder module is converted into a semantic tensor suitable for channel transmission through end-to-end network training.

[0026] (3) Classification / regression prediction module: The semantic vector extracted by the universal encoder module is converted into a token length classification or output token length prediction value through the residual module and the fully connected layer .

[0027] During the training process, the predicted value is compared with the actual output token length The difference between them is used as the loss function and the L1 norm is used for calculation: , This guides the model to gradually learn the mapping relationship between semantic input and its corresponding output scale. When it is converted to a classification problem, the loss function needs to use cross entropy.

[0028] The joint lightweight model is trained in the following way: Collect historical inference requests, the corresponding complete large language model inference output token sequence and token sequence length label; input the inference request into the universal encoder module to extract the semantic vector, and then obtain the original semantic tensor and reconstructed semantic tensor of the semantic information through the semantic processing module to calculate the semantic reconstruction loss. ; Input the semantic vector output by the universal encoder module into the classification / regression prediction module to obtain the token length prediction value, and calculate the length prediction loss based on the token length prediction value and the token sequence length label To further improve the system performance, a joint training strategy is adopted, which uses a shared BERT encoder based on semantic reconstruction loss. and length prediction loss Total loss To optimize the model parameters, Expressed as: , in, and Represent the weight coefficients, According to the similarity index of semantic reconstruction, namely BLEU score, L1 norm or cross entropy is used to measure the difference between the predicted output and the true label.

[0029] To prevent underfitting of small data tasks and ensure that the model takes into account task performance while improving communication efficiency, the dataset used for semantic reconstruction tasks is much smaller than the dataset for length prediction tasks, and the loss parameters are set to meet , in order to better train the model.

[0030] After model training is complete, the general encoder module, semantic processing module, and classification / regression prediction module are compressed into a lightweight model and deployed to the edge device. In actual operation, whenever an inference request is issued, the edge device can quickly predict the output length based on the local model and upload it along with other metadata (such as channel status and retry count) to the scheduling server to assist in optimizing subsequent batch scheduling strategies.

[0031] Through this step, the system can obtain key prior information of the inference task without significantly increasing the burden on edge devices, providing strong support for efficient use of graphics card resources, reducing padding redundancy and lowering average waiting latency.

[0032] In S2, the large language model server sorts the waiting requests according to the semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality, and historical retransmission times. The request group with the highest priority is selected based on the sorting, and the corresponding edge device is instructed by the communication base station to upload the semantic tensor of the inference request extracted by the joint lightweight model.

[0033] In the embodiment, efficient inference scheduling is achieved based on edge output prediction, and a dynamic batch construction mechanism guided by prediction is implemented. While ensuring communication reliability, the allocation and scheduling order of model inference resources are optimized, effectively improving the overall system throughput and response performance.

[0034] Specifically, before each edge device initiates an inference request, it first extracts the semantic vector through the locally deployed joint lightweight model and further predicts the output token length prediction value, which is recorded as At the same time, the device will detect the current channel quality Number of request retransmissions , forming a metadata package ,This metadata package does not contain the original semantic content and has the advantages of lightweight transmission and privacy protection.

[0035] All edge devices will be metadata The data is uploaded to the large language model inference server via a wireless channel via a communication base station. After receiving the batch metadata, the server prioritizes each request based on the following semantic scheduling scoring function, taking into account the impact of channel quality and scheduling length on latency. The number of retransmissions is used to limit the number of requests waiting too long. To ensure scheduling fairness and efficiency, the server prioritizes each request based on the following semantic scheduling scoring function: , in, Indicates the The output token length of each request, Indicates transmission The corresponding link channel quality, Indicates the number of retransmissions. 、 and These are the set scheduling weight factors used to balance the impact of output length, channel quality, and transmission stability in scheduling, dynamically adapting to network and model status. The scheduling goal is to minimize the average terminal wait time and inference latency. After scheduling, the large language model server prioritizes requests for centralized inference based on their scores. Lower scores indicate more appropriate priority processing.

[0036] Furthermore, according to the scheduling score Sort all pending requests in ascending order and divide them into batches based on the principle of "similar output length". The difference in output token length of inference requests in each batch is controlled within the preset threshold. Within, record as: , in, and Respectively represent batches The output token length of any two inference requests in , Indicates the requested index. This strategy can effectively avoid extending the overall batch inference time due to a single long task, reduce graphics card resource waste, and improve batch processing efficiency.

[0037] S3, the large language model server receives the semantic tensor and reconstructs the information for batch reasoning.

[0038] In the embodiment, the edge device selected by the server and issued with the scheduling notification sends its semantic tensor Upload to the cloud. The server decodes the received semantic data and completes the final input construction.

[0039] All requests in the same batch are processed as a unified input and then fed into a large language model for batch inference. The optimization strategy for batch inference dynamically determines the batch size for each round of inference while ensuring system performance. The optimization goal is to improve throughput while avoiding memory overflows. When executing batch inference, the server takes into account the variability of graphics card resources and introduces the following batch optimization strategy: , in, Indicates the actual batch size, Indicates the hard batch limit set by the server. Indicates the maximum allowed safe batch size calculated in real time by the memory monitoring module based on the current available memory. It is a dynamically changing value. Represents the current set of pending requests The actual batch size cannot exceed the total number of pending requests.

[0040] At the same time, the model processing time It is subject to the inference time of the longest request in the batch, that is: .

[0041] After the inference is completed, the server will package the output results and return them to the corresponding edge devices, and record indicators such as scheduling efficiency, waiting time, and prediction error for subsequent strategy optimization.

[0042] The system continuously collects the difference between the predicted length and the actual output, builds an error model to optimize the edge prediction module; at the same time, it periodically fine-tunes the weight parameters of the scheduling scoring function to adapt to dynamic changing factors such as system operating status, network environment and user distribution.

[0043] Through the batch inference dynamic scheduling strategy provided in this embodiment, the system can achieve the following goals: (1) Using prediction information to reduce the difference in task output length within an inference batch, thereby shortening the overall inference time; (2) Introducing channel quality and retransmission times as scheduling factors to improve the communication robustness of task scheduling; (3) Implement scheduling and sorting based on the scoring mechanism to achieve dynamic adaptation of resource allocation; (4) Balance graphics card utilization and user experience, significantly reducing average waiting time.

[0044] In summary, by constructing a semantic prediction-guided scheduling scoring system and a dynamic batch construction mechanism, parallel computing can be performed while ensuring resource security, achieving the goal of joint scheduling of communication perception and reasoning resources. It has good versatility, scalability and practical application value, and can greatly improve the reasoning efficiency and service capabilities of large language models while ensuring communication quality. It is suitable for large language model reasoning service optimization tasks in multi-user and highly concurrent edge semantic communication environments.

[0045] Based on the same inventive concept, Figure 4 As shown, an embodiment of the present invention further provides a large language model reasoning scheduling device 400 based on semantic communication, including: an edge semantic extraction module 410, a request scheduling upload module 420, and a dynamic batch reasoning module 430.

[0046] The edge semantic extraction module 410 is used by the edge device to extract the semantic information of the user request through a joint lightweight model, predict the output token length, and upload the output token length to the large language model server via the communication base station.

[0047] The request scheduling upload module 420 is used by the large language model server to sort the waiting requests according to the semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality and historical retransmission times. The request group with the highest priority is selected according to the sorting and the corresponding edge device is instructed by the communication base station to upload the semantic tensor of the inference request extracted by the joint lightweight model.

[0048] The dynamic batch reasoning module 430 is used for the large language model server to receive semantic tensors and reconstruct information for batch reasoning.

[0049] It should be noted that the large language model inference scheduling device based on semantic communication provided in the above embodiment belongs to the same inventive concept as the large language model inference scheduling method based on semantic communication. The specific implementation process is detailed in the embodiment of the large language model inference scheduling method based on semantic communication, which will not be repeated here.

[0050] The specific implementation methods described above provide a detailed description of the technical solutions and beneficial effects of the present invention. It should be understood that the above is only the most preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, supplements and equivalent substitutions made within the scope of the principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A large language model reasoning scheduling method based on semantic communication, characterized in that: A semantic communication reasoning system applied to a communication base station equipped with a large language model server and multiple distributed edge devices includes the following steps: The edge device extracts the semantic information of the large language model inference request through the joint lightweight model and predicts the output token length. The output token length is uploaded to the large language model server via the communication base station. The large language model server sorts waiting requests according to a semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality, and historical retransmission count. Based on the sorting, the request group with the highest priority is selected and the corresponding edge device is instructed to upload the semantic tensor of the inference request extracted by the joint lightweight model through the communication base station. The large language model server receives the semantic tensor and reconstructs the information for batch inference.

2. The large language model reasoning and scheduling method based on semantic communication according to claim 1 is characterized in that: The joint lightweight model includes: The general encoder module maps the inference request into a semantic vector through the encoder; The semantic processing module converts the semantic vector extracted by the general encoder module into a semantic tensor suitable for channel transmission through the encoder; The classification / regression prediction module converts the semantic vector extracted by the universal encoder module into a token length classification or outputs a token length prediction value through the residual module and the fully connected layer.

3. The large language model reasoning and scheduling method based on semantic communication according to claim 1 is characterized in that: The semantic scheduling scoring function is used to measure the scheduling priority of each request in the current system state, and its form satisfies the following function expression: ， in, Indicates the The output token length of each request, Indicates transmission The corresponding link channel quality, Indicates the number of retransmissions. 、 and are the set scheduling weight factors respectively. The scheduling goal is to minimize the average waiting time and inference delay of the terminal. After scheduling, the large language model server selects priority requests according to the scores for centralized inference.

4. The large language model reasoning and scheduling method based on semantic communication according to claim 1, characterized in that: The optimization strategy for batch inference is used to dynamically determine the batch size for each round of inference. Its optimization goal is to improve throughput efficiency while avoiding memory overflow. The optimization strategy expression is as follows: ， in, Indicates the actual batch size, Indicates the hard batch limit set by the server. Indicates the maximum allowed safe batch size calculated in real time by the memory monitoring module based on the current available memory. It is a dynamically changing value. Represents the current set of pending requests The actual batch size cannot exceed the total number of pending requests.

5. The large language model reasoning and scheduling method based on semantic communication according to claim 1 or 4, characterized in that: Control the difference in output token length of inference requests in each batch within a preset threshold To avoid prolonging the overall batch inference time due to a single long task, it is recorded as: ， in, and Respectively represent batches The output token length of any two inference requests in , Indicates the requested index.

6. The large language model reasoning and scheduling method based on semantic communication according to claim 1 or 2, characterized in that: The joint lightweight model is trained in the following way: Collect historical inference requests, the corresponding complete large language model inference output token sequence and token sequence length label; input the inference request into the universal encoder module to extract the semantic vector, and then obtain the original semantic tensor and reconstructed semantic tensor of the semantic information through the semantic processing module to calculate the semantic reconstruction loss. ; Input the semantic vector output by the universal encoder module into the classification / regression prediction module to obtain the token length prediction value, and calculate the length prediction loss based on the token length prediction value and the token sequence length label ; Based on the joint training strategy, the shared BERT encoder is based on the semantic reconstruction loss and length prediction loss Total loss To optimize the model parameters, Expressed as: ，， in, and Represent the weight coefficients respectively.

7. The method for large language model reasoning and scheduling based on semantic communication according to claim 6, characterized in that: Semantic reconstruction loss The length prediction loss is calculated based on the similarity index of semantic reconstruction, namely BLEU score. L1 norm or cross entropy is used to measure the difference between the predicted output and the true label.

8. The method for large language model reasoning and scheduling based on semantic communication according to claim 6, characterized in that: The dataset for semantic reconstruction is much smaller than that for length prediction, and the loss parameter is set to satisfy .

9. A large language model reasoning and scheduling device based on semantic communication, implemented using the large language model reasoning and scheduling method based on semantic communication according to any one of claims 1 to 8, characterized in that: The system is applied to a semantic communication reasoning system that includes a communication base station equipped with a large language model server and multiple distributed edge devices, including: an edge semantic extraction module, a request scheduling upload module, and a dynamic batch reasoning module; The edge semantic extraction module is used to use the edge device to extract the semantic information of the large language model inference request through the joint lightweight model, predict the output token length, and upload the output token length to the large language model server via the communication base station; The request scheduling upload module is used to use the large language model server to sort the waiting requests according to the semantic scheduling scoring function. The scoring function is dynamically weighted based on the output token length, channel quality, and historical retransmission count. The request group with the highest priority is selected according to the sorting and the corresponding edge device is instructed to upload the semantic tensor of the inference request extracted by the joint lightweight model through the communication base station; The dynamic batch reasoning module is used to utilize the large language model server to receive semantic tensors and reconstruct information for batch reasoning.

Images