Heterogeneous LLM reliable optimization collaborative reasoning method for edge device end side

Abstract

A heterogeneous LLM reliable optimization collaborative reasoning method for an edge device end side comprises the following steps: selecting a base station subset participating in collaborative reasoning through a gating optimization allocation algorithm based on unsupervised learning, transmitting an input prompt to each base station in the base station subset according to uplink transmitting power, and performing parallel autoregression reasoning and result uploading by each base station; and the UE performs semantic alignment and unsupervised dynamic weighted fusion on the outputs from the plurality of heterogeneous LLMs at the local end to generate a collaborative reasoning result with high accuracy and low delay. According to the invention, a unified relative semantic space and an unsupervised integrated weight estimation mechanism are adopted to generate an integrated weight, and a discrete base station selection and continuous UE transmitting power distribution problem is converted into a differentiable end-to-end optimization task, so that the existing heterogeneous LLM and infrastructure thereof do not need to be modified; and efficient, robust and low-delay heterogeneous LLM collaborative reasoning can be realized under strict communication constraints.

Description

Technical Field

[0001] This invention relates to a technology in the field of wireless communication, specifically a reliable optimization collaborative reasoning method for heterogeneous large language model (LLM) on the edge device side. Background Technology

[0002] Large Language Models (LLMs) face several technical challenges in practical deployment. First, LLMs with different architectures and training data possess independent vocabularies and output spaces, making direct comparison or integration of their output probability distributions impossible, thus creating a fundamental obstacle to collaborative reasoning. Second, the lack of abundant, accurately labeled data in real-world edge scenarios makes traditional methods relying on supervised learning for model selection and integration modules difficult to implement. Third, limited device resources (transmission power) and dynamically changing wireless channel conditions make selecting the optimal subset from numerous LLMs for collaboration a complex problem, tightly coupled with communication constraints.

[0003] Existing semantic processing techniques for heterogeneous data do not consider the strict communication resource constraints (such as transmit power, dynamic channels, and latency) in edge wireless network scenarios. Their collaborative reasoning process lacks modeling and optimization of the coupling relationship between the quality of the underlying communication link, resource allocation, and inference performance. Furthermore, it does not address how to unsupervisedly and dynamically estimate the reliability (weights) of each heterogeneous model in edge scenarios lacking a large amount of accurately labeled data, and how to select models and fuse outputs accordingly. Moreover, existing technologies cannot achieve direct comparison and deep fusion of heterogeneous LLM output probabilities within a unified relative semantic space, nor can they achieve joint optimization by dynamically selecting the optimal model subset based on task semantics and accurately allocating resources. In addition, the collaborative process lacks an inherent measurement and utilization mechanism for the quality of model outputs. Summary of the Invention

[0004] This invention addresses the semantic misalignment problem caused by vocabulary differences in existing technologies by proposing a reliable optimization collaborative reasoning method for heterogeneous LLMs on the edge device side. It adopts a unified relative semantic space and an unsupervised integrated weight estimation mechanism to generate integrated weights, transforming the discrete base station selection and continuous UE transmit power allocation problem into a differentiable end-to-end optimization task. Therefore, it does not require modification of existing heterogeneous LLMs and their infrastructure, and can achieve efficient, robust and low-latency heterogeneous LLM collaborative reasoning under strict communication constraints.

[0005] This invention is achieved through the following technical solution:

[0006] This invention relates to a heterogeneous LLM reliable optimization collaborative inference method on the edge device side, comprising:

[0007] Step 1: When a UE initiates a natural language inference request (such as question answering, summary generation, or multi-turn dialogue), the UE first dynamically evaluates and selects a subset of base stations to participate in collaborative inference based on the current radio channel state, the potential performance of each LLM in the context of the request, and resource constraints. ,in: For input prompts, For the entire set of base stations, For the selected set of base stations, This is a gated optimization allocation algorithm based on unsupervised learning. For relative to the first Normalized path loss coefficient for each base station This is the set of normalized path loss coefficients for all base stations. Due to the maximum transmit power constraint, To constrain maximum latency, uplink transmit power allocation is optimized.

[0008] Step 2: After the UE measures the instantaneous channel state information of the base stations in each base station subset, it calculates the transmission capacity of each uplink and evaluates whether the overall transmission delay meets the constraints. Then, based on the transmit power allocated in Step 1, it transmits the input prompt through the uplink radio link. Send concurrently to the selected base station.

[0009] The uplink transmission capacity ,in: For UE to the The uplink capacity of each base station, B is the bandwidth, and the bandwidth for each UE is the same for each base station. Is the UE assigned to the first The transmit power of each base station, This represents the power spectral density of additive white Gaussian noise.

[0010] The aforementioned assessment of overall transmission latency refers to the fact that the uplink latency is determined by the slowest transmission rate among the selected set of base stations, therefore the latency... ,in: Input prompts The size of the data, The set of transmit power allocated to the selected base station for the UE. This is the minimum uplink capacity among the selected base station sets.

[0011] Step 3, Receive the same input The base station uses its locally deployed LLM to perform parallel autoregressive inference. After outputting the probability distribution of the corresponding token position to the UE at each decoding step, the UE performs semantic alignment and unsupervised dynamic weighted fusion of the outputs from multiple heterogeneous LLMs on the local end to generate a collaborative inference result with both high accuracy and low latency.

[0012] The semantic alignment and unsupervised dynamic weighted fusion mentioned above refer to mapping the probability distribution of token positions to a shared relative semantic space, that is, for each LLM, the probability vector... Through relative matrix Projecting onto the relative semantic space specifically involves: ,in: A set of anchor tokens shared across models. For the original lexical space The cosine similarity between the embedding vector of the middle token and the embedding vector of the shared anchor token. Therefore, the result of the ensemble of the output probabilities of each model. ,in: To be assigned to the The weights of each model are determined. The entire process requires no modification to existing models or communication infrastructure, enabling efficient, flexible, and resource-aware collaborative services for large edge models.

[0013] This invention relates to a heterogeneous LLM reliable optimization collaborative reasoning method system on the edge device side, comprising: a dynamic gating and resource allocation module, an uplink communication management module, and a semantic alignment and weight fusion module, wherein: the dynamic gating and resource allocation module is based on input prompts The set of normalized path loss coefficients for all base stations Maximum transmit power constraint and maximum delay constraint Through an unsupervised learning-based gating optimization allocation algorithm The process is performed to output the final integrated weight set. Selected set of base stations and the transmit power allocated by the UE to each selected base station The uplink communication management module uses the selected base station set output by the dynamic gating and resource allocation module. and the transmit power allocated to each selected base station Bandwidth B and power spectral density of additive white Gaussian noise The uplink capacity is calculated based on Shannon's formula. And the maximum transmission latency D in the uplink, and manage the prompt text. Concurrent transmission to selected base stations; the semantic alignment and weight fusion module, based on the same prompt text returned by each selected base station, performs the following: Heterogeneous LLM output probability vector Using the relative matrices corresponding to each LLM Each probability vector Projecting onto the relative semantic space yields semantic vectors. Subsequently, based on the final integrated weight set provided by the dynamic gating and resource allocation module... Then based on semantic vectors and the final integrated weight set We perform a weighted summation to obtain the semantically aligned ensemble probability vector. It also decodes and generates natural language output results for collaborative reasoning.

[0014] Technical effect

[0015] This invention is based on an unsupervised ensemble weight estimation mechanism, using the obtained weights simultaneously for model selection and subsequent output ensemble. During GAT training, GAT is combined with the Gumbel-Softmax technique, transforming it into a unified framework that can be trained end-to-end, solving both the discrete base station (model) selection problem and the continuous device transmit power allocation problem. Unlike traditional ADMM updates, this method employs asynchronous ADMM updates, performing GAT forward inference once every two GAT network parameter updates, and updating auxiliary variables based on the GAT forward inference results. Lagrange dual variables The obtained weights are used for both model selection and subsequent output integration. Compared to the method where the obtained weights are only used for model selection, the use of a simple averaging baseline scheme during integration brings additional accuracy improvements and ensures that the output scheme strictly meets the maximum delay and maximum power constraints. Attached Figure Description

[0016] Figure 1 This is a scene diagram for an example embodiment;

[0017] Figure 2 This is a flowchart of the present invention;

[0018] Figures 3-7 This is a schematic diagram illustrating the effect of an example. Detailed Implementation

[0019] This embodiment is in a... Figure 1 The diagram illustrates a user-centric wireless communication network scenario, which includes a UE and Each base station is co-located with an edge server, which deploys a pre-trained, fixed-parameter heterogeneous LLM model. The models differ from each other in terms of architecture, vocabulary, or capabilities.

[0020] like Figure 2 As shown, this embodiment presents a heterogeneous LLM reliable optimization collaborative inference method based on the above scenario. It selects a subset of base stations participating in collaborative inference using a gating optimization allocation algorithm based on unsupervised learning. Subsequently, input prompts are transmitted to each base station in the base station subset according to the uplink transmit power, and each base station performs parallel autoregressive inference and uploads the results. The UE performs semantic alignment and unsupervised dynamic weighted fusion of the outputs from multiple heterogeneous LLMs on the local end to generate collaborative inference results with both high accuracy and low latency.

[0021] The unsupervised learning-based gating optimization allocation algorithm specifically includes:

[0022] Step 1, Semantic Alignment: Align input prompts and all historical outputs of LLM Each is a matrix and Projecting onto the relative semantic space yields input prompts. Projection in relative semantic space and historical output Projection in relative semantic space ;

[0023] Step 2, Unsupervised ensemble weight estimation: Estimating weights in the relative space using potential true values... Compared with historical output The set of projections in the relative semantic space A probabilistic graphical model is established between them to estimate the integrated weights, and then a closed estimator is derived to average the nearest neighbors as a set to obtain the smooth weights of the current request.

[0024] In the aforementioned probabilistic graphical model, probability ,in: It is a logarithmic function.

[0025] The closed-form estimator is specifically: ,in: It is the first The first LLM and the first The squared Euclidean distance of each LLM in the relative semantic space. The smoothing weights. ,in: For the current Based on historical database and Euclidean distance retrieval between A set of nearest neighbors.

[0026] Step 3: Based on the smoothing weight of the current request obtained in Step 2. The set of normalized path loss coefficients for all base stations Power allocation is obtained through power allocation based on graph attention network (GAT). Based on the smoothing weight of the current request obtained in step 2 and power distribution We use ranking-based joint gating to obtain an initial subset of base stations under constraints, and then use GAT training and ADMM asynchronous update to optimize power allocation and base station selection end-to-end.

[0027] The ranking-based joint gating specifically includes:

[0028] i) Construct a mean square error minimization problem under resource constraints Resource constraints are , ,pass This can be further transformed into an optimization problem of finding the optimal ensemble weights under resource constraints. .

[0029] ii) Approximate the time delay as a differentiable logarithmic summation exponential function. ,in: It is a smoothing hyperparameter; subsequently, an augmented Lagrangian function is constructed for the optimization problem. ,in: , Let the variables be Lagrange dual variables. It is an auxiliary variable. It is a positive penalty parameter.

[0030] The power allocation mentioned refers to the set concatenated by a graph attention network (GAT) based on the smoothing weights of the current request and the normalized path loss coefficients of all base stations. To obtain power allocation Then, the optimal subset of base stations under the constraints is selected, and the power allocation and base station selection are optimized end-to-end using GAT training and ADMM asynchronous update.

[0031] The graph attention network described has 8 layers, 4 attention heads, and uses the LeakyReLU activation function. Set it to 1000.

[0032] The aforementioned ,in: It is the output of the last layer of GAT.

[0033] The optimal subset of base stations is obtained in the following way: in total In the round selection process, each round selects base stations based on their augmented Lagrange function ranking, resulting in a new set: ,in: For the previously selected set of base stations, The remaining base station options are as follows. An empty set, meaning no new base stations are added;

[0034] The aforementioned GAT training refers to: performing GAT forward inference once every two GAT parameter updates, and updating the GAT auxiliary variables based on the GAT forward inference results. Lagrange dual variables The weights obtained from training are used for both model selection and subsequent output integration, compared to the weights obtained from training which are only used for model selection.

[0035] The loss function used in the GAT training is: ,in: For trajectory.

[0036] The GAT training described above uses Gumbel-Softmax for sampling during the forward propagation process; and employs a pass-through estimator (STE) during the backpropagation process to avoid non-differentiable argmax operations. Specifically: ,in: This is a hyperparameter.

[0037] The hyperparameters of the pass-through estimator Smoothing hyperparameters of summation of logarithmic and exponential functions There is an annealing process. They are initially assigned higher values ​​to encourage exploration, and then gradually reduced to approximate discrete decisions.

[0038] The aforementioned asynchronous update of ADMM refers to updating the auxiliary variables of the augmented Lagrange function after performing one inference process. Lagrange dual variables Specifically: , Among them: penalty factor During training, the values ​​are gradually increased from small to large. Initially, the smaller values ​​encourage extensive exploration, while at convergence, the larger values ​​impose strict delay constraints.

[0039] Through practical application experiments, on a hardware platform equipped with an Intel Xeon multi-core processor server, four NVIDIA RTX4090 GPUs and four NVIDIA RTX A6000 GPUs, and in a simulation environment built based on Python 3.10 and PyTorch 2.7.0+cu118 framework, the heterogeneous LLM reliable optimization collaborative inference method for the edge device side of this invention was run in Monte Carlo simulation mode. The experimental data obtained are as follows: the obtained weights are used for both model selection and subsequent output integration, compared to the obtained weights being used only for model selection, and a simple averaging baseline scheme is used during integration, which brings additional accuracy improvement: 0.4% improvement when selecting two LLMs, 0.65% improvement when selecting three LLMs, 0.2% improvement when selecting four LLMs, and 0.25% improvement when selecting five LLMs. Figure 7 As shown, this joint optimization framework consistently outperforms model-first or channel-first baseline schemes under dynamic channels and tasks, achieving higher inference accuracy, up to 76.40%, under the same time delay constraints. This represents a 2.95% improvement over a single best model baseline, and all results are achieved within the set resource constraints.

[0040] The evaluation is conducted over a square region with sides of 400m. The base station distribution is obtained through Poisson point simulation, as follows: Figure 3 The image shown is a simulation scenario diagram. The base station coordinates are... , , , and User coordinates are The fading effect is modeled as having a standard deviation of The model follows a zero-mean log-normal distribution. Parameter configurations include: an evaluation area of ​​400m × 400m, 5 base stations, a base station height of 35m, a bandwidth of 20MHz, a maximum transmit power limit of 27dBm, a path loss (dB) of 35+35log10(d), a noise power spectral density of -174 dBm / Hz, a standard deviation of 3 dB for shadow fading, a LLM cue size of 1.5Mbytes, datasets SQuAD and TriviaQA, and LLMs Mistral-7B, Vicuna-7B, Nous Capybara-7B, Gemma-7B, and Llama-2-7B used. Training steps are 8000, ADMM update frequency is every 2 steps, initial values ​​for μ (0), κ (0), ρ (1e2, 1e4), τ (1e7, 1e-1), and ζ (1e7, 1e1).

[0041] This embodiment first calculates the weight of each available LLM for the problem using an unsupervised weight estimation mechanism. For each question, only select the current weight. Ranked A single large language model is used, and the output of that single model is used to answer the question, with the following results: Figure 4 As shown in (a). Furthermore, the Spearman rank correlation coefficient between the true accuracy ranking of each independent model on the test dataset and the ranking based on weights was calculated, as shown in... Figure 4 As shown in (b).

[0042] like Figure 4 As shown in (a), the estimated weights There is a strong positive correlation between the accuracy and the accuracy of each individual LLM. Specifically, according to The highest-ranked model had a 0.5% higher accuracy than the lowest-ranked model. Figure 4 (b) shows that the average Spearman correlation coefficient between the model’s true performance and the weights is 0.7, which statistically confirms that the estimated weights can reliably reflect the model’s performance.

[0043] like Figure 5 As shown, different gating and integration strategies were compared. The following baselines were constructed: the optimal model baseline was a fixed selection of the single LLM with the best overall performance on the test set to handle all requests, representing the performance ceiling of a single model; the random gating + average weight baseline was a randomly selected... Each LLM is used, and its output is simply averaged and weighted; the proposed gating + average weighting baseline is based on... Select the top rankings The proposed LLM method uses average weights during fusion; the proposed gating + weighting scheme is based on weights. Before choosing The LLM, and using the same weights. Weighted fusion was performed. The number of LLMs participating in the collaboration was systematically varied, and the final inference accuracy of each of the above strategies under this setting was tested.

[0044] like Figure 5 As shown, all ensemble methods outperform the best model baseline, with the highest accuracy improvement reaching 2.95%. This result demonstrates the effectiveness of the relative semantic space and the advantages of multi-LLM collaboration. Under the same gating policy, as the number of LLMs participating in the collaboration increases, the additional accuracy improvement of the ensemble scheme of this invention compared to the average weighted scheme is as follows: 0.4% improvement when selecting two LLMs, 0.65% improvement when selecting three LLMs, 0.2% improvement when selecting four LLMs, and 0.25% improvement when selecting five LLMs.

[0045] like Figure 6 As shown, this invention, through asynchronous update and annealing strategies, enables the Gumbel-Softmax process to smoothly converge from exploration to a definite decision satisfying the constraints, dynamically and jointly outputting the optimal subset of selected base stations and power allocation. In the simulation, after the first round of selection, more power budget is retained compared to the baseline considering only model performance, thus enabling the selection of more base stations in the second round.

[0046] This invention is compared with two baseline schemes. The following baselines were constructed, with the model-first baseline being based solely on weights. To select a model, channel conditions are ignored; the channel-priority baseline is based solely on channel state conditions. To select a model, model performance is ignored; this invention jointly optimizes model selection and power allocation. First, dynamic gating based on Gumbel-Softmax is visualized. In order to select base stations The remaining power. The experimental configuration for the right axis is as follows: and .

[0047] like Figure 7 As shown, further, under different maximum time delay constraints The performance of each scheme was tested. The similar trends on both the left and right axes prove that minimizing the inference loss in the original problem has been transformed into maximizing it. The rationality of the proposed scheme is demonstrated. Under all latency settings, the proposed scheme matches or outperforms the two baseline schemes. When latency constraints are small, all schemes perform consistently, i.e., only one base station is selected. As latency constraints increase, the proposed scheme and the model-preferred baseline scheme will select... A higher LLM (Limited Least Mean Time) improves inference accuracy. In contrast, the channel-first approach remains constrained to the base station with the best channel until a larger delay budget is available. However, due to its channel-insensitive selection method, the model-first approach exhibits instability and typically selects fewer base stations than this invention under the same delay constraints.

[0048] Compared to existing technical solutions that typically only address one or two of the three problems—misalignment of semantic space in heterogeneous large models, missing labels in edge scenarios, or limited wireless resources—making it difficult to balance overall performance, this invention provides a collaborative inference framework for heterogeneous large language models for 6G edge networks. This framework systematically solves all three core challenges simultaneously. Furthermore, it requires no structural modifications or retraining of existing heterogeneous large language models or communication infrastructure, allowing for direct deployment and application in live network environments. This demonstrates significant engineering practicality and promotional value.

[0049] The above-described specific implementations can be partially adjusted by those skilled in the art in different ways without departing from the principles and purpose of the present invention. The scope of protection of the present invention is defined by the claims and is not limited to the above-described specific implementations. All implementation schemes within the scope of the claims are bound by the present invention.

Claims

1. A heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side, characterized in that, A gating optimization allocation algorithm based on unsupervised learning is used to select a subset of base stations to participate in collaborative inference. Subsequently, input prompts are transmitted to each base station in the base station subset according to the uplink transmit power, and each base station performs parallel autoregressive inference and uploads the results. The UE performs semantic alignment and unsupervised dynamic weighted fusion of the outputs from multiple heterogeneous LLMs on the local end to generate collaborative inference results with both high accuracy and low latency.

2. The heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side according to claim 1, characterized in that, The unsupervised learning-based gating optimization allocation algorithm specifically includes: Step 1, Semantic Alignment: Align input prompts and all historical outputs of LLM Each is a matrix and Projecting onto the relative semantic space yields input prompts. Projection in relative semantic space and historical output Projection in relative semantic space ; Step 2, Unsupervised ensemble weight estimation: Estimating weights in the relative space using potential true values... Compared with historical output The set of projections in the relative semantic space A probabilistic graphical model is established between them to estimate the integrated weights, and then a closed estimator is derived to average the nearest neighbors as a set to obtain the smooth weights of the current request. Step 3: Based on the smoothing weight of the current request obtained in Step 2. The set of normalized path loss coefficients for all base stations Power allocation is obtained through power allocation based on graph attention network (GAT). Based on the smoothing weight of the current request obtained in step 2 and power distribution We use ranking-based joint gating to obtain an initial subset of base stations under constraints, and then use GAT training and ADMM asynchronous update to optimize power allocation and base station selection end-to-end.

3. The heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side according to claim 2, characterized in that, In the aforementioned probabilistic graphical model, probability ,in: It is a logarithmic function; The closed-form estimator is specifically: ,in: It is the first The first LLM and the first The squared Euclidean distance of each LLM in the relative semantic space, the smoothing weight ,in: For the current Based on historical database and Euclidean distance retrieval between A set of nearest neighbors.

4. The heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side according to claim 2, characterized in that, The ranking-based joint gating specifically includes: i) Construct a mean square error minimization problem under resource constraints Resource constraints are , ,pass This can be further transformed into an optimization problem of finding the optimal ensemble weights under resource constraints. ; ii) Approximate the time delay as a differentiable logarithmic summation exponential function. ,in: After smoothing the hyperparameters, construct an augmented Lagrangian function for the optimization problem. ,in: , Let the variables be Lagrange dual variables. It is an auxiliary variable. It is a positive penalty parameter; The smoothing hyperparameter of the logarithmic summation exponential function Set up the annealing process.

5. The heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side according to claim 2, characterized in that, The power allocation mentioned refers to the set concatenated by a graph attention network (GAT) based on the smoothing weights of the current request and the normalized path loss coefficients of all base stations. To obtain power allocation After selecting the optimal subset of base stations under the constraints, power allocation and base station selection are optimized end-to-end using GAT training and ADMM asynchronous updates, specifically as follows: ,in: It is the output of the last layer of GAT.

6. The heterogeneous LLM reliable optimization collaborative reasoning method on the edge device side according to claim 5, characterized in that, The optimal subset of base stations is obtained in the following way: in total In the round selection process, each round selects base stations based on their augmented Lagrange function ranking, resulting in a new set: ,in: For the previously selected set of base stations, The remaining base station options are as follows. An empty set, meaning no new base stations are added.

7. The heterogeneous LLM reliable optimization collaborative reasoning method for the edge device side according to claim 2 or 5, characterized in that, The graph attention network described has 8 layers, 4 attention heads, and uses the LeakyReLU activation function. Set to 1000; The graph attention network is trained as follows: after every two GAT parameter updates, GAT forward inference is performed once, and the auxiliary variables of GAT are updated based on the GAT forward inference results. Lagrange dual variables The weights obtained from training are used for both model selection and subsequent output integration, compared to the weights obtained from training being used only for model selection. The loss function used in the training is: ,in: For trajectory; In the forward propagation, Gumbel-Softmax is used to implement the sampling process; in the backpropagation, a pass-through estimator (STE) is used to avoid non-differentiable argmax operations, specifically: ,in: For hyperparameters; The hyperparameters of the pass-through estimator An annealing process is included.

8. The heterogeneous LLM reliable optimization collaborative reasoning method for edge devices as described in claim 5, characterized in that, The aforementioned asynchronous update of ADMM refers to updating the auxiliary variables of the augmented Lagrange function after performing one inference process. Lagrange dual variables Specifically: , Among them: penalty factor During training, the values ​​are gradually increased from small to large. Initially, the smaller values ​​encourage extensive exploration, while at convergence, the larger values ​​impose strict delay constraints.

9. The heterogeneous LLM reliable optimization collaborative reasoning method for the edge device side according to any one of claims 1-8, characterized in that, specifically include: Step 1: When a UE initiates a natural language inference request, the UE first dynamically evaluates and selects a subset of base stations to participate in collaborative inference based on the current radio channel state, the potential performance of each LLM in the context of the request, and resource constraints. ,in: For input prompts, For the entire set of base stations, For the selected set of base stations, This is a gated optimization allocation algorithm based on unsupervised learning. For relative to the first Normalized path loss coefficient for each base station This is the set of normalized path loss coefficients for all base stations. Due to the maximum transmit power constraint, To maximize latency constraints, while optimizing the allocation of uplink transmit power; Step 2: After the UE measures the instantaneous channel state information of the base stations in each base station subset, it calculates the transmission capacity of each uplink and evaluates whether the overall transmission delay meets the constraints. Then, based on the transmit power allocated in Step 1, it transmits the input prompt through the uplink radio link. Send concurrently to the selected base station; Step 3, Receive the same input The base station uses its locally deployed LLM to perform parallel autoregressive inference, and after outputting the probability distribution of the corresponding token position to the UE in each decoding step, the UE performs semantic alignment and unsupervised dynamic weighted fusion of the outputs from multiple heterogeneous LLMs on the local end to generate a collaborative inference result with both high accuracy and low latency. The semantic alignment and unsupervised dynamic weighted fusion mentioned above refer to mapping the probability distribution of token positions to a shared relative semantic space, that is, for each LLM, the probability vector... Through relative matrix Projecting onto the relative semantic space specifically involves: ,in: A set of anchored tokens shared across models. For the original lexical space The cosine similarity between the embedding vector of the middle token and the embedding vector of the shared anchor token is used to determine the ensemble result of the output probabilities of each model. ,in: To be assigned to the The weights of each model are determined, and the entire process does not require modification of existing models or communication infrastructure, achieving efficient, flexible, and resource-aware collaborative services for large edge models.

10. A heterogeneous LLM reliable optimization collaborative inference system at the edge device side implementing the method of any one of claims 1-9, characterized in that, include: The system includes a dynamic gating and resource allocation module, an uplink communication management module, and a semantic alignment and weight fusion module. The dynamic gating and resource allocation module responds to input prompts. The set of normalized path loss coefficients for all base stations Maximum transmit power constraint and maximum delay constraint Through an unsupervised learning-based gating optimization allocation algorithm The process is performed to output the final integrated weight set. Selected set of base stations and the transmit power allocated by the UE to each selected base station The uplink communication management module uses the selected base station set output by the dynamic gating and resource allocation module. and the transmit power allocated to each selected base station Bandwidth B and power spectral density of additive white Gaussian noise The uplink capacity is calculated based on Shannon's formula. And the maximum transmission latency D in the uplink, and manage the prompt text. Concurrent transmission to the selected base station; The semantic alignment and weight fusion module aligns the text returned by each selected base station based on the same prompt text. Heterogeneous LLM output probability vector Using the relative matrices corresponding to each LLM Each probability vector Projecting onto the relative semantic space yields semantic vectors. Subsequently, based on the final integrated weight set provided by the dynamic gating and resource allocation module... Then based on semantic vectors and the final integrated weight set We perform a weighted summation to obtain the semantically aligned ensemble probability vector. It also decodes and generates natural language output results for collaborative reasoning.

Images