Wireless sensing speculative decoding method and system for edge-end collaborative large model inference

Abstract

This invention provides a wireless sensing speculative decoding method and system for edge-device collaborative large model inference. The method includes deploying a draft model on a terminal device and a target model on an edge server; the terminal generates a candidate lexical sequence and transmits the candidate lexical sequence index information to the edge server; the edge server performs forward verification of the target model to obtain the hidden state and lexical probability distribution of the target model and acquire the current channel state information; the edge server inputs the obtained draft model hidden state, target model hidden state, and channel state information into a pre-trained semantic verification decision head to calculate the semantic rejection probability; by determining whether a candidate lexical is accepted, the confirmed lexical sequence is updated, and the process is iteratively executed until an end marker is generated or the maximum generation length is reached. This invention can reduce edge-device interaction overhead and end-to-end inference latency while ensuring inference accuracy, thereby improving the inference efficiency of large models in wireless edge-device collaborative scenarios.

Description

Technical Field

[0001] This invention relates to the intersection of communication technology and large model inference, specifically to a wireless sensing semantic verification speculative decoding method and system for edge-to-end collaborative 6G large model inference. Background Technology

[0002] With the deep integration of large language models and 6G wireless communication technology, edge-device collaborative distributed inference has become an important technical direction for low-latency, high-concurrency AI services. Under the realities of limited terminal computing power, high pressure on centralized inference in the cloud, and dynamic fluctuations in wireless channels, a collaborative architecture that deploys lightweight draft models on terminals and high-precision target models on edge servers can effectively share the computing load and improve service response speed, becoming the mainstream technical path for deploying large models in wireless scenarios.

[0003] To improve the efficiency of autoregressive generation, existing speculative decoding and distributed edge-end collaborative solutions generally adopt a terminal generation and edge verification model. A small terminal model quickly generates candidate lexical units, while a large edge model performs parallel verification to reduce the overhead of word-by-word generation by the large model. However, existing technologies have the following significant drawbacks:

[0004] First, traditional verification mechanisms employ strict word-level matching rules, directly rejecting candidates whose words do not match the target model's most probable word. This leads to the misjudgment and discarding of many semantically reasonable but word-different results, resulting in low word acceptance rates, short average acceptance lengths, and a significant increase in rollback and interaction rounds. Second, existing distributed speculative decoding does not fully consider the dynamic characteristics of wireless links, employing a fixed transmission strategy regardless of channel conditions. This either results in wasted uplink bandwidth due to full uploading of the hidden state or frequent retransmissions leading to a sharp increase in latency, a problem particularly pronounced in weak network and high-latency scenarios. Third, existing semantic verification research only focuses on semantic consistency at the model output level, failing to incorporate wireless channel conditions such as signal-to-noise ratio, channel quality, round-trip time, and available bandwidth into the decision model. This prevents an adaptive trade-off between verification accuracy and communication costs based on link conditions. Furthermore, existing technologies lack refined hidden state transmission mechanisms for mismatch locations, generally employing a simple strategy of full uploading or no uploading, causing unnecessary communication overhead or insufficient verification information, further exacerbating multi-round interactions and increased latency.

[0005] The aforementioned issues collectively result in high end-to-end latency, poor stability, and unreasonable resource utilization for edge-to-end collaborative large model inference in wireless environments, making it difficult to meet the low-latency and high-reliability inference requirements of 6G scenarios.

[0006] A patent search revealed invention patent CN120031128A, which discloses an LLM speculative decoding optimization method and system for downstream tasks. The method includes the following steps: constructing multiple aligned datasets for different tasks and using these datasets to build a heterogeneous draft model pool; obtaining the cue text of the downstream task, classifying the input task based on the cue text using a task classification mechanism, and obtaining the classification result; selecting the optimal draft model from the heterogeneous draft model pool based on the classification result; generating a sequence of guessed tokens using the selected draft model, and inputting the guessed tokens sequence into the target model for parallel verification to complete the speculative decoding optimization. This patent lacks a hidden state dynamic upload mechanism and channel information fusion, only optimizing task adaptability, and does not address the communication overhead and latency issues in wireless scenarios.

[0007] In summary, given the problems of the existing technologies, researching a wireless sensing speculative decoding method and system for edge-to-edge collaborative large-model inference has become a critical task that urgently needs to be addressed. Summary of the Invention

[0008] To address the shortcomings of existing technologies, the purpose of this invention is to provide a wireless sensing speculative decoding method and system for edge-to-edge collaborative large model inference.

[0009] A wireless sensing speculative decoding method for edge-to-edge collaborative large model inference, provided by the present invention, includes the following steps: Step S1: Deploy the draft model on the terminal device and the target model on the edge server; Step S2: The terminal device generates a candidate word sequence based on the confirmed word sequence using an autoregressive method, and retains the draft model hidden state of each candidate word position; according to the wireless link conditions, the index information of the candidate word sequence is transmitted to the edge server. Step S3: The edge server receives the index information of the candidate word sequence, performs forward verification of the target model, obtains the hidden state of the target model corresponding to each candidate word position and the word probability distribution output by the target model; and obtains the current wireless channel state information. Step S4: The edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit; Step S5: The edge server inputs the draft model hidden state, the target model hidden state, and the channel state information into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate word. Step S6: The edge server determines whether each candidate word is accepted based on the comparison between the semantic rejection probability of each candidate word and the preset threshold. Step S7: Based on the determination result, update the confirmed word sequence and return to step S2 to continue the next round of iteration until an end marker is generated or the maximum generation length is reached.

[0010] Preferably, in step S2, the terminal device simultaneously obtains the draft model hidden state of the candidate word position when generating each candidate word.

[0011] Preferably, step S2, which involves transmitting the index information of the candidate lexical sequence to the edge server based on the wireless link conditions, includes: When the round-trip time of the wireless link is less than the first threshold and the uplink transmission rate is lower than the second threshold, a selective hidden state upload mechanism is adopted: the terminal device only uploads the index information of the candidate word sequence; When the round-trip time of the wireless link is greater than or equal to the first threshold and the uplink transmission rate is higher than or equal to the second threshold, a fully hidden state upload mechanism is adopted: the terminal device uploads the draft model hidden state of each candidate word position to the edge server along with the index information of the candidate word sequence.

[0012] Preferably, in step S3, the channel state information includes one or more of the following: signal-to-noise ratio, reference signal received power, channel quality indication, round-trip time, and uplink available bandwidth.

[0013] Preferably, in step S3, the edge server compares each candidate word in the candidate word sequence with the word with the highest probability at the same candidate word position in the word probability distribution output by the target model; if the two are different, the candidate word position is marked as a mismatch position; and all candidate words are traversed to obtain a set of mismatch positions.

[0014] Preferably, in step S4, the edge server obtains the hidden state of the draft model corresponding to each candidate lexical position in the following way: If a selective hidden state upload mechanism is adopted, the edge server requests the hidden state of the draft model at the corresponding position from the terminal device based on the mismatch position set. After receiving the request, the terminal device sends the hidden state of the draft model at the specified position back to the edge server. If a fully hidden state upload mechanism is adopted, the edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit from the data received in step S2.

[0015] Preferably, in step S5, the semantic verification decision head is a multilayer perceptron or a small neural network, whose input is a concatenated vector of the draft model hidden state, the target model hidden state and the channel state information, and whose output is a binary classification probability.

[0016] Preferably, the semantic verification decision head training process includes: Step U1: Input the inference task text into the draft model and the target model respectively to obtain the first word sequence and the second word sequence; Step U2: Record the positions where the lexical units in the first lexical unit sequence and the second lexical unit sequence are inconsistent, and record the hidden layer features of the draft model and the target model at the lexical unit position during the inference process. Replace the lexical unit at the position in the first lexical unit sequence with the lexical unit at the position in the second lexical unit sequence to obtain the corrected first lexical unit sequence. Step U3: Input the corrected first word sequence into the target model to continue generating subsequent words and obtain the final output result; Step U4: Compare the final output with the actual result of the reasoning task text. If they are inconsistent, mark the word corresponding to the replacement position in step U2 as an important word, and otherwise mark it as an unimportant word. Step U5: Obtain channel state information and adjust the importance labels of words using a Lagrange random strategy. The lower the channel quality, the lower the importance of important words. Obtain the relabeled word importance labels and combine them with the hidden layer features recorded in step U2 to form the relabeled training dataset. Step U6: Supervised training of the semantic verification decision head is performed using the re-labeled training dataset, enabling the semantic verification decision head to learn the acceptance or rejection decisions of candidate lexical units under different channel states.

[0017] Preferably, in step S7, the method for updating the confirmed word sequence is as follows: If every candidate word in the candidate word sequence is accepted, the terminal device will merge the candidate word sequence into the confirmed word sequence. If there are rejected candidate words in the candidate word sequence, the first rejected position is taken as the earliest rejection position, and all accepted candidate words before the earliest rejection position are merged into the confirmed word sequence; the edge server returns a correction word corresponding to the earliest rejection position to the terminal device; the terminal device merges the correction word into the confirmed word sequence; all candidate words after the earliest rejection position are discarded.

[0018] This invention also provides a wireless sensing speculative decoding system based on edge-end collaborative large model inference, employing the aforementioned wireless sensing speculative decoding method based on edge-end collaborative large model inference, comprising: Module M1 deploys the draft model on the terminal device and the target model on the edge server; Module M2: The terminal device generates candidate word sequences based on the confirmed word sequence using an autoregressive method, and retains the draft model hidden state of each candidate word position; according to the wireless link conditions, it transmits the index information of the candidate word sequence to the edge server. Module M3, the edge server, receives the index information of the candidate lexical sequence, performs forward verification of the target model, obtains the hidden state of the target model corresponding to each candidate lexical position and the lexical probability distribution output by the target model; and obtains the current wireless channel state information; Module M4, the edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit; Module M5, the edge server inputs the draft model hidden state, the target model hidden state and the channel state information into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate word. Module M6, the edge server determines whether each candidate word is accepted based on the comparison between the semantic rejection probability of each candidate word and the preset threshold. Module M7 updates the confirmed word sequence based on the judgment result and returns to module M2 to continue the next round of iteration until an end marker is generated or the maximum generation length is reached.

[0019] Compared with the prior art, the present invention has the following beneficial effects: 1. This invention separates the candidate lexical generation and verification tasks by deploying a draft model on the terminal and the target model on the edge server in an edge-end collaborative architecture. This fully utilizes the computing power of the terminal and reduces the centralized computing pressure on the edge server, avoids the single-node computing power bottleneck, and improves the overall computing power utilization efficiency and scalability of the system.

[0020] 2. Based on the round-trip delay of the wireless link and the uplink transmission rate, this invention adaptively adopts a selective hidden state uplink mechanism or a fully hidden state uplink mechanism. When the channel quality is good, unnecessary hidden state transmission is reduced, and when the channel quality is poor, frequent retransmissions and additional interactions are avoided, effectively reducing the amount of uplink transmission data and communication redundancy in wireless scenarios.

[0021] 3. This invention introduces a semantic verification decision head driven by the hidden state of the draft model, the hidden state of the target model, and channel state information. It replaces the traditional strict lexical matching verification with semantic acceptability, which can retain candidate results that are semantically consistent but whose lexical matching is not complete, improve the average lexical acceptance length, and reduce the number of rollbacks and interaction rounds caused by frequent rejections.

[0022] 4. In the target model verification stage, the present invention first marks the set of mismatch positions and only requests the hidden state of the draft model at the corresponding position as needed, avoiding the transmission of the full hidden state, further reducing the amount of data interaction between the edge and the device, and the latency improvement effect is significant, especially in weak network and high latency scenarios.

[0023] 5. The semantic verification decision head of this invention combines the differences in real word sequences to mark important words during the training phase, and dynamically adjusts the weights according to the channel quality, so that the decision model is more in line with the fluctuation characteristics of the wireless channel and improves the accuracy and stability of semantic verification under different link conditions.

[0024] 6. After the word acceptance judgment, the present invention updates the confirmed word sequence according to the rules, directly confirms the valid word before the first rejected position and corrects only the word at that position, reduces the transmission and calculation of invalid candidate words, further reduces end-to-end inference latency, and improves the real-time performance and deployment feasibility of large models in wireless edge-end collaborative scenarios. Attached Figure Description

[0025] Other features, objects, and advantages of the present invention will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings: Figure 1 This is the edge-to-edge collaborative reasoning structure of Embodiment 1 of the present invention; Figure 2 This is the edge-to-edge collaborative reasoning structure of Embodiment 2 of the present invention. Detailed Implementation

[0026] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make several changes and improvements without departing from the concept of the present invention. These all fall within the protection scope of the present invention.

[0027] This invention discloses a wireless sensing speculative decoding method and system for edge-device collaborative large model inference, aiming to solve the problems of low draft model lexical acceptance rate, numerous edge-device interaction rounds, and high end-to-end inference latency in edge-device collaborative distributed speculative decoding. The method deploys a draft model on the terminal device and a target model on the edge server. The terminal generates a candidate lexical sequence based on the confirmed lexical sequence through autoregression and retains the hidden state of the draft model at each position. Depending on the wireless link conditions, it uses a selective hidden state upload or full hidden state upload mechanism to transmit the candidate lexical sequence index information to the edge server. The edge server performs forward verification of the target model on the candidate lexical sequence to obtain the target model hidden state and lexical probability distribution, and acquires the current channel state information. After acquiring the corresponding draft model hidden state, the edge server inputs the draft model hidden state, the target model hidden state, and the channel state information into a pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate lexical. Based on the comparison between the semantic rejection probability and a preset threshold, it sequentially determines whether the candidate lexical is accepted and updates the confirmed lexical sequence according to the determination results. This process is iteratively executed until an end marker is generated or the maximum generation length is reached. This invention can reduce edge-to-edge interaction overhead and end-to-end inference latency while ensuring inference accuracy, thereby improving the inference efficiency of large models in wireless edge-to-edge collaborative scenarios.

[0028] Example 1: In this embodiment, the round-trip latency of the wireless link is relatively small (less than 5ms) and the uplink transmission rate is relatively low (less than 10Mbps). Based on the communication and computing characteristics of this scenario, this embodiment adopts a selective hidden state uploading mechanism to complete distributed speculative decoding, so as to minimize the uplink hidden state transmission burden while controlling additional interaction latency.

[0029] Figure 1 This is the edge-to-edge collaborative reasoning structure of Embodiment 1 of the present invention.

[0030] like Figure 1 As shown in the figure, the wireless sensing speculative decoding method for edge-to-edge collaborative large model inference provided in this embodiment includes the following steps: Step S1: Deploy a lightweight draft model on the terminal device and a high-precision target model on the edge server.

[0031] Step S2: Based on the confirmed lexical sequence, the draft model continuously generates candidate lexical sequences using an autoregressive approach. The candidate lexical sequence is a continuous sequence of candidate lexical units of length K (K=5 in this embodiment), and the draft model's hidden state at each candidate lexical position is stored locally (the final layer hidden state in this embodiment).

[0032] According to the wireless link conditions of this embodiment (round-trip latency less than 5ms and uplink transmission rate less than 10Mbps), the conditions for the selective hidden state upload mechanism are met. Therefore, the selective hidden state upload mechanism is adopted: the terminal device only uploads the index information of the candidate word sequence, and does not immediately upload the draft model hidden state of each candidate word position, thereby reducing the uplink transmission overhead under low-rate link.

[0033] In step S3, the edge server receives the index information of the candidate word sequence, performs forward verification of the target model based on the confirmed word sequence and the received candidate words, and obtains the hidden state of the target model corresponding to each candidate word position and the word probability distribution output by the target model.

[0034] Subsequently, the edge server compares each candidate word in the candidate word sequence with the word with the highest probability at the same candidate word position in the word probability distribution output by the target model; if the two are different, the candidate word position is marked as a mismatch position; all candidate words are traversed to obtain a set of mismatch positions.

[0035] At the same time, the edge server obtains the current wireless channel status information.

[0036] Specifically, channel state information includes one or more of the following: signal-to-noise ratio, reference signal received power, channel quality indication, round-trip time, and uplink available bandwidth.

[0037] In this embodiment, the current channel state information is: signal-to-noise ratio (SNR) = 8dB, round-trip time (RTT) = 4ms.

[0038] Step S4: The edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit; Specifically, the edge server obtains the hidden state of the draft model corresponding to each candidate lexical position as follows: If a selective hidden state upload mechanism is adopted, the edge server requests the hidden state of the draft model at the corresponding position from the terminal device based on the set of mismatch positions. After receiving the request, the terminal device extracts the hidden state of the draft model at these specified positions from the local cache and sends it back to the edge server. If a fully hidden state upload mechanism is adopted, the edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit from the data received in step S2.

[0039] In step S5, the edge server concatenates the hidden state of the draft model, the hidden state of the target model, and the current channel state information, and inputs them into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate lexical.

[0040] In this embodiment, the semantic verification decision head is a three-layer multilayer perceptron. Its input is a concatenated vector of the draft model hidden state, the target model hidden state, and the channel state information, and its output is a binary classification probability.

[0041] In step S6, the edge server determines whether each candidate word is accepted based on the comparison between the semantic rejection probability of each candidate word and a preset threshold (e.g., 0.5). Specifically, if the semantic rejection probability is lower than the threshold, the candidate word is accepted; if the semantic rejection probability is higher than the threshold, it is rejected.

[0042] Step S7: Based on the determination result, update the confirmed word sequence and return to step S2 to continue the next round of iteration until an end marker is generated or the maximum generation length is reached.

[0043] Specifically, the methods for updating the confirmed lexical sequence include: If every candidate lexical in the candidate lexical sequence is accepted, the terminal device will merge the candidate lexical sequence into the confirmed lexical sequence.

[0044] If there are rejected candidate words in the candidate word sequence, the first rejected position is taken as the earliest rejection position, and all accepted candidate words before the earliest rejection position are merged into the confirmed word sequence; the edge server returns a correction word corresponding to the earliest rejection position to the terminal device; the terminal device merges the correction word into the confirmed word sequence; all candidate words after the earliest rejection position are discarded.

[0045] Before performing the above online inference steps, the semantic verification decision head needs to be pre-trained. This embodiment provides a preferred training method as follows: Step U1: Input the inference task text into the draft model and the target model respectively to obtain the first word sequence and the second word sequence; Step U2: Record the positions where the lexical units in the first lexical unit sequence and the second lexical unit sequence are inconsistent, and record the hidden layer features of the draft model and the target model at the lexical unit position during the inference process. Replace the lexical unit at the position in the first lexical unit sequence with the lexical unit at the position in the second lexical unit sequence to obtain the corrected first lexical unit sequence. Step U3: Input the corrected first word sequence into the target model to continue generating subsequent words and obtain the final output result; Step U4: Compare the final output with the actual result of the reasoning task text. If they are inconsistent, mark the word corresponding to the replacement position in step U2 as an important word, and otherwise mark it as an unimportant word. Step U5: Obtain channel state information and adjust the importance labels of words using a Lagrange random strategy. The lower the channel quality, the lower the importance of important words. Obtain the relabeled word importance labels and combine them with the hidden layer features recorded in step U2 to form the relabeled training dataset. Step U6: Supervised training of the semantic verification decision head is performed using the re-labeled training dataset, enabling the semantic verification decision head to learn the acceptance or rejection decisions of candidate lexical units under different channel states.

[0046] After training, the decision head is deployed on the edge server. When the channel quality is lower than the preset threshold (e.g., SNR < 10dB), the semantic verification decision head will naturally exhibit the characteristic of relaxing the acceptance condition because the channel cost has been introduced through the Lagrange strategy during training. That is, the rejection probability is lower under the same semantic matching degree.

[0047] Example 2: In this embodiment, a wireless link scenario with a large round-trip latency (greater than 20ms) and a high uplink transmission rate (greater than 50Mbps) is considered. Based on the communication and computing characteristics in this scenario, a fully hidden state upload mechanism is adopted to complete distributed speculative decoding, thereby reducing the overhead of additional interaction rounds and reducing the latency accumulation caused by multiple round-trip communications under high RTT conditions.

[0048] Figure 2 This is the edge-to-edge collaborative reasoning structure of Embodiment 2 of the present invention.

[0049] like Figure 2 As shown, the processing flow of this embodiment differs from that of Embodiment 1 in that: Step S2: Based on the confirmed lexical sequence, the draft model continuously generates candidate lexical sequences using an autoregressive approach, and retains the hidden state of the draft model at each candidate lexical position locally. According to the wireless link conditions of this embodiment (round-trip latency greater than 20ms and uplink transmission rate greater than 50Mbps), the conditions for a fully hidden state upload mechanism are met. Therefore, a fully hidden state upload mechanism is adopted: the terminal device uploads the hidden state of the draft model at each candidate lexical position along with the index information of the candidate lexical sequence to the edge server, without waiting for position-by-position requests from the edge side. This strategy avoids the extra round-trip of "request-response" in high RTT scenarios, significantly reducing the total latency.

[0050] In step S3, the edge server receives the index information of the candidate lexical sequence and the draft model hidden state of each candidate lexical position. Based on the confirmed lexical sequence and the received candidate lexicals, it performs forward verification of the target model to obtain the target model hidden state corresponding to each candidate lexical position and the lexical probability distribution output by the target model.

[0051] Subsequently, the edge server compares each candidate word in the candidate word sequence with the word with the highest probability at the same candidate word position in the word probability distribution output by the target model; if the two are different, the candidate word position is marked as a mismatch position; all candidate words are traversed to obtain a set of mismatch positions.

[0052] At the same time, the edge server obtains the current wireless channel status information.

[0053] In step S4, due to the adoption of the fully hidden state upload mechanism, the edge server directly obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit from the data received in step S2, without the need for additional requests.

[0054] The other steps are the same as the corresponding steps in Example 1.

[0055] Example 3: The present invention also provides a wireless sensing speculative decoding system for edge-end collaborative large model inference. The wireless sensing speculative decoding system for edge-end collaborative large model inference can be implemented by executing the process steps of the wireless sensing speculative decoding method for edge-end collaborative large model inference. That is, those skilled in the art can understand the wireless sensing speculative decoding method for edge-end collaborative large model inference as a preferred embodiment of the wireless sensing speculative decoding system for edge-end collaborative large model inference.

[0056] Specifically, this edge-to-edge collaborative large-model inference wireless sensing speculative decoding system includes: Module M1 deploys the draft model on the terminal device and the target model on the edge server; Module M2: The terminal device generates candidate word sequences based on the confirmed word sequence using an autoregressive method, and retains the draft model hidden state of each candidate word position; according to the wireless link conditions, it transmits the index information of the candidate word sequence to the edge server. Module M3, the edge server, receives the index information of the candidate lexical sequence, performs forward verification of the target model, obtains the hidden state of the target model corresponding to each candidate lexical position and the lexical probability distribution output by the target model; and obtains the current wireless channel state information; Module M4, the edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit; Module M5, the edge server inputs the draft model hidden state, the target model hidden state and the channel state information into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate word. Module M6, the edge server determines whether each candidate word is accepted based on the comparison between the semantic rejection probability of each candidate word and the preset threshold. Module M7 updates the confirmed word sequence based on the judgment result and returns to module M2 to continue the next round of iteration until an end marker is generated or the maximum generation length is reached.

[0057] Those skilled in the art will understand that, besides implementing the system and its various devices, modules, and units provided by this invention in the form of purely computer-readable program code, the same functions can be achieved entirely through logical programming of the method steps, making the system and its various devices, modules, and units of this invention function in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, the system and its various devices, modules, and units provided by this invention can be considered as a hardware component, and the devices, modules, and units included therein for implementing various functions can also be considered as structures within the hardware component; alternatively, the devices, modules, and units for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0058] Specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various changes or modifications within the scope of the claims, which do not affect the essence of the present invention. Unless otherwise specified, the embodiments and features described in this application can be arbitrarily combined with each other.

Claims

1. A wireless sensing speculative decoding method for edge-to-edge collaborative large-scale model inference, characterized in that, Includes the following steps: Step S1: Deploy the draft model on the terminal device and the target model on the edge server; Step S2: The terminal device generates a candidate word sequence based on the confirmed word sequence using an autoregressive method, and retains the draft model hidden state of each candidate word position; Based on the wireless link conditions, the index information of the candidate lexical sequence is transmitted to the edge server; Step S3: The edge server receives the index information of the candidate lexical sequence, performs forward verification of the target model, obtains the hidden state of the target model corresponding to each candidate lexical position and the lexical probability distribution output by the target model; and obtains the current wireless channel state information. Step S4: The edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit; Step S5: The edge server inputs the draft model hidden state, the target model hidden state, and the channel state information into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate lexical. Step S6: The edge server determines whether each candidate word is accepted based on the comparison between the semantic rejection probability of each candidate word and a preset threshold. Step S7: Based on the determination result, update the confirmed word sequence and return to step S2 to continue the next iteration until an end marker is generated or the maximum generation length is reached.

2. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S2, the terminal device simultaneously obtains the draft model hidden state of the candidate word position when generating each candidate word.

3. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, Step S2, which involves transmitting the index information of the candidate lexical sequence to the edge server based on the wireless link conditions, includes: When the round-trip time of the wireless link is less than the first threshold and the uplink transmission rate is lower than the second threshold, a selective hidden state upload mechanism is adopted: the terminal device only uploads the index information of the candidate word sequence; When the round-trip time of the wireless link is greater than or equal to the first threshold and the uplink transmission rate is higher than or equal to the second threshold, a fully hidden state upload mechanism is adopted: the terminal device uploads the draft model hidden state of each candidate word position together with the index information of the candidate word sequence to the edge server.

4. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S3, the channel state information includes one or more of the following: signal-to-noise ratio, reference signal received power, channel quality indication, round-trip time, and uplink available bandwidth.

5. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S3, the edge server compares each candidate word in the candidate word sequence with the word with the highest probability at the same candidate word position in the word probability distribution output by the target model; if the two are different, the candidate word position is marked as a mismatch position; and all candidate words are traversed to obtain a set of mismatch positions.

6. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 3, characterized in that, In step S4, the edge server obtains the hidden state of the draft model corresponding to each candidate lexical position in the following way: If a selective hidden state upload mechanism is adopted, the edge server requests the hidden state of the draft model at the corresponding position from the terminal device according to the mismatch position set. After receiving the request, the terminal device sends the hidden state of the draft model at the specified position back to the edge server. If a fully hidden state upload mechanism is adopted, the edge server obtains the hidden state of the draft model corresponding to the position of each candidate lexical unit from the data received in step S2.

7. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S5, the semantic verification decision head is a multilayer perceptron or a small neural network. Its input is a concatenated vector of the draft model hidden state, the target model hidden state, and the channel state information, and its output is a binary classification probability.

8. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S6, if the semantic rejection probability of a candidate word is lower than a preset threshold, the candidate word is determined to be accepted; if the semantic rejection probability of a candidate word is higher than the preset threshold, it is determined to be rejected.

9. The wireless sensing speculative decoding method for edge-to-edge collaborative large model inference according to claim 1, characterized in that, In step S7, the confirmed word sequence is updated in the following way: If each candidate word in the candidate word sequence is accepted, the terminal device will merge the candidate word sequence into the confirmed word sequence. If there are rejected candidate words in the candidate word sequence, the first rejected position is taken as the earliest rejection position, and all accepted candidate words before the earliest rejection position are incorporated into the confirmed word sequence; the edge server returns a corrected word corresponding to the earliest rejection position to the terminal device; the terminal device incorporates the corrected word into the confirmed word sequence; all candidate words after the earliest rejection position are discarded.

10. A wireless sensing speculative decoding system for edge-end collaborative large model inference, employing the wireless sensing speculative decoding method for edge-end collaborative large model inference as described in any one of claims 1-9, characterized in that, include: Module M1 deploys the draft model on the terminal device and the target model on the edge server; Module M2, the terminal device generates candidate word sequences based on the confirmed word sequence using an autoregressive method, and retains the draft model hidden state of each candidate word position; Based on the wireless link conditions, the index information of the candidate lexical sequence is transmitted to the edge server; Module M3, the edge server receives the index information of the candidate lexical sequence, performs forward verification of the target model, obtains the hidden state of the target model corresponding to each candidate lexical position and the lexical probability distribution output by the target model; and obtains the current wireless channel state information; Module M4, the edge server obtains the draft model hidden state corresponding to the position of each candidate lexical unit; Module M5, the edge server inputs the draft model hidden state, the target model hidden state and the channel state information into the pre-trained semantic verification decision head to calculate the semantic rejection probability of each candidate word; Module M6, the edge server determines whether each candidate word is accepted based on the comparison result of the semantic rejection probability of each candidate word with a preset threshold; Module M7 updates the confirmed word sequence based on the judgment result and returns to module M2 to continue the next round of iteration until an end marker is generated or the maximum generation length is reached.

Images